Buildout

Merge lp:~gary/zc.buildout/betafix8 into lp:zc.buildout

  1. betafix8
  2. Merge into trunk
Proposed by Gary Poster
Status: Needs review
Proposed branch: lp:~gary/zc.buildout/betafix8
Merge into: lp:zc.buildout
Prerequisite: lp:~gary/zc.buildout/betafix7
Diff against target: 526 lines (+155/-80)
6 files modified
CHANGES.txt (+14/-8)
src/zc/buildout/bootstrap.txt (+43/-26)
src/zc/buildout/buildout.py (+42/-17)
src/zc/buildout/buildout.txt (+12/-13)
src/zc/buildout/tests.py (+11/-9)
src/zc/buildout/update.txt (+33/-7)
To merge this branch: bzr merge lp:~gary/zc.buildout/betafix8
Reviewer Review Type Date Requested Status
Francis J. Lacoste (community) Approve
Review via email: mp+33077@code.launchpad.net

Description of the change

In the review of betafix6 (https://code.edge.launchpad.net/~gary/zc.buildout/betafix6/+merge/31996), Francis had some concerns about the usability of the switch to use testing versions (alphas, betas) of build dependencies. This branch addresses those concerns.

I am slightly abusing command-line overrides for the new option, but I think it is a reasonable compromise in practice.

In light of these changes, I also changed the pertinent option names. Before, we had tightly related behavior called "--accept-early-release" in bootstrap and "prefer-final-build-system" in buildout.cfg. I changed these to "--accept-buildout-test-releases" and "accept-buildout-test-releases" respectively. I think the similarity is appropriate. Moreover, I prefer boolean options to default to false, so this name in buildout appeals to me. The only mild downside is that the option in buildout to prefer final releases of the dependencies of your own software (the software buildout is supposed to be helping you develop or deploy) is "prefer-final." The old name ("prefer-final-build-system") was a better parallel.

To post a comment you must log in.
Revision history for this message
Francis J. Lacoste (flacoste) wrote :

This is great, thanks for doing this.

I think losing the parallel with prefer-final is not important since that's really an 'internal' configuration value. The API is the command-line switch.

review: Approve
Reply

Unmerged revisions

565. By Gary Poster

make it much simpler to try out new buildout versions: instead of having to make a gesture in bootstrap *and* buildout.cfg, you can make the gesture once in bootstrap. Changed pertinent option names to reflect new thinking.

564. By Gary Poster

merge from trunk/previous branches

563. By Gary Poster

docs, test fixes and cleanups

562. By Gary Poster

By default, Buildout and the bootstrap script now prefer final versions of Buildout, recipes, and extensions.

561. By Gary Poster

get tests passing for Python 2.7. Also includes Lennart Regebro changes to use the standard library doctest.

560. By Gary Poster

small Python 2.6 bugfix and Distribute test clean-ups

559. By Gary Poster

fix tests for changes

558. By Gary Poster

mimic standard site.py behavior for inclusion of .pth files

557. By Gary Poster

eliminate spurious warning if you are using distribute.

556. By Gary Poster

add files so releases can be made from a non-SVN checkout

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'CHANGES.txt'
2--- CHANGES.txt 2010-08-19 02:27:43 +0000
3+++ CHANGES.txt 2010-08-19 02:27:43 +0000
4@@ -9,17 +9,23 @@
5 - zc.buildout supports Python 2.7.
6
7 - By default, Buildout and the bootstrap script now prefer final versions of
8- Buildout, recipes, and extensions. This can be changed by setting
9- ``prefer-final-build-system = false`` in your configuration's
10- [buildout] section, and by using the --accept-early-release flag when
11- calling bootstrap. This will hopefully allow beta releases to be more
12- easily and safely made in the future. Note that dependencies of your
13- software do not have this behavior: use the pre-existing switch
14- ``prefer-final = true`` to get this behavior.
15+ Buildout, recipes, and extensions. This can be changed by using the
16+ --accept-buildout-test-releases flag (or -t for short) when calling
17+ bootstrap. This will hopefully allow beta releases of these items to
18+ be more easily and safely made in the future.
19+
20+ NOTE: dependencies of your own software are not affected by this new
21+ behavior. Buildout continues to choose the newest available versions
22+ of your dependencies regardless of whether they are final releases. To
23+ prevent this, use the pre-existing switch ``prefer-final = true`` in
24+ the [buildout] section of your configuration file (see
25+ http://pypi.python.org/pypi/zc.buildout#preferring-final-releases) or
26+ pin your versions using a versions section (see
27+ http://pypi.python.org/pypi/zc.buildout#repeatable-buildouts-controlling-eggs-used).
28
29 Bugs fixed:
30
31-- You can now again use virtualenv with zc.buildout. The new features to let
32+- You can now again use virtualenv with Buildout. The new features to let
33 buildout be used with a system Python are disabled in this configuration,
34 and the previous script generation behavior (1.4.3) is used, even if
35 the new function ``zc.buildout.easy_install.sitepackage_safe_scripts``
36
37=== modified file 'src/zc/buildout/bootstrap.txt'
38--- src/zc/buildout/bootstrap.txt 2010-08-19 02:27:43 +0000
39+++ src/zc/buildout/bootstrap.txt 2010-08-19 02:27:43 +0000
40@@ -93,23 +93,14 @@
41 specify an explicit version (using --version here and specifying the
42 version in the buildout configuration file using the
43 ``buildout-version`` option or the ``versions`` option) or specify that you
44-accept early releases.
45-
46-You accept early releases by using ``--accept-early-release`` on the
47-bootstrap script and specifying ``prefer-final-build-system = false`` in the
48-buildout configuration file. You must do both.
49+accept early releases by using ``--accept-buildout-test-releases`` on the
50+bootstrap script.
51
52 Here's an example.
53
54- >>> write('buildout.cfg',
55- ... '''
56- ... [buildout]
57- ... parts =
58- ... prefer-final-build-system = false
59- ... ''')
60 >>> ignored = system(
61 ... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
62- ... 'bootstrap.py --accept-early-release')
63+ ... 'bootstrap.py --accept-buildout-test-releases')
64 >>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
65 "...
66 buildout_paths = [
67@@ -120,13 +111,41 @@
68
69 Notice we are now using zc.buildout 100.0b1, a non-final release.
70
71-Now we'll go back to the default of preferring final versions.
72-
73- >>> write('buildout.cfg',
74- ... '''
75- ... [buildout]
76- ... parts =
77- ... ''')
78+The buildout script remembers the decision to accept early releases, and
79+alerts the user.
80+
81+ >>> print system(join('bin', 'buildout')),
82+ ... # doctest: +NORMALIZE_WHITESPACE
83+ NOTE: Accepting early releases of build system packages. Rerun bootstrap
84+ without --accept-buildout-test-releases (-t) to return to default
85+ behavior.
86+
87+This is accomplished within the script itself.
88+
89+ >>> print open(buildout_script).read() # doctest: +ELLIPSIS
90+ #...
91+ sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true')
92+ print ('NOTE: Accepting early releases of build system packages. Rerun '
93+ 'bootstrap without --accept-buildout-test-releases (-t) to return to '
94+ 'default behavior.')
95+ ...
96+
97+As the note says, to undo, you just need to re-run bootstrap without
98+--accept-buildout-test-releases.
99+
100+ >>> ignored = system(
101+ ... zc.buildout.easy_install._safe_arg(sys.executable)+' '+
102+ ... 'bootstrap.py')
103+ >>> print open(buildout_site_py).read() # doctest: +ELLIPSIS
104+ "...
105+ buildout_paths = [
106+ '/sample/eggs/setuptools-...egg',
107+ '/sample/eggs/zc.buildout-99.99-pyN.N.egg'
108+ ]
109+ ...
110+ >>> ('buildout:accept-buildout-test-releases=true' in
111+ ... open(buildout_script).read())
112+ False
113
114 Now we will try the `--version` option, which lets you define a version for
115 `zc.buildout`. If not provided, bootstrap will look for the latest one.
116@@ -329,14 +348,12 @@
117 --eggs=EGGS Specify a directory for storing eggs. Defaults to a
118 temporary directory that is deleted when the bootstrap
119 script completes.
120- --accept-early-release
121+ -t, --accept-buildout-test-releases
122 Normally, if you do not specify a --version, the
123- bootstrap script gets the newest *final* versions of
124- zc.buildout for you. If you use this flag, bootstrap
125+ bootstrap script and buildout gets the newest *final*
126+ versions of zc.buildout and its recipes and extensions
127+ for you. If you use this flag, bootstrap and buildout
128 will get the newest releases even if they are alphas
129- or betas. Note that, if you do want to use early
130- buildout releases, you probably want to also set
131- ``prefer-final-build-system= false`` in the [buildout]
132- section of your configuration file.
133+ or betas.
134 -c CONFIG_FILE Specify the path to the buildout configuration file to
135 be used.
136
137=== modified file 'src/zc/buildout/buildout.py'
138--- src/zc/buildout/buildout.py 2010-08-19 02:27:43 +0000
139+++ src/zc/buildout/buildout.py 2010-08-19 02:27:43 +0000
140@@ -116,6 +116,7 @@
141 return data
142
143 _buildout_default_options = _annotate_section({
144+ 'accept-buildout-test-releases': 'false',
145 'allow-hosts': '*',
146 'allow-picked-versions': 'true',
147 'bin-directory': 'bin',
148@@ -131,7 +132,6 @@
149 'offline': 'false',
150 'parts-directory': 'parts',
151 'prefer-final': 'false',
152- 'prefer-final-build-system': 'true',
153 'python': 'buildout',
154 'relative-paths': 'false',
155 'socket-timeout': '',
156@@ -235,8 +235,8 @@
157 self._logger = logging.getLogger('zc.buildout')
158 self.offline = (buildout_section['offline'] == 'true')
159 self.newest = (buildout_section['newest'] == 'true')
160- self.prefer_final_build_system = (
161- buildout_section['prefer-final-build-system'] == 'true')
162+ self.accept_buildout_test_releases = (
163+ buildout_section['accept-buildout-test-releases'] == 'true')
164
165 ##################################################################
166 ## WARNING!!!
167@@ -281,8 +281,8 @@
168 self.newest = options.get_bool('newest')
169 zc.buildout.easy_install.prefer_final(
170 options.get_bool('prefer-final'))
171- self.prefer_final_build_system = options.get_bool(
172- 'prefer-final-build-system')
173+ self.accept_buildout_test_releases = options.get_bool(
174+ 'accept-buildout-test-releases')
175 zc.buildout.easy_install.use_dependency_links(
176 options.get_bool('use-dependency-links'))
177 zc.buildout.easy_install.allow_picked_versions(
178@@ -338,7 +338,7 @@
179 [options['develop-eggs-directory'],
180 options['eggs-directory']],
181 include_site_packages=_sys_executable_has_broken_dash_S,
182- prefer_final = self.prefer_final_build_system,
183+ prefer_final=not self.accept_buildout_test_releases,
184 )
185 else:
186 ws = zc.buildout.easy_install.install(
187@@ -350,7 +350,7 @@
188 newest=self.newest,
189 allow_hosts=self._allow_hosts,
190 include_site_packages=_sys_executable_has_broken_dash_S,
191- prefer_final = self.prefer_final_build_system,
192+ prefer_final=not self.accept_buildout_test_releases,
193 )
194
195 # Now copy buildout and setuptools eggs, and record destination eggs:
196@@ -373,7 +373,9 @@
197 else:
198 shutil.copy2(dist.location, dest)
199
200- # Create buildout script
201+ # Create buildout script.
202+ # Ideally the (possibly) new version of buildout would get a
203+ # chance to write the script. Not sure how to do that.
204 ws = pkg_resources.WorkingSet(entries)
205 ws.require('zc.buildout')
206 partsdir = os.path.join(options['parts-directory'], 'buildout')
207@@ -386,12 +388,19 @@
208 else:
209 assert relative_paths == 'false'
210 relative_paths = ''
211- # Ideally the (possibly) new version of buildout would get a
212- # chance to write the script. Not sure how to do that.
213+ if (self.accept_buildout_test_releases and
214+ self._annotated['buildout']['accept-buildout-test-releases'][1] ==
215+ 'COMMAND_LINE_VALUE'):
216+ # Bootstrap was called with '--accept-buildout-test-releases'.
217+ # Continue to honor that setting.
218+ script_initialization = _early_release_initialization_code
219+ else:
220+ script_initialization = ''
221 zc.buildout.easy_install.sitepackage_safe_scripts(
222 options['bin-directory'], ws, options['executable'], partsdir,
223 reqs=['zc.buildout'], relative_paths=relative_paths,
224- include_site_packages=_sys_executable_has_broken_dash_S)
225+ include_site_packages=_sys_executable_has_broken_dash_S,
226+ script_initialization=script_initialization,)
227
228 init = bootstrap
229
230@@ -838,7 +847,7 @@
231 path = [options['develop-eggs-directory']],
232 allow_hosts = self._allow_hosts,
233 include_site_packages=_sys_executable_has_broken_dash_S,
234- prefer_final=self.prefer_final_build_system,
235+ prefer_final=not self.accept_buildout_test_releases,
236 )
237
238 upgraded = []
239@@ -886,18 +895,27 @@
240
241 # the new dist is different, so we've upgraded.
242 # Update the scripts and return True
243+ # Ideally the new version of buildout would get a chance to write the
244+ # script. Not sure how to do that.
245 partsdir = os.path.join(options['parts-directory'], 'buildout')
246 if os.path.exists(partsdir):
247 # This is primarily for unit tests, in which .py files change too
248 # fast for Python to know to regenerate the .pyc/.pyo files.
249 shutil.rmtree(partsdir)
250 os.mkdir(partsdir)
251- # Ideally the new version of buildout would get a chance to write the
252- # script. Not sure how to do that.
253+ if (self.accept_buildout_test_releases and
254+ self._annotated['buildout']['accept-buildout-test-releases'][1] ==
255+ 'COMMAND_LINE_VALUE'):
256+ # Bootstrap was called with '--accept-buildout-test-releases'.
257+ # Continue to honor that setting.
258+ script_initialization = _early_release_initialization_code
259+ else:
260+ script_initialization = ''
261 zc.buildout.easy_install.sitepackage_safe_scripts(
262 options['bin-directory'], ws, sys.executable, partsdir,
263 reqs=['zc.buildout'],
264- include_site_packages=_sys_executable_has_broken_dash_S)
265+ include_site_packages=_sys_executable_has_broken_dash_S,
266+ script_initialization=script_initialization)
267
268 # Restart
269 args = map(zc.buildout.easy_install._safe_arg, sys.argv)
270@@ -939,7 +957,7 @@
271 index = self['buildout'].get('index'),
272 newest=self.newest, allow_hosts=self._allow_hosts,
273 include_site_packages=_sys_executable_has_broken_dash_S,
274- prefer_final=self.prefer_final_build_system)
275+ prefer_final=not self.accept_buildout_test_releases)
276
277 # Clear cache because extensions might now let us read pages we
278 # couldn't read before.
279@@ -1055,7 +1073,7 @@
280 newest=buildout.newest,
281 allow_hosts=buildout._allow_hosts,
282 include_site_packages=_sys_executable_has_broken_dash_S,
283- prefer_final=buildout.prefer_final_build_system)
284+ prefer_final=not buildout.accept_buildout_test_releases)
285
286 __doing__ = 'Loading %s recipe entry %s:%s.', group, spec, entry
287 return pkg_resources.load_entry_point(
288@@ -1526,6 +1544,13 @@
289 % (section, ' '.join(map(repr, unused)))
290 )
291
292+_early_release_initialization_code = """\
293+sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true')
294+print ('NOTE: Accepting early releases of build system packages. Rerun '
295+ 'bootstrap without --accept-buildout-test-releases (-t) to return to '
296+ 'default behavior.')
297+"""
298+
299 _usage = """\
300 Usage: buildout [options] [assignments] [command [command arguments]]
301
302
303=== modified file 'src/zc/buildout/buildout.txt'
304--- src/zc/buildout/buildout.txt 2010-08-19 02:27:43 +0000
305+++ src/zc/buildout/buildout.txt 2010-08-19 02:27:43 +0000
306@@ -729,6 +729,8 @@
307 ==================
308 <BLANKLINE>
309 [buildout]
310+ accept-buildout-test-releases= false
311+ DEFAULT_VALUE
312 allow-hosts= *
313 DEFAULT_VALUE
314 allow-picked-versions= true
315@@ -765,8 +767,6 @@
316 DEFAULT_VALUE
317 prefer-final= false
318 DEFAULT_VALUE
319- prefer-final-build-system= true
320- DEFAULT_VALUE
321 python= buildout
322 DEFAULT_VALUE
323 relative-paths= false
324@@ -2229,6 +2229,7 @@
325 <BLANKLINE>
326 Configuration data:
327 [buildout]
328+ accept-buildout-test-releases = false
329 allow-hosts = *
330 allow-picked-versions = true
331 bin-directory = /sample-buildout/bin
332@@ -2246,7 +2247,6 @@
333 parts =
334 parts-directory = /sample-buildout/parts
335 prefer-final = false
336- prefer-final-build-system = true
337 python = buildout
338 relative-paths = false
339 socket-timeout =
340@@ -2502,19 +2502,18 @@
341 releases that satisfy distribution requirements, then those releases
342 are used even if newer non-final releases are available.
343
344+In buildout version 2, all final releases will be preferred by
345+default--that is ``prefer-final`` will also default to 'true'. You will
346+then need to use a 'false' value for ``prefer-final`` to get the newest
347+releases.
348+
349 A separate option controls the behavior of the build system itself.
350 When buildout looks for recipes, extensions, and for updates to itself,
351 it does prefer final releases by default, as of the 1.5.0 release. The
352-``prefer-final-build-system`` option will let you override this behavior.
353-
354- [buildout]
355- ...
356- prefer-final-build-system = false
357-
358-In buildout version 2, all final releases will be preferred by
359-default--that is ``prefer-final`` will also default to 'true'. You will
360-then need to use a 'false' value for ``prefer-final`` to get the newest
361-releases, like with ``prefer-final-build-system``.
362+``accept-buildout-test-releases`` option will let you override this behavior.
363+However, it is typically changed by the --accept-buildout-test-releases
364+option to the bootstrap script, since bootstrapping is the first step to
365+selecting a buildout.
366
367 Finding distributions
368 ---------------------
369
370=== modified file 'src/zc/buildout/tests.py'
371--- src/zc/buildout/tests.py 2010-08-19 02:27:43 +0000
372+++ src/zc/buildout/tests.py 2010-08-19 02:27:43 +0000
373@@ -3128,9 +3128,11 @@
374
375 def buildout_prefer_final_build_system_option():
376 """
377-The prefer-final-build-system buildout option can be used for overriding
378+The accept-buildout-test-releases buildout option can be used for overriding
379 the default preference for final distributions for recipes, buildout
380-extensions, and buildout itself.
381+extensions, and buildout itself. It is usually controlled via the bootstrap
382+script rather than in the configuration file, but we will test the machinery
383+using the file.
384
385 Set up. This creates sdists for demorecipe 1.0 and 1.1b1, and for
386 demoextension 1.0 and 1.1b1.
387@@ -3138,7 +3140,7 @@
388 >>> create_sample_recipe_sdists(sample_eggs)
389 >>> create_sample_extension_sdists(sample_eggs)
390
391-The default is prefer-final-build-system = true:
392+The default is accept-buildout-test-releases = false:
393
394 >>> write('buildout.cfg',
395 ... '''
396@@ -3161,7 +3163,7 @@
397 Here we see that the final versions of demorecipe and demoextension were used.
398
399 We get the same behavior if we explicitly state that
400-prefer-final-build-system = true.
401+accept-buildout-test-releases = false.
402
403 >>> write('buildout.cfg',
404 ... '''
405@@ -3169,7 +3171,7 @@
406 ... parts = demo
407 ... find-links = %(link_server)s
408 ... extensions = demoextension
409- ... prefer-final-build-system = true
410+ ... accept-buildout-test-releases = false
411 ...
412 ... [demo]
413 ... recipe = demorecipe
414@@ -3182,7 +3184,7 @@
415 Picked: demorecipe = 1.0
416 ...
417
418-If we specify prefer-final-build-system = false, we'll get the newest
419+If we specify accept-buildout-test-releases = true, we'll get the newest
420 distributions in the build system:
421
422 >>> write('buildout.cfg',
423@@ -3191,7 +3193,7 @@
424 ... parts = demo
425 ... find-links = %(link_server)s
426 ... extensions = demoextension
427- ... prefer-final-build-system = false
428+ ... accept-buildout-test-releases = true
429 ...
430 ... [demo]
431 ... recipe = demorecipe
432@@ -3212,7 +3214,7 @@
433 ... parts = demo
434 ... find-links = %(link_server)s
435 ... extensions = demoextension
436- ... prefer-final-build-system = no
437+ ... accept-buildout-test-releases = no
438 ...
439 ... [demo]
440 ... recipe = demorecipe
441@@ -3221,7 +3223,7 @@
442 >>> print system(buildout+' -v'), # doctest: +ELLIPSIS
443 While:
444 Initializing.
445- Error: Invalid value for prefer-final-build-system option: no
446+ Error: Invalid value for accept-buildout-test-releases option: no
447
448 """
449
450
451=== modified file 'src/zc/buildout/update.txt'
452--- src/zc/buildout/update.txt 2010-08-19 02:27:43 +0000
453+++ src/zc/buildout/update.txt 2010-08-19 02:27:43 +0000
454@@ -81,8 +81,10 @@
455
456 Notice that, even though we have a newer beta version of zc.buildout
457 available, the final "99.99" was selected. If you want to get non-final
458-versions, specify a specific version in your buildout's versions section,
459-or use the ``prefer-final-build-system = false`` discussed below.
460+versions, specify a specific version in your buildout's versions
461+section, you typically want to use the --accept-buildout-test-releases
462+option to the bootstrap script, which internally uses the
463+``accept-buildout-test-releases = true`` discussed below.
464
465 Our buildout script's site.py has been updated to use the new eggs:
466
467@@ -194,9 +196,14 @@
468
469 >>> ls('bin')
470
471-Notice that, as mentioned above, the ``prefer-final-build-system =
472-false`` means that newer non-final versions of these dependencies are
473-preferred.
474+As mentioned above, the ``accept-buildout-test-releases = true`` means that
475+newer non-final versions of these dependencies are preferred. Typically
476+users are not expected to actually manipulate this value. Instead, the
477+bootstrap script creates a buildout buildout script that passes in the
478+value as a command line override. This then results in the buildout
479+script being rewritten to remember the decision.
480+
481+We'll mimic this by passing the argument actually in the command line.
482
483 >>> cd(sample_buildout)
484 >>> write(sample_buildout, 'buildout.cfg',
485@@ -206,20 +213,39 @@
486 ... index = %(new_releases)s
487 ... parts = show-versions
488 ... develop = showversions
489- ... prefer-final-build-system = false
490 ...
491 ... [show-versions]
492 ... recipe = showversions
493 ... """ % dict(new_releases=new_releases))
494
495- >>> print system(buildout),
496+ >>> print system(buildout +
497+ ... ' buildout:accept-buildout-test-releases=true'),
498+ ... # doctest: +NORMALIZE_WHITESPACE
499 Getting distribution for 'zc.buildout'.
500 Got zc.buildout 100.0b1.
501 Upgraded:
502 zc.buildout version 100.0b1,
503 setuptools version 99.99;
504 restarting.
505+ Generated script '/sample-buildout/bin/buildout'.
506+ NOTE: Accepting early releases of build system packages. Rerun bootstrap
507+ without --accept-buildout-test-releases (-t) to return to default
508+ behavior.
509 Develop: '/sample-buildout/showversions'
510 Updating show-versions.
511 zc.buildout 100.0b1
512 setuptools 99.99
513+
514+The buildout script shows the change.
515+
516+ >>> buildout_script = join(sample_buildout, 'bin', 'buildout')
517+ >>> import sys
518+ >>> if sys.platform.startswith('win'):
519+ ... buildout_script += '-script.py'
520+ >>> print open(buildout_script).read() # doctest: +ELLIPSIS
521+ #...
522+ sys.argv.insert(1, 'buildout:accept-buildout-test-releases=true')
523+ print ('NOTE: Accepting early releases of build system packages. Rerun '
524+ 'bootstrap without --accept-buildout-test-releases (-t) to return to '
525+ 'default behavior.')
526+ ...

Subscribers

People subscribed via source and target branches

to all changes: