Merge lp:~gary/z3c.recipe.filetemplate/support-system-python into lp:~gary/z3c.recipe.filetemplate/trunk
- support-system-python
- Merge into trunk
Status: | Needs review |
---|---|
Proposed branch: | lp:~gary/z3c.recipe.filetemplate/support-system-python |
Merge into: | lp:~gary/z3c.recipe.filetemplate/trunk |
Diff against target: | None lines |
To merge this branch: | bzr merge lp:~gary/z3c.recipe.filetemplate/support-system-python |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Francis J. Lacoste (community) | Approve | ||
Review via email: mp+8616@code.launchpad.net |
Commit message
Description of the change
Gary Poster (gary) wrote : | # |
Sorry, this change was removed from the branch, but I mistakenly added it to the comment. (I removed it because, upon trying it in Launchpad, I decided that it caused annoyances for little gain).
NOT PART OF THIS BRANCH:
- Support specifying local options in templates without braces (e.g.,
"Hello $world" is now equivalent to "Hello ${world}".
Francis J. Lacoste (flacoste) wrote : | # |
On July 11, 2009, Gary Poster wrote:
> Gary Poster has proposed merging
> lp:~gary/z3c.recipe.filetemplate/support-system-python into
> lp:~gary/z3c.recipe.filetemplate/trunk.
>
> Requested reviews:
> Francis J. Lacoste (flacoste)
>
> Support the new zc.buildout 1.4.0+ include-
>
Hi Gary,
This all looks good. My only concern is the new write_and_wait which I don't
understand why it's necessary.
> === modified file 'z3c/recipe/
> @@ -195,17 +202,19 @@
> 'Destinations already exist: %s. Please make sure that '
> 'you really want to generate these automatically. Then '
> 'move them away.', ', '.join(
> + seen = [] # we throw this away right now, but could move template
Sentence should start with a capital.
> + # processing up to __init__ if valuable. That would mean that
templates
> + # would be rewritten even if a value in another section had been
> + # referenced; however, it would also mean that __init__ would do
> + # virtually all of the work, with install only doing the writing.
> for rel_path, last_mod, st_mode in self.actions:
> source = os.path.
> dest = os.path.
> mode=stat.
> - template=
> - template=
> - template)
> - self._create_
> # we process the file first so that it won't be created if
there
> # is a problem.
> - processed = self.options.
> + processed = Template(
> + self._create_
> result=open(dest, "wt")
> result.
> result.close()
> @@ -221,3 +230,74 @@
>
> def update(self):
> pass
> +
> +
> +class Template:
> + # hacked from string.Template
> + pattern = re.compile(r"""
> + \$(?:
> + \${(?P<
delimiters.
> + {(?P<braced_
> + # Delimiter and a braced local
option
> + {(?P<braced_
> + # Delimiter and a braced fully
> + # qualified option (that is, with
> + # explicit section).
> + {(?P<invalid>
exprs.
> + )
> + """, re.IGNORECASE | re.VERBOSE)
> +
> + def __init__(self, source):
> + self.source = source
> + self.template = open(source).read()
> +
> + def _get_colno_
> + lines = self.template[
> + if not lines:
> + colno = 1
> + lineno = 1
> + else:
> + colno = i - len(''.
> + ...
Gary Poster (gary) wrote : | # |
On Jul 13, 2009, at 5:21 PM, Francis J. Lacoste wrote:
> On July 11, 2009, Gary Poster wrote:
>> Gary Poster has proposed merging
>> lp:~gary/z3c.recipe.filetemplate/support-system-python into
>> lp:~gary/z3c.recipe.filetemplate/trunk.
>>
>> Requested reviews:
>> Francis J. Lacoste (flacoste)
>>
>> Support the new zc.buildout 1.4.0+ include-
>> and more.
>>
>
> Hi Gary,
>
> This all looks good. My only concern is the new write_and_wait which
> I don't
> understand why it's necessary.
Cool. I'll discuss below.
>
>> === modified file 'z3c/recipe/
>
>> @@ -195,17 +202,19 @@
>> 'Destinations already exist: %s. Please make sure
>> that '
>> 'you really want to generate these automatically.
>> Then '
>> 'move them away.', ', '.join(
>> + seen = [] # we throw this away right now, but could move
>> template
>
> Sentence should start with a capital.
Ack. See incremental diff.
>
>> + # processing up to __init__ if valuable. That would mean
>> that
> templates
>> + # would be rewritten even if a value in another section
>> had been
>> + # referenced; however, it would also mean that __init__
>> would do
>> + # virtually all of the work, with install only doing the
>> writing.
>> for rel_path, last_mod, st_mode in self.actions:
>> source = os.path.
>> dest = os.path.
>> mode=stat.
>> - template=
>> - template=
>> self.name,
>> - template)
>> - self._create_
>> # we process the file first so that it won't be created
>> if
> there
>> # is a problem.
>> - processed = self.options.
>> + processed = Template(
>> + self._create_
>> result=open(dest, "wt")
>> result.
>> result.close()
>> @@ -221,3 +230,74 @@
>>
>> def update(self):
>> pass
>> +
>> +
>> +class Template:
>> + # hacked from string.Template
>> + pattern = re.compile(r"""
>> + \$(?:
>> + \${(?P<
>> of two
> delimiters.
>> + {(?P<braced_
>> + # Delimiter and a braced
>> local
> option
>> + {(?P<braced_
>> + # Delimiter and a braced
>> fully
>> + # qualified option (that
>> is, with
>> + # explicit section).
>> + {(?P<invalid>
>> delimiter
> exprs.
>> + )
>> + """, re.IGNORECASE | re.VERBOSE)
>> +
>> + def __init__(self, source):
>> + self.source = source
>> + self.template = open(source)....
- 21. By Gary Poster
-
changes from flacosate review, merged from svn
Francis J. Lacoste (flacoste) wrote : | # |
On July 13, 2009, Gary Poster wrote:
> In general, the necessity is caused by the mtime precision. On my
> Mac, the precision is 1 second. Here's an example run of a slightly
> modified version of the helper (adding a print), in the interactive
> prompt.
>
> >>> def write_and_wait(dir, *args):
> ... path = os.path.join(dir, *(args[:-1]))
> ... original = os.stat(
> ... while os.stat(
> ... print original
> ... f = open(path, 'w')
> ... f.write(args[-1])
> ... f.flush()
> ... os.fsync(
> ... f.close()
> ...
> >>> f = open('/
> >>> f.write('test')
> >>> f.close()
> >>> write_and_
> 1247536625.0
> >>> write_and_
> 1247536627.0
> >>> write_and_
> 1247536628.0
> >>> write_and_
> 1247536629.0
> 1247536629.0
> 1247536629.0
> [...there are 299 of these...]
> >>>
>
> So, the precision is the point. I tried a variety of things to
> trigger the mtime, but writing an empty string didn't work, and
> directly writing the mtime is OS-specific/
> really want to go there. This is a test harness. Good enough, was my
> thinking.
>
> As far as all the flushing and so on, that's what Jim had for his
> ``write`` helper function, so I suspect there's a point there too. It
> could probably be somewhat refactored but...it's a test harness.
Simply add a docstring explaining that we use this helper to update files to
make sure that the mtime of the file is updated so that buildout pick up the
changes, with a note that the resolution of mtime is system dependant. (And
possibly rename it to update_file() or something like that?)
And instead of busy-waiting, why don't you add a time.sleep(0.5) in there?
Cheers
review approve
--
Francis J. Lacoste
<email address hidden>
Gary Poster (gary) wrote : | # |
On Jul 14, 2009, at 9:27 AM, Francis J. Lacoste wrote:
> Review: Approve
> On July 13, 2009, Gary Poster wrote:
>> In general, the necessity is caused by the mtime precision. On my
>> Mac, the precision is 1 second. Here's an example run of a slightly
>> modified version of the helper (adding a print), in the interactive
>> prompt.
>>
>>>>> def write_and_wait(dir, *args):
>> ... path = os.path.join(dir, *(args[:-1]))
>> ... original = os.stat(
>> ... while os.stat(
>> ... print original
>> ... f = open(path, 'w')
>> ... f.write(args[-1])
>> ... f.flush()
>> ... os.fsync(
>> ... f.close()
>> ...
>>>>> f = open('/
>>>>> f.write('test')
>>>>> f.close()
>>>>> write_and_
>> 1247536625.0
>>>>> write_and_
>> 1247536627.0
>>>>> write_and_
>> 1247536628.0
>>>>> write_and_
>> 1247536629.0
>> 1247536629.0
>> 1247536629.0
>> [...there are 299 of these...]
>>>>>
>>
>> So, the precision is the point. I tried a variety of things to
>> trigger the mtime, but writing an empty string didn't work, and
>> directly writing the mtime is OS-specific/
>> really want to go there. This is a test harness. Good enough, was
>> my
>> thinking.
>>
>> As far as all the flushing and so on, that's what Jim had for his
>> ``write`` helper function, so I suspect there's a point there too.
>> It
>> could probably be somewhat refactored but...it's a test harness.
>
> Simply add a docstring explaining that we use this helper to update
> files to
> make sure that the mtime of the file is updated so that buildout
> pick up the
> changes, with a note that the resolution of mtime is system
> dependant. (And
> possibly rename it to update_file() or something like that?)
>
> And instead of busy-waiting, why don't you add a time.sleep(0.5) in
> there?
OK, cool, I like the name change in particular.
I actually left out the time.sleep as a policy-ish decision, wanting
the tests to go as fast as possible, but it was an experiment, and I'm
happy to switch back to the time.sleep approach.
Incremental diff below.
Thanks
Gary
Index: z3c/recipe/
=======
--- z3c/recipe/
+++ z3c/recipe/
@@ -13,14 +13,21 @@
#######
import os
+import time
import zc.buildout.testing
import zc.buildout.tests
from zope.testing import doctest
-def write_and_wait(dir, *args):
+def update_file(dir, *args):
+ """Update a file.
+
+ Make sure that the mtime of the file is updated so that buildout
notices
+ the changes. The resolution of mtime is system dependent, so we
keep
+ trying to write until mtime has actually changed."""
path = os.path.join(dir, *(args[:-1]))
original = os.stat(
wh...
Francis J. Lacoste (flacoste) wrote : | # |
On July 14, 2009, Gary Poster wrote:
> I actually left out the time.sleep as a policy-ish decision, wanting
> the tests to go as fast as possible, but it was an experiment, and I'm
> happy to switch back to the time.sleep approach.
>
Right, but know that busy waiting is bad in general because it makes it harder
for the OS to schedule other task (since you are competing for the CPU).
>
> Index: z3c/recipe/
> =======
> --- z3c/recipe/
> +++ z3c/recipe/
> @@ -13,14 +13,21 @@
>
> #######
>###
>
> import os
> +import time
> import zc.buildout.testing
> import zc.buildout.tests
> from zope.testing import doctest
>
> -def write_and_wait(dir, *args):
> +def update_file(dir, *args):
> + """Update a file.
> +
> + Make sure that the mtime of the file is updated so that buildout
> notices
> + the changes. The resolution of mtime is system dependent, so we
> keep
> + trying to write until mtime has actually changed."""
> path = os.path.join(dir, *(args[:-1]))
> original = os.stat(
> while os.stat(
> + time.sleep(0.2)
> f = open(path, 'w')
> f.write(args[-1])
> f.flush()
If you want the test to run as fast as possible I suggest:
while True:
# Write the change
# ...
if os.stat(
break
time.sleep(0.2)
Use it if you like it.
--
Francis J. Lacoste
<email address hidden>
Gary Poster (gary) wrote : | # |
On Jul 14, 2009, at 10:09 AM, Francis J. Lacoste wrote:
> On July 14, 2009, Gary Poster wrote:
>> I actually left out the time.sleep as a policy-ish decision, wanting
>> the tests to go as fast as possible, but it was an experiment, and
>> I'm
>> happy to switch back to the time.sleep approach.
>>
>
> Right, but know that busy waiting is bad in general because it makes
> it harder
> for the OS to schedule other task (since you are competing for the
> CPU).
Bah, get more cores. :-)
>> Index: z3c/recipe/
>> =======
>> --- z3c/recipe/
>> +++ z3c/recipe/
>> @@ -13,14 +13,21 @@
>>
>> #######
>> ###
>>
>> import os
>> +import time
>> import zc.buildout.testing
>> import zc.buildout.tests
>> from zope.testing import doctest
>>
>> -def write_and_wait(dir, *args):
>> +def update_file(dir, *args):
>> + """Update a file.
>> +
>> + Make sure that the mtime of the file is updated so that buildout
>> notices
>> + the changes. The resolution of mtime is system dependent, so we
>> keep
>> + trying to write until mtime has actually changed."""
>> path = os.path.join(dir, *(args[:-1]))
>> original = os.stat(
>> while os.stat(
>> + time.sleep(0.2)
>> f = open(path, 'w')
>> f.write(args[-1])
>> f.flush()
>
>
> If you want the test to run as fast as possible I suggest:
>
> while True:
> # Write the change
> # ...
> if os.stat(
> break
> time.sleep(0.2)
>
> Use it if you like it.
Sure, cool. Done.
Thanks
Gary
Unmerged revisions
- 21. By Gary Poster
-
changes from flacosate review, merged from svn
- 20. By gary
-
revert some of the features; they were annoying to me in practice. Also make error messages better.
- 19. By gary
-
Support the new zc.buildout 1.4.0+ include-
site-packages option, and more. --------
Features
--------- Support the new zc.buildout 1.4.0+ include-
site-packages option. - Use the new zc.buildout 1.4.0+ path sorting algorithm, which puts
site-package dependency paths after the other dependency paths (but before
extra paths, as before). This can reduce or eliminate problems with
packages in site-packages causing other dependencies to be masked with the
versions in site-packages.- Support escaping "$" with "$$" in templates. This is particularly useful
for *NIX shell scripts.- Support specifying local options in templates without braces (e.g.,
"Hello $world" is now equivalent to "Hello ${world}".-----
Fixes
------ Clarify that the recipe does not support the relative-paths options.
- Make tests less susceptible to timing errors.
- 18. By gary
-
make a branch for supporting zc.buildout 1.4.0 (hopefully) that also has some additional features
Preview Diff
1 | === modified file 'CHANGES.txt' |
2 | --- CHANGES.txt 2009-07-02 18:29:12 +0000 |
3 | +++ CHANGES.txt 2009-07-10 02:59:42 +0000 |
4 | @@ -9,7 +9,24 @@ |
5 | Features |
6 | -------- |
7 | |
8 | -- None yet. |
9 | +- Support the new zc.buildout 1.4.0+ include-site-packages option. |
10 | + |
11 | +- Use the new zc.buildout 1.4.0+ path sorting algorithm, which puts |
12 | + site-package dependency paths after the other dependency paths (but before |
13 | + extra paths, as before). This can reduce or eliminate problems with |
14 | + packages in site-packages causing other dependencies to be masked with the |
15 | + versions in site-packages. |
16 | + |
17 | +- Support escaping "${...}" with "$${...}" in templates. This is particularly |
18 | + useful for *NIX shell scripts. |
19 | + |
20 | +----- |
21 | +Fixes |
22 | +----- |
23 | + |
24 | +- Clarify that the recipe does not support the relative-paths options. |
25 | + |
26 | +- Make tests less susceptible to timing errors. |
27 | |
28 | |
29 | 2.0.3 (2009-07-02) |
30 | |
31 | === modified file 'buildout.cfg' |
32 | --- buildout.cfg 2007-09-30 19:14:00 +0000 |
33 | +++ buildout.cfg 2009-07-09 18:20:56 +0000 |
34 | @@ -1,7 +1,17 @@ |
35 | [buildout] |
36 | develop = . |
37 | + zc.buildout |
38 | + zc.buildout/zc.recipe.egg_ |
39 | parts = test |
40 | + interpreter |
41 | +include-site-packages = false |
42 | |
43 | [test] |
44 | recipe = zc.recipe.testrunner |
45 | eggs = z3c.recipe.filetemplate |
46 | + |
47 | +[interpreter] |
48 | +recipe = zc.recipe.egg |
49 | +interpreter = py |
50 | +eggs = z3c.recipe.filetemplate |
51 | + |
52 | |
53 | === modified file 'setup.py' |
54 | --- setup.py 2009-05-04 17:31:38 +0000 |
55 | +++ setup.py 2009-07-09 18:20:56 +0000 |
56 | @@ -42,8 +42,8 @@ |
57 | packages=find_packages(), |
58 | namespace_packages=['z3c', 'z3c.recipe'], |
59 | install_requires=['setuptools', |
60 | - 'zc.buildout', |
61 | - 'zc.recipe.egg', |
62 | + 'zc.buildout>=1.4.0dev', |
63 | + 'zc.recipe.egg>=1.3.0dev', |
64 | ], |
65 | zip_safe=True, |
66 | entry_points=""" |
67 | |
68 | === modified file 'z3c/recipe/filetemplate/README.txt' |
69 | --- z3c/recipe/filetemplate/README.txt 2009-04-30 21:38:40 +0000 |
70 | +++ z3c/recipe/filetemplate/README.txt 2009-07-10 02:59:42 +0000 |
71 | @@ -40,7 +40,7 @@ |
72 | ... world = Philipp |
73 | ... """) |
74 | |
75 | -After executing buildout, we can see that ``$world`` has indeed been |
76 | +After executing buildout, we can see that ``${world}`` has indeed been |
77 | replaced by ``Philipp``: |
78 | |
79 | >>> print system(buildout) |
80 | @@ -49,6 +49,35 @@ |
81 | >>> cat(sample_buildout, 'helloworld.txt') |
82 | Hello Philipp! |
83 | |
84 | +If you need to escape the ${...} pattern, you can do so by repeating the dollar |
85 | +sign. |
86 | + |
87 | + >>> write_and_wait(sample_buildout, 'helloworld.txt.in', |
88 | + ... """ |
89 | + ... Hello world! The double $${dollar-sign} escapes! |
90 | + ... """) |
91 | + |
92 | + >>> print system(buildout) |
93 | + Uninstalling message. |
94 | + Installing message. |
95 | + |
96 | + >>> cat(sample_buildout, 'helloworld.txt') |
97 | + Hello world! The double ${dollar-sign} escapes! |
98 | + |
99 | +Note that dollar signs alone, without curly braces, are not parsed. |
100 | + |
101 | + >>> write_and_wait(sample_buildout, 'helloworld.txt.in', |
102 | + ... """ |
103 | + ... $Hello $$world! $$$profit! |
104 | + ... """) |
105 | + |
106 | + >>> print system(buildout) |
107 | + Uninstalling message. |
108 | + Installing message. |
109 | + |
110 | + >>> cat(sample_buildout, 'helloworld.txt') |
111 | + $Hello $$world! $$$profit! |
112 | + |
113 | Note that the output file uses the same permission bits as found on the input |
114 | file. |
115 | |
116 | @@ -327,13 +356,20 @@ |
117 | >>> cat(sample_buildout, 'helloworld.txt') # doctest:+ELLIPSIS |
118 | Hello! Here are the paths for the demo<0.3 eggs. |
119 | OS paths: |
120 | - .../eggs/demo-0.2...egg:.../eggs/demoneeded-1.2c1...egg |
121 | + .../eggs/demo-0.2...egg:.../eggs/demoneeded-1.2c1...egg:... |
122 | --- |
123 | String paths: |
124 | - '.../eggs/demo-0.2...egg', '.../eggs/demoneeded-1.2c1...egg' |
125 | + '.../eggs/demo-0.2...egg', '.../eggs/demoneeded-1.2c1...egg', '...' |
126 | --- |
127 | Space paths: |
128 | - .../eggs/demo-0.2...egg .../eggs/demoneeded-1.2c1...egg |
129 | + .../eggs/demo-0.2...egg .../eggs/demoneeded-1.2c1...egg ... |
130 | + |
131 | +Notice that included multiple paths. In fact, it includes the site packages |
132 | +and the standard library, so these are appropriate for entirely replacing |
133 | +sys.path. |
134 | + |
135 | +You can eliminate the site packages from the paths by specifying |
136 | +"include-site-packages = false" in the buildout or the specific section. |
137 | |
138 | You can specify extra-paths as well, which will go at the end of the egg paths. |
139 | |
140 | @@ -359,13 +395,13 @@ |
141 | >>> cat(sample_buildout, 'helloworld.txt') # doctest:+ELLIPSIS |
142 | Hello! Here are the paths for the demo<0.3 eggs. |
143 | OS paths: |
144 | - ...demo...:...demoneeded...:.../sample-buildout/foo |
145 | + ...demo...:...demoneeded...:.../sample-buildout/foo:... |
146 | --- |
147 | String paths: |
148 | - '...demo...', '...demoneeded...', '.../sample-buildout/foo' |
149 | + '...demo...', '...demoneeded...', '.../sample-buildout/foo', '...' |
150 | --- |
151 | Space paths: |
152 | - ...demo... ...demoneeded... .../sample-buildout/foo |
153 | + ...demo... ...demoneeded... .../sample-buildout/foo ... |
154 | |
155 | Defining options in Python |
156 | ========================== |
157 | @@ -420,8 +456,8 @@ |
158 | |
159 | >>> cat(sample_buildout, 'helloworld.txt') # doctest:+ELLIPSIS |
160 | hello world! |
161 | - duplicate-os-paths: ...demo-0.2...egg:...demoneeded-1.2c1...egg |
162 | - foo-paths: ...demo-0.2...eggFOO...demoneeded-1.2c1...egg |
163 | + duplicate-os-paths: ...demo-0.2...egg:...demoneeded-1.2c1...egg:... |
164 | + foo-paths: ...demo-0.2...eggFOO...demoneeded-1.2c1...eggFOO... |
165 | silly-range: [0, 1, 2, 3, 4] |
166 | first-interpreted-option: duplicate-os-paths=(os.pathsep).join(paths) |
167 | message-reversed-is-egassem: egassem |
168 | |
169 | === modified file 'z3c/recipe/filetemplate/__init__.py' |
170 | --- z3c/recipe/filetemplate/__init__.py 2009-06-06 12:31:22 +0000 |
171 | +++ z3c/recipe/filetemplate/__init__.py 2009-07-10 02:59:42 +0000 |
172 | @@ -22,6 +22,7 @@ |
173 | import traceback |
174 | import zc.recipe.egg |
175 | import zc.buildout |
176 | +import zc.buildout.buildout |
177 | import zc.buildout.easy_install |
178 | |
179 | ABS_PATH_ERROR = ('%s is an absolute path. Paths must be ' |
180 | @@ -44,15 +45,21 @@ |
181 | self.options.setdefault(key, value) |
182 | # set up paths for eggs, if given |
183 | if 'eggs' in self.options: |
184 | + relative_paths = self.options.get( |
185 | + 'relative-paths', |
186 | + buildout['buildout'].get('relative-paths', 'false') |
187 | + ) |
188 | + if relative_paths != 'false': |
189 | + self._user_error( |
190 | + 'This recipe does not support relative-paths.') |
191 | + # Why? Because the relative path tricks rely on Python |
192 | + # at runtime, and we're offering path values for arbitrary |
193 | + # files (for instance, including bash files). |
194 | self.eggs = zc.recipe.egg.Scripts(buildout, name, options) |
195 | orig_distributions, ws = self.eggs.working_set() |
196 | - # we want ws, eggs.extra_paths, eggs._relative_paths |
197 | - all_paths = [ |
198 | - zc.buildout.easy_install.realpath(dist.location) |
199 | - for dist in ws] |
200 | - all_paths.extend( |
201 | - zc.buildout.easy_install.realpath(path) |
202 | - for path in self.eggs.extra_paths) |
203 | + all_paths = zc.buildout.easy_install.get_path( |
204 | + ws, self.options['executable'], self.eggs.extra_paths, |
205 | + self.eggs.include_site_packages) |
206 | else: |
207 | all_paths = [] |
208 | paths = [path for path in all_paths if not path.endswith('.zip')] |
209 | @@ -195,17 +202,19 @@ |
210 | 'Destinations already exist: %s. Please make sure that ' |
211 | 'you really want to generate these automatically. Then ' |
212 | 'move them away.', ', '.join(already_exists)) |
213 | + seen = [] # we throw this away right now, but could move template |
214 | + # processing up to __init__ if valuable. That would mean that templates |
215 | + # would be rewritten even if a value in another section had been |
216 | + # referenced; however, it would also mean that __init__ would do |
217 | + # virtually all of the work, with install only doing the writing. |
218 | for rel_path, last_mod, st_mode in self.actions: |
219 | source = os.path.join(self.source_dir, rel_path) |
220 | dest = os.path.join(self.destination_dir, rel_path[:-3]) |
221 | mode=stat.S_IMODE(st_mode) |
222 | - template=open(source).read() |
223 | - template=re.sub(r"\$\{([^:]+?)\}", r"${%s:\1}" % self.name, |
224 | - template) |
225 | - self._create_paths(os.path.dirname(dest)) |
226 | # we process the file first so that it won't be created if there |
227 | # is a problem. |
228 | - processed = self.options._sub(template, []) |
229 | + processed = Template(source).substitute(self, seen) |
230 | + self._create_paths(os.path.dirname(dest)) |
231 | result=open(dest, "wt") |
232 | result.write(processed) |
233 | result.close() |
234 | @@ -221,3 +230,74 @@ |
235 | |
236 | def update(self): |
237 | pass |
238 | + |
239 | + |
240 | +class Template: |
241 | + # hacked from string.Template |
242 | + pattern = re.compile(r""" |
243 | + \$(?: |
244 | + \${(?P<escaped>[^}]*)} | # Escape sequence of two delimiters. |
245 | + {(?P<braced_single>[-a-z0-9 ._]+)} | |
246 | + # Delimiter and a braced local option |
247 | + {(?P<braced_double>[-a-z0-9 ._]+:[-a-z0-9 ._]+)} | |
248 | + # Delimiter and a braced fully |
249 | + # qualified option (that is, with |
250 | + # explicit section). |
251 | + {(?P<invalid>[^}]*}) # Other ill-formed delimiter exprs. |
252 | + ) |
253 | + """, re.IGNORECASE | re.VERBOSE) |
254 | + |
255 | + def __init__(self, source): |
256 | + self.source = source |
257 | + self.template = open(source).read() |
258 | + |
259 | + def _get_colno_lineno(self, i): |
260 | + lines = self.template[:i].splitlines(True) |
261 | + if not lines: |
262 | + colno = 1 |
263 | + lineno = 1 |
264 | + else: |
265 | + colno = i - len(''.join(lines[:-1])) |
266 | + lineno = len(lines) |
267 | + return colno, lineno |
268 | + |
269 | + def _get(self, options, section, option, seen, start): |
270 | + value = options.get(option, None, seen) |
271 | + if value is None: |
272 | + colno, lineno = self._get_colno_lineno(start) |
273 | + raise zc.buildout.buildout.MissingOption( |
274 | + "Option '%s:%s', referenced in line %d, col %d of %s, " |
275 | + "does not exist." % |
276 | + (section, option, lineno, colno, self.source)) |
277 | + return value |
278 | + |
279 | + def substitute(self, recipe, seen): |
280 | + # Helper function for .sub() |
281 | + def convert(mo): |
282 | + # Check the most common path first. |
283 | + option = mo.group('braced_single') |
284 | + if option is not None: |
285 | + val = self._get(recipe.options, recipe.name, option, seen, |
286 | + mo.start('braced_single')) |
287 | + # We use this idiom instead of str() because the latter will |
288 | + # fail if val is a Unicode containing non-ASCII characters. |
289 | + return '%s' % (val,) |
290 | + double = mo.group('braced_double') |
291 | + if double is not None: |
292 | + section, option = double.split(':') |
293 | + val = self._get(recipe.buildout[section], section, option, seen, |
294 | + mo.start('braced_double')) |
295 | + return '%s' % (val,) |
296 | + escaped = mo.group('escaped') |
297 | + if escaped is not None: |
298 | + return '${%s}' % (escaped,) |
299 | + invalid = mo.group('invalid') |
300 | + if invalid is not None: |
301 | + colno, lineno = self._get_colno_lineno(mo.start('invalid')) |
302 | + raise ValueError( |
303 | + 'Invalid placeholder %r in line %d, col %d of %s' % |
304 | + (mo.group('invalid'), lineno, colno, self.source)) |
305 | + raise ValueError('Unrecognized named group in pattern', |
306 | + self.pattern) # programmer error, AFAICT |
307 | + return self.pattern.sub(convert, self.template) |
308 | + |
309 | |
310 | === modified file 'z3c/recipe/filetemplate/tests.py' |
311 | --- z3c/recipe/filetemplate/tests.py 2009-04-30 17:56:10 +0000 |
312 | +++ z3c/recipe/filetemplate/tests.py 2009-07-09 18:20:56 +0000 |
313 | @@ -12,12 +12,24 @@ |
314 | # |
315 | ############################################################################## |
316 | |
317 | +import os |
318 | import zc.buildout.testing |
319 | import zc.buildout.tests |
320 | from zope.testing import doctest |
321 | |
322 | +def write_and_wait(dir, *args): |
323 | + path = os.path.join(dir, *(args[:-1])) |
324 | + original = os.stat(path).st_mtime |
325 | + while os.stat(path).st_mtime == original: |
326 | + f = open(path, 'w') |
327 | + f.write(args[-1]) |
328 | + f.flush() |
329 | + os.fsync(f.fileno()) |
330 | + f.close() |
331 | + |
332 | def setUp(test): |
333 | zc.buildout.tests.easy_install_SetUp(test) |
334 | + test.globs['write_and_wait'] = write_and_wait |
335 | zc.buildout.testing.install_develop('z3c.recipe.filetemplate', test) |
336 | |
337 | def test_suite(): |
338 | |
339 | === modified file 'z3c/recipe/filetemplate/tests.txt' |
340 | --- z3c/recipe/filetemplate/tests.txt 2009-04-30 21:54:08 +0000 |
341 | +++ z3c/recipe/filetemplate/tests.txt 2009-07-10 02:59:42 +0000 |
342 | @@ -148,11 +148,12 @@ |
343 | ... files = missing.txt |
344 | ... """) |
345 | |
346 | - >>> print system(buildout) |
347 | + >>> print system(buildout) # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
348 | Installing missing. |
349 | While: |
350 | Installing missing. |
351 | - Error: Referenced option does not exist: missing world |
352 | + Error: Option 'missing:world', referenced in line 2, col 8 of |
353 | + .../sample-buildout/missing.txt.in, does not exist. |
354 | |
355 | No changes means just an update |
356 | ------------------------------- |
Support the new zc.buildout 1.4.0+ include- site-packages option, and more.
--------
Features
--------
- Support the new zc.buildout 1.4.0+ include- site-packages option.
- Use the new zc.buildout 1.4.0+ path sorting algorithm, which puts
site-package dependency paths after the other dependency paths (but before
extra paths, as before). This can reduce or eliminate problems with
packages in site-packages causing other dependencies to be masked with the
versions in site-packages.
- Support escaping "$" with "$$" in templates. This is particularly useful
for *NIX shell scripts.
- Support specifying local options in templates without braces (e.g.,
"Hello $world" is now equivalent to "Hello ${world}".
-----
Fixes
-----
- Clarify that the recipe does not support the relative-paths options.
- Make tests less susceptible to timing errors.