Merge lp:~spiv/bzr-builder/merge-subdirs-479705 into lp:~james-w/bzr-builder/trunk-old

Proposed by Andrew Bennetts
Status: Work in progress
Proposed branch: lp:~spiv/bzr-builder/merge-subdirs-479705
Merge into: lp:~james-w/bzr-builder/trunk-old
Prerequisite: lp:~spiv/bzr-builder/refactor
Diff against target: 2784 lines (+1496/-600)
10 files modified
TODO (+0/-3)
__init__.py (+33/-377)
cmds.py (+492/-0)
ppa.py (+124/-0)
recipe.py (+421/-150)
setup.py (+13/-12)
tests/__init__.py (+4/-3)
tests/test_blackbox.py (+62/-16)
tests/test_ppa.py (+28/-0)
tests/test_recipe.py (+319/-39)
To merge this branch: bzr merge lp:~spiv/bzr-builder/merge-subdirs-479705
Reviewer Review Type Date Requested Status
Andrew Bennetts (community) Disapprove
James Westby Needs Fixing
Review via email: mp+14979@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Andrew Bennetts (spiv) wrote :

This fixes bug 479705. It allows a merge line of a builder recipe to specify a subdirectory to merge, rather than the whole named branch. The obvious application of this is to allow merging just the debian/ directory from a branch that contains more than just a debian/ directory.

Revision history for this message
James Westby (james-w) wrote :

Hi,

Thanks for working on this.

I have some concerns about the utility of it. If the branch
you are merging doesn't make any modifications outside of
./debian/ then this makes no difference, therefore it must
be for the case where it does.

Is this then to take the packaging without any modifications
to the source, so that it is a vanilla packaging?

If so, then it is just assumed that this won't be done if
some of the modifications are needed, or if the packaging
depends on some of the modifications?

That would be acceptable, but I wonder why we want to encourage
people to do things this way, instead of say, layering another
branch on top that undoes the unwanted things? Obviously the two
are not equivalent, but the latter seems to fit better with the
model to me.

As for a code review, two things mean that this is not mergeable
as is:

  * Needs a format bump.
  * Needs an update to the documentation so that people know it is possible.

Other than that the changes look fine with one obvious exception:

45 + :param subpath: XXX

Thanks,

James

review: Needs Resubmitting
Revision history for this message
Robert Collins (lifeless) wrote :

> Hi,
>
> Thanks for working on this.
>
> I have some concerns about the utility of it. If the branch
> you are merging doesn't make any modifications outside of
> ./debian/ then this makes no difference, therefore it must
> be for the case where it does.

This is false. If the branches are not related, merging a subdir can avoid very many unnecessary conflicts while still generating a single tree.

> Is this then to take the packaging without any modifications
> to the source, so that it is a vanilla packaging?

Or to merge unrelated branches.

> As for a code review, two things mean that this is not mergeable
> as is:
>
> * Needs a format bump.

I don't see why it needs this: old versions will error appropriately, new versions will work.

If old versions would misbehave then I'd agree on the need to bump the format.

Revision history for this message
James Westby (james-w) wrote :

On Tue Nov 24 00:59:18 UTC 2009 Robert Collins wrote:
> > Hi,
> >
> > Thanks for working on this.
> >
> > I have some concerns about the utility of it. If the branch
> > you are merging doesn't make any modifications outside of
> > ./debian/ then this makes no difference, therefore it must
> > be for the case where it does.
>
> This is false. If the branches are not related, merging a subdir can avoid very
> many unnecessary conflicts while still generating a single tree.

If the branches are not related there is no way in builder to merge
them at this point anyway (no way to specify a merge base). I haven't
tested to make sure that the API works the same as the UI here, so
apologies if I mispoke.

I appreciate trying to improve this situation, but I'm not sure that
this is the correct fix, at least at this time. I would like to know
more about the design though, as I feel that there is something I am
missing.

Thanks,

James

Revision history for this message
Andrew Bennetts (spiv) wrote :

> > This is false. If the branches are not related, merging a subdir can avoid
> very
> > many unnecessary conflicts while still generating a single tree.
>
> If the branches are not related there is no way in builder to merge
> them at this point anyway (no way to specify a merge base). I haven't
> tested to make sure that the API works the same as the UI here, so
> apologies if I mispoke.

You're right about that. It would be easy to catch UnrelatedBranches and if a subpath is specified try again with a base of NULL_REVISION. Or we could add a 'mergeunrelated' directive. Or further extend the syntax to allow specifying bases.

Do you have a preference for one of those options, or do you feel this is the wrong track entirely? It seems to me that however it is specified a partial merge is a fairly neat way to provide "please give me this branch, plus the debian/ dir from another branch."

I'm sure there are many cases where blindly combining a debian/ dir from an existing package with an arbitrary version of upstream won't work. I think it's likely it will
work ok for many nightly packages though: have a recipe that specifies "current trunk + packaging from $recent-packaging". I'm not certain which use-case Robert had in mind, but that's what I'm thinking of.

Revision history for this message
James Westby (james-w) wrote :

Hi,

I'm sorry the discussion on this has stalled, but I'm not sure what to do
about it.

I'd rather know the concrete use cases for the change and how this would fix it
before merging so that I can better support them in future. Help understanding that
would be greatly appreciated.

Thanks,

James

Revision history for this message
Robert Collins (lifeless) wrote :

So, in two parts:
 - Andrew is looking at making sure it works with unrelated branches.

Use cases: as we discussed, for launchpads 'build from branch' to be successful for upstreams, they need easy access to debian/ in package branches that are likely to have no history relationship: This syntax allows them to specify that that is what they want without us resorting to guessing.

Revision history for this message
James Westby (james-w) wrote :

Hi,

I think I would like to see catching UnrelatedBranches, and then if a subdir
is specified, try again with NULL_REVISION as the base. If there are then
any conflicts it should bail out as normal.

This should have it work for the simple case, which is a start.

Why specify the subdirectory separate from the URL? Could we not do an
open_containing and use the subpath returned from that?

Thanks,

James

64. By Andrew Bennetts

Make merge_unrelated_branch work using my merge-into-merger branch of bzrlib.

65. By Andrew Bennetts

Define a new 'merge-into' instruction.

66. By Andrew Bennetts

Expand docstrings, and allow specifying a target subdir.

67. By Andrew Bennetts

Bump format, add tests for parsing of merge-into, revert unused change to allow 'merge' instructions to take subpaths.

68. By Andrew Bennetts

Reduce cruft slightly.

69. By Andrew Bennetts

Rename 'merge-into' instruction to 'merge-part'.

Revision history for this message
James Westby (james-w) wrote :

Hi Andrew,

This is great now, thanks.

merge-part NAME BRANCH REVISION SUBDIR [TARGET-DIR]

I think I might prefer

merge-part NAME BRANCH SUBDIR REVISION [TARGET-DIR]

(Maybe even:

merge-part NAME BRANCH SUBDIR [REVISION [TARGET-DIR]]

or

merge-part NAME BRANCH SUBDIR [TARGET-DIR [REVISION]])

We can't do

from bzrlib.cleanup import OperationWithCleanups

unfortunately, as it is too new for the bzr that we need to work with.
We could backport newer bzr's for building older distroseries, but that's
quite a bit of work.

MergeIntoMerger only seems to be in newer bzr's too? Is there any way that
we can avoid depending on that?

108 + if target_subdir is None:
109 + target_subdir = os.path.basename(subpath)

I wouldn't expect that, I would think that it would just be "subpath".

I'm happy to do the work to merge this in, the only blocker right now
is MergeIntoMerger, so if you could comment on that then we can move
forward.

Thanks,

James

review: Needs Fixing
Revision history for this message
James Westby (james-w) wrote :

Right, sorry, I now see that MergeIntoMerger is proposed
for adding to bzr.

I think that's the right way to go as it should be in bzr,
but it does make this hard for deploying on older distroseries
in Launchpad.

Thanks,

James

Revision history for this message
Andrew Bennetts (spiv) wrote :

James Westby wrote:
> Right, sorry, I now see that MergeIntoMerger is proposed
> for adding to bzr.
>
> I think that's the right way to go as it should be in bzr,
> but it does make this hard for deploying on older distroseries
> in Launchpad.

Ah, I didn't realise that was a requirement. I can extract that out in a way
that will work with older bzrs without much trouble. Would you like that in
bzr-builder or a separate plugin?

Also, what's the oldest bzr version that bzr-builder wants to work with?

Revision history for this message
James Westby (james-w) wrote :

On Fri, 09 Jul 2010 00:49:38 -0000, Andrew Bennetts <email address hidden> wrote:
> James Westby wrote:
> > Right, sorry, I now see that MergeIntoMerger is proposed
> > for adding to bzr.
> >
> > I think that's the right way to go as it should be in bzr,
> > but it does make this hard for deploying on older distroseries
> > in Launchpad.
>
> Ah, I didn't realise that was a requirement. I can extract that out in a way
> that will work with older bzrs without much trouble. Would you like that in
> bzr-builder or a separate plugin?

Great. bzr-builder is easier.

Do you think it should still go in bzr with a "backport" inside
bzr-builder?

> Also, what's the oldest bzr version that bzr-builder wants to work with?

Well, dapper is still "supported" so 0.8.2.

Really though we're probably only going to have hardy and later,
meaning 1.3.1.

Previously bzr-builder used very simple APIs, so this wasn't really an
issue. If we keep running in to this we can review our deployment and
perhaps use a newer bzr as well as a newer bzr-builder (given that the
plugin isn't even in hardy).

Thanks,

James

Revision history for this message
Andrew Bennetts (spiv) wrote :

James Westby wrote:
> On Fri, 09 Jul 2010 00:49:38 -0000, Andrew Bennetts <email address hidden> wrote:
[...]
> Great. bzr-builder is easier.
>
> Do you think it should still go in bzr with a "backport" inside
> bzr-builder?

Yes, exactly.

> > Also, what's the oldest bzr version that bzr-builder wants to work with?
>
> Well, dapper is still "supported" so 0.8.2.

Ouch!

> Really though we're probably only going to have hardy and later,
> meaning 1.3.1.
>
> Previously bzr-builder used very simple APIs, so this wasn't really an
> issue. If we keep running in to this we can review our deployment and
> perhaps use a newer bzr as well as a newer bzr-builder (given that the
> plugin isn't even in hardy).

Considering that so many branches on Launchpad are 2a format now it
seems to me that bzr >= 2.0 is almost a requirement anyway. Please tell
me I'm right ;)

A compromise be to say "merge-part" is not available unless you
are using a new-enough bzr. (Again, probably >= 2.0 with the help of a
backport in bzr-builder.)

Revision history for this message
James Westby (james-w) wrote :

On Fri, 09 Jul 2010 05:25:53 -0000, Andrew Bennetts <email address hidden> wrote:
> Considering that so many branches on Launchpad are 2a format now it
> seems to me that bzr >= 2.0 is almost a requirement anyway. Please tell
> me I'm right ;)

That's a good point. It may be that the archive we are using to backport
bzr-builder already has a bzr in for other reasons, and so that is
getting picked up.

Checking...

It does not, so indeed the newer branch formats can't be built on older
releases.

Filed

  https://bugs.edge.launchpad.net/launchpad-code/+bug/603615

about this.

Depending on how we fix this we could then not put the new classes in to
bzr-builder.

> A compromise be to say "merge-part" is not available unless you
> are using a new-enough bzr. (Again, probably >= 2.0 with the help of a
> backport in bzr-builder.)

I don't fancy that, but it would be an option, yes.

Thanks,

James

Revision history for this message
Daniel Hahler (blueyed) wrote :

I am looking forward to using this, but wonder why it used with branching ("merge") and not nesting ("nest")?

Andrew wrote:
> have a recipe that specifies "current trunk + packaging from $recent-packaging".

That's the most common use case and looks like "nesting", not merging.

As far as I understand the following will take of this / make it behave like "nest":
> I think I would like to see catching UnrelatedBranches, and then if
> a subdir is specified, try again with NULL_REVISION as the base.

Revision history for this message
Andrew Bennetts (spiv) wrote :

Daniel: I agree, nest-part is probably a better description for users. It happens that the implementation in bzr reuses a bunch of merge infrastructure, but I think it's fair to say that's an implementation detail that is irrelevant to users. So I've updated the patch accordingly, thank you!

James: So what is the plan for dealing or not with older bzr versions for now? The relevant bzr change is with PQM for landing in 2.2 right now. I'll going resubmit this patch for merging without any backwards compat for bzr < 2.2 included, but if you do want it I can make an additional proposal to copy the MergeIntoMerge bits into bzr-builder. At this point I'm basically just asking for a clear "Yes, do that" or "No, don't bother" instruction and then I'll go and do as you say, the current plan is a bit unclear with bug 603615 not going anywhere atm.

(Because the trunk branch has changed since I first created this proposal I'm going to mark this one as rejected and create a new one. But I really just resubmitting this patch.)

Revision history for this message
Andrew Bennetts (spiv) wrote :

The new merge proposal is at <https://code.edge.launchpad.net/~spiv/bzr-builder/merge-subdirs-479705/+merge/31251>. For some reason LP won't let me mark this proposal as Rejected, though :(

review: Disapprove
70. By Andrew Bennetts

Rename merge-part instruction to nest-part.

71. By Andrew Bennetts

Merge trunk.

72. By Andrew Bennetts

Change the syntax of nest-part to have the revspec at the end, and make it optional.

73. By Andrew Bennetts

Tweak plugin help text.

Unmerged revisions

73. By Andrew Bennetts

Tweak plugin help text.

72. By Andrew Bennetts

Change the syntax of nest-part to have the revspec at the end, and make it optional.

71. By Andrew Bennetts

Merge trunk.

70. By Andrew Bennetts

Rename merge-part instruction to nest-part.

69. By Andrew Bennetts

Rename 'merge-into' instruction to 'merge-part'.

68. By Andrew Bennetts

Reduce cruft slightly.

67. By Andrew Bennetts

Bump format, add tests for parsing of merge-into, revert unused change to allow 'merge' instructions to take subpaths.

66. By Andrew Bennetts

Expand docstrings, and allow specifying a target subdir.

65. By Andrew Bennetts

Define a new 'merge-into' instruction.

64. By Andrew Bennetts

Make merge_unrelated_branch work using my merge-into-merger branch of bzrlib.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'TODO'
--- TODO 2009-04-15 14:10:19 +0000
+++ TODO 2010-08-13 01:14:42 +0000
@@ -1,6 +1,3 @@
1- Implement building the source package from the tree, including adding
2 the changelog entry with the desired version.
3- Implement uploading to a PPA.
4- Documentation.1- Documentation.
5- Work out how to get tarballs for non-native packages if we want that.2- Work out how to get tarballs for non-native packages if we want that.
6- Decide on error on existing target directory vs. re-use and pull changes.3- Decide on error on existing target directory vs. re-use and pull changes.
74
=== modified file '__init__.py'
--- __init__.py 2010-08-13 01:14:42 +0000
+++ __init__.py 2010-08-13 01:14:42 +0000
@@ -21,7 +21,7 @@
2121
22A recipe is just a text file that starts with a line such as::22A recipe is just a text file that starts with a line such as::
2323
24 # bzr-builder format 0.2 deb-version 1.0+{revno}-{revno:packaging}24 # bzr-builder format 0.3 deb-version 1.0+{revno}-{revno:packaging}
2525
26The format specifier is there to allow the syntax to be changed in later26The format specifier is there to allow the syntax to be changed in later
27versions, and the meaning of "deb-version" will be explained later.27versions, and the meaning of "deb-version" will be explained later.
@@ -70,6 +70,20 @@
70for the revisionspec is identical to that taken by the "--revision" argument70for the revisionspec is identical to that taken by the "--revision" argument
71to many bzr commands. See "bzr help revisionspec" for details.71to many bzr commands. See "bzr help revisionspec" for details.
7272
73You can also merge specific subdirectories from a branch with a "nest-part"
74line like
75
76nest-part packaging lp:~foo-dev/foo/packaging debian
77
78which specifies that the only the debian/ subdirectory should be merged. This
79works even if the branches share no revision history. You can optionally
80specify the subdirectory and revision in the target with a line like
81
82nest-part libfoo lp:libfoo src lib/foo tag:release-1.2
83
84which will put the "src" directory of libfoo in "lib/foo", using the revision
85of libfoo tagged "release-1.2".
86
73It is also possible to run an arbitrary command at a particular point in the87It is also possible to run an arbitrary command at a particular point in the
74construction process. For example::88construction process. For example::
7589
@@ -104,14 +118,28 @@
104118
105 * {time} will be substituted with the current date and time, such as119 * {time} will be substituted with the current date and time, such as
106 200908191512.120 200908191512.
121 * {date} will be substituted with just the current date, such as
122 20090819.
107 * {revno} will be the revno of the base branch (the first specified).123 * {revno} will be the revno of the base branch (the first specified).
108 * {revno:<branch name>} will be substituted with the revno for the124 * {revno:<branch name>} will be substituted with the revno for the
109 branch named <branch name> in the recipe.125 branch named <branch name> in the recipe.
126 * {debupstream} will be replaced by the upstream portion of the version
127 number taken from debian/changelog in the final tree. If when the
128 tree is built the top of debian/changelog has a version number of
129 "1.0-1" then this would evaluate to "1.0".
130
131Instruction syntax summary:
132
133 * nest NAME BRANCH TARGET-DIR [REVISION]
134 * merge NAME BRANCH [REVISION]
135 * nest-part NAME BRANCH SUBDIR [TARGET-DIR [REVISION]]
136 * run COMMAND
110137
111Format versions:138Format versions:
112139
113 0.1 - original format.140 0.1 - original format.
114 0.2 - added "run" instruction.141 0.2 - added "run" instruction.
142 0.3 - added "nest-part" instruction.
115"""143"""
116144
117if __name__ == '__main__':145if __name__ == '__main__':
@@ -123,382 +151,10 @@
123 shell=True, env={"BZR_PLUGIN_PATH": dir})151 shell=True, env={"BZR_PLUGIN_PATH": dir})
124 sys.exit(retcode)152 sys.exit(retcode)
125153
126import datetime154
127from email import utils155from bzrlib.commands import plugin_cmds
128import os156plugin_cmds.register_lazy("cmd_build", [], "bzrlib.plugins.builder.cmds")
129import pwd157plugin_cmds.register_lazy("cmd_dailydeb", [], "bzrlib.plugins.builder.cmds")
130import re
131import socket
132import shutil
133import subprocess
134import tempfile
135
136from debian_bundle import changelog
137
138from bzrlib import (
139 errors,
140 trace,
141 transport,
142 )
143from bzrlib.commands import Command, register_command
144from bzrlib.option import Option
145
146from bzrlib.plugins.builder.recipe import (
147 build_manifest,
148 build_tree,
149 RecipeParser,
150 resolve_revisions,
151 )
152
153
154def write_manifest_to_path(path, base_branch):
155 parent_dir = os.path.dirname(path)
156 if parent_dir != '' and not os.path.exists(parent_dir):
157 os.makedirs(parent_dir)
158 manifest_f = open(path, 'wb')
159 try:
160 manifest_f.write(build_manifest(base_branch))
161 finally:
162 manifest_f.close()
163
164
165def get_branch_from_recipe_file(recipe_file):
166 recipe_transport = transport.get_transport(os.path.dirname(recipe_file))
167 try:
168 recipe_contents = recipe_transport.get_bytes(
169 os.path.basename(recipe_file))
170 except errors.NoSuchFile:
171 raise errors.BzrCommandError("Specified recipe does not exist: "
172 "%s" % recipe_file)
173 parser = RecipeParser(recipe_contents, filename=recipe_file)
174 return parser.parse()
175
176
177def get_old_recipe(if_changed_from):
178 old_manifest_transport = transport.get_transport(os.path.dirname(
179 if_changed_from))
180 try:
181 old_manifest_contents = old_manifest_transport.get_bytes(
182 os.path.basename(if_changed_from))
183 except errors.NoSuchFile:
184 return None
185 old_recipe = RecipeParser(old_manifest_contents,
186 filename=if_changed_from).parse()
187 return old_recipe
188
189
190def get_maintainer():
191 """
192 Create maintainer string using the same algorithm as in dch
193 """
194 env = os.environ
195 regex = re.compile(r"^(.*)\s+<(.*)>$")
196
197 # Split email and name
198 if 'DEBEMAIL' in env:
199 match_obj = regex.match(env['DEBEMAIL'])
200 if match_obj:
201 if not 'DEBFULLNAME' in env:
202 env['DEBFULLNAME'] = match_obj.group(1)
203 env['DEBEMAIL'] = match_obj.group(2)
204 if 'DEBEMAIL' not in env or 'DEBFULLNAME' not in env:
205 if 'EMAIL' in env:
206 match_obj = regex.match(env['EMAIL'])
207 if match_obj:
208 if not 'DEBFULLNAME' in env:
209 env['DEBFULLNAME'] = match_obj.group(1)
210 env['EMAIL'] = match_obj.group(2)
211
212 # Get maintainer's name
213 if 'DEBFULLNAME' in env:
214 maintainer = env['DEBFULLNAME']
215 elif 'NAME' in env:
216 maintainer = env['NAME']
217 else:
218 # Use password database if no data in environment variables
219 try:
220 maintainer = re.sub(r',.*', '', pwd.getpwuid(os.getuid()).pw_gecos)
221 except KeyError, AttributeError:
222 # TBD: Use last changelog entry value
223 maintainer = "bzr-builder"
224
225 # Get maintainer's mail address
226 if 'DEBEMAIL' in env:
227 email = env['DEBEMAIL']
228 elif 'EMAIL' in env:
229 email = env['EMAIL']
230 else:
231 addr = None
232 if os.path.exists('/etc/mailname'):
233 f = open('/etc/mailname')
234 try:
235 addr = f.readline().strip()
236 finally:
237 f.close()
238 if not addr:
239 addr = socket.getfqdn()
240 if addr:
241 user = pwd.getpwuid(os.getuid()).pw_name
242 if not user:
243 addr = None
244 else:
245 addr = "%s@%s" % (user, addr)
246
247 if addr:
248 email = addr
249 else:
250 # TBD: Use last changelog entry value
251 email = "none@example.org"
252
253 return (maintainer, email)
254
255
256def add_changelog_entry(base_branch, basedir, distribution=None,
257 package=None):
258 debian_dir = os.path.join(basedir, "debian")
259 if not os.path.exists(debian_dir):
260 os.makedirs(debian_dir)
261 cl_path = os.path.join(debian_dir, "changelog")
262 if os.path.exists(cl_path):
263 cl_f = open(cl_path)
264 try:
265 cl = changelog.Changelog(file=cl_f)
266 finally:
267 cl_f.close()
268 else:
269 cl = changelog.Changelog()
270 if len(cl._blocks) > 0:
271 if distribution is None:
272 distribution = cl._blocks[0].distributions.split()[0]
273 if package is None:
274 package = cl._blocks[0].package
275 if "{debupstream}" in base_branch.deb_version:
276 cl_version = cl._blocks[0].version
277 base_branch.deb_version = base_branch.deb_version.replace(
278 "{debupstream}", cl_version.upstream_version)
279 else:
280 if package is None:
281 raise errors.BzrCommandError("No previous changelog to "
282 "take the package name from, and --package not "
283 "specified.")
284 if "{debupstream}" in base_branch.deb_version:
285 raise errors.BzrCommandError("No previous changelog to "
286 "take the upstream version from - {debupstream} was "
287 "used.")
288 if distribution is None:
289 distribution = "jaunty"
290 # Use debian packaging environment variables
291 # or default values if they don't exist
292 author = "%s <%s>" % get_maintainer()
293
294 date = utils.formatdate(localtime=True)
295 cl.new_block(package=package, version=base_branch.deb_version,
296 distributions=distribution, urgency="low",
297 changes=['', ' * Auto build.', ''],
298 author=author, date=date)
299 cl_f = open(cl_path, 'wb')
300 try:
301 cl.write_to_open_file(cl_f)
302 finally:
303 cl_f.close()
304
305
306def build_source_package(basedir):
307 trace.note("Building the source package")
308 command = ["/usr/bin/debuild", "--no-tgz-check", "-i", "-I", "-S",
309 "-uc", "-us"]
310 proc = subprocess.Popen(command, cwd=basedir,
311 stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
312 stdin=subprocess.PIPE)
313 proc.stdin.close()
314 retcode = proc.wait()
315 if retcode != 0:
316 output = proc.stdout.read()
317 raise errors.BzrCommandError("Failed to build the source package: "
318 "%s" % output)
319
320
321def sign_source_package(basedir, key_id):
322 trace.note("Signing the source package")
323 command = ["/usr/bin/debsign", "-S", "-k%s" % key_id]
324 proc = subprocess.Popen(command, cwd=basedir,
325 stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
326 stdin=subprocess.PIPE)
327 proc.stdin.close()
328 retcode = proc.wait()
329 if retcode != 0:
330 output = proc.stdout.read()
331 raise errors.BzrCommandError("Signing the package failed: "
332 "%s" % output)
333
334
335def dput_source_package(basedir, target):
336 trace.note("Uploading the source package")
337 command = ["/usr/bin/debrelease", "-S", "--dput", target]
338 proc = subprocess.Popen(command, cwd=basedir,
339 stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
340 stdin=subprocess.PIPE)
341 proc.stdin.close()
342 retcode = proc.wait()
343 if retcode != 0:
344 output = proc.stdout.read()
345 raise errors.BzrCommandError("Uploading the package failed: "
346 "%s" % output)
347
348
349class cmd_build(Command):
350 """Build a tree based on a 'recipe'.
351
352 Pass the name of the recipe file and the directory to work in.
353
354 See "bzr help builder" for more information on what a recipe is.
355 """
356 takes_args = ["recipe_file", "working_directory"]
357 takes_options = [
358 Option('manifest', type=str, argname="path",
359 help="Path to write the manifest to."),
360 Option('if-changed-from', type=str, argname="path",
361 help="Only build if the outcome would be different "
362 "to that specified in the specified manifest."),
363 ]
364
365 def _substitute_stuff(self, recipe_file, if_changed_from):
366 """Common code to substitute stuff
367
368 :return: A tuple with (retcode, base_branch). If retcode is None
369 then the command execution should continue.
370 """
371 base_branch = get_branch_from_recipe_file(recipe_file)
372 time = datetime.datetime.utcnow()
373 base_branch.substitute_time(time)
374 old_recipe = None
375 if if_changed_from is not None:
376 old_recipe = get_old_recipe(if_changed_from)
377 # Save the unsubstituted version for dailydeb.
378 self._template_version = base_branch.deb_version
379 changed = resolve_revisions(base_branch, if_changed_from=old_recipe)
380 if not changed:
381 trace.note("Unchanged")
382 return 0, base_branch
383 return None, base_branch
384
385 def run(self, recipe_file, working_directory, manifest=None,
386 if_changed_from=None):
387 result, base_branch = self._substitute_stuff(recipe_file,
388 if_changed_from)
389 if result is not None:
390 return result
391 manifest_path = manifest or os.path.join(working_directory,
392 "bzr-builder.manifest")
393 build_tree(base_branch, working_directory)
394 write_manifest_to_path(manifest_path, base_branch)
395
396
397register_command(cmd_build)
398
399
400class cmd_dailydeb(cmd_build):
401 """Build a deb based on a 'recipe'.
402
403 See "bzr help builder" for more information on what a recipe is.
404
405 If you do not specify a working directory then a temporary
406 directory will be used and it will be removed when the command
407 finishes.
408 """
409
410 takes_options = cmd_build.takes_options + [
411 Option("package", type=str,
412 help="The package name to use in the changelog entry. "
413 "If not specified then the package from the "
414 "previous changelog entry will be used, so it "
415 "must be specified if there is no changelog."),
416 Option("distribution", type=str,
417 help="The distribution to target. If not specified "
418 "then the same distribution as the last entry "
419 "in debian/changelog will be used."),
420 Option("dput", type=str, argname="target",
421 help="dput the built package to the specified "
422 "dput target."),
423 Option("key-id", type=str, short_name="k",
424 help="Sign the packages with the specified GnuPG key, "
425 "must be specified if you use --dput."),
426 ]
427
428 takes_args = ["recipe_file", "working_basedir?"]
429
430 def run(self, recipe_file, working_basedir=None, manifest=None,
431 if_changed_from=None, package=None, distribution=None,
432 dput=None, key_id=None):
433
434 if dput is not None and key_id is None:
435 raise errors.BzrCommandError("You must specify --key-id if you "
436 "specify --dput.")
437 result, base_branch = self._substitute_stuff(recipe_file,
438 if_changed_from)
439 if result is not None:
440 return result
441 if working_basedir is None:
442 temp_dir = tempfile.mkdtemp(prefix="bzr-builder-")
443 working_basedir = temp_dir
444 else:
445 temp_dir = None
446 if not os.path.exists(working_basedir):
447 os.makedirs(working_basedir)
448 self._calculate_package_name(recipe_file, package)
449 working_directory = os.path.join(working_basedir,
450 "%s-%s" % (self._package_name, self._template_version))
451 try:
452 # we want to use a consistent package_dir always to support
453 # updates in place, but debuild etc want PACKAGE-UPSTREAMVERSION
454 # on disk, so we build_tree with the unsubstituted version number
455 # and do a final rename-to step before calling into debian build
456 # tools. We then rename the working dir back.
457 manifest_path = os.path.join(working_directory, "debian",
458 "bzr-builder.manifest")
459 build_tree(base_branch, working_directory)
460 write_manifest_to_path(manifest_path, base_branch)
461 # Add changelog also substitutes {debupstream}.
462 add_changelog_entry(base_branch, working_directory,
463 distribution=distribution, package=package)
464 package_dir = self._calculate_package_dir(base_branch,
465 working_basedir)
466 # working_directory -> package_dir: after this debian stuff works.
467 os.rename(working_directory, package_dir)
468 try:
469 build_source_package(package_dir)
470 if key_id is not None:
471 sign_source_package(package_dir, key_id)
472 if dput is not None:
473 dput_source_package(package_dir, dput)
474 finally:
475 # package_dir -> working_directory
476 os.rename(package_dir, working_directory)
477 # Note that this may write a second manifest.
478 if manifest is not None:
479 write_manifest_to_path(manifest, base_branch)
480 finally:
481 if temp_dir is not None:
482 shutil.rmtree(temp_dir)
483
484 def _calculate_package_dir(self, base_branch, working_basedir):
485 """Calculate the directory name that should be used while debuilding."""
486 version = base_branch.deb_version
487 if "-" in version:
488 version = version[:version.rindex("-")]
489 package_basedir = "%s-%s" % (self._package_name, version)
490 package_dir = os.path.join(working_basedir, package_basedir)
491 return package_dir
492
493 def _calculate_package_name(self, recipe_file, package):
494 """Calculate the directory name that should be used while debuilding."""
495 recipe_name = os.path.basename(recipe_file)
496 if recipe_name.endswith(".recipe"):
497 recipe_name = recipe_name[:-len(".recipe")]
498 self._package_name = package or recipe_name
499
500
501register_command(cmd_dailydeb)
502158
503159
504def test_suite():160def test_suite():
505161
=== added file 'cmds.py'
--- cmds.py 1970-01-01 00:00:00 +0000
+++ cmds.py 2010-08-13 01:14:42 +0000
@@ -0,0 +1,492 @@
1# bzr-builder: a bzr plugin to constuct trees based on recipes
2# Copyright 2009 Canonical Ltd.
3
4# This program is free software: you can redistribute it and/or modify it
5# under the terms of the GNU General Public License version 3, as published
6# by the Free Software Foundation.
7
8# This program is distributed in the hope that it will be useful, but
9# WITHOUT ANY WARRANTY; without even the implied warranties of
10# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11# PURPOSE. See the GNU General Public License for more details.
12
13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.
15
16"""Subcommands provided by bzr-builder."""
17
18import datetime
19from email import utils
20import errno
21import os
22import pwd
23import re
24import socket
25import shutil
26import subprocess
27import tempfile
28
29try:
30 from debian import changelog
31except ImportError:
32 # In older versions of python-debian the main package was named
33 # debian_bundle
34 from debian_bundle import changelog
35
36from bzrlib import (
37 errors,
38 trace,
39 transport,
40 )
41from bzrlib.commands import Command
42from bzrlib.option import Option
43
44from bzrlib.plugins.builder.recipe import (
45 build_tree,
46 DEBUPSTREAM_VAR,
47 RecipeParser,
48 resolve_revisions,
49 )
50
51
52# The default distribution used by add_changelog_entry()
53DEFAULT_UBUNTU_DISTRIBUTION = "lucid"
54
55
56class MissingDependency(errors.BzrError):
57 pass
58
59
60def write_manifest_to_path(path, base_branch):
61 parent_dir = os.path.dirname(path)
62 if parent_dir != '' and not os.path.exists(parent_dir):
63 os.makedirs(parent_dir)
64 manifest_f = open(path, 'wb')
65 try:
66 manifest_f.write(str(base_branch))
67 finally:
68 manifest_f.close()
69
70
71def get_branch_from_recipe_file(recipe_file):
72 recipe_transport = transport.get_transport(os.path.dirname(recipe_file))
73 try:
74 recipe_contents = recipe_transport.get_bytes(
75 os.path.basename(recipe_file))
76 except errors.NoSuchFile:
77 raise errors.BzrCommandError("Specified recipe does not exist: "
78 "%s" % recipe_file)
79 parser = RecipeParser(recipe_contents, filename=recipe_file)
80 return parser.parse()
81
82
83def get_old_recipe(if_changed_from):
84 old_manifest_transport = transport.get_transport(os.path.dirname(
85 if_changed_from))
86 try:
87 old_manifest_contents = old_manifest_transport.get_bytes(
88 os.path.basename(if_changed_from))
89 except errors.NoSuchFile:
90 return None
91 old_recipe = RecipeParser(old_manifest_contents,
92 filename=if_changed_from).parse()
93 return old_recipe
94
95
96def get_maintainer():
97 """Create maintainer string using the same algorithm as in dch.
98 """
99 env = os.environ
100 regex = re.compile(r"^(.*)\s+<(.*)>$")
101
102 # Split email and name
103 if 'DEBEMAIL' in env:
104 match_obj = regex.match(env['DEBEMAIL'])
105 if match_obj:
106 if not 'DEBFULLNAME' in env:
107 env['DEBFULLNAME'] = match_obj.group(1)
108 env['DEBEMAIL'] = match_obj.group(2)
109 if 'DEBEMAIL' not in env or 'DEBFULLNAME' not in env:
110 if 'EMAIL' in env:
111 match_obj = regex.match(env['EMAIL'])
112 if match_obj:
113 if not 'DEBFULLNAME' in env:
114 env['DEBFULLNAME'] = match_obj.group(1)
115 env['EMAIL'] = match_obj.group(2)
116
117 # Get maintainer's name
118 if 'DEBFULLNAME' in env:
119 maintainer = env['DEBFULLNAME']
120 elif 'NAME' in env:
121 maintainer = env['NAME']
122 else:
123 # Use password database if no data in environment variables
124 try:
125 maintainer = re.sub(r',.*', '', pwd.getpwuid(os.getuid()).pw_gecos)
126 except (KeyError, AttributeError):
127 # TBD: Use last changelog entry value
128 maintainer = "bzr-builder"
129
130 # Get maintainer's mail address
131 if 'DEBEMAIL' in env:
132 email = env['DEBEMAIL']
133 elif 'EMAIL' in env:
134 email = env['EMAIL']
135 else:
136 addr = None
137 if os.path.exists('/etc/mailname'):
138 f = open('/etc/mailname')
139 try:
140 addr = f.readline().strip()
141 finally:
142 f.close()
143 if not addr:
144 addr = socket.getfqdn()
145 if addr:
146 user = pwd.getpwuid(os.getuid()).pw_name
147 if not user:
148 addr = None
149 else:
150 addr = "%s@%s" % (user, addr)
151
152 if addr:
153 email = addr
154 else:
155 # TBD: Use last changelog entry value
156 email = "none@example.org"
157
158 return (maintainer, email)
159
160
161def add_changelog_entry(base_branch, basedir, distribution=None,
162 package=None, author_name=None, author_email=None,
163 append_version=None):
164 debian_dir = os.path.join(basedir, "debian")
165 if not os.path.exists(debian_dir):
166 os.makedirs(debian_dir)
167 cl_path = os.path.join(debian_dir, "changelog")
168 file_found = False
169 if os.path.exists(cl_path):
170 file_found = True
171 cl_f = open(cl_path)
172 try:
173 contents = cl_f.read()
174 finally:
175 cl_f.close()
176 cl = changelog.Changelog(file=contents)
177 else:
178 cl = changelog.Changelog()
179 if len(cl._blocks) > 0:
180 if distribution is None:
181 distribution = cl._blocks[0].distributions.split()[0]
182 if package is None:
183 package = cl._blocks[0].package
184 if DEBUPSTREAM_VAR in base_branch.deb_version:
185 cl_version = cl._blocks[0].version
186 base_branch.substitute_debupstream(cl_version)
187 else:
188 if file_found:
189 if len(contents.strip()) > 0:
190 reason = ("debian/changelog didn't contain any "
191 "parseable stanzas")
192 else:
193 reason = "debian/changelog was empty"
194 else:
195 reason = "debian/changelog was not present"
196 if package is None:
197 raise errors.BzrCommandError("No previous changelog to "
198 "take the package name from, and --package not "
199 "specified: %s." % reason)
200 if DEBUPSTREAM_VAR in base_branch.deb_version:
201 raise errors.BzrCommandError("No previous changelog to "
202 "take the upstream version from as %s was "
203 "used: %s." % (DEBUPSTREAM_VAR, reason))
204 if distribution is None:
205 distribution = DEFAULT_UBUNTU_DISTRIBUTION
206 # Use debian packaging environment variables
207 # or default values if they don't exist
208 if author_name is None or author_email is None:
209 author_name, author_email = get_maintainer()
210 author = "%s <%s>" % (author_name, author_email)
211
212 date = utils.formatdate(localtime=True)
213 version = base_branch.deb_version
214 if append_version is not None:
215 version += append_version
216 try:
217 changelog.Version(version)
218 except (changelog.VersionError, ValueError), e:
219 raise errors.BzrCommandError("Invalid deb-version: %s: %s"
220 % (version, e))
221 cl.new_block(package=package, version=version,
222 distributions=distribution, urgency="low",
223 changes=['', ' * Auto build.', ''],
224 author=author, date=date)
225 cl_f = open(cl_path, 'wb')
226 try:
227 cl.write_to_open_file(cl_f)
228 finally:
229 cl_f.close()
230
231
232def calculate_package_dir(base_branch, package_name, working_basedir):
233 """Calculate the directory name that should be used while debuilding."""
234 version = base_branch.deb_version
235 if "-" in version:
236 version = version[:version.rindex("-")]
237 package_basedir = "%s-%s" % (package_name, version)
238 package_dir = os.path.join(working_basedir, package_basedir)
239 return package_dir
240
241
242def _run_command(command, basedir, msg, error_msg,
243 not_installed_msg=None):
244 """ Run a command in a subprocess.
245
246 :param command: list with command and parameters
247 :param msg: message to display to the user
248 :param error_msg: message to display if something fails.
249 :param not_installed_msg: the message to display if the command
250 isn't available.
251 """
252 trace.note(msg)
253 # Hide output if -q is in use.
254 quiet = trace.is_quiet()
255 if quiet:
256 kwargs = {"stderr": subprocess.STDOUT, "stdout": subprocess.PIPE}
257 else:
258 kwargs = {}
259 try:
260 proc = subprocess.Popen(command, cwd=basedir,
261 stdin=subprocess.PIPE, **kwargs)
262 except OSError, e:
263 if e.errno != errno.ENOENT:
264 raise
265 if not_installed_msg is None:
266 raise
267 raise MissingDependency(msg=not_installed_msg)
268 output = proc.communicate()
269 if proc.returncode != 0:
270 if quiet:
271 raise errors.BzrCommandError("%s: %s" % (error_msg, output))
272 else:
273 raise errors.BzrCommandError(error_msg)
274
275
276def build_source_package(basedir):
277 command = ["/usr/bin/debuild", "--no-tgz-check", "-i", "-I", "-S",
278 "-uc", "-us"]
279 _run_command(command, basedir,
280 "Building the source package",
281 "Failed to build the source package",
282 not_installed_msg="debuild is not installed, please install "
283 "the devscripts package.")
284
285
286def sign_source_package(basedir, key_id):
287 command = ["/usr/bin/debsign", "-S", "-k%s" % key_id]
288 _run_command(command, basedir,
289 "Signing the source package",
290 "Signing the package failed",
291 not_installed_msg="debsign is not installed, please install "
292 "the devscripts package.")
293
294
295def dput_source_package(basedir, target):
296 command = ["/usr/bin/debrelease", "-S", "--dput", target]
297 _run_command(command, basedir,
298 "Uploading the source package",
299 "Uploading the package failed",
300 not_installed_msg="debrelease is not installed, please "
301 "install the devscripts package.")
302
303
304class cmd_build(Command):
305 """Build a tree based on a 'recipe'.
306
307 Pass the name of the recipe file and the directory to work in.
308
309 See "bzr help builder" for more information on what a recipe is.
310 """
311 takes_args = ["recipe_file", "working_directory"]
312 takes_options = [
313 Option('manifest', type=str, argname="path",
314 help="Path to write the manifest to."),
315 Option('if-changed-from', type=str, argname="path",
316 help="Only build if the outcome would be different "
317 "to that specified in the specified manifest."),
318 ]
319
320 def _get_prepared_branch_from_recipe(self, recipe_file,
321 if_changed_from=None):
322 """Common code to prepare a branch and do substitutions.
323
324 :param recipe_file: a path to a recipe file to work from.
325 :param if_changed_from: an optional path to a manifest to
326 compare the recipe against.
327 :return: A tuple with (retcode, base_branch). If retcode is None
328 then the command execution should continue.
329 """
330 base_branch = get_branch_from_recipe_file(recipe_file)
331 time = datetime.datetime.utcnow()
332 base_branch.substitute_time(time)
333 old_recipe = None
334 if if_changed_from is not None:
335 old_recipe = get_old_recipe(if_changed_from)
336 # Save the unsubstituted version for dailydeb.
337 self._template_version = base_branch.deb_version
338 changed = resolve_revisions(base_branch, if_changed_from=old_recipe)
339 if not changed:
340 trace.note("Unchanged")
341 return 0, base_branch
342 return None, base_branch
343
344 def run(self, recipe_file, working_directory, manifest=None,
345 if_changed_from=None):
346 result, base_branch = self._get_prepared_branch_from_recipe(recipe_file,
347 if_changed_from=if_changed_from)
348 if result is not None:
349 return result
350 manifest_path = manifest or os.path.join(working_directory,
351 "bzr-builder.manifest")
352 build_tree(base_branch, working_directory)
353 write_manifest_to_path(manifest_path, base_branch)
354
355
356class cmd_dailydeb(cmd_build):
357 """Build a deb based on a 'recipe'.
358
359 See "bzr help builder" for more information on what a recipe is.
360
361 If you do not specify a working directory then a temporary
362 directory will be used and it will be removed when the command
363 finishes.
364 """
365
366 takes_options = cmd_build.takes_options + [
367 Option("package", type=str,
368 help="The package name to use in the changelog entry. "
369 "If not specified then the package from the "
370 "previous changelog entry will be used, so it "
371 "must be specified if there is no changelog."),
372 Option("distribution", type=str,
373 help="The distribution to target. If not specified "
374 "then the same distribution as the last entry "
375 "in debian/changelog will be used."),
376 Option("dput", type=str, argname="target",
377 help="dput the built package to the specified "
378 "dput target."),
379 Option("key-id", type=str, short_name="k",
380 help="Sign the packages with the specified GnuPG key. "
381 "Must be specified if you use --dput."),
382 Option("no-build",
383 help="Just ready the source package and don't "
384 "actually build it."),
385 Option("watch-ppa", help="Watch the PPA the package was "
386 "dput to and exit with 0 only if it builds and "
387 "publishes successfully."),
388 Option("append-version", type=str, help="Append the "
389 "specified string to the end of the version used "
390 "in debian/changelog."),
391 ]
392
393 takes_args = ["recipe_file", "working_basedir?"]
394
395 def run(self, recipe_file, working_basedir=None, manifest=None,
396 if_changed_from=None, package=None, distribution=None,
397 dput=None, key_id=None, no_build=None, watch_ppa=False,
398 append_version=None):
399
400 if dput is not None and key_id is None:
401 raise errors.BzrCommandError("You must specify --key-id if you "
402 "specify --dput.")
403 if watch_ppa:
404 if not dput:
405 raise errors.BzrCommandError(
406 "cannot watch a ppa without doing dput.")
407 else:
408 # Check we can calculate a PPA url.
409 target_from_dput(dput)
410
411 result, base_branch = self._get_prepared_branch_from_recipe(recipe_file,
412 if_changed_from=if_changed_from)
413 if result is not None:
414 return result
415 if working_basedir is None:
416 temp_dir = tempfile.mkdtemp(prefix="bzr-builder-")
417 working_basedir = temp_dir
418 else:
419 temp_dir = None
420 if not os.path.exists(working_basedir):
421 os.makedirs(working_basedir)
422 package_name = self._calculate_package_name(recipe_file, package)
423 working_directory = os.path.join(working_basedir,
424 "%s-%s" % (package_name, self._template_version))
425 try:
426 # we want to use a consistent package_dir always to support
427 # updates in place, but debuild etc want PACKAGE-UPSTREAMVERSION
428 # on disk, so we build_tree with the unsubstituted version number
429 # and do a final rename-to step before calling into debian build
430 # tools. We then rename the working dir back.
431 manifest_path = os.path.join(working_directory, "debian",
432 "bzr-builder.manifest")
433 build_tree(base_branch, working_directory)
434 write_manifest_to_path(manifest_path, base_branch)
435 # Add changelog also substitutes {debupstream}.
436 add_changelog_entry(base_branch, working_directory,
437 distribution=distribution, package=package,
438 append_version=append_version)
439 package_dir = calculate_package_dir(base_branch,
440 package_name, working_basedir)
441 # working_directory -> package_dir: after this debian stuff works.
442 os.rename(working_directory, package_dir)
443 if no_build:
444 if manifest is not None:
445 write_manifest_to_path(manifest, base_branch)
446 return 0
447 try:
448 build_source_package(package_dir)
449 if key_id is not None:
450 sign_source_package(package_dir, key_id)
451 if dput is not None:
452 dput_source_package(package_dir, dput)
453 finally:
454 # package_dir -> working_directory
455 # FIXME: may fail in error unwind, masking the original exception.
456 os.rename(package_dir, working_directory)
457 # Note that this may write a second manifest.
458 if manifest is not None:
459 write_manifest_to_path(manifest, base_branch)
460 finally:
461 if temp_dir is not None:
462 shutil.rmtree(temp_dir)
463 if watch_ppa:
464 from bzrlib.plugins.builder.ppa import watch
465 target = target_from_dput(dput)
466 if not watch(target, self.package, base_branch.deb_version):
467 return 2
468
469 def _calculate_package_name(self, recipe_file, package):
470 """Calculate the directory name that should be used while debuilding."""
471 recipe_name = os.path.basename(recipe_file)
472 if recipe_name.endswith(".recipe"):
473 recipe_name = recipe_name[:-len(".recipe")]
474 return package or recipe_name
475
476
477def target_from_dput(dput):
478 """Convert a dput specification to a LP API specification.
479
480 :param dput: A dput command spec like ppa:team-name.
481 :return: A LP API target like team-name/ppa.
482 """
483 ppa_prefix = 'ppa:'
484 if not dput.startswith(ppa_prefix):
485 raise errors.BzrCommandError('%r does not appear to be a PPA. '
486 'A dput target like \'%suser[/name]\' must be used.'
487 % (dput, ppa_prefix))
488 base, _, suffix = dput[len(ppa_prefix):].partition('/')
489 if not suffix:
490 suffix = 'ppa'
491 return base + '/' + suffix
492
0493
=== added file 'ppa.py'
--- ppa.py 1970-01-01 00:00:00 +0000
+++ ppa.py 2010-08-13 01:14:42 +0000
@@ -0,0 +1,124 @@
1# ppa support for bzr builder.
2#
3# Copyright: Canonical Ltd. (C) 2009
4#
5# This program is free software: you can redistribute it and/or modify it
6# under the terms of the GNU General Public License version 3, as published
7# by the Free Software Foundation.
8
9# This program is distributed in the hope that it will be useful, but
10# WITHOUT ANY WARRANTY; without even the implied warranties of
11# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
12# PURPOSE. See the GNU General Public License for more details.
13
14# You should have received a copy of the GNU General Public License along
15# with this program. If not, see <http://www.gnu.org/licenses/>.
16
17import os
18import time
19
20
21from launchpadlib.launchpad import (
22 Launchpad,
23 EDGE_SERVICE_ROOT,
24 )
25from launchpadlib.credentials import Credentials
26
27from bzrlib import (
28 errors,
29 trace,
30 )
31
32
33def get_lp():
34 credentials = Credentials()
35 oauth_file = os.path.expanduser('~/.cache/launchpadlib/bzr-builder')
36 if os.path.exists(oauth_file):
37 f = open(oauth_file)
38 try:
39 credentials.load(f)
40 finally:
41 f.close()
42 launchpad = Launchpad(credentials, EDGE_SERVICE_ROOT)
43 else:
44 launchpad = Launchpad.get_token_and_login('bzr-builder',
45 EDGE_SERVICE_ROOT)
46 f = open(oauth_file, 'wb')
47 try:
48 launchpad.credentials.save(f)
49 finally:
50 f.close()
51 return launchpad
52
53
54def watch(owner_name, archive_name, package_name, version):
55 """Watch a package build.
56
57 :return: True once the package built and published, or False if it fails
58 or there is a timeout waiting.
59 """
60 version = str(version)
61 trace.note("Logging into Launchpad")
62
63 launchpad = get_lp()
64 owner = launchpad.people[owner_name]
65 archive = owner.getPPAByName(name=archive_name)
66 end_states = ['FAILEDTOBUILD', 'FULLYBUILT']
67 important_arches = ['amd64', 'i386', 'armel']
68 trace.note("Waiting for version %s of %s to build." % (version, package_name))
69 start = time.time()
70 while True:
71 sourceRecords = list(archive.getPublishedSources(
72 source_name=package_name, version=version))
73 if not sourceRecords:
74 if time.time() - 900 > start:
75 # Over 15 minutes and no source yet, upload FAIL.
76 raise errors.BzrCommandError("No source record in %s/%s for "
77 "package %s=%s after 15 minutes." % (owner_name,
78 archive_name, package_name, version))
79 return False
80 trace.note("Source not available yet - waiting.")
81 time.sleep(60)
82 continue
83 pkg = sourceRecords[0]
84 if pkg.status.lower() not in ('published', 'pending'):
85 trace.note("Package status: %s" % (pkg.status,))
86 time.sleep(60)
87 continue
88 # FIXME: LP should export this as an attribute.
89 source_id = pkg.self_link.rsplit('/', 1)[1]
90 buildSummaries = archive.getBuildSummariesForSourceIds(
91 source_ids=[source_id])[source_id]
92 if buildSummaries['status'] in end_states:
93 break
94 if buildSummaries['status'] == 'NEEDSBUILD':
95 # We ignore non-virtual PPA architectures that are sparsely
96 # supplied with buildds.
97 missing = []
98 for build in buildSummaries['builds']:
99 arch = build['arch_tag']
100 if arch in important_arches:
101 missing.append(arch)
102 if not missing:
103 break
104 extra = ' on ' + ', '.join(missing)
105 else:
106 extra = ''
107 trace.note("%s is still in %s%s" % (pkg.display_name,
108 buildSummaries['status'], extra))
109 time.sleep(60)
110 trace.note("%s is now %s" % (pkg.display_name, buildSummaries['status']))
111 result = True
112 if pkg.status.lower() != 'published':
113 result = False # should this perhaps keep waiting?
114 if buildSummaries['status'] != 'FULLYBUILT':
115 if buildSummaries['status'] == 'NEEDSBUILD':
116 # We're stopping early cause the important_arches are built.
117 builds = pkg.getBuilds()
118 for build in builds:
119 if build.arch_tag in important_arches:
120 if build.buildstate != 'Successfully built':
121 result = False
122 else:
123 result = False
124 return result
0125
=== modified file 'recipe.py'
--- recipe.py 2010-08-13 01:14:42 +0000
+++ recipe.py 2010-08-13 01:14:42 +0000
@@ -22,22 +22,53 @@
22 bzrdir,22 bzrdir,
23 errors,23 errors,
24 merge,24 merge,
25 revision,
25 revisionspec,26 revisionspec,
26 tag,27 tag,
28 trace,
27 transport,29 transport,
28 ui,
29 urlutils,30 urlutils,
30 )31 )
3132
3233
34try:
35 from debian import changelog
36except ImportError:
37 from debian_bundle import changelog
38
39
33def subprocess_setup():40def subprocess_setup():
34 signal.signal(signal.SIGPIPE, signal.SIG_DFL)41 signal.signal(signal.SIGPIPE, signal.SIG_DFL)
3542
3643
37MERGE_INSTRUCTION = "merge"44MERGE_INSTRUCTION = "merge"
45NEST_PART_INSTRUCTION = "nest-part"
38NEST_INSTRUCTION = "nest"46NEST_INSTRUCTION = "nest"
39RUN_INSTRUCTION = "run"47RUN_INSTRUCTION = "run"
4048
49TIME_VAR = "{time}"
50DATE_VAR = "{date}"
51REVNO_VAR = "{revno}"
52REVNO_PARAM_VAR = "{revno:%s}"
53DEBUPSTREAM_VAR = "{debupstream}"
54
55ok_to_preserve = [DEBUPSTREAM_VAR]
56# The variables that don't require substitution in their name
57simple_vars = [TIME_VAR, DATE_VAR, REVNO_VAR, DEBUPSTREAM_VAR]
58
59
60def check_expanded_deb_version(base_branch):
61 checked_version = base_branch.deb_version
62 for token in ok_to_preserve:
63 checked_version = checked_version.replace(token, "")
64 if "{" in checked_version:
65 available_tokens = simple_vars
66 for name in base_branch.list_branch_names():
67 available_tokens.append(REVNO_PARAM_VAR % name)
68 raise errors.BzrCommandError("deb-version not fully "
69 "expanded: %s. Valid substitutions are: %s"
70 % (base_branch.deb_version, available_tokens))
71
4172
42class CommandFailedError(errors.BzrError):73class CommandFailedError(errors.BzrError):
4374
@@ -113,10 +144,10 @@
113 # We do a "pull"144 # We do a "pull"
114 if tree_to is not None:145 if tree_to is not None:
115 # FIXME: should these pulls overwrite?146 # FIXME: should these pulls overwrite?
116 result = tree_to.pull(br_from, stop_revision=revision_id,147 tree_to.pull(br_from, stop_revision=revision_id,
117 possible_transports=possible_transports)148 possible_transports=possible_transports)
118 else:149 else:
119 result = br_to.pull(br_from, stop_revision=revision_id,150 br_to.pull(br_from, stop_revision=revision_id,
120 possible_transports=possible_transports)151 possible_transports=possible_transports)
121 tree_to = br_to.bzrdir.create_workingtree()152 tree_to = br_to.bzrdir.create_workingtree()
122 # Ugh, we have to assume that the caller replaces their reference153 # Ugh, we have to assume that the caller replaces their reference
@@ -146,7 +177,7 @@
146 return tree_to, br_to177 return tree_to, br_to
147178
148179
149def merge_branch(child_branch, tree_to, br_to):180def merge_branch(child_branch, tree_to, br_to, possible_transports=None):
150 """Merge the branch specified by child_branch.181 """Merge the branch specified by child_branch.
151182
152 :param child_branch: the RecipeBranch to retrieve the branch and revision to183 :param child_branch: the RecipeBranch to retrieve the branch and revision to
@@ -154,82 +185,129 @@
154 :param tree_to: the WorkingTree to merge in to.185 :param tree_to: the WorkingTree to merge in to.
155 :param br_to: the Branch to merge in to.186 :param br_to: the Branch to merge in to.
156 """187 """
188 if child_branch.branch is None:
189 child_branch.branch = branch.Branch.open(child_branch.url,
190 possible_transports=possible_transports)
191 child_branch.branch.lock_read()
192 try:
193 tag._merge_tags_if_possible(child_branch.branch, br_to)
194 if child_branch.revspec is not None:
195 merge_revspec = revisionspec.RevisionSpec.from_string(
196 child_branch.revspec)
197 try:
198 merge_revid = merge_revspec.as_revision_id(
199 child_branch.branch)
200 except errors.InvalidRevisionSpec, e:
201 # Give the user a hint if they didn't mean to speciy
202 # a revspec.
203 e.extra = (". Did you not mean to specify a revspec "
204 "at the end of the merge line?")
205 raise e
206 else:
207 merge_revid = child_branch.branch.last_revision()
208 child_branch.revid = merge_revid
209 merger = merge.Merger.from_revision_ids(None, tree_to, merge_revid,
210 other_branch=child_branch.branch, tree_branch=br_to)
211 merger.merge_type = merge.Merge3Merger
212 if (merger.base_rev_id == merger.other_rev_id and
213 merger.other_rev_id is not None):
214 # Nothing to do.
215 return
216 conflict_count = merger.do_merge()
217 merger.set_pending()
218 if conflict_count:
219 # FIXME: better reporting
220 raise errors.BzrCommandError("Conflicts from merge")
221 tree_to.commit("Merge %s" %
222 urlutils.unescape_for_display(
223 child_branch.url, 'utf-8'))
224 finally:
225 child_branch.branch.unlock()
226
227
228def nest_part_branch(op, child_branch, tree_to, br_to, subpath,
229 target_subdir=None):
230 """Merge the branch subdirectory specified by child_branch.
231
232 :param child_branch: the RecipeBranch to retrieve the branch and revision to
233 merge from.
234 :param tree_to: the WorkingTree to merge in to.
235 :param br_to: the Branch to merge in to.
236 :param subpath: only merge files from branch that are from this path.
237 e.g. subpath='/debian' will only merge changes from that directory.
238 :param target_subdir: (optional) directory in target to merge that
239 subpath into. Defaults to basename of subpath.
240 """
157 merge_from = branch.Branch.open(child_branch.url)241 merge_from = branch.Branch.open(child_branch.url)
158 merge_from.lock_read()242 merge_from.lock_read()
159 try:243 op.add_cleanup(merge_from.unlock)
160 pb = ui.ui_factory.nested_progress_bar()244 if child_branch.revspec is not None:
161 try:245 merge_revspec = revisionspec.RevisionSpec.from_string(
162 tag._merge_tags_if_possible(merge_from, br_to)246 child_branch.revspec)
163 if child_branch.revspec is not None:247 merge_revid = merge_revspec.as_revision_id(merge_from)
164 merge_revspec = revisionspec.RevisionSpec.from_string(248 else:
165 child_branch.revspec)249 merge_revid = merge_from.last_revision()
166 merge_revid = merge_revspec.as_revision_id(merge_from)250 child_branch.revid = merge_revid
167 else:251 other_tree = merge_from.basis_tree()
168 merge_revid = merge_from.last_revision()252 other_tree.lock_read()
169 child_branch.revid = merge_revid253 op.add_cleanup(other_tree.unlock)
170 merger = merge.Merger.from_revision_ids(pb, tree_to, merge_revid,254 if target_subdir is None:
171 other_branch=merge_from, tree_branch=br_to)255 target_subdir = os.path.basename(subpath)
172 merger.merge_type = merge.Merge3Merger256 merger = merge.MergeIntoMerger(this_tree=tree_to, other_tree=other_tree,
173 if (merger.base_rev_id == merger.other_rev_id and257 other_branch=merge_from, target_subdir=target_subdir,
174 merger.other_rev_id is not None):258 source_subpath=subpath, other_rev_id=merge_revid)
175 # Nothing to do.259 merger.set_base_revision(revision.NULL_REVISION, merge_from)
176 return260 conflict_count = merger.do_merge()
177 conflict_count = merger.do_merge()261 merger.set_pending()
178 merger.set_pending()262 if conflict_count:
179 if conflict_count:263 # FIXME: better reporting
180 # FIXME: better reporting264 raise errors.BzrCommandError("Conflicts from merge")
181 raise errors.BzrCommandError("Conflicts from merge")265 tree_to.commit("Merge %s of %s" %
182 tree_to.commit("Merge %s" %266 (subpath, urlutils.unescape_for_display(child_branch.url, 'utf-8')))
183 urlutils.unescape_for_display(267
184 child_branch.url, 'utf-8'))268
185 finally:269def update_branch(base_branch, tree_to, br_to, to_transport,
186 pb.finished()270 possible_transports=None):
187 finally:271 if base_branch.branch is None:
188 merge_from.unlock()272 base_branch.branch = branch.Branch.open(base_branch.url,
189273 possible_transports=possible_transports)
190274 base_branch.branch.lock_read()
191def update_branch(base_branch, tree_to, br_to, to_transport):
192 from_location = base_branch.url
193 accelerator_tree, br_from = bzrdir.BzrDir.open_tree_or_branch(
194 from_location)
195 br_from.lock_read()
196 try:275 try:
197 if base_branch.revspec is not None:276 if base_branch.revspec is not None:
198 revspec = revisionspec.RevisionSpec.from_string(277 revspec = revisionspec.RevisionSpec.from_string(
199 base_branch.revspec)278 base_branch.revspec)
200 revision_id = revspec.as_revision_id(br_from)279 revision_id = revspec.as_revision_id(base_branch.branch)
201 else:280 else:
202 revision_id = br_from.last_revision()281 revision_id = base_branch.branch.last_revision()
203 base_branch.revid = revision_id282 base_branch.revid = revision_id
204 tree_to, br_to = pull_or_branch(tree_to, br_to, br_from,283 tree_to, br_to = pull_or_branch(tree_to, br_to, base_branch.branch,
205 to_transport, revision_id,284 to_transport, revision_id,
206 accelerator_tree=accelerator_tree,285 possible_transports=possible_transports)
207 possible_transports=[to_transport])
208 finally:286 finally:
209 br_from.unlock()287 base_branch.branch.unlock()
210 return tree_to, br_to288 return tree_to, br_to
211289
212290
213def _resolve_revisions_recurse(new_branch, substitute_revno,291def _resolve_revisions_recurse(new_branch, substitute_revno,
214 if_changed_from=None):292 if_changed_from=None):
215 changed = False293 changed = False
216 br_from = branch.Branch.open(new_branch.url)294 new_branch.branch = branch.Branch.open(new_branch.url)
217 br_from.lock_read()295 new_branch.branch.lock_read()
218 try:296 try:
219 if new_branch.revspec is not None:297 if new_branch.revspec is not None:
220 revspec = revisionspec.RevisionSpec.from_string(298 revspec = revisionspec.RevisionSpec.from_string(
221 new_branch.revspec)299 new_branch.revspec)
222 revision_id = revspec.as_revision_id(br_from)300 revision_id = revspec.as_revision_id(new_branch.branch)
223 else:301 else:
224 revision_id = br_from.last_revision()302 revision_id = new_branch.branch.last_revision()
225 new_branch.revid = revision_id303 new_branch.revid = revision_id
226 def get_revno():304 def get_revno():
227 try:305 try:
228 revno = br_from.revision_id_to_revno(revision_id)306 revno = new_branch.branch.revision_id_to_revno(revision_id)
229 return str(revno)307 return str(revno)
230 except errors.NoSuchRevision:308 except errors.NoSuchRevision:
231 # We need to load and use the full revno map after all309 # We need to load and use the full revno map after all
232 result = br_from.get_revision_id_to_revno_map().get(310 result = new_branch.branch.get_revision_id_to_revno_map().get(
233 revision_id)311 revision_id)
234 if result is None:312 if result is None:
235 return result313 return result
@@ -242,14 +320,13 @@
242 changed_revspec = revisionspec.RevisionSpec.from_string(320 changed_revspec = revisionspec.RevisionSpec.from_string(
243 if_changed_from.revspec)321 if_changed_from.revspec)
244 changed_revision_id = changed_revspec.as_revision_id(322 changed_revision_id = changed_revspec.as_revision_id(
245 br_from)323 new_branch.branch)
246 else:324 else:
247 changed_revision_id = br_from.last_revision()325 changed_revision_id = new_branch.branch.last_revision()
248 if revision_id != changed_revision_id:326 if revision_id != changed_revision_id:
249 changed = True327 changed = True
250 for index, instruction in enumerate(new_branch.child_branches):328 for index, instruction in enumerate(new_branch.child_branches):
251 child_branch = instruction.recipe_branch329 child_branch = instruction.recipe_branch
252 nest_location = instruction.nest_path
253 if_changed_child = None330 if_changed_child = None
254 if if_changed_from is not None:331 if if_changed_from is not None:
255 if_changed_child = if_changed_from.child_branches[index].recipe_branch332 if_changed_child = if_changed_from.child_branches[index].recipe_branch
@@ -261,7 +338,7 @@
261 changed = child_changed338 changed = child_changed
262 return changed339 return changed
263 finally:340 finally:
264 br_from.unlock()341 new_branch.branch.unlock()
265342
266343
267def resolve_revisions(base_branch, if_changed_from=None):344def resolve_revisions(base_branch, if_changed_from=None):
@@ -289,30 +366,20 @@
289 if_changed_from=if_changed_from_revisions)366 if_changed_from=if_changed_from_revisions)
290 if not changed:367 if not changed:
291 changed = changed_revisions368 changed = changed_revisions
292 ok_to_preserve = ["{debupstream}"]369 check_expanded_deb_version(base_branch)
293 checked_version = base_branch.deb_version
294 for token in ok_to_preserve:
295 checked_version = checked_version.replace(token, "")
296 if "{" in checked_version:
297 raise errors.BzrCommandError("deb-version not fully "
298 "expanded: %s" % base_branch.deb_version)
299 if if_changed_from is not None and not changed:370 if if_changed_from is not None and not changed:
300 return False371 return False
301 return True372 return True
302373
303374
304def build_tree(base_branch, target_path):375def _build_inner_tree(base_branch, target_path, possible_transports=None):
305 """Build the RecipeBranch at a path.376 revision_of = ""
306377 if base_branch.revspec is not None:
307 Follow the instructions embodied in RecipeBranch and build a tree378 revision_of = "revision '%s' of " % base_branch.revspec
308 based on them rooted at target_path. If target_path exists and379 trace.note("Retrieving %s'%s' to put at '%s'."
309 is the root of the branch then the branch will be updated based on380 % (revision_of, base_branch.url, target_path))
310 what the RecipeBranch requires.381 to_transport = transport.get_transport(target_path,
311382 possible_transports=possible_transports)
312 :param base_branch: a RecipeBranch to build.
313 :param target_path: the path to the base of the desired output.
314 """
315 to_transport = transport.get_transport(target_path)
316 try:383 try:
317 tree_to, br_to = bzrdir.BzrDir.open_tree_or_branch(target_path)384 tree_to, br_to = bzrdir.BzrDir.open_tree_or_branch(target_path)
318 # Should we commit any changes in the tree here? If we don't385 # Should we commit any changes in the tree here? If we don't
@@ -327,7 +394,7 @@
327 br_to.lock_write()394 br_to.lock_write()
328 try:395 try:
329 tree_to, br_to = update_branch(base_branch, tree_to, br_to,396 tree_to, br_to = update_branch(base_branch, tree_to, br_to,
330 to_transport)397 to_transport, possible_transports=possible_transports)
331 for instruction in base_branch.child_branches:398 for instruction in base_branch.child_branches:
332 instruction.apply(target_path, tree_to, br_to)399 instruction.apply(target_path, tree_to, br_to)
333 finally:400 finally:
@@ -339,49 +406,25 @@
339 tree_to.unlock()406 tree_to.unlock()
340407
341408
342def _add_child_branches_to_manifest(child_branches, indent_level):409def build_tree(base_branch, target_path, possible_transports=None):
343 manifest = ""410 """Build the RecipeBranch at a path.
344 for instruction in child_branches:411
345 child_branch = instruction.recipe_branch412 Follow the instructions embodied in RecipeBranch and build a tree
346 nest_location = instruction.nest_path413 based on them rooted at target_path. If target_path exists and
347 if child_branch is None:414 is the root of the branch then the branch will be updated based on
348 manifest += "%s%s %s\n" % (" " * indent_level, RUN_INSTRUCTION,415 what the RecipeBranch requires.
349 nest_location)416
350 else:417 :param base_branch: a RecipeBranch to build.
351 assert child_branch.revid is not None, "Branch hasn't been built"418 :param target_path: the path to the base of the desired output.
352 if nest_location is not None:419 """
353 manifest += "%s%s %s %s %s revid:%s\n" % \420 trace.note("Building tree.")
354 (" " * indent_level, NEST_INSTRUCTION,421 _build_inner_tree(base_branch, target_path,
355 child_branch.name,422 possible_transports=possible_transports)
356 child_branch.url, nest_location,
357 child_branch.revid)
358 manifest += _add_child_branches_to_manifest(
359 child_branch.child_branches, indent_level+1)
360 else:
361 manifest += "%s%s %s %s revid:%s\n" % \
362 (" " * indent_level, MERGE_INSTRUCTION,
363 child_branch.name,
364 child_branch.url, child_branch.revid)
365 return manifest
366
367
368def build_manifest(base_branch):
369 manifest = "# bzr-builder format %s deb-version " % str(base_branch.format)
370 # TODO: should we store the expanded version that was used?
371 manifest += "%s\n" % (base_branch.deb_version,)
372 assert base_branch.revid is not None, "Branch hasn't been built"
373 manifest += "%s revid:%s\n" % (base_branch.url, base_branch.revid)
374 manifest += _add_child_branches_to_manifest(base_branch.child_branches, 0)
375 # Sanity check.
376 # TODO: write a function that compares the result of this parse with
377 # the branch that we built it from.
378 RecipeParser(manifest).parse()
379 return manifest
380423
381424
382class ChildBranch(object):425class ChildBranch(object):
383 """A child branch in a recipe.426 """A child branch in a recipe.
384 427
385 If the nest path is not None it is the path relative to the recipe branch428 If the nest path is not None it is the path relative to the recipe branch
386 where the child branch should be placed. If it is None then the child429 where the child branch should be placed. If it is None then the child
387 branch should be merged instead of nested.430 branch should be merged instead of nested.
@@ -390,8 +433,8 @@
390 def __init__(self, recipe_branch, nest_path=None):433 def __init__(self, recipe_branch, nest_path=None):
391 self.recipe_branch = recipe_branch434 self.recipe_branch = recipe_branch
392 self.nest_path = nest_path435 self.nest_path = nest_path
393 436
394 def apply(self, target_path, tree_to, br_to):437 def apply(self, target_path, tree_to, br_to, possible_transports=None):
395 raise NotImplementedError(self.apply)438 raise NotImplementedError(self.apply)
396439
397 def as_tuple(self):440 def as_tuple(self):
@@ -400,8 +443,9 @@
400443
401class CommandInstruction(ChildBranch):444class CommandInstruction(ChildBranch):
402445
403 def apply(self, target_path, tree_to, br_to):446 def apply(self, target_path, tree_to, br_to, possible_transports=None):
404 # it's a command447 # it's a command
448 trace.note("Running '%s' in '%s'." % (self.nest_path, target_path))
405 proc = subprocess.Popen(self.nest_path, cwd=target_path,449 proc = subprocess.Popen(self.nest_path, cwd=target_path,
406 preexec_fn=subprocess_setup, shell=True, stdin=subprocess.PIPE)450 preexec_fn=subprocess_setup, shell=True, stdin=subprocess.PIPE)
407 proc.communicate()451 proc.communicate()
@@ -411,16 +455,36 @@
411455
412class MergeInstruction(ChildBranch):456class MergeInstruction(ChildBranch):
413457
458 def apply(self, target_path, tree_to, br_to, possible_transports=None):
459 revision_of = ""
460 if self.recipe_branch.revspec is not None:
461 revision_of = "revision '%s' of " % self.recipe_branch.revspec
462 trace.note("Merging %s'%s' in to '%s'."
463 % (revision_of, self.recipe_branch.url, target_path))
464 merge_branch(self.recipe_branch, tree_to, br_to,
465 possible_transports=possible_transports)
466
467
468class NestPartInstruction(ChildBranch):
469
470 def __init__(self, recipe_branch, subpath, target_subdir):
471 ChildBranch.__init__(self, recipe_branch)
472 self.subpath = subpath
473 self.target_subdir = target_subdir
474
414 def apply(self, target_path, tree_to, br_to):475 def apply(self, target_path, tree_to, br_to):
415 merge_branch(self.recipe_branch, tree_to, br_to)476 from bzrlib.cleanup import OperationWithCleanups
477 op = OperationWithCleanups(nest_part_branch)
478 op.run(self.recipe_branch, tree_to, br_to, self.subpath,
479 self.target_subdir)
416480
417481
418class NestInstruction(ChildBranch):482class NestInstruction(ChildBranch):
419483
420 def apply(self, target_path, tree_to, br_to):484 def apply(self, target_path, tree_to, br_to, possible_transports=None):
421 # FIXME: pass possible_transports around485 _build_inner_tree(self.recipe_branch,
422 build_tree(self.recipe_branch,486 target_path=os.path.join(target_path, self.nest_path),
423 target_path=os.path.join(target_path, self.nest_path))487 possible_transports=possible_transports)
424488
425489
426class RecipeBranch(object):490class RecipeBranch(object):
@@ -430,9 +494,12 @@
430 root branch), and optionally child branches that are either merged494 root branch), and optionally child branches that are either merged
431 or nested.495 or nested.
432496
433 The child_branches attribute is a list of tuples of ChildBranch objects. 497 The child_branches attribute is a list of tuples of ChildBranch objects.
434 The revid attribute records the revid that the url and revspec resolved498 The revid attribute records the revid that the url and revspec resolved
435 to when the RecipeBranch was built, or None if it has not been built.499 to when the RecipeBranch was built, or None if it has not been built.
500
501 :ivar revid: after this recipe branch has been built this is set to the
502 revision ID that was merged/nested from the branch at self.url.
436 """503 """
437504
438 def __init__(self, name, url, revspec=None):505 def __init__(self, name, url, revspec=None):
@@ -448,6 +515,7 @@
448 self.revspec = revspec515 self.revspec = revspec
449 self.child_branches = []516 self.child_branches = []
450 self.revid = None517 self.revid = None
518 self.branch = None
451519
452 def merge_branch(self, branch):520 def merge_branch(self, branch):
453 """Merge a child branch in to this one.521 """Merge a child branch in to this one.
@@ -456,6 +524,18 @@
456 """524 """
457 self.child_branches.append(MergeInstruction(branch))525 self.child_branches.append(MergeInstruction(branch))
458526
527 def nest_part_branch(self, branch, subpath=None, target_subdir=None):
528 """Merge subdir of a child branch into this one.
529
530 :param branch: the RecipeBranch to merge.
531 :param subpath: only merge files from branch that are from this path.
532 e.g. subpath='/debian' will only merge changes from that directory.
533 :param target_subdir: (optional) directory in target to merge that
534 subpath into. Defaults to basename of subpath.
535 """
536 self.child_branches.append(
537 NestPartInstruction(branch, subpath, target_subdir))
538
459 def nest_branch(self, location, branch):539 def nest_branch(self, location, branch):
460 """Nest a child branch in to this one.540 """Nest a child branch in to this one.
461541
@@ -492,10 +572,32 @@
492 if ((child_branch is None and other_child_branch is not None)572 if ((child_branch is None and other_child_branch is not None)
493 or (child_branch is not None and other_child_branch is None)):573 or (child_branch is not None and other_child_branch is None)):
494 return True574 return True
495 if child_branch.different_shape_to(other_child_branch):575 # if child_branch is None then other_child_branch must be
576 # None too, meaning that they are both run instructions,
577 # we would compare their nest locations (commands), but
578 # that has already been done, so just guard
579 if (child_branch is not None
580 and child_branch.different_shape_to(other_child_branch)):
496 return True581 return True
497 return False582 return False
498583
584 def _list_child_names(self):
585 child_names = []
586 for instruction in self.child_branches:
587 child_branch = instruction.recipe_branch
588 if child_branch is None:
589 continue
590 child_names += child_branch.list_branch_names()
591 return child_names
592
593 def list_branch_names(self):
594 """List all of the branch names under this one.
595
596 :return: a list of the branch names.
597 :rtype: list(str)
598 """
599 return [self.name] + self._list_child_names()
600
499601
500class BaseRecipeBranch(RecipeBranch):602class BaseRecipeBranch(RecipeBranch):
501 """The RecipeBranch that is at the root of a recipe."""603 """The RecipeBranch that is at the root of a recipe."""
@@ -521,9 +623,9 @@
521 needed.623 needed.
522 """624 """
523 if branch_name is None:625 if branch_name is None:
524 subst_string = "{revno}"626 subst_string = REVNO_VAR
525 else:627 else:
526 subst_string = "{revno:%s}" % branch_name628 subst_string = REVNO_PARAM_VAR % branch_name
527 if subst_string in self.deb_version:629 if subst_string in self.deb_version:
528 revno = get_revno_cb()630 revno = get_revno_cb()
529 if revno is None:631 if revno is None:
@@ -537,9 +639,75 @@
537639
538 :param time: a datetime.datetime with the desired time.640 :param time: a datetime.datetime with the desired time.
539 """641 """
540 if "{time}" in self.deb_version:642 if TIME_VAR in self.deb_version:
541 self.deb_version = self.deb_version.replace("{time}",643 self.deb_version = self.deb_version.replace(TIME_VAR,
542 time.strftime("%Y%m%d%H%M"))644 time.strftime("%Y%m%d%H%M"))
645 if DATE_VAR in self.deb_version:
646 self.deb_version = self.deb_version.replace(DATE_VAR,
647 time.strftime("%Y%m%d"))
648
649 def substitute_debupstream(self, version):
650 """Substitute {debupstream} in to deb_version if needed.
651
652 :param version: the Version object to take the upstream version
653 from.
654 """
655 if DEBUPSTREAM_VAR in self.deb_version:
656 # Should we include the epoch?
657 self.deb_version = self.deb_version.replace(DEBUPSTREAM_VAR,
658 version.upstream_version)
659
660 def _add_child_branches_to_manifest(self, child_branches, indent_level):
661 manifest = ""
662 for instruction in child_branches:
663 child_branch = instruction.recipe_branch
664 nest_location = instruction.nest_path
665 if child_branch is None:
666 manifest += "%s%s %s\n" % (" " * indent_level, RUN_INSTRUCTION,
667 nest_location)
668 else:
669 if child_branch.revid is not None:
670 revid_part = " revid:%s" % child_branch.revid
671 elif child_branch.revspec is not None:
672 revid_part = " %s" % child_branch.revspec
673 else:
674 revid_part = ""
675 if nest_location is not None:
676 manifest += "%s%s %s %s %s%s\n" % \
677 (" " * indent_level, NEST_INSTRUCTION,
678 child_branch.name,
679 child_branch.url, nest_location,
680 revid_part)
681 manifest += self._add_child_branches_to_manifest(
682 child_branch.child_branches, indent_level+1)
683 else:
684 manifest += "%s%s %s %s%s\n" % \
685 (" " * indent_level, MERGE_INSTRUCTION,
686 child_branch.name,
687 child_branch.url, revid_part)
688 return manifest
689
690
691 def __str__(self):
692 manifest = "# bzr-builder format %s deb-version " % str(self.format)
693 # TODO: should we store the expanded version that was used?
694 manifest += "%s\n" % (self.deb_version,)
695 if self.revid is not None:
696 manifest += "%s revid:%s\n" % (self.url, self.revid)
697 elif self.revspec is not None:
698 manifest += "%s %s\n" % (self.url, self.revspec)
699 else:
700 manifest += "%s\n" % (self.url,)
701 manifest += self._add_child_branches_to_manifest(self.child_branches,
702 0)
703 # Sanity check.
704 # TODO: write a function that compares the result of this parse with
705 # the branch that we built it from.
706 RecipeParser(manifest).parse()
707 return manifest
708
709 def list_branch_names(self):
710 return self._list_child_names()
543711
544712
545class RecipeParseError(errors.BzrError):713class RecipeParseError(errors.BzrError):
@@ -550,6 +718,13 @@
550 problem=problem)718 problem=problem)
551719
552720
721class ForbiddenInstructionError(RecipeParseError):
722
723 def __init__(self, filename, line, char, problem, instruction_name=None):
724 RecipeParseError.__init__(self, filename, line, char, problem)
725 self.instruction_name = instruction_name
726
727
553class RecipeParser(object):728class RecipeParser(object):
554 """Parse a recipe.729 """Parse a recipe.
555730
@@ -560,7 +735,7 @@
560 eol_char = "\n"735 eol_char = "\n"
561 digit_chars = ("0", "1", "2", "3", "4", "5", "6", "7", "8", "9")736 digit_chars = ("0", "1", "2", "3", "4", "5", "6", "7", "8", "9")
562737
563 NEWEST_VERSION = 0.2738 NEWEST_VERSION = 0.3
564739
565 def __init__(self, f, filename=None):740 def __init__(self, f, filename=None):
566 """Create a RecipeParser.741 """Create a RecipeParser.
@@ -578,9 +753,12 @@
578 if filename is None:753 if filename is None:
579 self.filename = "recipe"754 self.filename = "recipe"
580755
581 def parse(self):756 def parse(self, forbidden_instructions=None):
582 """Parse the recipe.757 """Parse the recipe.
583758
759 :param forbidden_instructions: a list of instructions that you
760 don't want to allow. Defaults to None allowing them all.
761 :type forbidden_instructions: list(str) or None
584 :return: a RecipeBranch representing the recipe.762 :return: a RecipeBranch representing the recipe.
585 """763 """
586 self.lines = self.text.split("\n")764 self.lines = self.text.split("\n")
@@ -588,12 +766,21 @@
588 self.line_index = 0766 self.line_index = 0
589 self.current_line = self.lines[self.line_index]767 self.current_line = self.lines[self.line_index]
590 self.current_indent_level = 0768 self.current_indent_level = 0
769 self.seen_nicks = set()
770 self.seen_paths = {".": 1}
591 (version, deb_version) = self.parse_header()771 (version, deb_version) = self.parse_header()
592 self.version = version772 self.version = version
593 last_instruction = None773 last_instruction = None
594 active_branches = []774 active_branches = []
595 last_branch = None775 last_branch = None
596 while self.line_index < len(self.lines):776 while self.line_index < len(self.lines):
777 if self.is_blankline():
778 self.new_line()
779 continue
780 comment = self.parse_comment_line()
781 if comment is not None:
782 self.new_line()
783 continue
597 old_indent_level = self.parse_indent()784 old_indent_level = self.parse_indent()
598 if old_indent_level is not None:785 if old_indent_level is not None:
599 if (old_indent_level < self.current_indent_level786 if (old_indent_level < self.current_indent_level
@@ -605,10 +792,6 @@
605 else:792 else:
606 unindent = self.current_indent_level - old_indent_level793 unindent = self.current_indent_level - old_indent_level
607 active_branches = active_branches[:unindent]794 active_branches = active_branches[:unindent]
608 comment = self.parse_comment_line()
609 if comment is not None:
610 self.new_line()
611 continue
612 if last_instruction is None:795 if last_instruction is None:
613 url = self.take_to_whitespace("branch to start from")796 url = self.take_to_whitespace("branch to start from")
614 revspec = self.parse_optional_revspec()797 revspec = self.parse_optional_revspec()
@@ -618,7 +801,8 @@
618 active_branches = [last_branch]801 active_branches = [last_branch]
619 last_instruction = ""802 last_instruction = ""
620 else:803 else:
621 instruction = self.parse_instruction()804 instruction = self.parse_instruction(
805 forbidden_instructions=forbidden_instructions)
622 if instruction == RUN_INSTRUCTION:806 if instruction == RUN_INSTRUCTION:
623 self.parse_whitespace("the command")807 self.parse_whitespace("the command")
624 command = self.take_to_newline().strip()808 command = self.take_to_newline().strip()
@@ -629,13 +813,25 @@
629 url = self.parse_branch_url()813 url = self.parse_branch_url()
630 if instruction == NEST_INSTRUCTION:814 if instruction == NEST_INSTRUCTION:
631 location = self.parse_branch_location()815 location = self.parse_branch_location()
632 revspec = self.parse_optional_revspec()816 if instruction == NEST_PART_INSTRUCTION:
817 path = self.parse_subpath()
818 target_subdir = self.parse_optional_path()
819 if target_subdir == '':
820 target_subdir = None
821 revspec = None
822 else:
823 revspec = self.parse_optional_revspec()
824 else:
825 revspec = self.parse_optional_revspec()
633 self.new_line()826 self.new_line()
634 last_branch = RecipeBranch(branch_id, url, revspec=revspec)827 last_branch = RecipeBranch(branch_id, url, revspec=revspec)
635 if instruction == NEST_INSTRUCTION:828 if instruction == NEST_INSTRUCTION:
636 active_branches[-1].nest_branch(location, last_branch)829 active_branches[-1].nest_branch(location, last_branch)
637 else:830 elif instruction == MERGE_INSTRUCTION:
638 active_branches[-1].merge_branch(last_branch)831 active_branches[-1].merge_branch(last_branch)
832 elif instruction == NEST_PART_INSTRUCTION:
833 active_branches[-1].nest_part_branch(
834 last_branch, path, target_subdir)
639 last_instruction = instruction835 last_instruction = instruction
640 if len(active_branches) == 0:836 if len(active_branches) == 0:
641 self.throw_parse_error("Empty recipe")837 self.throw_parse_error("Empty recipe")
@@ -655,18 +851,28 @@
655 self.new_line()851 self.new_line()
656 return version, deb_version852 return version, deb_version
657853
658 def parse_instruction(self):854 def parse_instruction(self, forbidden_instructions=None):
659 if self.version < 0.2:855 if self.version < 0.2:
660 options = (MERGE_INSTRUCTION, NEST_INSTRUCTION)856 options = (MERGE_INSTRUCTION, NEST_INSTRUCTION)
661 options_str = "'%s' or '%s'" % options857 options_str = "'%s' or '%s'" % options
662 else:858 elif self.version < 0.3:
663 options = (MERGE_INSTRUCTION, NEST_INSTRUCTION, RUN_INSTRUCTION)859 options = (MERGE_INSTRUCTION, NEST_INSTRUCTION, RUN_INSTRUCTION)
664 options_str = "'%s', '%s' or '%s'" % options860 options_str = "'%s', '%s' or '%s'" % options
861 else:
862 options = (MERGE_INSTRUCTION, NEST_INSTRUCTION,
863 NEST_PART_INSTRUCTION, RUN_INSTRUCTION)
864 options_str = "'%s', '%s', '%s' or '%s'" % options
665 instruction = self.peek_to_whitespace()865 instruction = self.peek_to_whitespace()
666 if instruction is None:866 if instruction is None:
667 self.throw_parse_error("End of line while looking for %s"867 self.throw_parse_error("End of line while looking for %s"
668 % options_str)868 % options_str)
669 if instruction in options:869 if instruction in options:
870 if forbidden_instructions is not None:
871 if instruction in forbidden_instructions:
872 self.throw_parse_error("The '%s' instruction is "
873 "forbidden." % instruction,
874 cls=ForbiddenInstructionError,
875 instruction_name=instruction)
670 self.take_chars(len(instruction))876 self.take_chars(len(instruction))
671 return instruction877 return instruction
672 self.throw_parse_error("Expecting %s, got '%s'"878 self.throw_parse_error("Expecting %s, got '%s'"
@@ -674,7 +880,15 @@
674880
675 def parse_branch_id(self):881 def parse_branch_id(self):
676 self.parse_whitespace("the branch id")882 self.parse_whitespace("the branch id")
677 branch_id = self.take_to_whitespace("the branch id")883 branch_id = self.peek_to_whitespace()
884 if branch_id is None:
885 self.throw_parse_error("End of line while looking for the "
886 "branch id")
887 if branch_id in self.seen_nicks:
888 self.throw_parse_error("'%s' was already used to identify "
889 "a branch." % branch_id)
890 self.take_chars(len(branch_id))
891 self.seen_nicks.add(branch_id)
678 return branch_id892 return branch_id
679893
680 def parse_branch_url(self):894 def parse_branch_url(self):
@@ -685,8 +899,34 @@
685 def parse_branch_location(self):899 def parse_branch_location(self):
686 # FIXME: Needs a better term900 # FIXME: Needs a better term
687 self.parse_whitespace("the location to nest")901 self.parse_whitespace("the location to nest")
688 location = self.take_to_whitespace("the location to nest")902 location = self.peek_to_whitespace()
689 return location903 if location is None:
904 self.throw_parse_error("End of line while looking for the "
905 "location to nest")
906 norm_location = os.path.normpath(location)
907 if norm_location in self.seen_paths:
908 self.throw_parse_error("The path '%s' is a duplicate of "
909 "the one used on line %d." % (location,
910 self.seen_paths[norm_location]))
911 if os.path.isabs(norm_location):
912 self.throw_parse_error("Absolute paths are not allowed: %s"
913 % location)
914 if norm_location.startswith(".."):
915 self.throw_parse_error("Paths outside the current directory "
916 "are not allowed: %s" % location)
917 self.take_chars(len(location))
918 self.seen_paths[norm_location] = self.line_index + 1
919 return location
920
921 def parse_subpath(self):
922 self.parse_whitespace("the subpath to merge")
923 location = self.take_to_whitespace("the subpath to merge")
924 return location
925
926 def parse_revspec(self):
927 self.parse_whitespace("the revspec")
928 revspec = self.take_to_whitespace("the revspec")
929 return revspec
690930
691 def parse_optional_revspec(self):931 def parse_optional_revspec(self):
692 self.parse_whitespace(None, require=False)932 self.parse_whitespace(None, require=False)
@@ -695,9 +935,18 @@
695 self.take_chars(len(revspec))935 self.take_chars(len(revspec))
696 return revspec936 return revspec
697937
698 def throw_parse_error(self, problem):938 def parse_optional_path(self):
699 raise RecipeParseError(self.filename, self.line_index + 1,939 self.parse_whitespace(None, require=False)
700 self.index + 1, problem)940 path = self.peek_to_whitespace()
941 if path is not None:
942 self.take_chars(len(path))
943 return path
944
945 def throw_parse_error(self, problem, cls=None, **kwargs):
946 if cls is None:
947 cls = RecipeParseError
948 raise cls(self.filename, self.line_index + 1,
949 self.index + 1, problem, **kwargs)
701950
702 def throw_expecting_error(self, expected, actual):951 def throw_expecting_error(self, expected, actual):
703 self.throw_parse_error("Expecting '%s', got '%s'"952 self.throw_parse_error("Expecting '%s', got '%s'"
@@ -720,6 +969,12 @@
720 else:969 else:
721 self.current_line = self.lines[self.line_index]970 self.current_line = self.lines[self.line_index]
722971
972 def is_blankline(self):
973 whitespace = self.peek_whitespace()
974 if whitespace is None:
975 return True
976 return self.peek_char(skip=len(whitespace)) is None
977
723 def take_char(self):978 def take_char(self):
724 if self.index >= len(self.current_line):979 if self.index >= len(self.current_line):
725 return None980 return None
@@ -789,6 +1044,18 @@
789 actual = self.peek_char()1044 actual = self.peek_char()
790 return ret1045 return ret
7911046
1047 def peek_whitespace(self):
1048 ret = ""
1049 char = self.peek_char()
1050 if char is None:
1051 return char
1052 count = 0
1053 while char is not None and char in self.whitespace_chars:
1054 ret += char
1055 count += 1
1056 char = self.peek_char(skip=count)
1057 return ret
1058
792 def parse_word(self, expected, require_whitespace=True):1059 def parse_word(self, expected, require_whitespace=True):
793 self.parse_whitespace("'%s'" % expected, require=require_whitespace)1060 self.parse_whitespace("'%s'" % expected, require=require_whitespace)
794 length = len(expected)1061 length = len(expected)
@@ -865,10 +1132,14 @@
865 return text1132 return text
8661133
867 def parse_comment_line(self):1134 def parse_comment_line(self):
868 if self.peek_char() is None:1135 whitespace = self.peek_whitespace()
869 return ""1136 if whitespace is None:
870 if self.peek_char() != "#":1137 return ""
1138 if self.peek_char(skip=len(whitespace)) is None:
1139 return ""
1140 if self.peek_char(skip=len(whitespace)) != "#":
871 return None1141 return None
1142 self.parse_whitespace(None, require=False)
872 comment = self.current_line[self.index:]1143 comment = self.current_line[self.index:]
873 self.index += len(comment)1144 self.index += len(comment)
874 return comment1145 return comment
8751146
=== modified file 'setup.py'
--- setup.py 2009-08-25 11:43:42 +0000
+++ setup.py 2010-08-13 01:14:42 +0000
@@ -2,15 +2,16 @@
22
3from distutils.core import setup3from distutils.core import setup
44
5setup(name="bzr-builder",5if __name__ == '__main__':
6 version="0.1",6 setup(name="bzr-builder",
7 description="Turn a recipe in to a bzr branch",7 version="0.2",
8 author="James Westby",8 description="Turn a recipe in to a bzr branch",
9 author_email="james.westby@canonical.com",9 author="James Westby",
10 license="GNU GPL v3",10 author_email="james.westby@canonical.com",
11 url="http://launchpad.net/bzr-builder",11 license="GNU GPL v3",
12 packages=['bzrlib.plugins.builder',12 url="http://launchpad.net/bzr-builder",
13 'bzrlib.plugins.builder.tests',13 packages=['bzrlib.plugins.builder',
14 ],14 'bzrlib.plugins.builder.tests',
15 package_dir={'bzrlib.plugins.builder': '.'},15 ],
16 )16 package_dir={'bzrlib.plugins.builder': '.'},
17 )
1718
=== modified file 'tests/__init__.py'
--- tests/__init__.py 2009-04-16 20:23:39 +0000
+++ tests/__init__.py 2010-08-13 01:14:42 +0000
@@ -20,9 +20,10 @@
20 loader = TestUtil.TestLoader()20 loader = TestUtil.TestLoader()
21 suite = TestSuite()21 suite = TestSuite()
22 testmod_names = [22 testmod_names = [
23 'test_blackbox',23 'blackbox',
24 'test_recipe',24 'ppa',
25 'recipe',
25 ]26 ]
26 suite.addTest(loader.loadTestsFromModuleNames(["%s.%s" % (__name__, i)27 suite.addTest(loader.loadTestsFromModuleNames(["%s.test_%s" % (__name__, i)
27 for i in testmod_names]))28 for i in testmod_names]))
28 return suite29 return suite
2930
=== modified file 'tests/test_blackbox.py'
--- tests/test_blackbox.py 2010-08-13 01:14:42 +0000
+++ tests/test_blackbox.py 2010-08-13 01:14:42 +0000
@@ -55,7 +55,7 @@
55 self.build_tree(["source/a"])55 self.build_tree(["source/a"])
56 source.add(["a"])56 source.add(["a"])
57 revid = source.commit("one")57 revid = source.commit("one")
58 self.run_bzr("build recipe working")58 self.run_bzr("build -q recipe working")
59 self.failUnlessExists("working/a")59 self.failUnlessExists("working/a")
60 tree = workingtree.WorkingTree.open("working")60 tree = workingtree.WorkingTree.open("working")
61 self.assertEqual(revid, tree.last_revision())61 self.assertEqual(revid, tree.last_revision())
@@ -71,7 +71,7 @@
71 self.build_tree(["source/a"])71 self.build_tree(["source/a"])
72 source.add(["a"])72 source.add(["a"])
73 revid = source.commit("one")73 revid = source.commit("one")
74 self.run_bzr("build recipe working --manifest manifest")74 self.run_bzr("build -q recipe working --manifest manifest")
75 self.failUnlessExists("working/a")75 self.failUnlessExists("working/a")
76 self.failUnlessExists("manifest")76 self.failUnlessExists("manifest")
77 self.check_file_contents("manifest", "# bzr-builder format 0.1 "77 self.check_file_contents("manifest", "# bzr-builder format 0.1 "
@@ -83,7 +83,7 @@
83 source = self.make_branch_and_tree("source")83 source = self.make_branch_and_tree("source")
84 self.build_tree(["source/a"])84 self.build_tree(["source/a"])
85 source.add(["a"])85 source.add(["a"])
86 revid = source.commit("one")86 source.commit("one")
87 out, err = self.run_bzr("build recipe working "87 out, err = self.run_bzr("build recipe working "
88 "--if-changed-from manifest")88 "--if-changed-from manifest")
8989
@@ -111,7 +111,7 @@
111 "deb-version 1\nsource 1\n")])111 "deb-version 1\nsource 1\n")])
112 self.build_tree_contents([("old-manifest", "# bzr-builder format 0.1 "112 self.build_tree_contents([("old-manifest", "# bzr-builder format 0.1 "
113 "deb-version 1\nsource revid:foo\n")])113 "deb-version 1\nsource revid:foo\n")])
114 out, err = self.run_bzr("build recipe working --manifest manifest "114 out, err = self.run_bzr("build -q recipe working --manifest manifest "
115 "--if-changed-from old-manifest")115 "--if-changed-from old-manifest")
116 self.failUnlessExists("working/a")116 self.failUnlessExists("working/a")
117 self.failUnlessExists("manifest")117 self.failUnlessExists("manifest")
@@ -130,7 +130,7 @@
130 revid = source.commit("one")130 revid = source.commit("one")
131 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "131 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "
132 "deb-version 1\nsource 1\n")])132 "deb-version 1\nsource 1\n")])
133 out, err = self.run_bzr("dailydeb test.recipe working "133 out, err = self.run_bzr("dailydeb -q test.recipe working "
134 "--manifest manifest --package foo")134 "--manifest manifest --package foo")
135 self.failIfExists("working/a")135 self.failIfExists("working/a")
136 package_root = "working/foo-1/"136 package_root = "working/foo-1/"
@@ -149,13 +149,14 @@
149 cl_f = open(cl_path)149 cl_f = open(cl_path)
150 try:150 try:
151 line = cl_f.readline()151 line = cl_f.readline()
152 self.assertEqual("foo (1) jaunty; urgency=low\n", line)152 self.assertEqual("foo (1) lucid; urgency=low\n", line)
153 finally:153 finally:
154 cl_f.close()154 cl_f.close()
155155
156 def test_cmd_dailydeb_no_work_dir(self):156 def test_cmd_dailydeb_no_work_dir(self):
157 #TODO: define a test feature for debuild and require it here.157 #TODO: define a test feature for debuild and require it here.
158 self.permit_dir('/') # Allow the made working dir to be accessed.158 if getattr(self, "permit_dir", None) is not None:
159 self.permit_dir('/') # Allow the made working dir to be accessed.
159 source = self.make_branch_and_tree("source")160 source = self.make_branch_and_tree("source")
160 self.build_tree(["source/a", "source/debian/"])161 self.build_tree(["source/a", "source/debian/"])
161 self.build_tree_contents([("source/debian/rules",162 self.build_tree_contents([("source/debian/rules",
@@ -163,15 +164,16 @@
163 ("source/debian/control",164 ("source/debian/control",
164 "Source: foo\nMaintainer: maint maint@maint.org\n")])165 "Source: foo\nMaintainer: maint maint@maint.org\n")])
165 source.add(["a", "debian/", "debian/rules", "debian/control"])166 source.add(["a", "debian/", "debian/rules", "debian/control"])
166 revid = source.commit("one")167 source.commit("one")
167 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "168 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "
168 "deb-version 1\nsource 1\n")])169 "deb-version 1\nsource 1\n")])
169 out, err = self.run_bzr("dailydeb test.recipe "170 out, err = self.run_bzr("dailydeb -q test.recipe "
170 "--manifest manifest --package foo")171 "--manifest manifest --package foo")
171172
172 def test_cmd_dailydeb_if_changed_from_non_existant(self):173 def test_cmd_dailydeb_if_changed_from_non_existant(self):
173 #TODO: define a test feature for debuild and require it here.174 #TODO: define a test feature for debuild and require it here.
174 self.permit_dir('/') # Allow the made working dir to be accessed.175 if getattr(self, "permit_dir", None) is not None:
176 self.permit_dir('/') # Allow the made working dir to be accessed.
175 source = self.make_branch_and_tree("source")177 source = self.make_branch_and_tree("source")
176 self.build_tree(["source/a", "source/debian/"])178 self.build_tree(["source/a", "source/debian/"])
177 self.build_tree_contents([("source/debian/rules",179 self.build_tree_contents([("source/debian/rules",
@@ -179,10 +181,10 @@
179 ("source/debian/control",181 ("source/debian/control",
180 "Source: foo\nMaintainer: maint maint@maint.org\n")])182 "Source: foo\nMaintainer: maint maint@maint.org\n")])
181 source.add(["a", "debian/", "debian/rules", "debian/control"])183 source.add(["a", "debian/", "debian/rules", "debian/control"])
182 revid = source.commit("one")184 source.commit("one")
183 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "185 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "
184 "deb-version 1\nsource 1\n")])186 "deb-version 1\nsource 1\n")])
185 out, err = self.run_bzr("dailydeb test.recipe "187 out, err = self.run_bzr("dailydeb -q test.recipe "
186 "--manifest manifest --package foo --if-changed-from bar")188 "--manifest manifest --package foo --if-changed-from bar")
187189
188 def make_simple_package(self):190 def make_simple_package(self):
@@ -201,12 +203,29 @@
201 source.commit("one")203 source.commit("one")
202 return source204 return source
203205
206 def test_cmd_dailydeb_no_build(self):
207 self.make_simple_package()
208 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "
209 "deb-version 1\nsource 1\n")])
210 out, err = self.run_bzr("dailydeb -q test.recipe "
211 "--manifest manifest --no-build working")
212 new_cl_contents = ("package (1) unstable; urgency=low\n\n"
213 " * Auto build.\n\n -- M. Maintainer <maint@maint.org> ")
214 f = open("working/test-1/debian/changelog")
215 try:
216 actual_cl_contents = f.read()
217 finally:
218 f.close()
219 self.assertStartsWith(actual_cl_contents, new_cl_contents)
220 for fn in os.listdir("working"):
221 self.assertFalse(fn.endswith(".changes"))
222
204 def test_cmd_dailydeb_with_package_from_changelog(self):223 def test_cmd_dailydeb_with_package_from_changelog(self):
205 #TODO: define a test feature for debuild and require it here.224 #TODO: define a test feature for debuild and require it here.
206 source = self.make_simple_package()225 self.make_simple_package()
207 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "226 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "
208 "deb-version 1\nsource 1\n")])227 "deb-version 1\nsource 1\n")])
209 out, err = self.run_bzr("dailydeb test.recipe "228 out, err = self.run_bzr("dailydeb -q test.recipe "
210 "--manifest manifest --if-changed-from bar working")229 "--manifest manifest --if-changed-from bar working")
211 new_cl_contents = ("package (1) unstable; urgency=low\n\n"230 new_cl_contents = ("package (1) unstable; urgency=low\n\n"
212 " * Auto build.\n\n -- M. Maintainer <maint@maint.org> ")231 " * Auto build.\n\n -- M. Maintainer <maint@maint.org> ")
@@ -218,10 +237,10 @@
218 self.assertStartsWith(actual_cl_contents, new_cl_contents)237 self.assertStartsWith(actual_cl_contents, new_cl_contents)
219238
220 def test_cmd_dailydeb_with_upstream_version_from_changelog(self):239 def test_cmd_dailydeb_with_upstream_version_from_changelog(self):
221 source = self.make_simple_package()240 self.make_simple_package()
222 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "241 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "
223 "deb-version {debupstream}-2\nsource 1\n")])242 "deb-version {debupstream}-2\nsource 1\n")])
224 out, err = self.run_bzr("dailydeb test.recipe working")243 out, err = self.run_bzr("dailydeb -q test.recipe working")
225 new_cl_contents = ("package (0.1-2) unstable; urgency=low\n\n"244 new_cl_contents = ("package (0.1-2) unstable; urgency=low\n\n"
226 " * Auto build.\n\n -- M. Maintainer <maint@maint.org> ")245 " * Auto build.\n\n -- M. Maintainer <maint@maint.org> ")
227 f = open("working/test-{debupstream}-2/debian/changelog")246 f = open("working/test-{debupstream}-2/debian/changelog")
@@ -230,3 +249,30 @@
230 finally:249 finally:
231 f.close()250 f.close()
232 self.assertStartsWith(actual_cl_contents, new_cl_contents)251 self.assertStartsWith(actual_cl_contents, new_cl_contents)
252
253 def test_cmd_dailydeb_with_append_version(self):
254 self.make_simple_package()
255 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "
256 "deb-version 1\nsource 1\n")])
257 out, err = self.run_bzr("dailydeb -q test.recipe working "
258 "--append-version ~ppa1")
259 new_cl_contents = ("package (1~ppa1) unstable; urgency=low\n\n"
260 " * Auto build.\n\n -- M. Maintainer <maint@maint.org> ")
261 f = open("working/test-1/debian/changelog")
262 try:
263 actual_cl_contents = f.read()
264 finally:
265 f.close()
266 self.assertStartsWith(actual_cl_contents, new_cl_contents)
267
268 def test_cmd_dailydeb_with_invalid_version(self):
269 source = self.make_branch_and_tree("source")
270 self.build_tree(["source/a"])
271 source.add(["a"])
272 revid = source.commit("one")
273 self.build_tree_contents([("test.recipe", "# bzr-builder format 0.1 "
274 "deb-version $\nsource 1\n")])
275 err = self.run_bzr("dailydeb -q test.recipe working --package foo",
276 retcode=3)[1]
277 self.assertEqual("bzr: ERROR: Invalid deb-version: $: "
278 "Could not parse version: $\n", err)
233279
=== added file 'tests/test_ppa.py'
--- tests/test_ppa.py 1970-01-01 00:00:00 +0000
+++ tests/test_ppa.py 2010-08-13 01:14:42 +0000
@@ -0,0 +1,28 @@
1# bzr-builder: a bzr plugin to construct trees based on recipes
2# Copyright 2009 Canonical Ltd.
3
4# This program is free software: you can redistribute it and/or modify it
5# under the terms of the GNU General Public License version 3, as published
6# by the Free Software Foundation.
7
8# This program is distributed in the hope that it will be useful, but
9# WITHOUT ANY WARRANTY; without even the implied warranties of
10# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
11# PURPOSE. See the GNU General Public License for more details.
12
13# You should have received a copy of the GNU General Public License along
14# with this program. If not, see <http://www.gnu.org/licenses/>.
15
16from bzrlib.plugins.builder.cmds import target_from_dput
17from bzrlib.tests import (
18 TestCase,
19 )
20
21
22class TestTargetFromDPut(TestCase):
23
24 def test_default_ppa(self):
25 self.assertEqual('team-name/ppa', target_from_dput('ppa:team-name'))
26
27 def test_named_ppa(self):
28 self.assertEqual('team/ppa2', target_from_dput('ppa:team/ppa2'))
029
=== modified file 'tests/test_recipe.py'
--- tests/test_recipe.py 2010-08-13 01:14:42 +0000
+++ tests/test_recipe.py 2010-08-13 01:14:42 +0000
@@ -27,26 +27,28 @@
27 )27 )
28from bzrlib.plugins.builder.recipe import (28from bzrlib.plugins.builder.recipe import (
29 BaseRecipeBranch,29 BaseRecipeBranch,
30 build_manifest,
31 build_tree,30 build_tree,
32 ensure_basedir,31 ensure_basedir,
32 ForbiddenInstructionError,
33 pull_or_branch,33 pull_or_branch,
34 RecipeParser,34 RecipeParser,
35 RecipeBranch,35 RecipeBranch,
36 RecipeParseError,36 RecipeParseError,
37 resolve_revisions,37 resolve_revisions,
38 RUN_INSTRUCTION,
38 )39 )
3940
4041
41class RecipeParserTests(TestCaseInTempDir):42class RecipeParserTests(TestCaseInTempDir):
4243
43 deb_version = "0.1-{revno}"44 deb_version = "0.1-{revno}"
44 basic_header = ("# bzr-builder format 0.2 deb-version "45 basic_header = ("# bzr-builder format 0.3 deb-version "
45 + deb_version +"\n")46 + deb_version +"\n")
46 basic_header_and_branch = basic_header + "http://foo.org/\n"47 basic_branch = "http://foo.org/"
48 basic_header_and_branch = basic_header + basic_branch + "\n"
4749
48 def get_recipe(self, recipe_text):50 def get_recipe(self, recipe_text, **kwargs):
49 return RecipeParser(recipe_text).parse()51 return RecipeParser(recipe_text).parse(**kwargs)
5052
51 def assertParseError(self, line, char, problem, callable, *args,53 def assertParseError(self, line, char, problem, callable, *args,
52 **kwargs):54 **kwargs):
@@ -55,6 +57,7 @@
55 self.assertEqual(line, exc.line)57 self.assertEqual(line, exc.line)
56 self.assertEqual(char, exc.char)58 self.assertEqual(char, exc.char)
57 self.assertEqual("recipe", exc.filename)59 self.assertEqual("recipe", exc.filename)
60 return exc
5861
59 def check_recipe_branch(self, branch, name, url, revspec=None,62 def check_recipe_branch(self, branch, name, url, revspec=None,
60 num_child_branches=0, revid=None):63 num_child_branches=0, revid=None):
@@ -130,8 +133,8 @@
130 self.basic_header + "http://foo.org/ 2 foo")133 self.basic_header + "http://foo.org/ 2 foo")
131134
132 def tests_rejects_unknown_instruction(self):135 def tests_rejects_unknown_instruction(self):
133 self.assertParseError(3, 1, "Expecting 'merge', 'nest' or 'run', "136 self.assertParseError(3, 1, "Expecting 'merge', 'nest', 'nest-part' "
134 "got 'cat'", self.get_recipe,137 "or 'run', got 'cat'", self.get_recipe,
135 self.basic_header + "http://foo.org/\n" + "cat")138 self.basic_header + "http://foo.org/\n" + "cat")
136139
137 def test_rejects_merge_no_name(self):140 def test_rejects_merge_no_name(self):
@@ -149,6 +152,21 @@
149 "got 'bar'", self.get_recipe,152 "got 'bar'", self.get_recipe,
150 self.basic_header_and_branch + "merge foo url 2 bar")153 self.basic_header_and_branch + "merge foo url 2 bar")
151154
155 def test_rejects_nest_part_no_name(self):
156 self.assertParseError(3, 11, "End of line while looking for "
157 "the branch id", self.get_recipe,
158 self.basic_header_and_branch + "nest-part ")
159
160 def test_rejects_nest_part_no_url(self):
161 self.assertParseError(3, 15, "End of line while looking for "
162 "the branch url", self.get_recipe,
163 self.basic_header_and_branch + "nest-part foo ")
164
165 def test_rejects_nest_part_no_subpath(self):
166 self.assertParseError(3, 22, "End of line while looking for "
167 "the subpath to merge", self.get_recipe,
168 self.basic_header_and_branch + "nest-part foo url:// ")
169
152 def test_rejects_nest_no_name(self):170 def test_rejects_nest_no_name(self):
153 self.assertParseError(3, 6, "End of line while looking for "171 self.assertParseError(3, 6, "End of line while looking for "
154 "the branch id", self.get_recipe,172 "the branch id", self.get_recipe,
@@ -202,6 +220,12 @@
202 self.assertParseError(3, 1, "Empty recipe", self.get_recipe,220 self.assertParseError(3, 1, "Empty recipe", self.get_recipe,
203 self.basic_header)221 self.basic_header)
204222
223 def test_rejects_non_unique_ids(self):
224 self.assertParseError(4, 7, "'foo' was already used to identify "
225 "a branch.", self.get_recipe,
226 self.basic_header_and_branch + "merge foo url\n"
227 + "merge foo other-url\n")
228
205 def test_builds_simplest_recipe(self):229 def test_builds_simplest_recipe(self):
206 base_branch = self.get_recipe(self.basic_header_and_branch)230 base_branch = self.get_recipe(self.basic_header_and_branch)
207 self.check_base_recipe_branch(base_branch, "http://foo.org/")231 self.check_base_recipe_branch(base_branch, "http://foo.org/")
@@ -220,6 +244,42 @@
220 self.assertEqual(None, location)244 self.assertEqual(None, location)
221 self.check_recipe_branch(child_branch, "bar", "http://bar.org")245 self.check_recipe_branch(child_branch, "bar", "http://bar.org")
222246
247 def test_builds_recipe_with_nest_part(self):
248 base_branch = self.get_recipe(self.basic_header_and_branch
249 + "nest-part bar http://bar.org some/path")
250 self.check_base_recipe_branch(base_branch, "http://foo.org/",
251 num_child_branches=1)
252 instruction = base_branch.child_branches[0]
253 self.assertEqual(None, instruction.nest_path)
254 self.check_recipe_branch(
255 instruction.recipe_branch, "bar", "http://bar.org")
256 self.assertEqual("some/path", instruction.subpath)
257 self.assertEqual(None, instruction.target_subdir)
258
259 def test_builds_recipe_with_nest_part_subdir(self):
260 base_branch = self.get_recipe(self.basic_header_and_branch
261 + "nest-part bar http://bar.org some/path target-subdir")
262 self.check_base_recipe_branch(base_branch, "http://foo.org/",
263 num_child_branches=1)
264 instruction = base_branch.child_branches[0]
265 self.assertEqual(None, instruction.nest_path)
266 self.check_recipe_branch(
267 instruction.recipe_branch, "bar", "http://bar.org")
268 self.assertEqual("some/path", instruction.subpath)
269 self.assertEqual("target-subdir", instruction.target_subdir)
270
271 def test_builds_recipe_with_nest_part_subdir_and_revspec(self):
272 base_branch = self.get_recipe(self.basic_header_and_branch
273 + "nest-part bar http://bar.org some/path target-subdir 1234")
274 self.check_base_recipe_branch(base_branch, "http://foo.org/",
275 num_child_branches=1)
276 instruction = base_branch.child_branches[0]
277 self.assertEqual(None, instruction.nest_path)
278 self.check_recipe_branch(
279 instruction.recipe_branch, "bar", "http://bar.org", "1234")
280 self.assertEqual("some/path", instruction.subpath)
281 self.assertEqual("target-subdir", instruction.target_subdir)
282
223 def test_builds_recipe_with_nest(self):283 def test_builds_recipe_with_nest(self):
224 base_branch = self.get_recipe(self.basic_header_and_branch284 base_branch = self.get_recipe(self.basic_header_and_branch
225 + "nest bar http://bar.org baz")285 + "nest bar http://bar.org baz")
@@ -306,6 +366,60 @@
306 self.assertEqual(None, child_branch)366 self.assertEqual(None, child_branch)
307 self.assertEqual("touch test", command)367 self.assertEqual("touch test", command)
308368
369 def test_accepts_blank_line_during_nest(self):
370 base_branch = self.get_recipe(self.basic_header_and_branch
371 + "nest foo http://bar.org bar\n merge baz baz.org\n\n"
372 " merge zap zap.org\n")
373 self.check_base_recipe_branch(base_branch, self.basic_branch,
374 num_child_branches=1)
375 nested_branch, location = base_branch.child_branches[0].as_tuple()
376 self.assertEqual("bar", location)
377 self.check_recipe_branch(nested_branch, "foo", "http://bar.org",
378 num_child_branches=2)
379 child_branch, location = nested_branch.child_branches[0].as_tuple()
380 self.assertEqual(None, location)
381 self.check_recipe_branch(child_branch, "baz", "baz.org")
382 child_branch, location = nested_branch.child_branches[1].as_tuple()
383 self.assertEqual(None, location)
384 self.check_recipe_branch(child_branch, "zap", "zap.org")
385
386 def test_accepts_blank_line_at_start_of_nest(self):
387 base_branch = self.get_recipe(self.basic_header_and_branch
388 + "nest foo http://bar.org bar\n\n merge baz baz.org\n")
389 self.check_base_recipe_branch(base_branch, self.basic_branch,
390 num_child_branches=1)
391 nested_branch, location = base_branch.child_branches[0].as_tuple()
392 self.assertEqual("bar", location)
393 self.check_recipe_branch(nested_branch, "foo", "http://bar.org",
394 num_child_branches=1)
395 child_branch, location = nested_branch.child_branches[0].as_tuple()
396 self.assertEqual(None, location)
397 self.check_recipe_branch(child_branch, "baz", "baz.org")
398
399 def test_accepts_blank_line_as_only_thing_in_nest(self):
400 base_branch = self.get_recipe(self.basic_header_and_branch
401 + "nest foo http://bar.org bar\n\nmerge baz baz.org\n")
402 self.check_base_recipe_branch(base_branch, self.basic_branch,
403 num_child_branches=2)
404 nested_branch, location = base_branch.child_branches[0].as_tuple()
405 self.assertEqual("bar", location)
406 self.check_recipe_branch(nested_branch, "foo", "http://bar.org")
407 child_branch, location = base_branch.child_branches[1].as_tuple()
408 self.assertEqual(None, location)
409 self.check_recipe_branch(child_branch, "baz", "baz.org")
410
411 def test_accepts_comment_line_with_any_number_of_spaces(self):
412 base_branch = self.get_recipe(self.basic_header_and_branch
413 + "nest foo http://bar.org bar\n #foo\nmerge baz baz.org\n")
414 self.check_base_recipe_branch(base_branch, self.basic_branch,
415 num_child_branches=2)
416 nested_branch, location = base_branch.child_branches[0].as_tuple()
417 self.assertEqual("bar", location)
418 self.check_recipe_branch(nested_branch, "foo", "http://bar.org")
419 child_branch, location = base_branch.child_branches[1].as_tuple()
420 self.assertEqual(None, location)
421 self.check_recipe_branch(child_branch, "baz", "baz.org")
422
309 def test_old_format_rejects_run(self):423 def test_old_format_rejects_run(self):
310 header = ("# bzr-builder format 0.1 deb-version "424 header = ("# bzr-builder format 0.1 deb-version "
311 + self.deb_version +"\n")425 + self.deb_version +"\n")
@@ -313,6 +427,45 @@
313 , self.get_recipe, header + "http://foo.org/\n"427 , self.get_recipe, header + "http://foo.org/\n"
314 + "run touch test \n")428 + "run touch test \n")
315429
430 def test_error_on_forbidden_instructions(self):
431 exc = self.assertParseError(3, 1, "The 'run' instruction is "
432 "forbidden.", self.get_recipe, self.basic_header_and_branch
433 + "run touch test\n",
434 forbidden_instructions=[RUN_INSTRUCTION])
435 self.assertTrue(isinstance(exc, ForbiddenInstructionError))
436 self.assertEqual("run", exc.instruction_name)
437
438 def test_error_on_duplicate_path(self):
439 exc = self.assertParseError(3, 15, "The path '.' is a duplicate "
440 "of the one used on line 1.", self.get_recipe,
441 self.basic_header_and_branch + "nest nest url .\n")
442
443 def test_error_on_duplicate_path_with_another_nest(self):
444 exc = self.assertParseError(4, 16, "The path 'foo' is a duplicate "
445 "of the one used on line 3.", self.get_recipe,
446 self.basic_header_and_branch + "nest nest url foo\n"
447 + "nest nest2 url foo\n")
448
449 def test_duplicate_path_check_uses_normpath(self):
450 exc = self.assertParseError(3, 15, "The path 'foo/..' is a duplicate "
451 "of the one used on line 1.", self.get_recipe,
452 self.basic_header_and_branch + "nest nest url foo/..\n")
453
454 def test_error_absolute_path(self):
455 exc = self.assertParseError(3, 15, "Absolute paths are not allowed: "
456 "/etc/passwd", self.get_recipe, self.basic_header_and_branch
457 + "nest nest url /etc/passwd\n")
458
459 def test_error_simple_parent_dir(self):
460 exc = self.assertParseError(3, 15, "Paths outside the current "
461 "directory are not allowed: ../foo", self.get_recipe,
462 self.basic_header_and_branch + "nest nest url ../foo\n")
463
464 def test_error_complex_parent_dir(self):
465 exc = self.assertParseError(3, 15, "Paths outside the current "
466 "directory are not allowed: ./foo/../..", self.get_recipe,
467 self.basic_header_and_branch + "nest nest url ./foo/../..\n")
468
316469
317class BuildTreeTests(TestCaseWithTransport):470class BuildTreeTests(TestCaseWithTransport):
318471
@@ -352,7 +505,7 @@
352 def test_build_tree_single_branch_existing_branch(self):505 def test_build_tree_single_branch_existing_branch(self):
353 source = self.make_branch_and_tree("source")506 source = self.make_branch_and_tree("source")
354 revid = source.commit("one")507 revid = source.commit("one")
355 target = self.make_branch_and_tree("target")508 self.make_branch_and_tree("target")
356 base_branch = BaseRecipeBranch("source", "1", 0.2)509 base_branch = BaseRecipeBranch("source", "1", 0.2)
357 build_tree(base_branch, "target")510 build_tree(base_branch, "target")
358 self.failUnlessExists("target")511 self.failUnlessExists("target")
@@ -360,15 +513,17 @@
360 self.assertEqual(revid, tree.last_revision())513 self.assertEqual(revid, tree.last_revision())
361 self.assertEqual(revid, base_branch.revid)514 self.assertEqual(revid, base_branch.revid)
362515
516 def make_source_branch(self, relpath):
517 """Make a branch with one file and one commit."""
518 source1 = self.make_branch_and_tree(relpath)
519 self.build_tree([relpath + "/a"])
520 source1.add(["a"])
521 source1.commit("one")
522 return source1
523
363 def test_build_tree_nested(self):524 def test_build_tree_nested(self):
364 source1 = self.make_branch_and_tree("source1")525 source1_rev_id = self.make_source_branch("source1").last_revision()
365 self.build_tree(["source1/a"])526 source2_rev_id = self.make_source_branch("source2").last_revision()
366 source1.add(["a"])
367 source1_rev_id = source1.commit("one")
368 source2 = self.make_branch_and_tree("source2")
369 self.build_tree(["source2/a"])
370 source2.add(["a"])
371 source2_rev_id = source2.commit("one")
372 base_branch = BaseRecipeBranch("source1", "1", 0.2)527 base_branch = BaseRecipeBranch("source1", "1", 0.2)
373 nested_branch = RecipeBranch("nested", "source2")528 nested_branch = RecipeBranch("nested", "source2")
374 base_branch.nest_branch("sub", nested_branch)529 base_branch.nest_branch("sub", nested_branch)
@@ -382,10 +537,8 @@
382 self.assertEqual(source2_rev_id, nested_branch.revid)537 self.assertEqual(source2_rev_id, nested_branch.revid)
383538
384 def test_build_tree_merged(self):539 def test_build_tree_merged(self):
385 source1 = self.make_branch_and_tree("source1")540 source1 = self.make_source_branch("source1")
386 self.build_tree(["source1/a"])541 source1_rev_id = source1.last_revision()
387 source1.add(["a"])
388 source1_rev_id = source1.commit("one")
389 source2 = source1.bzrdir.sprout("source2").open_workingtree()542 source2 = source1.bzrdir.sprout("source2").open_workingtree()
390 self.build_tree_contents([("source2/a", "other change")])543 self.build_tree_contents([("source2/a", "other change")])
391 source2_rev_id = source2.commit("one")544 source2_rev_id = source2.commit("one")
@@ -403,11 +556,60 @@
403 self.assertEqual(source1_rev_id, base_branch.revid)556 self.assertEqual(source1_rev_id, base_branch.revid)
404 self.assertEqual(source2_rev_id, merged_branch.revid)557 self.assertEqual(source2_rev_id, merged_branch.revid)
405558
559 def test_build_tree_nest_part(self):
560 """A recipe can specify a merge of just part of an unrelated tree."""
561 source1 = self.make_source_branch("source1")
562 source2 = self.make_source_branch("source2")
563 source1.lock_read()
564 self.addCleanup(source1.unlock)
565 source1_rev_id = source1.last_revision()
566 # Add 'b' to source2.
567 self.build_tree_contents([
568 ("source2/b", "new file"), ("source2/not-b", "other file")])
569 source2.add(["b", "not-b"])
570 source2_rev_id = source2.commit("two")
571 base_branch = BaseRecipeBranch("source1", "1", 0.2)
572 merged_branch = RecipeBranch("merged", "source2")
573 # Merge just 'b' from source2; 'a' is untouched.
574 base_branch.nest_part_branch(merged_branch, "b")
575 build_tree(base_branch, "target")
576 file_id = source1.path2id("a")
577 self.check_file_contents("target/a", source1.get_file_text(file_id))
578 self.check_file_contents("target/b", "new file")
579 self.assertNotInWorkingTree("not-b", "target")
580 self.assertEqual(source1_rev_id, base_branch.revid)
581 self.assertEqual(source2_rev_id, merged_branch.revid)
582
583 def test_build_tree_nest_part_explicit_target(self):
584 """A recipe can specify a merge of just part of an unrelated tree into
585 a specific subdirectory of the target tree.
586 """
587 source1 = self.make_branch_and_tree("source1")
588 self.build_tree(["source1/dir/"])
589 source1.add(["dir"])
590 source1.commit("one")
591 source2 = self.make_source_branch("source2")
592 source1.lock_read()
593 self.addCleanup(source1.unlock)
594 source1_rev_id = source1.last_revision()
595 # Add 'b' to source2.
596 self.build_tree_contents([
597 ("source2/b", "new file"), ("source2/not-b", "other file")])
598 source2.add(["b", "not-b"])
599 source2_rev_id = source2.commit("two")
600 base_branch = BaseRecipeBranch("source1", "1", 0.2)
601 merged_branch = RecipeBranch("merged", "source2")
602 # Merge just 'b' from source2; 'a' is untouched.
603 base_branch.nest_part_branch(merged_branch, "b", "dir/b")
604 build_tree(base_branch, "target")
605 self.check_file_contents("target/dir/b", "new file")
606 self.assertNotInWorkingTree("dir/not-b", "target")
607 self.assertEqual(source1_rev_id, base_branch.revid)
608 self.assertEqual(source2_rev_id, merged_branch.revid)
609
406 def test_build_tree_merge_twice(self):610 def test_build_tree_merge_twice(self):
407 source1 = self.make_branch_and_tree("source1")611 source1 = self.make_source_branch("source1")
408 self.build_tree(["source1/a"])612 source1_rev_id = source1.last_revision()
409 source1.add(["a"])
410 source1_rev_id = source1.commit("one")
411 source2 = source1.bzrdir.sprout("source2").open_workingtree()613 source2 = source1.bzrdir.sprout("source2").open_workingtree()
412 self.build_tree_contents([("source2/a", "other change")])614 self.build_tree_contents([("source2/a", "other change")])
413 source2_rev_id = source2.commit("one")615 source2_rev_id = source2.commit("one")
@@ -436,10 +638,7 @@
436 self.assertEqual(source3_rev_id, merged_branch2.revid)638 self.assertEqual(source3_rev_id, merged_branch2.revid)
437639
438 def test_build_tree_merged_with_conflicts(self):640 def test_build_tree_merged_with_conflicts(self):
439 source1 = self.make_branch_and_tree("source1")641 source1 = self.make_source_branch("source1")
440 self.build_tree(["source1/a"])
441 source1.add(["a"])
442 source1_rev_id = source1.commit("one")
443 source2 = source1.bzrdir.sprout("source2").open_workingtree()642 source2 = source1.bzrdir.sprout("source2").open_workingtree()
444 self.build_tree_contents([("source2/a", "other change\n")])643 self.build_tree_contents([("source2/a", "other change\n")])
445 source2_rev_id = source2.commit("one")644 source2_rev_id = source2.commit("one")
@@ -448,7 +647,7 @@
448 base_branch = BaseRecipeBranch("source1", "1", 0.2)647 base_branch = BaseRecipeBranch("source1", "1", 0.2)
449 merged_branch = RecipeBranch("merged", "source2")648 merged_branch = RecipeBranch("merged", "source2")
450 base_branch.merge_branch(merged_branch)649 base_branch.merge_branch(merged_branch)
451 e = self.assertRaises(errors.BzrCommandError, build_tree,650 self.assertRaises(errors.BzrCommandError, build_tree,
452 base_branch, "target")651 base_branch, "target")
453 self.failUnlessExists("target")652 self.failUnlessExists("target")
454 tree = workingtree.WorkingTree.open("target")653 tree = workingtree.WorkingTree.open("target")
@@ -465,10 +664,8 @@
465 self.assertEqual(source2_rev_id, merged_branch.revid)664 self.assertEqual(source2_rev_id, merged_branch.revid)
466665
467 def test_build_tree_with_revspecs(self):666 def test_build_tree_with_revspecs(self):
468 source1 = self.make_branch_and_tree("source1")667 source1 = self.make_source_branch("source1")
469 self.build_tree(["source1/a"])668 source1_rev_id = source1.last_revision()
470 source1.add(["a"])
471 source1_rev_id = source1.commit("one")
472 source2 = source1.bzrdir.sprout("source2").open_workingtree()669 source2 = source1.bzrdir.sprout("source2").open_workingtree()
473 self.build_tree_contents([("source2/a", "other change\n")])670 self.build_tree_contents([("source2/a", "other change\n")])
474 source2_rev_id = source2.commit("one")671 source2_rev_id = source2.commit("one")
@@ -622,6 +819,20 @@
622 self.assertEqual(revid, tree.last_revision())819 self.assertEqual(revid, tree.last_revision())
623 self.assertEqual(revid, base_branch.revid)820 self.assertEqual(revid, base_branch.revid)
624821
822 def test_error_on_merge_revspec(self):
823 # See bug 416950
824 source = self.make_branch_and_tree("source")
825 revid = source.commit("one")
826 base_branch = BaseRecipeBranch("source", "1", 0.2)
827 merged_branch = RecipeBranch("merged", "source", revspec="debian")
828 base_branch.merge_branch(merged_branch)
829 e = self.assertRaises(errors.InvalidRevisionSpec,
830 build_tree, base_branch, "target")
831 self.assertTrue(str(e).startswith("Requested revision: 'debian' "
832 "does not exist in branch: "))
833 self.assertTrue(str(e).endswith(". Did you not mean to specify a "
834 "revspec at the end of the merge line?"))
835
625836
626class ResolveRevisionsTests(TestCaseWithTransport):837class ResolveRevisionsTests(TestCaseWithTransport):
627838
@@ -704,7 +915,7 @@
704915
705 def test_changed_command(self):916 def test_changed_command(self):
706 source =self.make_branch_and_tree("source")917 source =self.make_branch_and_tree("source")
707 revid = source.commit("one")918 source.commit("one")
708 branch1 = BaseRecipeBranch("source", "{revno}", 0.2)919 branch1 = BaseRecipeBranch("source", "{revno}", 0.2)
709 branch2 = BaseRecipeBranch("source", "{revno}", 0.2)920 branch2 = BaseRecipeBranch("source", "{revno}", 0.2)
710 branch1.run_command("touch test1")921 branch1.run_command("touch test1")
@@ -713,6 +924,17 @@
713 if_changed_from=branch2))924 if_changed_from=branch2))
714 self.assertEqual("source", branch1.url)925 self.assertEqual("source", branch1.url)
715926
927 def test_unchanged_command(self):
928 source =self.make_branch_and_tree("source")
929 source.commit("one")
930 branch1 = BaseRecipeBranch("source", "{revno}", 0.2)
931 branch2 = BaseRecipeBranch("source", "{revno}", 0.2)
932 branch1.run_command("touch test1")
933 branch2.run_command("touch test1")
934 self.assertEqual(False, resolve_revisions(branch1,
935 if_changed_from=branch2))
936 self.assertEqual("source", branch1.url)
937
716 def test_substitute(self):938 def test_substitute(self):
717 source =self.make_branch_and_tree("source")939 source =self.make_branch_and_tree("source")
718 revid1 = source.commit("one")940 revid1 = source.commit("one")
@@ -737,13 +959,20 @@
737 resolve_revisions(branch1)959 resolve_revisions(branch1)
738 self.assertEqual("{debupstream}-2", branch1.deb_version)960 self.assertEqual("{debupstream}-2", branch1.deb_version)
739961
740962 def test_subsitute_not_fully_expanded(self):
741class BuildManifestTests(TestCaseInTempDir):963 source =self.make_branch_and_tree("source")
964 source.commit("one")
965 source.commit("two")
966 branch1 = BaseRecipeBranch("source", "{revno:packaging}", 0.2)
967 self.assertRaises(errors.BzrCommandError, resolve_revisions, branch1)
968
969
970class StringifyTests(TestCaseInTempDir):
742971
743 def test_simple_manifest(self):972 def test_simple_manifest(self):
744 base_branch = BaseRecipeBranch("base_url", "1", 0.1)973 base_branch = BaseRecipeBranch("base_url", "1", 0.1)
745 base_branch.revid = "base_revid"974 base_branch.revid = "base_revid"
746 manifest = build_manifest(base_branch)975 manifest = str(base_branch)
747 self.assertEqual("# bzr-builder format 0.1 deb-version 1\n"976 self.assertEqual("# bzr-builder format 0.1 deb-version 1\n"
748 "base_url revid:base_revid\n", manifest)977 "base_url revid:base_revid\n", manifest)
749978
@@ -759,7 +988,7 @@
759 merged_branch = RecipeBranch("merged", "merged_url")988 merged_branch = RecipeBranch("merged", "merged_url")
760 merged_branch.revid = "merged_revid"989 merged_branch.revid = "merged_revid"
761 base_branch.merge_branch(merged_branch)990 base_branch.merge_branch(merged_branch)
762 manifest = build_manifest(base_branch)991 manifest = str(base_branch)
763 self.assertEqual("# bzr-builder format 0.2 deb-version 2\n"992 self.assertEqual("# bzr-builder format 0.2 deb-version 2\n"
764 "base_url revid:base_revid\n"993 "base_url revid:base_revid\n"
765 "nest nested1 nested1_url nested revid:nested1_revid\n"994 "nest nested1 nested1_url nested revid:nested1_revid\n"
@@ -770,11 +999,40 @@
770 base_branch = BaseRecipeBranch("base_url", "1", 0.2)999 base_branch = BaseRecipeBranch("base_url", "1", 0.2)
771 base_branch.revid = "base_revid"1000 base_branch.revid = "base_revid"
772 base_branch.run_command("touch test")1001 base_branch.run_command("touch test")
773 manifest = build_manifest(base_branch)1002 manifest = str(base_branch)
774 self.assertEqual("# bzr-builder format 0.2 deb-version 1\n"1003 self.assertEqual("# bzr-builder format 0.2 deb-version 1\n"
775 "base_url revid:base_revid\n"1004 "base_url revid:base_revid\n"
776 "run touch test\n", manifest)1005 "run touch test\n", manifest)
7771006
1007 def test_recipe_with_no_revspec(self):
1008 base_branch = BaseRecipeBranch("base_url", "1", 0.1)
1009 manifest = str(base_branch)
1010 self.assertEqual("# bzr-builder format 0.1 deb-version 1\n"
1011 "base_url\n", manifest)
1012
1013 def test_recipe_with_tag_revspec(self):
1014 base_branch = BaseRecipeBranch("base_url", "1", 0.1,
1015 revspec="tag:foo")
1016 manifest = str(base_branch)
1017 self.assertEqual("# bzr-builder format 0.1 deb-version 1\n"
1018 "base_url tag:foo\n", manifest)
1019
1020 def test_recipe_with_child(self):
1021 base_branch = BaseRecipeBranch("base_url", "2", 0.2)
1022 nested_branch1 = RecipeBranch("nested1", "nested1_url",
1023 revspec="tag:foo")
1024 base_branch.nest_branch("nested", nested_branch1)
1025 nested_branch2 = RecipeBranch("nested2", "nested2_url")
1026 nested_branch1.nest_branch("nested2", nested_branch2)
1027 merged_branch = RecipeBranch("merged", "merged_url")
1028 base_branch.merge_branch(merged_branch)
1029 manifest = str(base_branch)
1030 self.assertEqual("# bzr-builder format 0.2 deb-version 2\n"
1031 "base_url\n"
1032 "nest nested1 nested1_url nested tag:foo\n"
1033 " nest nested2 nested2_url nested2\n"
1034 "merge merged merged_url\n", manifest)
1035
7781036
779class RecipeBranchTests(TestCaseInTempDir):1037class RecipeBranchTests(TestCaseInTempDir):
7801038
@@ -817,6 +1075,14 @@
817 base_branch.substitute_time(time)1075 base_branch.substitute_time(time)
818 self.assertEqual("1-197001010000", base_branch.deb_version)1076 self.assertEqual("1-197001010000", base_branch.deb_version)
8191077
1078 def test_substitute_date(self):
1079 time = datetime.datetime.utcfromtimestamp(1)
1080 base_branch = BaseRecipeBranch("base_url", "1-{date}", 0.2)
1081 base_branch.substitute_time(time)
1082 self.assertEqual("1-19700101", base_branch.deb_version)
1083 base_branch.substitute_time(time)
1084 self.assertEqual("1-19700101", base_branch.deb_version)
1085
820 def test_substitute_revno(self):1086 def test_substitute_revno(self):
821 base_branch = BaseRecipeBranch("base_url", "1", 0.2)1087 base_branch = BaseRecipeBranch("base_url", "1", 0.2)
822 base_branch.substitute_revno(None, None)1088 base_branch.substitute_revno(None, None)
@@ -838,3 +1104,17 @@
838 self.assertEqual("3", base_branch.deb_version)1104 self.assertEqual("3", base_branch.deb_version)
839 base_branch.substitute_revno("foo", lambda: "3")1105 base_branch.substitute_revno("foo", lambda: "3")
840 self.assertEqual("3", base_branch.deb_version)1106 self.assertEqual("3", base_branch.deb_version)
1107
1108 def test_list_branch_names(self):
1109 base_branch = BaseRecipeBranch("base_url", "1", 0.2)
1110 base_branch.merge_branch(RecipeBranch("merged", "merged_url"))
1111 nested_branch = RecipeBranch("nested", "nested_url")
1112 nested_branch.merge_branch(
1113 RecipeBranch("merged_into_nested", "another_url"))
1114 base_branch.nest_branch("subdir", nested_branch)
1115 base_branch.merge_branch(
1116 RecipeBranch("another_nested", "yet_another_url"))
1117 base_branch.run_command("a command")
1118 self.assertEqual(
1119 ["merged", "nested", "merged_into_nested", "another_nested"],
1120 base_branch.list_branch_names())

Subscribers

People subscribed via source and target branches