Launchpad itself

Merge lp:~abentley/launchpad/build-recipe into lp:launchpad/db-devel

  1. build-recipe
  2. Merge into db-devel
Proposed by Aaron Bentley
Status: Merged
Approved by: Muharem Hrnjadovic
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~abentley/launchpad/build-recipe
Merge into: lp:launchpad/db-devel
Prerequisite: lp:~wgrant/launchpad/lp-buildd-generalisation
Diff against target: 645 lines (+400/-30)
13 files modified
daemons/buildd-slave.tac (+5/-2)
lib/canonical/buildd/binarypackage.py (+7/-6)
lib/canonical/buildd/buildrecipe (+131/-0)
lib/canonical/buildd/check-implicit-pointer-functions (+1/-0)
lib/canonical/buildd/debian.py (+30/-16)
lib/canonical/buildd/debian/changelog (+12/-0)
lib/canonical/buildd/debian/rules (+3/-3)
lib/canonical/buildd/debian/upgrade-config (+15/-1)
lib/canonical/buildd/slave.py (+10/-1)
lib/canonical/buildd/sourcepackagerecipe.py (+152/-0)
lib/canonical/buildd/template-buildd-slave.conf (+3/-0)
lib/canonical/buildd/test_buildd_recipe (+30/-0)
lib/canonical/buildd/tests/harness.py (+1/-1)
To merge this branch: bzr merge lp:~abentley/launchpad/build-recipe
Reviewer Review Type Date Requested Status
Muharem Hrnjadovic (community) code Approve
Review via email: mp+17346@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Aaron Bentley (abentley) wrote :

= Summary =
Implement building from recipe in launchpad-buildd

== Pre-implementation notes ==

== Implementation details ==
This provides a script, buildrecipe, that actually performs the build, and a SourcePackageRecipeBuildManager that integrates it into the existing build infrastructure.

== Tests ==
None. (Not really testable, as it involves buildds and chroots.)

== Demo and Q/A ==
This must be tested by hand, as it involves buildds and chroots.

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/canonical/buildd/binarypackage.py
  lib/canonical/buildd/debian.py

== Pylint notices ==

lib/canonical/buildd/binarypackage.py
    20: [C0301] Line too long (115/78)
    27: [C0301] Line too long (95/78)
    28: [C0301] Line too long (96/78)
    29: [C0301] Line too long (100/78)
    30: [C0301] Line too long (102/78)
    45: [C0301] Line too long (82/78)

Revision history for this message
Muharem Hrnjadovic (al-maisan) wrote :

Looks good!

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'daemons/buildd-slave.tac'
2--- daemons/buildd-slave.tac 2010-02-02 20:22:28 +0000
3+++ daemons/buildd-slave.tac 2010-02-02 20:22:31 +0000
4@@ -1,4 +1,4 @@
5-# Copyright 2009 Canonical Ltd. This software is licensed under the
6+# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
7 # GNU Affero General Public License version 3 (see the file LICENSE).
8
9 # Author: Daniel Silverstone <daniel.silverstone@canonical.com>
10@@ -10,6 +10,8 @@
11 from twisted.application import service, strports
12 from canonical.buildd import XMLRPCBuildDSlave
13 from canonical.buildd.binarypackage import BinaryPackageBuildManager
14+from canonical.buildd.sourcepackagerecipe import (
15+ SourcePackageRecipeBuildManager)
16 from canonical.launchpad.daemons import tachandler
17
18 from twisted.web import server, resource, static
19@@ -26,6 +28,7 @@
20 # 'debian' is the old name. It remains here for compatibility.
21 slave.registerBuilder(BinaryPackageBuildManager, "debian")
22 slave.registerBuilder(BinaryPackageBuildManager, "binarypackage")
23+slave.registerBuilder(SourcePackageRecipeBuildManager, "sourcepackagerecipe")
24
25 application = service.Application('BuildDSlave')
26 builddslaveService = service.IServiceCollection(application)
27@@ -46,5 +49,5 @@
28 #
29 # python
30 # import xmlrpclib
31-# s = xmlrpclib.Server("http://localhost:8221/")
32+# s = xmlrpclib.ServerProxy("http://localhost:8221/rpc")
33 # s.echo("Hello World")
34
35=== modified file 'lib/canonical/buildd/binarypackage.py'
36--- lib/canonical/buildd/binarypackage.py 2010-02-02 20:22:28 +0000
37+++ lib/canonical/buildd/binarypackage.py 2010-02-02 20:22:31 +0000
38@@ -1,6 +1,9 @@
39-# Copyright 2009 Canonical Ltd. This software is licensed under the
40+# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
41 # GNU Affero General Public License version 3 (see the file LICENSE).
42
43+
44+import re
45+
46 from canonical.buildd.debian import DebianBuildManager, DebianBuildState
47
48
49@@ -95,20 +98,18 @@
50
51 def iterate_SBUILD(self, success):
52 """Finished the sbuild run."""
53+ tmpLog = self.getTmpLogContents()
54 if success != SBuildExitCodes.OK:
55- tmpLogHandle = open(os.path.join(self._cachepath, "buildlog"))
56- tmpLog = tmpLogHandle.read()
57- tmpLogHandle.close()
58 if (success == SBuildExitCodes.DEPFAIL or
59 success == SBuildExitCodes.PACKAGEFAIL):
60 for rx in BuildLogRegexes.GIVENBACK:
61- mo=re.search(rx, tmpLog, re.M)
62+ mo = re.search(rx, tmpLog, re.M)
63 if mo:
64 success = SBuildExitCodes.GIVENBACK
65
66 if success == SBuildExitCodes.DEPFAIL:
67 for rx, dep in BuildLogRegexes.DEPFAIL:
68- mo=re.search(rx, tmpLog, re.M)
69+ mo = re.search(rx, tmpLog, re.M)
70 if mo:
71 if not self.alreadyfailed:
72 print("Returning build status: DEPFAIL")
73
74=== added file 'lib/canonical/buildd/buildrecipe'
75--- lib/canonical/buildd/buildrecipe 1970-01-01 00:00:00 +0000
76+++ lib/canonical/buildd/buildrecipe 2010-02-02 20:22:31 +0000
77@@ -0,0 +1,131 @@
78+#!/usr/bin/env python
79+# Copyright 2010 Canonical Ltd. This software is licensed under the
80+# GNU Affero General Public License version 3 (see the file LICENSE).
81+
82+"""A script that builds a package from a recipe and a chroot."""
83+
84+__metaclass__ = type
85+
86+
87+import os.path
88+import pwd
89+from subprocess import call
90+import sys
91+
92+
93+RETCODE_SUCCESS = 0
94+RETCODE_FAILURE_INSTALL = 200
95+RETCODE_FAILURE_BUILD_TREE = 201
96+RETCODE_FAILURE_INSTALL_BUILD_DEPS = 202
97+RETCODE_FAILURE_BUILD_SOURCE_PACKAGE = 203
98+
99+
100+class RecipeBuilder:
101+ """Builds a package from a recipe."""
102+
103+ def __init__(self, build_id, author_name, author_email,
104+ package_name, suite):
105+ """Constructor.
106+
107+ :param build_id: The id of the build (a str).
108+ :param author_name: The name of the author (a str).
109+ :param author_email: The email address of the author (a str).
110+ :param package_name: The name of the package (a str).
111+ :param suite: The suite the package should be built for (a str).
112+ """
113+ self.build_id = build_id
114+ self.author_name = author_name
115+ self.author_email = author_email
116+ self.package_name = package_name
117+ self.suite = suite
118+ self.base_branch = None
119+ self.chroot_path = get_build_path(build_id, 'chroot-autobuild')
120+ self.work_dir_relative = os.environ['HOME'] + '/work'
121+ self.work_dir = os.path.join(self.chroot_path,
122+ self.work_dir_relative[1:])
123+ self.tree_path = os.path.join(self.work_dir, 'tree')
124+ self.username = pwd.getpwuid(os.getuid())[0]
125+
126+ def install(self):
127+ """Install all the requirements for building recipes.
128+
129+ :return: A retcode from apt.
130+ """
131+ return self.chroot(['apt-get', 'install', '-y', 'pbuilder',
132+ 'bzr-builder', 'sudo'])
133+
134+ def buildTree(self):
135+ """Build the recipe into a source tree.
136+
137+ As a side-effect, sets self.source_dir_relative.
138+ :return: a retcode from `bzr dailydeb`.
139+ """
140+ assert not os.path.exists(self.tree_path)
141+ recipe_path_relative = os.path.join(self.work_dir_relative, 'recipe')
142+ self.tree_path_relative = os.path.join(self.work_dir_relative, 'tree')
143+ manifest_path_relative = os.path.join(
144+ self.tree_path_relative, 'manifest')
145+ retcode = self.chroot([
146+ 'sudo', '-u', self.username, 'DEBEMAIL=%s' % self.author_email,
147+ 'DEBFULLNAME=%s' % self.author_name, 'bzr', 'dailydeb',
148+ '--no-build', recipe_path_relative, self.tree_path_relative,
149+ '--manifest', manifest_path_relative])
150+ if retcode != 0:
151+ return retcode
152+ (source,) = [name for name in os.listdir(self.tree_path)
153+ if name != 'manifest']
154+ self.source_dir_relative = os.path.join(self.tree_path_relative,
155+ source)
156+ return retcode
157+
158+ def installBuildDeps(self):
159+ """Install the build-depends of the source tree."""
160+ return self.chroot(['sh', '-c', 'cd %s &&'
161+ '/usr/lib/pbuilder/pbuilder-satisfydepends'
162+ % self.source_dir_relative])
163+
164+ def chroot(self, args):
165+ """Run a command in the chroot.
166+
167+ :param args: the command and arguments to run.
168+ :return: the status code.
169+ """
170+ return call([
171+ '/usr/bin/sudo', '/usr/sbin/chroot', self.chroot_path] + args)
172+
173+ def buildSourcePackage(self):
174+ """Build the source package.
175+
176+ :return: a retcode from dpkg-buildpackage.
177+ """
178+ retcode = self.chroot([
179+ 'su', '-c', 'cd %s && /usr/bin/dpkg-buildpackage -i -I -us -uc -S'
180+ % self.source_dir_relative, self.username])
181+ for filename in os.listdir(self.tree_path):
182+ path = os.path.join(self.tree_path, filename)
183+ if os.path.isfile(path):
184+ os.rename(path, get_build_path(self.build_id, filename))
185+ return retcode
186+
187+
188+def get_build_path(build_id, *extra):
189+ """Generate a path within the build directory.
190+
191+ :param build_id: the build id to use.
192+ :param extra: the extra path segments within the build directory.
193+ :return: the generated path.
194+ """
195+ return os.path.join(
196+ os.environ["HOME"], "build-" + build_id, *extra)
197+
198+if __name__ == '__main__':
199+ builder = RecipeBuilder(*sys.argv[1:])
200+ if builder.install() != 0:
201+ sys.exit(RETCODE_FAILURE_INSTALL)
202+ if builder.buildTree() != 0:
203+ sys.exit(RETCODE_FAILURE_BUILD_TREE)
204+ if builder.installBuildDeps() != 0:
205+ sys.exit(RETCODE_FAILURE_INSTALL_BUILD_DEPS)
206+ if builder.buildSourcePackage() != 0:
207+ sys.exit(RETCODE_FAILURE_BUILD_SOURCE_PACKAGE)
208+ sys.exit(RETCODE_SUCCESS)
209
210=== modified file 'lib/canonical/buildd/check-implicit-pointer-functions'
211--- lib/canonical/buildd/check-implicit-pointer-functions 2010-02-02 20:22:28 +0000
212+++ lib/canonical/buildd/check-implicit-pointer-functions 2010-02-02 20:22:31 +0000
213@@ -3,6 +3,7 @@
214 #
215 # Copyright (c) 2004 Hewlett-Packard Development Company, L.P.
216 # David Mosberger <davidm@hpl.hp.com>
217+# Copyright 2010 Canonical Ltd.
218 #
219 # Permission is hereby granted, free of charge, to any person
220 # obtaining a copy of this software and associated documentation
221
222=== modified file 'lib/canonical/buildd/debian.py'
223--- lib/canonical/buildd/debian.py 2010-02-02 20:22:28 +0000
224+++ lib/canonical/buildd/debian.py 2010-02-02 20:22:31 +0000
225@@ -1,4 +1,4 @@
226-# Copyright 2009 Canonical Ltd. This software is licensed under the
227+# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
228 # GNU Affero General Public License version 3 (see the file LICENSE).
229
230 # Authors: Daniel Silverstone <daniel.silverstone@canonical.com>
231@@ -9,10 +9,9 @@
232 __metaclass__ = type
233
234 import os
235-import re
236
237 from canonical.buildd.slave import (
238- BuildManager, RunCapture
239+ BuildManager,
240 )
241
242
243@@ -33,7 +32,7 @@
244 """Base behaviour for Debian chrooted builds."""
245
246 def __init__(self, slave, buildid):
247- BuildManager.__init__(self,slave,buildid)
248+ BuildManager.__init__(self, slave, buildid)
249 self._updatepath = slave._config.get("debianmanager", "updatepath")
250 self._scanpath = slave._config.get("debianmanager", "processscanpath")
251 self._ogrepath = slave._config.get("debianmanager", "ogrepath")
252@@ -114,30 +113,27 @@
253 filename = line.split(' ')[-1]
254 yield filename
255
256+ def getChangesFilename(self):
257+ changes = (
258+ self._dscfile[:-4] + "_" + self._slave.getArch() + ".changes")
259+ return get_build_path(self._buildid, changes)
260+
261 def gatherResults(self):
262 """Gather the results of the build and add them to the file cache.
263
264 The primary file we care about is the .changes file. We key from there.
265 """
266- changes = self._dscfile[:-4] + "_" + self._slave.getArch() + ".changes"
267- # XXX: dsilvers 2005-03-17: This join thing needs to be split out
268- # into a method and unit tested.
269- path = os.path.join(os.environ["HOME"], "build-"+self._buildid,
270- changes)
271+ path = self.getChangesFilename()
272+ name = os.path.basename(path)
273 chfile = open(path, "r")
274- filemap = {}
275- filemap[changes] = self._slave.storeFile(chfile.read())
276+ self._slave.waitingfiles[name] = self._slave.storeFile(chfile.read())
277 chfile.seek(0)
278 seenfiles = False
279
280 for fn in self._parseChangesFile(chfile):
281- path = os.path.join(os.environ["HOME"], "build-"+self._buildid, fn)
282- f = open(path, "r")
283- filemap[fn] = self._slave.storeFile(f.read())
284- f.close()
285+ self._slave.addWaitingFile(get_build_path(self._buildid, fn))
286
287 chfile.close()
288- self._slave.waitingfiles = filemap
289
290 def iterate(self, success):
291 # When a Twisted ProcessControl class is killed by SIGTERM,
292@@ -212,6 +208,13 @@
293 self._state = DebianBuildState.UPDATE
294 self.doUpdateChroot()
295
296+ def getTmpLogContents(self):
297+ try:
298+ tmpLogHandle = open(os.path.join(self._cachepath, "buildlog"))
299+ return tmpLogHandle.read()
300+ finally:
301+ tmpLogHandle.close()
302+
303 def iterate_SOURCES(self, success):
304 """Just finished overwriting sources.list."""
305 if success != 0:
306@@ -261,3 +264,14 @@
307 if not self.alreadyfailed:
308 self._slave.buildOK()
309 self._slave.buildComplete()
310+
311+
312+def get_build_path(build_id, *extra):
313+ """Generate a path within the build directory.
314+
315+ :param build_id: the build id to use.
316+ :param extra: the extra path segments within the build directory.
317+ :return: the generated path.
318+ """
319+ return os.path.join(
320+ os.environ["HOME"], "build-" + build_id, *extra)
321
322=== modified file 'lib/canonical/buildd/debian/changelog'
323--- lib/canonical/buildd/debian/changelog 2010-02-02 20:22:28 +0000
324+++ lib/canonical/buildd/debian/changelog 2010-02-02 20:22:31 +0000
325@@ -1,3 +1,15 @@
326+launchpad-buildd (58~1) karmic; urgency=low
327+
328+ * Misc fixes to match APIs.
329+
330+ -- Aaron Bentley <aaron@aaronbentley.com> Fri, 15 Jan 2010 10:03:07 +1300
331+
332+launchpad-buildd (58~0) karmic; urgency=low
333+
334+ * Include buildrecipe.py.
335+
336+ -- Aaron Bentley <aaron@aaronbentley.com> Wed, 13 Jan 2010 17:06:59 +1300
337+
338 launchpad-buildd (57) hardy-cat; urgency=low
339
340 * Split the sbuild wrapper from DebianBuildManager into a new
341
342=== modified file 'lib/canonical/buildd/debian/rules'
343--- lib/canonical/buildd/debian/rules 2010-02-02 20:22:28 +0000
344+++ lib/canonical/buildd/debian/rules 2010-02-02 20:22:31 +0000
345@@ -1,6 +1,6 @@
346 #!/usr/bin/make -f
347 #
348-# Copyright 2009 Canonical Ltd. This software is licensed under the
349+# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
350 # GNU Affero General Public License version 3 (see the file LICENSE).
351
352 export DH_COMPAT=4
353@@ -21,10 +21,10 @@
354 targetshare = $(target)/usr/share/launchpad-buildd
355 pytarget = $(targetshare)/canonical/buildd
356
357-pyfiles = debian.py slave.py binarypackage.py utils.py __init__.py
358+pyfiles = debian.py slave.py binarypackage.py utils.py __init__.py sourcepackagerecipe.py
359 slavebins = unpack-chroot mount-chroot update-debian-chroot sbuild-package \
360 scan-for-processes umount-chroot remove-build apply-ogre-model \
361-override-sources-list
362+override-sources-list buildrecipe
363
364 BUILDDUID=65500
365 BUILDDGID=65500
366
367=== modified file 'lib/canonical/buildd/debian/upgrade-config'
368--- lib/canonical/buildd/debian/upgrade-config 2010-02-02 20:22:28 +0000
369+++ lib/canonical/buildd/debian/upgrade-config 2010-02-02 20:22:31 +0000
370@@ -1,6 +1,6 @@
371 #!/usr/bin/python
372 #
373-# Copyright 2009 Canonical Ltd. This software is licensed under the
374+# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
375 # GNU Affero General Public License version 3 (see the file LICENSE).
376
377 """Upgrade a launchpad-buildd configuration file."""
378@@ -70,6 +70,18 @@
379 in_file.close()
380 out_file.close()
381
382+def upgrade_to_58():
383+ print "Upgrading %s to version 58" % conf_file
384+ subprocess.call(["mv", conf_file, conf_file+"-prev58~"])
385+ in_file = open(conf_file+"-prev58~", "r")
386+ out_file = open(conf_file, "w")
387+ out_file.write(in_file.read())
388+ out_file.write(
389+ '\n[sourcepackagerecipemanager]\n'
390+ 'buildrecipepath = /usr/share/launchpad-buildd'
391+ '/slavebin/buildrecipe\n')
392+
393+
394 if __name__ == "__main__":
395 if old_version.find("~") > 0:
396 old_version = old_version[:old_version.find("~")]
397@@ -81,4 +93,6 @@
398 upgrade_to_39()
399 if int(old_version) < 57:
400 upgrade_to_57()
401+ if int(old_version) < 58:
402+ upgrade_to_58()
403
404
405=== modified file 'lib/canonical/buildd/slave.py'
406--- lib/canonical/buildd/slave.py 2009-06-25 05:30:52 +0000
407+++ lib/canonical/buildd/slave.py 2010-02-02 20:22:31 +0000
408@@ -1,4 +1,4 @@
409-# Copyright 2009 Canonical Ltd. This software is licensed under the
410+# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
411 # GNU Affero General Public License version 3 (see the file LICENSE).
412
413 # Authors: Daniel Silverstone <daniel.silverstone@canonical.com>
414@@ -325,6 +325,15 @@
415 f.close()
416 return sha1sum
417
418+ def addWaitingFile(self, path):
419+ """Add a file to the cache and store its details for reporting."""
420+ fn = os.path.basename(path)
421+ f = open(path)
422+ try:
423+ self.waitingfiles[fn] = self.storeFile(f.read())
424+ finally:
425+ f.close()
426+
427 def fetchFile(self, sha1sum):
428 """Fetch the file of the given sha1sum."""
429 present, info = self.ensurePresent(sha1sum)
430
431=== added file 'lib/canonical/buildd/sourcepackagerecipe.py'
432--- lib/canonical/buildd/sourcepackagerecipe.py 1970-01-01 00:00:00 +0000
433+++ lib/canonical/buildd/sourcepackagerecipe.py 2010-02-02 20:22:31 +0000
434@@ -0,0 +1,152 @@
435+# Copyright 2010 Canonical Ltd. This software is licensed under the
436+# GNU Affero General Public License version 3 (see the file LICENSE).
437+
438+"""The manager class for building packages from recipes."""
439+
440+import os
441+import re
442+
443+from canonical.buildd.debian import (
444+ DebianBuildManager,
445+ DebianBuildState,
446+ get_build_path,
447+)
448+RETCODE_SUCCESS = 0
449+RETCODE_FAILURE_INSTALL = 200
450+RETCODE_FAILURE_BUILD_TREE = 201
451+RETCODE_FAILURE_INSTALL_BUILD_DEPS = 202
452+RETCODE_FAILURE_BUILD_SOURCE_PACKAGE = 203
453+
454+
455+def splat_file(path, contents):
456+ """Write a string to the specified path.
457+
458+ :param path: The path to store the string in.
459+ :param contents: The string to write to the file.
460+ """
461+ file_obj = open(path, 'w')
462+ try:
463+ file_obj.write(contents)
464+ finally:
465+ file_obj.close()
466+
467+
468+def get_chroot_path(build_id, *extra):
469+ """Return a path within the chroot.
470+
471+ :param build_id: The build_id of the build.
472+ :param extra: Additional path elements.
473+ """
474+ return get_build_path(
475+ build_id, 'chroot-autobuild', os.environ['HOME'][1:], *extra)
476+
477+
478+class SourcePackageRecipeBuildState(DebianBuildState):
479+ """The set of states that a recipe build can be in."""
480+ BUILD_RECIPE = "BUILD_RECIPE"
481+
482+
483+class SourcePackageRecipeBuildManager(DebianBuildManager):
484+ """Build a source package from a bzr-builder recipe."""
485+
486+ initial_build_state = SourcePackageRecipeBuildState.BUILD_RECIPE
487+
488+ def __init__(self, slave, buildid):
489+ """Constructor.
490+
491+ :param slave: A build slave device.
492+ :param buildid: The id of the build (a str).
493+ """
494+ DebianBuildManager.__init__(self, slave, buildid)
495+ self.build_recipe_path = slave._config.get(
496+ "sourcepackagerecipemanager", "buildrecipepath")
497+
498+ def initiate(self, files, chroot, extra_args):
499+ """Initiate a build with a given set of files and chroot.
500+
501+ :param files: The files sent by the manager with the request.
502+ :param chroot: The sha1sum of the chroot to use.
503+ :param extra_args: A dict of extra arguments.
504+ """
505+ self.recipe_text = extra_args['recipe_text']
506+ self.suite = extra_args['suite']
507+ self.component = extra_args['ogrecomponent']
508+ self.package_name = extra_args['package_name']
509+ self.author_name = extra_args['author_name']
510+ self.author_email = extra_args['author_email']
511+ self.archive_purpose = extra_args['archive_purpose']
512+
513+ super(SourcePackageRecipeBuildManager, self).initiate(
514+ files, chroot, extra_args)
515+
516+ def doRunBuild(self):
517+ """Run the build process to build the source package."""
518+ currently_building = get_build_path(
519+ self._buildid, 'chroot-autobuild/CurrentlyBuilding')
520+ splat_file(currently_building,
521+ 'Package: %s\n'
522+ 'Suite: %s\n'
523+ 'Component: %s\n'
524+ 'Purpose: %s\n'
525+ 'Build-Debug-Symbols: no\n' %
526+ (self.package_name, self.suite, self.component,
527+ self.archive_purpose))
528+ os.makedirs(get_chroot_path(self._buildid, 'work'))
529+ recipe_path = get_chroot_path(self._buildid, 'work/recipe')
530+ splat_file(recipe_path, self.recipe_text)
531+ args = [
532+ "buildrecipe.py", self._buildid, self.author_name,
533+ self.author_email, self.package_name, self.suite]
534+ self.runSubProcess(self.build_recipe_path, args)
535+
536+ def iterate_BUILD_RECIPE(self, retcode):
537+ """Move from BUILD_RECIPE to the next logical state."""
538+ if retcode == RETCODE_SUCCESS:
539+ self.gatherResults()
540+ print("Returning build status: OK")
541+ elif retcode == RETCODE_FAILURE_INSTALL_BUILD_DEPS:
542+ if not self.alreadyfailed:
543+ tmpLog = self.getTmpLogContents()
544+ rx = (
545+ 'The following packages have unmet dependencies:\n'
546+ '.*: Depends: ([^ ]*( \([^)]*\))?)')
547+ mo = re.search(rx, tmpLog, re.M)
548+ if mo:
549+ self._slave.depFail(mo.group(1))
550+ print("Returning build status: DEPFAIL")
551+ print("Dependencies: " + mo.group(1))
552+ else:
553+ print("Returning build status: Build failed")
554+ self._slave.buildFail()
555+ self.alreadyfailed = True
556+ elif (
557+ retcode >= RETCODE_FAILURE_INSTALL and
558+ retcode <= RETCODE_FAILURE_BUILD_SOURCE_PACKAGE):
559+ # XXX AaronBentley 2009-01-13: We should handle depwait separately
560+ if not self.alreadyfailed:
561+ self._slave.buildFail()
562+ print("Returning build status: Build failed.")
563+ self.alreadyfailed = True
564+ else:
565+ if not self.alreadyfailed:
566+ self._slave.builderFail()
567+ print("Returning build status: Builder failed.")
568+ self.alreadyfailed = True
569+ self._state = DebianBuildState.REAP
570+ self.doReapProcesses()
571+
572+ def getChangesFilename(self):
573+ """Return the path to the changes file."""
574+ work_path = get_build_path(self._buildid)
575+ for name in os.listdir(work_path):
576+ if name.endswith('_source.changes'):
577+ return os.path.join(work_path, name)
578+
579+ def gatherResults(self):
580+ """Gather the results of the build and add them to the file cache.
581+
582+ The primary file we care about is the .changes file.
583+ The manifest is also a useful record.
584+ """
585+ DebianBuildManager.gatherResults(self)
586+ self._slave.addWaitingFile(get_build_path(self._buildid, 'manifest'))
587
588=== modified file 'lib/canonical/buildd/template-buildd-slave.conf'
589--- lib/canonical/buildd/template-buildd-slave.conf 2010-02-02 20:22:28 +0000
590+++ lib/canonical/buildd/template-buildd-slave.conf 2010-02-02 20:22:31 +0000
591@@ -24,3 +24,6 @@
592 [binarypackagemanager]
593 sbuildpath = /usr/share/launchpad-buildd/slavebin/sbuild-package
594 sbuildargs = --nolog --batch --archive=ubuntu
595+
596+[sourcepackagerecipemanager]
597+buildrecipepath = /usr/share/launchpad-buildd/slavebin/buildrecipe
598
599=== added file 'lib/canonical/buildd/test_buildd_recipe'
600--- lib/canonical/buildd/test_buildd_recipe 1970-01-01 00:00:00 +0000
601+++ lib/canonical/buildd/test_buildd_recipe 2010-02-02 20:22:31 +0000
602@@ -0,0 +1,30 @@
603+#!/usr/bin/env python
604+# Copyright 2010 Canonical Ltd. This software is licensed under the
605+# GNU Affero General Public License version 3 (see the file LICENSE).
606+#
607+# This is a script to do end-to-end testing of the buildd with a bzr-builder
608+# recipe, without involving the BuilderBehaviour.
609+
610+from xmlrpclib import ServerProxy
611+proxy = ServerProxy('http://localhost:8221/rpc')
612+print proxy.echo('Hello World')
613+print proxy.info()
614+print proxy.status()
615+recipe_text = """# bzr-builder format 0.2 deb-version something
616+http://bazaar.launchpad.net/~james-w/bzr-builder/trunk/
617+merge packaging http://bazaar.launchpad.net/~james-w/bzr-builder/packaging"""
618+print proxy.build(
619+ '1-2', 'sourcepackagerecipe', 'f2afc3c0507dffba181638d70ae55ac675679460',
620+ {}, {'author_name': 'Steve',
621+ 'author_email': 'stevea@example.org',
622+ 'package_name': 'bzr-builder',
623+ 'suite': 'lucid',
624+ 'ogrecomponent': 'universe',
625+ 'archive_purpose': 'puppies',
626+ 'recipe_text': recipe_text,
627+ 'archives': [
628+ 'deb http://nz.archive.ubuntu.com/ubuntu lucid main universe']})
629+#status = proxy.status()
630+#for filename, sha1 in status[3].iteritems():
631+# print filename
632+#proxy.clean()
633
634=== modified file 'lib/canonical/buildd/tests/harness.py'
635--- lib/canonical/buildd/tests/harness.py 2009-06-25 05:30:52 +0000
636+++ lib/canonical/buildd/tests/harness.py 2010-02-02 20:22:31 +0000
637@@ -80,7 +80,7 @@
638 ['Hello World']
639
640 >>> s.info()
641- ['1.0', 'i386', ['debian']]
642+ ['1.0', 'i386', ['sourcepackagerecipe', 'binarypackage', 'debian']]
643
644 >>> s.status()
645 ['BuilderStatus.IDLE', '']

Subscribers

People subscribed via source and target branches

to status/vote changes: