Launchpad itself

Merge lp:~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild-2 into lp:launchpad/db-devel

  1. 567922-binarypackagebuild-packagebuild-2
  2. Merge into db-devel
Proposed by Michael Nelson
Status: Merged
Approved by: Michael Nelson
Approved revision: no longer in the source branch.
Merged at revision: 9405
Proposed branch: lp:~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild-2
Merge into: lp:launchpad/db-devel
Prerequisite: lp:~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild
Diff against target: 664 lines (+293/-68) (has conflicts)
11 files modified
lib/lp/buildmaster/interfaces/buildbase.py (+10/-1)
lib/lp/buildmaster/interfaces/buildfarmjob.py (+11/-1)
lib/lp/buildmaster/interfaces/packagebuild.py (+43/-0)
lib/lp/buildmaster/model/buildbase.py (+58/-28)
lib/lp/buildmaster/model/buildfarmjob.py (+4/-0)
lib/lp/buildmaster/model/packagebuild.py (+33/-0)
lib/lp/buildmaster/tests/test_buildbase.py (+61/-12)
lib/lp/buildmaster/tests/test_buildfarmjob.py (+1/-0)
lib/lp/buildmaster/tests/test_packagebuild.py (+38/-12)
lib/lp/soyuz/doc/buildd-slavescanner.txt (+2/-1)
lib/lp/soyuz/tests/test_build.py (+32/-13)
Text conflict in lib/lp/buildmaster/interfaces/buildbase.py
Text conflict in lib/lp/buildmaster/model/buildbase.py
Text conflict in lib/lp/buildmaster/tests/test_buildbase.py
To merge this branch: bzr merge lp:~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild-2
Reviewer Review Type Date Requested Status
Jeroen T. Vermeulen (community) code Approve
Brad Crittenden (community) Abstain
Jelmer Vernooij code Pending
Review via email: mp+24211@code.launchpad.net

Description of the change

This branch is part of a pipeline for

https://blueprints.edge.launchpad.net/soyuz/+spec/build-generalisation
https://dev.launchpad.net/LEP/GeneralBuildHistories

Overview
========
The changes in this branch continue the preparation to switch the BinaryPackageBuild class from the old build table to the new collection of 3 tables (buildfarmjob, packagebuild and binarypackagebuild, similar to http://people.ubuntu.com/~wgrant/launchpad/buildfarm/new-build-model-again.png).

The previous MP started this (adding the attributes with specific API versions):
https://code.edge.launchpad.net/~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild/+merge/24107

This branch mainly consists of making sure that the BuildFarmJob/PackageBuild classes provide the same interface methods as the current IBuildBase (which is *not* a database class), while at the same time,

(1) ensuring IBuildBase (and its implementation) still function correctly as it is used by other classes which will be updated in later work (ie. SourcePackageRecipeBuild), and
(2) we don't duplicate the implementation during the transition (so that there's no possibility of divergence).

To achieve these two points, I had to turn some of the BuildBase methods into staticmethods with the build as the first argument so the implementation can be used by both IBuildBase and IPackageBuild.

This branch is dependent on the pending schema patch in a previous branch.

To test
=======

First update the test db schema (required as the db patch still needs to be updated to remove the old build table):
psql launchpad_ftest_template -f database/schema/pending/michaeln-build-generalisation.sql
bin/py database/schema/security.py -d launchpad_ftest_template

And then:
bin/test -vvt test_packagebuild -t test_buildfarmjob -t doc/build.txt -t test_buildqueue -t test_sourcepackagerecipebuild -t test_buildpackagejob -t test_buildbase

(there is one intentional test failure in test_buildfarmjob for a method that can't be implemented until I switch BinaryPackageBuild to the new delegation model of IPackageBuild/IBuildFarmJob).

The next branch will continue to transfer the interface to IBuildFarmJob/IPackageBuild before finally switching the BinaryPackageBuild from the old build table to the new binarypackagebuild table (Note: IBuildBase will still be used by ISourcePackageRecipeBuild until we do a second db-schema change).

Outstanding questions
=================
Because BuildBase is currently shared by SPRecipeBuild and BinaryPackageBuild, I've basically just ensure that the equivalently shared model in the new schema - IPackageBuild - gets these attributes/methods, but they could sensibly be on IBuildFarmJob instead, so that they were available for all build farm jobs (ie. including TranslationsTemplateBuildJob)... I hope to get some input from jtv on that.

To post a comment you must log in.
Revision history for this message
Brad Crittenden (bac) wrote :

Michael I had some questions I wanted to discuss with you on IRC to help me understand what you're doing here.

Since you aren't around due to hitting EOD I am going to abstain on this branch so you can get a review first thing tomorrow, hopefully by someone who is already familiar with this effort.

review: Abstain
Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

Discussed on IRC. A fair portion of my notes were on docstrings that aren't new and have been handed down from times immemorial; might as well seize the opportunity to clear things up while we're here.

lib/lp/buildmaster/interfaces/buildfarmjob.py: makeJob

I thought this would be a problem with TranslationTemplatesBuildJob, which is based on BranchJob whose constructor likes to create its own Jobs. Turns out this has been taken into account, but the docstring has been clarified. Thanks!

lib/lp/buildmaster/interfaces/packagebuild.py: getUploaderCommand

Clarified: what does "the command to run as the uploader" mean? That all this is about on command, and it is to be run under the uploader's identity?

lib/lp/buildmaster/interfaces/packagebuild.py: getUploadLeaf

Docstring clarified.

The slave build id has been replaced with the slave build cookie. It is now a hash without (I hope) any distinguishable components.

lib/lp/buildmaster/tests/test_buildbase.py: BuildBaseTestCase

Don't derive base class for tests from TestCase. It's not a test. Make it a mixin instead, and use multiple inheritance in the leaf classes. Also saves you from deriving a test case from both TestCase *and* TestCaseWithFactory. Rename setUp so as not to confuse super().

At the time of writing, this is being resolved.

Good luck with this refactoring!

Jeroen

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/buildmaster/interfaces/buildbase.py'
2--- lib/lp/buildmaster/interfaces/buildbase.py 2010-05-05 15:48:34 +0000
3+++ lib/lp/buildmaster/interfaces/buildbase.py 2010-05-05 15:48:37 +0000
4@@ -212,11 +212,20 @@
5
6 title = exported(TextLine(title=_("Title"), required=False))
7
8+<<<<<<< TREE
9 def getUploaderCommand(upload_leaf, uploader_logfilename):
10 """Get the command to run as the uploader.
11
12 :return: A list of command line arguments, beginning with the
13 executable.
14+=======
15+ def processUpload(leaf, root, logger):
16+ """Process an upload.
17+
18+ :param leaf: Leaf for this particular upload
19+ :param root: Root directory for the uploads
20+ :param logger: A logger object
21+>>>>>>> MERGE-SOURCE
22 """
23
24 def handleStatus(status, librarian, slave_status):
25@@ -242,7 +251,7 @@
26 def estimateDuration():
27 """Estimate the build duration."""
28
29- def storeBuildInfo(librarian, slave_status):
30+ def storeBuildInfo(build, librarian, slave_status):
31 """Store available information for the build job.
32
33 Subclasses can override this as needed, and call it from custom status
34
35=== modified file 'lib/lp/buildmaster/interfaces/buildfarmjob.py'
36--- lib/lp/buildmaster/interfaces/buildfarmjob.py 2010-05-05 15:48:34 +0000
37+++ lib/lp/buildmaster/interfaces/buildfarmjob.py 2010-05-05 15:48:37 +0000
38@@ -91,7 +91,8 @@
39 date_finished = exported(
40 Datetime(
41 title=_("Date finished"), required=False, readonly=True,
42- description=_("The timestamp when the build farm job was finished.")),
43+ description=_(
44+ "The timestamp when the build farm job was finished.")),
45 ("1.0", dict(exported=True, exported_as="datebuilt")))
46
47 date_first_dispatched = exported(
48@@ -172,6 +173,15 @@
49 def jobAborted():
50 """'Job aborted' life cycle event, handle as appropriate."""
51
52+ def makeJob():
53+ """Create the specific job relating this with an lp.services.job.
54+
55+ XXX 2010-04-26 michael.nelson bug=567922
56+ Once all *Build classes are using BuildFarmJob we can lose the
57+ 'specific_job' attributes and simply have a reference to the
58+ services job directly on the BuildFarmJob.
59+ """
60+
61
62 class IBuildFarmJobDerived(Interface):
63 """Common functionality required by classes delegating IBuildFarmJob.
64
65=== modified file 'lib/lp/buildmaster/interfaces/packagebuild.py'
66--- lib/lp/buildmaster/interfaces/packagebuild.py 2010-05-05 15:48:34 +0000
67+++ lib/lp/buildmaster/interfaces/packagebuild.py 2010-05-05 15:48:37 +0000
68@@ -75,6 +75,49 @@
69 title=_("Distribution"), required=True,
70 description=_("Shortcut for its distribution.")))
71
72+ def getUploaderCommand(distro_series, upload_leaf, uploader_logfilename):
73+ """Get the command to run as the uploader.
74+
75+ :return: A list of command line arguments, beginning with the
76+ executable.
77+ """
78+
79+ def getUploadDirLeaf(build_cookie, now=None):
80+ """Return the directory-leaf where files to be uploaded are stored.
81+
82+ :param build_cookie: The build cookie as returned by the slave.
83+ :param now: The `datetime` to use when constructing the leaf
84+ directory name. If not provided, defaults to now.
85+ """
86+
87+ def getUploadDir(upload_leaf):
88+ """Return the full directory where files to be uploaded are stored.
89+
90+ :param upload_leaf: The leaf directory name where things will be
91+ stored.
92+ """
93+
94+ def getLogFromSlave():
95+ """Get last buildlog from slave. """
96+
97+ def getUploadLogContent(root, leaf):
98+ """Retrieve the upload log contents.
99+
100+ :param root: Root directory for the uploads
101+ :param leaf: Leaf for this particular upload
102+ :return: Contents of log file or message saying no log file was found.
103+ """
104+
105+ def estimateDuration():
106+ """Estimate the build duration."""
107+
108+ def storeBuildInfo(librarian, slave_status):
109+ """Store available information for the build job.
110+
111+ Derived classes can override this as needed, and call it from
112+ custom status handlers, but it should not be called externally.
113+ """
114+
115
116 class IPackageBuildSource(Interface):
117 """A utility of this interface used to create _things_."""
118
119=== modified file 'lib/lp/buildmaster/model/buildbase.py'
120--- lib/lp/buildmaster/model/buildbase.py 2010-05-04 15:38:08 +0000
121+++ lib/lp/buildmaster/model/buildbase.py 2010-05-05 15:48:37 +0000
122@@ -50,19 +50,16 @@
123 """
124 policy_name = 'buildd'
125
126- def getUploadLeaf(self, build_id, now=None):
127- """Return a directory name to store build things in.
128-
129- :param build_id: The id as returned by the slave, normally
130- $BUILD_ID-$BUILDQUEUE_ID
131- :param now: The `datetime` to use. If not provided, defaults to now.
132- """
133- # UPLOAD_LEAF: <TIMESTAMP>-<BUILD_ID>-<BUILDQUEUE_ID>
134+ @staticmethod
135+ def getUploadDirLeaf(build_cookie, now=None):
136+ """See `IPackageBuild`."""
137+ # UPLOAD_LEAF: <TIMESTAMP>-<BUILD-COOKIE>
138 if now is None:
139 now = datetime.datetime.now()
140- return '%s-%s' % (now.strftime("%Y%m%d-%H%M%S"), build_id)
141+ return '%s-%s' % (now.strftime("%Y%m%d-%H%M%S"), build_cookie)
142
143- def getUploadDir(self, upload_leaf):
144+ @staticmethod
145+ def getUploadDir(upload_leaf):
146 """Return the directory that things will be stored in."""
147 return os.path.join(config.builddmaster.root, 'incoming', upload_leaf)
148
149@@ -141,6 +138,37 @@
150
151 method(librarian, slave_status, logger)
152
153+<<<<<<< TREE
154+=======
155+ def processUpload(self, leaf, root, logger):
156+ """Process an upload.
157+
158+ :param leaf: Leaf for this particular upload
159+ :param root: Root directory for the uploads
160+ :param logger: A logger object
161+ """
162+ class ProcessUploadOptions(object):
163+
164+ def __init__(self, policy_name, distribution, distroseries, pocket,
165+ buildid):
166+ self.context = policy_name
167+ self.distro = distribution.name
168+ self.distroseries = distroseries.name + pocketsuffix[pocket]
169+ self.buildid = buildid
170+ self.announce = []
171+
172+ options = ProcessUploadOptions(self.policy_name, self.distribution,
173+ self.distroseries, self.pocket, self.id)
174+ # XXX JRV 20100317: This should not create a mock options
175+ # object and derive the policy from that but rather create a
176+ # policy object in a more sensible way.
177+ policy = findPolicyByOptions(options)
178+ processor = UploadProcessor(root, dry_run=False, no_mails=True,
179+ keep=False, policy_for_distro=lambda distro: policy,
180+ ztm=ZopelessTransactionManager, log=logger)
181+ processor.processUploadQueue(leaf)
182+
183+>>>>>>> MERGE-SOURCE
184 def _handleStatus_OK(self, librarian, slave_status, logger):
185 """Handle a package that built successfully.
186
187@@ -165,7 +193,7 @@
188 root = os.path.abspath(config.builddmaster.root)
189
190 # create a single directory to store build result files
191- upload_leaf = self.getUploadLeaf(
192+ upload_leaf = self.getUploadDirLeaf(
193 '%s-%s' % (self.id, self.buildqueue_record.id))
194 upload_dir = self.getUploadDir(upload_leaf)
195 logger.debug("Storing build result at '%s'" % upload_dir)
196@@ -262,7 +290,7 @@
197
198 # Store build information, build record was already updated during
199 # the binary upload.
200- self.storeBuildInfo(librarian, slave_status)
201+ self.storeBuildInfo(self, librarian, slave_status)
202
203 # Retrive the up-to-date build record and perform consistency
204 # checks. The build record should be updated during the binary
205@@ -307,7 +335,7 @@
206 remove Buildqueue entry.
207 """
208 self.buildstate = BuildStatus.FAILEDTOBUILD
209- self.storeBuildInfo(librarian, slave_status)
210+ self.storeBuildInfo(self, librarian, slave_status)
211 self.buildqueue_record.builder.cleanSlave()
212 self.notify()
213 self.buildqueue_record.destroySelf()
214@@ -320,7 +348,7 @@
215 entry and release builder slave for another job.
216 """
217 self.buildstate = BuildStatus.MANUALDEPWAIT
218- self.storeBuildInfo(librarian, slave_status)
219+ self.storeBuildInfo(self, librarian, slave_status)
220 logger.critical("***** %s is MANUALDEPWAIT *****"
221 % self.buildqueue_record.builder.name)
222 self.buildqueue_record.builder.cleanSlave()
223@@ -335,7 +363,7 @@
224 and release the builder.
225 """
226 self.buildstate = BuildStatus.CHROOTWAIT
227- self.storeBuildInfo(librarian, slave_status)
228+ self.storeBuildInfo(self, librarian, slave_status)
229 logger.critical("***** %s is CHROOTWAIT *****" %
230 self.buildqueue_record.builder.name)
231 self.buildqueue_record.builder.cleanSlave()
232@@ -354,7 +382,7 @@
233 self.buildqueue_record.builder.failBuilder(
234 "Builder returned BUILDERFAIL when asked for its status")
235 # simply reset job
236- self.storeBuildInfo(librarian, slave_status)
237+ self.storeBuildInfo(self, librarian, slave_status)
238 self.buildqueue_record.reset()
239
240 def _handleStatus_GIVENBACK(self, librarian, slave_status, logger):
241@@ -367,7 +395,7 @@
242 logger.warning("***** %s is GIVENBACK by %s *****"
243 % (self.buildqueue_record.specific_job.build.title,
244 self.buildqueue_record.builder.name))
245- self.storeBuildInfo(librarian, slave_status)
246+ self.storeBuildInfo(self, librarian, slave_status)
247 # XXX cprov 2006-05-30: Currently this information is not
248 # properly presented in the Web UI. We will discuss it in
249 # the next Paris Summit, infinity has some ideas about how
250@@ -375,28 +403,30 @@
251 self.buildqueue_record.builder.cleanSlave()
252 self.buildqueue_record.reset()
253
254- def getLogFromSlave(self):
255+ @staticmethod
256+ def getLogFromSlave(build):
257 """See `IBuildBase`."""
258- return self.buildqueue_record.builder.transferSlaveFileToLibrarian(
259- 'buildlog', self.buildqueue_record.getLogFileName(),
260- self.is_private)
261+ return build.buildqueue_record.builder.transferSlaveFileToLibrarian(
262+ 'buildlog', build.buildqueue_record.getLogFileName(),
263+ build.is_private)
264
265- def storeBuildInfo(self, librarian, slave_status):
266+ @staticmethod
267+ def storeBuildInfo(build, librarian, slave_status):
268 """See `IBuildBase`."""
269- self.buildlog = self.getLogFromSlave()
270- self.builder = self.buildqueue_record.builder
271+ build.buildlog = build.getLogFromSlave(build)
272+ build.builder = build.buildqueue_record.builder
273 # XXX cprov 20060615 bug=120584: Currently buildduration includes
274 # the scanner latency, it should really be asking the slave for
275 # the duration spent building locally.
276- self.datebuilt = UTC_NOW
277+ build.datebuilt = UTC_NOW
278 # We need dynamic datetime.now() instance to be able to perform
279 # the time operations for duration.
280 RIGHT_NOW = datetime.datetime.now(pytz.timezone('UTC'))
281- self.buildduration = RIGHT_NOW - self.buildqueue_record.date_started
282+ build.buildduration = RIGHT_NOW - build.buildqueue_record.date_started
283 if slave_status.get('dependencies') is not None:
284- self.dependencies = unicode(slave_status.get('dependencies'))
285+ build.dependencies = unicode(slave_status.get('dependencies'))
286 else:
287- self.dependencies = None
288+ build.dependencies = None
289
290 def storeUploadLog(self, content):
291 """See `IBuildBase`."""
292
293=== modified file 'lib/lp/buildmaster/model/buildfarmjob.py'
294--- lib/lp/buildmaster/model/buildfarmjob.py 2010-05-05 15:48:34 +0000
295+++ lib/lp/buildmaster/model/buildfarmjob.py 2010-05-05 15:48:37 +0000
296@@ -121,6 +121,10 @@
297 """See `IBuildFarmJob`."""
298 raise NotImplementedError
299
300+ def makeJob(self):
301+ """See `IBuildFarmJob`."""
302+ raise NotImplementedError
303+
304 def jobStarted(self):
305 """See `IBuildFarmJob`."""
306 if not self.has_concrete_build_farm_job:
307
308=== modified file 'lib/lp/buildmaster/model/packagebuild.py'
309--- lib/lp/buildmaster/model/packagebuild.py 2010-05-05 15:48:34 +0000
310+++ lib/lp/buildmaster/model/packagebuild.py 2010-05-05 15:48:37 +0000
311@@ -23,6 +23,7 @@
312 from lp.buildmaster.interfaces.buildfarmjob import IBuildFarmJobSource
313 from lp.buildmaster.interfaces.packagebuild import (
314 IPackageBuild, IPackageBuildSource)
315+from lp.buildmaster.model.buildbase import BuildBase
316 from lp.buildmaster.model.buildfarmjob import BuildFarmJobDerived
317 from lp.registry.interfaces.pocket import PackagePublishingPocket
318 from lp.soyuz.adapters.archivedependencies import (
319@@ -163,6 +164,38 @@
320 return None
321 return ProxiedLibraryFileAlias(self.log, self).http_url
322
323+ def getUploadDirLeaf(self, build_cookie, now=None):
324+ """See `IPackageBuild`."""
325+ return BuildBase.getUploadDirLeaf(build_cookie, now)
326+
327+ def getUploadDir(self, upload_leaf):
328+ """See `IPackageBuild`."""
329+ return BuildBase.getUploadDir(upload_leaf)
330+
331+ @staticmethod
332+ def getUploaderCommand(package_build, distro_series, upload_leaf,
333+ upload_logfilename):
334+ """See `IPackageBuild`."""
335+ return BuildBase.getUploaderCommand(
336+ package_build, distro_series, upload_leaf, upload_logfilename)
337+
338+ def getLogFromSlave(self):
339+ """See `IPackageBuild`."""
340+ return BuildBase.getLogFromSlave(self)
341+
342+ @staticmethod
343+ def getUploadLogContent(root, leaf):
344+ """See `IPackageBuild`."""
345+ return BuildBase.getUploadLogContent(root, leaf)
346+
347+ def estimateDuration(self):
348+ """See `IPackageBuild`."""
349+ raise NotImplementedError
350+
351+ def storeBuildInfo(self, librarian, slave_status):
352+ """See `IPackageBuild`."""
353+ return BuildBase.storeBuildInfo(self, librarian, slave_status)
354+
355
356 class PackageBuildDerived(BuildFarmJobDerived):
357 """Override the base delegate to use a build farm job specific to
358
359=== modified file 'lib/lp/buildmaster/tests/test_buildbase.py'
360--- lib/lp/buildmaster/tests/test_buildbase.py 2010-05-04 15:38:08 +0000
361+++ lib/lp/buildmaster/tests/test_buildbase.py 2010-05-05 15:48:37 +0000
362@@ -3,8 +3,13 @@
363
364 from __future__ import with_statement
365
366-"""Tests for `IBuildBase`."""
367+"""Tests for `IBuildBase`.
368
369+ XXX 2010-04-26 michael.nelson bug=567922.
370+ These tests should be moved into test_packagebuild when buildbase is
371+ deleted. For the moment, test_packagebuild inherits these tests to
372+ ensure the new classes pass too.
373+"""
374 __metaclass__ = type
375
376 from datetime import datetime
377@@ -24,30 +29,32 @@
378 from lp.testing.fakemethod import FakeMethod
379
380
381-class TestBuildBase(TestCase):
382+class TestBuildBaseMixin:
383 """Tests for `IBuildBase`."""
384
385- def test_getUploadLeaf(self):
386- # getUploadLeaf returns the current time, followed by the build id.
387- build_base = BuildBase()
388+ def test_getUploadDirLeaf(self):
389+ # getUploadDirLeaf returns the current time, followed by the build
390+ # cookie.
391 now = datetime.now()
392- build_id = self.factory.getUniqueInteger()
393- upload_leaf = build_base.getUploadLeaf(build_id, now=now)
394+ build_cookie = self.factory.getUniqueString()
395+ upload_leaf = self.package_build.getUploadDirLeaf(
396+ build_cookie, now=now)
397 self.assertEqual(
398- '%s-%s' % (now.strftime("%Y%m%d-%H%M%S"), build_id), upload_leaf)
399+ '%s-%s' % (now.strftime("%Y%m%d-%H%M%S"), build_cookie),
400+ upload_leaf)
401
402 def test_getUploadDir(self):
403 # getUploadDir is the absolute path to the directory in which things
404 # are uploaded to.
405- build_base = BuildBase()
406- build_id = self.factory.getUniqueInteger()
407- upload_leaf = build_base.getUploadLeaf(build_id)
408- upload_dir = build_base.getUploadDir(upload_leaf)
409+ build_cookie = self.factory.getUniqueInteger()
410+ upload_leaf = self.package_build.getUploadDirLeaf(build_cookie)
411+ upload_dir = self.package_build.getUploadDir(upload_leaf)
412 self.assertEqual(
413 os.path.join(config.builddmaster.root, 'incoming', upload_leaf),
414 upload_dir)
415
416
417+<<<<<<< TREE
418 class TestBuildBaseWithDatabase(TestCaseWithFactory):
419 """Tests for `IBuildBase` that need objects from the rest of Launchpad."""
420
421@@ -55,6 +62,48 @@
422
423 def test_getUploadLogContent_nolog(self):
424 """If there is no log file there, a string explanation is returned.
425+=======
426+class TestBuildBase(TestCase, TestBuildBaseMixin):
427+
428+ def setUp(self):
429+ """Create the package build for testing."""
430+ super(TestBuildBase, self).setUp()
431+ self.package_build = BuildBase()
432+
433+
434+class TestProcessUpload(TestCaseWithFactory):
435+ """Test the execution of process-upload."""
436+
437+ layer = LaunchpadZopelessLayer
438+
439+ def setUp(self):
440+ self.queue_location = tempfile.mkdtemp()
441+ self.leaf = "theleaf"
442+ os.mkdir(os.path.join(self.queue_location, self.leaf))
443+ super(TestProcessUpload, self).setUp()
444+ self.build_base = BuildBase()
445+ self.build_base.distroseries = self.factory.makeDistroSeries()
446+ self.build_base.distribution = self.build_base.distroseries.distribution
447+ self.build_base.pocket = self.factory.getAnyPocket()
448+ self.build_base.id = self.factory.getUniqueInteger()
449+ self.build_base.policy_name = "insecure"
450+
451+ def tearDown(self):
452+ super(TestProcessUpload, self).tearDown()
453+ shutil.rmtree(self.queue_location)
454+
455+ def assertQueuePath(self, path):
456+ """Check if given path exists within the current queue_location."""
457+ probe_path = os.path.join(self.queue_location, path)
458+ self.assertTrue(
459+ os.path.exists(probe_path), "'%s' does not exist." % path)
460+
461+ def testSimpleRun(self):
462+ """Try a simple process-upload run.
463+
464+ Observe it creating the required directory tree for a given
465+ empty queue_location.
466+>>>>>>> MERGE-SOURCE
467 """
468 self.useTempDir()
469 build_base = BuildBase()
470
471=== modified file 'lib/lp/buildmaster/tests/test_buildfarmjob.py'
472--- lib/lp/buildmaster/tests/test_buildfarmjob.py 2010-05-05 15:48:34 +0000
473+++ lib/lp/buildmaster/tests/test_buildfarmjob.py 2010-05-05 15:48:37 +0000
474@@ -87,6 +87,7 @@
475 self.assertRaises(NotImplementedError, self.build_farm_job.score)
476 self.assertRaises(NotImplementedError, self.build_farm_job.getName)
477 self.assertRaises(NotImplementedError, self.build_farm_job.getTitle)
478+ self.assertRaises(NotImplementedError, self.build_farm_job.makeJob)
479
480 def test_jobStarted(self):
481 # Starting a job sets the date_started and status, as well as
482
483=== modified file 'lib/lp/buildmaster/tests/test_packagebuild.py'
484--- lib/lp/buildmaster/tests/test_packagebuild.py 2010-05-05 15:48:34 +0000
485+++ lib/lp/buildmaster/tests/test_packagebuild.py 2010-05-05 15:48:37 +0000
486@@ -18,21 +18,17 @@
487 from lp.buildmaster.interfaces.packagebuild import (
488 IPackageBuild, IPackageBuildSource)
489 from lp.buildmaster.model.packagebuild import PackageBuild
490+from lp.buildmaster.tests.test_buildbase import TestBuildBaseMixin
491 from lp.registry.interfaces.pocket import PackagePublishingPocket
492 from lp.testing import TestCaseWithFactory
493
494
495-class TestPackageBuild(TestCaseWithFactory):
496- """Tests for the package build object."""
497-
498- layer = LaunchpadFunctionalLayer
499-
500- def setUp(self):
501- """Create a package build with which to test."""
502- super(TestPackageBuild, self).setUp()
503- joe = self.factory.makePerson(name="joe")
504- joes_ppa = self.factory.makeArchive(owner=joe)
505- self.package_build = self.makePackageBuild(archive=joes_ppa)
506+class TestPackageBuildBase(TestCaseWithFactory):
507+ """Provide a factory method for creating PackageBuilds.
508+
509+ This is not included in the launchpad test factory because
510+ only classes deriving from PackageBuild should be used.
511+ """
512
513 def makePackageBuild(self, archive=None):
514 if archive is None:
515@@ -44,6 +40,34 @@
516 archive=archive,
517 pocket=PackagePublishingPocket.RELEASE)
518
519+
520+class TestBuildBaseMethods(TestPackageBuildBase, TestBuildBaseMixin):
521+ """The new PackageBuild class provides the same methods as the BuildBase.
522+
523+ XXX 2010-04-21 michael.nelson bug=567922.
524+ Until the BuildBase class and its tests are removed, we re-use the tests
525+ here to ensure that there is no divergence. Once BuildBase is removed the
526+ tests can be moved into TestPackageBuild.
527+ """
528+ layer = LaunchpadFunctionalLayer
529+
530+ def setUp(self):
531+ super(TestBuildBaseMethods, self).setUp()
532+ self.package_build = self.makePackageBuild()
533+
534+
535+class TestPackageBuild(TestPackageBuildBase):
536+ """Tests for the package build object."""
537+
538+ layer = LaunchpadFunctionalLayer
539+
540+ def setUp(self):
541+ """Create a package build with which to test."""
542+ super(TestPackageBuild, self).setUp()
543+ joe = self.factory.makePerson(name="joe")
544+ joes_ppa = self.factory.makeArchive(owner=joe)
545+ self.package_build = self.makePackageBuild(archive=joes_ppa)
546+
547 def test_providesInterface(self):
548 # PackageBuild provides IPackageBuild
549 self.assertProvides(self.package_build, IPackageBuild)
550@@ -57,9 +81,11 @@
551 PackageBuild.id == self.package_build.id).one()
552 self.assertEqual(self.package_build, retrieved_build)
553
554- def test_getTitle_not_implemented(self):
555+ def test_unimplemented_methods(self):
556 # Classes deriving from PackageBuild must provide getTitle.
557 self.assertRaises(NotImplementedError, self.package_build.getTitle)
558+ self.assertRaises(
559+ NotImplementedError, self.package_build.estimateDuration)
560
561 def test_default_values(self):
562 # PackageBuild has a number of default values.
563
564=== modified file 'lib/lp/soyuz/doc/buildd-slavescanner.txt'
565--- lib/lp/soyuz/doc/buildd-slavescanner.txt 2010-05-04 21:39:00 +0000
566+++ lib/lp/soyuz/doc/buildd-slavescanner.txt 2010-05-05 15:48:37 +0000
567@@ -533,7 +533,8 @@
568
569 Collect and process the buildlog.
570
571- >>> logfile_alias = bqItem10.specific_job.build.getLogFromSlave()
572+ >>> build = bqItem10.specific_job.build
573+ >>> logfile_alias = build.getLogFromSlave(build)
574
575 Audit the /tmp for lost temporary files, there should not be any new
576 files. For the record, the procedure creates files with the
577
578=== modified file 'lib/lp/soyuz/tests/test_build.py'
579--- lib/lp/soyuz/tests/test_build.py 2010-04-09 15:46:09 +0000
580+++ lib/lp/soyuz/tests/test_build.py 2010-05-05 15:48:37 +0000
581@@ -15,9 +15,11 @@
582 from lp.services.job.model.job import Job
583 from lp.buildmaster.interfaces.buildbase import BuildStatus, IBuildBase
584 from lp.buildmaster.interfaces.builder import IBuilderSet
585+from lp.buildmaster.interfaces.buildqueue import IBuildQueue
586 from lp.buildmaster.model.buildqueue import BuildQueue
587 from lp.soyuz.interfaces.binarypackagebuild import (
588 IBinaryPackageBuild, IBinaryPackageBuildSet)
589+from lp.soyuz.interfaces.buildpackagejob import IBuildPackageJob
590 from lp.soyuz.interfaces.component import IComponentSet
591 from lp.soyuz.interfaces.publishing import PackagePublishingStatus
592 from lp.soyuz.model.buildpackagejob import BuildPackageJob
593@@ -27,25 +29,42 @@
594 from lp.testing import TestCaseWithFactory
595
596
597-class TestBuildInterface(TestCaseWithFactory):
598+class TestBinaryPackageBuild(TestCaseWithFactory):
599
600 layer = LaunchpadZopelessLayer
601
602+ def setUp(self):
603+ super(TestBinaryPackageBuild, self).setUp()
604+ publisher = SoyuzTestPublisher()
605+ publisher.prepareBreezyAutotest()
606+ gedit_spr = publisher.getPubSource(
607+ spr_only=True, sourcename="gedit",
608+ status=PackagePublishingStatus.PUBLISHED)
609+ self.build = gedit_spr.createBuild(
610+ distroarchseries=publisher.distroseries['i386'],
611+ archive=gedit_spr.upload_archive,
612+ pocket=gedit_spr.package_upload.pocket)
613+
614 def test_providesInterfaces(self):
615 # Build provides IBuildBase and IBuild.
616- publisher = SoyuzTestPublisher()
617- publisher.prepareBreezyAutotest()
618- gedit_src_hist = publisher.getPubSource(
619- sourcename="gedit", status=PackagePublishingStatus.PUBLISHED)
620- build = gedit_src_hist.createMissingBuilds()[0]
621
622 # The IBinaryPackageBuild.calculated_buildstart property asserts
623 # that both datebuilt and buildduration are set.
624- build.datebuilt = datetime.now(pytz.UTC)
625- build.buildduration = timedelta(0, 1)
626-
627- self.assertProvides(build, IBuildBase)
628- self.assertProvides(build, IBinaryPackageBuild)
629+ self.build.datebuilt = datetime.now(pytz.UTC)
630+ self.build.buildduration = timedelta(0, 1)
631+
632+ self.assertProvides(self.build, IBuildBase)
633+ self.assertProvides(self.build, IBinaryPackageBuild)
634+
635+ def test_queueBuild(self):
636+ # BinaryPackageBuild can create the queue entry for itself.
637+ bq = self.build.queueBuild()
638+ self.assertProvides(bq, IBuildQueue)
639+ self.assertProvides(bq.specific_job, IBuildPackageJob)
640+ self.failUnlessEqual(self.build.is_virtualized, bq.virtualized)
641+ self.failIfEqual(None, bq.processor)
642+ self.failUnless(bq, self.build.buildqueue_record)
643+
644
645 class TestBuildUpdateDependencies(TestCaseWithFactory):
646
647@@ -271,7 +290,7 @@
648
649 def testDependencies(self):
650 """Verify that storeBuildInfo sets any dependencies."""
651- self.build.storeBuildInfo(None, {'dependencies': 'somepackage'})
652+ self.build.storeBuildInfo(self.build, None, {'dependencies': 'somepackage'})
653 self.assertIsNot(None, self.build.buildlog)
654 self.assertEqual(self.builder, self.build.builder)
655 self.assertEqual(u'somepackage', self.build.dependencies)
656@@ -284,7 +303,7 @@
657 # empties it.
658 self.build.dependencies = u'something'
659
660- self.build.storeBuildInfo(None, {})
661+ self.build.storeBuildInfo(self.build, None, {})
662 self.assertIsNot(None, self.build.buildlog)
663 self.assertEqual(self.builder, self.build.builder)
664 self.assertIs(None, self.build.dependencies)

Subscribers

People subscribed via source and target branches

to status/vote changes: