Launchpad itself

Merge lp:~michael.nelson/launchpad/450124-findBuildCandidate_improvements into lp:launchpad

  1. 450124-findBuildCandidate_improvements
  2. Merge into devel
Proposed by Michael Nelson
Status: Merged
Approved by: Gavin Panella
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~michael.nelson/launchpad/450124-findBuildCandidate_improvements
Merge into: lp:launchpad
Diff against target: 340 lines
5 files modified
lib/lp/soyuz/doc/buildd-dispatching.txt (+12/-38)
lib/lp/soyuz/interfaces/builder.py (+4/-0)
lib/lp/soyuz/model/builder.py (+3/-3)
lib/lp/soyuz/tests/test_builder.py (+127/-0)
lib/lp/testing/factory.py (+29/-0)
To merge this branch: bzr merge lp:~michael.nelson/launchpad/450124-findBuildCandidate_improvements
Reviewer Review Type Date Requested Status
Gavin Panella (community) Approve
Review via email: mp+13334@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Michael Nelson (michael.nelson) wrote :

= Summary =

Ensures that private PPAs are able to have builds for the same
architecture building in parallel.

== Pre-implementation notes ==

Chatted with Julian about other options - see bug 450124 for
documentation of those discussions.

== Implementation details ==

I had to create a unittest as the documentation is getting unreadable. I
also reverted the additional specific test that I had added to the
documentation now that it is covered by the unittest.

== Tests ==

bin/test -vvt TestFindBuildCandidate

== Demo and Q/A ==

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/soyuz/model/builder.py
  lib/lp/testing/factory.py
  lib/lp/soyuz/doc/buildd-dispatching.txt
  lib/lp/soyuz/tests/test_builder.py
  lib/lp/soyuz/interfaces/builder.py

--
Michael

Revision history for this message
Gavin Panella (allenap) wrote :
Download full text (6.1 KiB)

Hi Michael,

I have a few comments about the tests which you can take or
leave. Other than that, all good as far as I can tell :)

Gavin.

[...]
> === added file 'lib/lp/soyuz/tests/test_builder.py'
> --- lib/lp/soyuz/tests/test_builder.py 1970-01-01 00:00:00 +0000
> +++ lib/lp/soyuz/tests/test_builder.py 2009-10-14 10:55:53 +0000
> @@ -0,0 +1,116 @@
> +# Copyright 2009 Canonical Ltd. This software is licensed under the
> +# GNU Affero General Public License version 3 (see the file LICENSE).
> +
> +"""Test Builder features."""
> +
> +import unittest
> +
> +from zope.component import getUtility
> +
> +from canonical.testing import LaunchpadZopelessLayer
> +from lp.soyuz.interfaces.archive import ArchivePurpose
> +from lp.soyuz.interfaces.builder import IBuilderSet
> +from lp.soyuz.interfaces.build import BuildStatus
> +from lp.soyuz.interfaces.publishing import PackagePublishingStatus
> +from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
> +from lp.testing import TestCaseWithFactory
> +
> +
> +class TestFindBuildCandidate(TestCaseWithFactory):
> +
> + layer = LaunchpadZopelessLayer
> +
> + def setUp(self):
> + """Publish some builds for the test archive."""
> + super(TestFindBuildCandidate, self).setUp()
> + self.publisher = SoyuzTestPublisher()
> + self.publisher.prepareBreezyAutotest()
> +
> + # Create two PPAs and add some builds to each.
> + self.ppa_joe = self.factory.makeArchive(name="joesppa")
> + self.ppa_jim = self.factory.makeArchive(name="jimsppa")
> +
> + self.publisher.getPubSource(
> + sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
> + archive=self.ppa_joe).createMissingBuilds()
> + self.publisher.getPubSource(
> + sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
> + archive=self.ppa_joe).createMissingBuilds()
> +
> + self.publisher.getPubSource(
> + sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
> + archive=self.ppa_jim).createMissingBuilds()
> + self.publisher.getPubSource(
> + sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
> + archive=self.ppa_jim).createMissingBuilds()
> +
> + # Create two i386 builders ready to build PPA builds.
> + builder_set = getUtility(IBuilderSet)
> + self.builder1 = self.factory.makeBuilder(name='bob2')
> + self.builder2 = self.factory.makeBuilder(name='frog2')
> +
> + # Grab the first build, ensure that it is what we expect
> + # (ie. the first build from joesppa) and set it building.
> + self.first_job = self.builder1.findBuildCandidate()
> + self.failUnlessEqual('joesppa', self.first_job.build.archive.name)
> + self.failUnlessEqual(
> + u'i386 build of gedit 666 in ubuntutest breezy-autotest RELEASE',
> + self.first_job.build.title)
> + self.first_job.build.buildstate = BuildStatus.BUILDING
> + self.first_job.build.builder = self.builder1

That looks like a lot of set-up. If there's anything there that's
specific to only one test_* method, consider moving it o...

Read more...

review: Approve
Revision history for this message
Michael Nelson (michael.nelson) wrote :
Download full text (6.4 KiB)

Gavin Panella wrote:
> Review: Approve
> Hi Michael,
>
> I have a few comments about the tests which you can take or
> leave. Other than that, all good as far as I can tell :)
>
> Gavin.

Thanks Gavin.

>
>
> [...]
>> === added file 'lib/lp/soyuz/tests/test_builder.py'
>> --- lib/lp/soyuz/tests/test_builder.py 1970-01-01 00:00:00 +0000
>> +++ lib/lp/soyuz/tests/test_builder.py 2009-10-14 10:55:53 +0000
>> @@ -0,0 +1,116 @@
>> +# Copyright 2009 Canonical Ltd. This software is licensed under the
>> +# GNU Affero General Public License version 3 (see the file LICENSE).
>> +
>> +"""Test Builder features."""
>> +
>> +import unittest
>> +
>> +from zope.component import getUtility
>> +
>> +from canonical.testing import LaunchpadZopelessLayer
>> +from lp.soyuz.interfaces.archive import ArchivePurpose
>> +from lp.soyuz.interfaces.builder import IBuilderSet
>> +from lp.soyuz.interfaces.build import BuildStatus
>> +from lp.soyuz.interfaces.publishing import PackagePublishingStatus
>> +from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
>> +from lp.testing import TestCaseWithFactory
>> +
>> +
>> +class TestFindBuildCandidate(TestCaseWithFactory):
>> +
>> + layer = LaunchpadZopelessLayer
>> +
>> + def setUp(self):
>> + """Publish some builds for the test archive."""
>> + super(TestFindBuildCandidate, self).setUp()
>> + self.publisher = SoyuzTestPublisher()
>> + self.publisher.prepareBreezyAutotest()
>> +
>> + # Create two PPAs and add some builds to each.
>> + self.ppa_joe = self.factory.makeArchive(name="joesppa")
>> + self.ppa_jim = self.factory.makeArchive(name="jimsppa")
>> +
>> + self.publisher.getPubSource(
>> + sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
>> + archive=self.ppa_joe).createMissingBuilds()
>> + self.publisher.getPubSource(
>> + sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
>> + archive=self.ppa_joe).createMissingBuilds()
>> +
>> + self.publisher.getPubSource(
>> + sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
>> + archive=self.ppa_jim).createMissingBuilds()
>> + self.publisher.getPubSource(
>> + sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
>> + archive=self.ppa_jim).createMissingBuilds()
>> +
>> + # Create two i386 builders ready to build PPA builds.
>> + builder_set = getUtility(IBuilderSet)
>> + self.builder1 = self.factory.makeBuilder(name='bob2')
>> + self.builder2 = self.factory.makeBuilder(name='frog2')
>> +
>> + # Grab the first build, ensure that it is what we expect
>> + # (ie. the first build from joesppa) and set it building.
>> + self.first_job = self.builder1.findBuildCandidate()
>> + self.failUnlessEqual('joesppa', self.first_job.build.archive.name)
>> + self.failUnlessEqual(
>> + u'i386 build of gedit 666 in ubuntutest breezy-autotest RELEASE',
>> + self.first_job.build.title)
>> + self.first_job.build.buildstate = BuildStatus.BUILDING
>> + self.first_job.build.builder = ...

Read more...

changes_allenap.diff
=== modified file 'lib/lp/soyuz/tests/test_builder.py'
--- lib/lp/soyuz/tests/test_builder.py 2009-10-14 09:43:42 +0000
+++ lib/lp/soyuz/tests/test_builder.py 2009-10-14 14:22:29 +0000
@@ -16,39 +16,46 @@
16from lp.testing import TestCaseWithFactory16from lp.testing import TestCaseWithFactory
1717
1818
19class TestFindBuildCandidate(TestCaseWithFactory):19class TestFindBuildCandidateBase(TestCaseWithFactory):
20 """Setup the test publisher and some builders."""
2021
21 layer = LaunchpadZopelessLayer22 layer = LaunchpadZopelessLayer
2223
23 def setUp(self):24 def setUp(self):
24 """Publish some builds for the test archive."""25 super(TestFindBuildCandidateBase, self).setUp()
25 super(TestFindBuildCandidate, self).setUp()
26 self.publisher = SoyuzTestPublisher()26 self.publisher = SoyuzTestPublisher()
27 self.publisher.prepareBreezyAutotest()27 self.publisher.prepareBreezyAutotest()
2828
29 # Create two PPAs and add some builds to each.
30 self.ppa_joe = self.factory.makeArchive(name="joesppa")
31 self.ppa_jim = self.factory.makeArchive(name="jimsppa")
32
33 self.publisher.getPubSource(
34 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
35 archive=self.ppa_joe).createMissingBuilds()
36 self.publisher.getPubSource(
37 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
38 archive=self.ppa_joe).createMissingBuilds()
39
40 self.publisher.getPubSource(
41 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
42 archive=self.ppa_jim).createMissingBuilds()
43 self.publisher.getPubSource(
44 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
45 archive=self.ppa_jim).createMissingBuilds()
46
47 # Create two i386 builders ready to build PPA builds.29 # Create two i386 builders ready to build PPA builds.
48 builder_set = getUtility(IBuilderSet)30 builder_set = getUtility(IBuilderSet)
49 self.builder1 = self.factory.makeBuilder(name='bob2')31 self.builder1 = self.factory.makeBuilder(name='bob2')
50 self.builder2 = self.factory.makeBuilder(name='frog2')32 self.builder2 = self.factory.makeBuilder(name='frog2')
5133
34
35class TestFindBuildCandidatePPA(TestFindBuildCandidateBase):
36
37 def setUp(self):
38 """Publish some builds for the test archive."""
39 super(TestFindBuildCandidatePPA, self).setUp()
40
41 # Create two PPAs and add some builds to each.
42 self.ppa_joe = self.factory.makeArchive(name="joesppa")
43 self.ppa_jim = self.factory.makeArchive(name="jimsppa")
44
45 self.publisher.getPubSource(
46 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
47 archive=self.ppa_joe).createMissingBuilds()
48 self.publisher.getPubSource(
49 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
50 archive=self.ppa_joe).createMissingBuilds()
51
52 self.publisher.getPubSource(
53 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
54 archive=self.ppa_jim).createMissingBuilds()
55 self.publisher.getPubSource(
56 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
57 archive=self.ppa_jim).createMissingBuilds()
58
52 # Grab the first build, ensure that it is what we expect59 # Grab the first build, ensure that it is what we expect
53 # (ie. the first build from joesppa) and set it building.60 # (ie. the first build from joesppa) and set it building.
54 self.first_job = self.builder1.findBuildCandidate()61 self.first_job = self.builder1.findBuildCandidate()
@@ -80,9 +87,14 @@
80 next_job = self.builder2.findBuildCandidate()87 next_job = self.builder2.findBuildCandidate()
81 self.failUnlessEqual('joesppa', next_job.build.archive.name)88 self.failUnlessEqual('joesppa', next_job.build.archive.name)
8289
83 def test_findBuildCandidate_for_non_ppa(self):90
84 # Normal archives are not restricted to serial builds per91class TestFindBuildCandidateDistroArchive(TestFindBuildCandidateBase):
85 # arch.92
93 def setUp(self):
94 """Publish some builds for the test archive."""
95 super(TestFindBuildCandidateDistroArchive, self).setUp()
96 # Create a primary archive and publish some builds for the
97 # queue.
86 non_ppa = self.factory.makeArchive(98 non_ppa = self.factory.makeArchive(
87 name="primary", purpose=ArchivePurpose.PRIMARY)99 name="primary", purpose=ArchivePurpose.PRIMARY)
88100
@@ -93,10 +105,9 @@
93 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,105 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
94 archive=non_ppa).createMissingBuilds()[0]106 archive=non_ppa).createMissingBuilds()[0]
95107
96 # Rescore our primary builds so that they'll be returned before108 def test_findBuildCandidate_for_non_ppa(self):
97 # the PPA ones.109 # Normal archives are not restricted to serial builds per
98 gedit_build.buildqueue_record.manualScore(3000)110 # arch.
99 firefox_build.buildqueue_record.manualScore(3000)
100111
101 next_job = self.builder2.findBuildCandidate()112 next_job = self.builder2.findBuildCandidate()
102 self.failUnlessEqual('primary', next_job.build.archive.name)113 self.failUnlessEqual('primary', next_job.build.archive.name)
Revision history for this message
Gavin Panella (allenap) wrote :

That's a beauty :)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/soyuz/doc/buildd-dispatching.txt'
--- lib/lp/soyuz/doc/buildd-dispatching.txt 2009-10-12 11:46:43 +0000
+++ lib/lp/soyuz/doc/buildd-dispatching.txt 2009-10-14 14:28:13 +0000
@@ -202,20 +202,15 @@
202202
203== PPA build dispatching ==203== PPA build dispatching ==
204204
205Create two Build records of the same source targeted for a PPA archive:205Create a new Build record of the same source targeted for a PPA archive:
206206
207 >>> from lp.registry.interfaces.person import IPersonSet207 >>> from lp.registry.interfaces.person import IPersonSet
208 >>> cprov = getUtility(IPersonSet).getByName('cprov')208 >>> cprov = getUtility(IPersonSet).getByName('cprov')
209 >>> warty = getUtility(IDistributionSet)['ubuntu']['warty']
210 >>> warty_i386 = warty['i386']
211209
212 # One for hoary and one for warty.
213 >>> ppa_build = sprf.sourcepackagerelease.createBuild(210 >>> ppa_build = sprf.sourcepackagerelease.createBuild(
214 ... hoary_i386, PackagePublishingPocket.RELEASE, cprov.archive)211 ... hoary_i386, PackagePublishingPocket.RELEASE, cprov.archive)
215 >>> ppa_build_2 = sprf.sourcepackagerelease.createBuild(
216 ... warty_i386, PackagePublishingPocket.RELEASE, cprov.archive)
217212
218Create BuildQueue records and inspect some parameters:213Create BuildQueue record and inspect some parameters:
219214
220 >>> ppa_job = ppa_build.createBuildQueueEntry()215 >>> ppa_job = ppa_build.createBuildQueueEntry()
221 >>> ppa_job.id216 >>> ppa_job.id
@@ -224,9 +219,6 @@
224 True219 True
225 >>> ppa_job.buildstart == None220 >>> ppa_job.buildstart == None
226 True221 True
227 >>> ppa_job_2 = ppa_build_2.createBuildQueueEntry()
228 >>> ppa_job_2.id
229 4
230222
231The build job's archive requires virtualized builds.223The build job's archive requires virtualized builds.
232224
@@ -248,10 +240,10 @@
248 >>> print job240 >>> print job
249 None241 None
250242
251In order to enable 'bob' to find and build the PPA jobs, we have to243In order to enable 'bob' to find and build the PPA job, we have to
252change it to virtualized. This is because PPA builds will only build244change it to virtualized. This is because PPA builds will only build
253on virtualized builders. We also need to make sure the builds' sources245on virtualized builders. We also need to make sure this build's source
254are published, or they will also be ignored (by superseding it). We can246is published, or it will also be ignored (by superseding it). We can
255do this by copying the existing publication in Ubuntu.247do this by copying the existing publication in Ubuntu.
256248
257 >>> from lp.soyuz.model.publishing import (249 >>> from lp.soyuz.model.publishing import (
@@ -261,11 +253,6 @@
261 ... sourcepackagerelease=ppa_job.build.sourcepackagerelease)253 ... sourcepackagerelease=ppa_job.build.sourcepackagerelease)
262 >>> new_pub = old_pub.copyTo(254 >>> new_pub = old_pub.copyTo(
263 ... old_pub.distroseries, old_pub.pocket, ppa_job.build.archive)255 ... old_pub.distroseries, old_pub.pocket, ppa_job.build.archive)
264 >>> [old_pub_2] = SourcePackagePublishingHistory.selectBy(
265 ... distroseries=ppa_job_2.build.distroseries,
266 ... sourcepackagerelease=ppa_job_2.build.sourcepackagerelease)
267 >>> new_pub_2 = old_pub_2.copyTo(
268 ... old_pub_2.distroseries, old_pub_2.pocket, ppa_job_2.build.archive)
269256
270 >>> bob_builder.virtualized = True257 >>> bob_builder.virtualized = True
271 >>> syncUpdate(bob_builder)258 >>> syncUpdate(bob_builder)
@@ -274,6 +261,9 @@
274 >>> ppa_job.id == job.id261 >>> ppa_job.id == job.id
275 True262 True
276263
264For further details regarding IBuilder.findBuildCandidate() please see
265lib/lp/soyuz/tests/test_builder.py.
266
277Start buildd-slave to be able to dispatch jobs.267Start buildd-slave to be able to dispatch jobs.
278268
279 >>> BuilddSlaveTestSetup().setUp()269 >>> BuilddSlaveTestSetup().setUp()
@@ -309,30 +299,14 @@
309 >>> ppa_job.buildstart == get_transaction_timestamp()299 >>> ppa_job.buildstart == get_transaction_timestamp()
310 True300 True
311301
312At this point, the second build job - for the same archive and architecture -
313will not be found as a build candidate for other builders. This is to
314avoid nightly builds using all the builders for an arch at once.
315
316 >>> print frog_builder.findBuildCandidate()
317 None
318
319But once the first build (for the same archive and architecture) finishes,
320the second build will be available as a build candidate for both builders.
321
322 >>> ppa_job.build.buildstate = BuildStatus.FAILEDTOBUILD
323 >>> ppa_job.destroySelf()
324 >>> print frog_builder.findBuildCandidate().id
325 4
326 >>> print bob_builder.findBuildCandidate().id
327 4
328
329Shutdown builder slave, mark the ppa build record as failed, remove the302Shutdown builder slave, mark the ppa build record as failed, remove the
330buildqueue record and make 'bob' builder non-virtual again, so the303buildqueue record and make 'bob' builder non-virtual again, so the
331environment is back to the initial state.304environment is back to the initial state.
332305
333 >>> BuilddSlaveTestSetup().tearDown()306 >>> BuilddSlaveTestSetup().tearDown()
334 >>> ppa_job_2.build.buildstate = BuildStatus.FAILEDTOBUILD307
335 >>> ppa_job_2.destroySelf()308 >>> ppa_job.build.buildstate = BuildStatus.FAILEDTOBUILD
309 >>> ppa_job.destroySelf()
336 >>> bob_builder.virtualized = False310 >>> bob_builder.virtualized = False
337 >>> flush_database_updates()311 >>> flush_database_updates()
338312
@@ -355,7 +329,7 @@
355329
356 >>> sec_job = sec_build.createBuildQueueEntry()330 >>> sec_job = sec_build.createBuildQueueEntry()
357 >>> sec_job.id331 >>> sec_job.id
358 5332 4
359 >>> print sec_job.builder333 >>> print sec_job.builder
360 None334 None
361 >>> print sec_job.buildstart335 >>> print sec_job.buildstart
362336
=== modified file 'lib/lp/soyuz/interfaces/builder.py'
--- lib/lp/soyuz/interfaces/builder.py 2009-08-27 14:48:38 +0000
+++ lib/lp/soyuz/interfaces/builder.py 2009-10-14 14:28:13 +0000
@@ -262,6 +262,10 @@
262262
263 The pending BuildQueue item with the highest score for this builder263 The pending BuildQueue item with the highest score for this builder
264 ProcessorFamily or None if no candidate is available.264 ProcessorFamily or None if no candidate is available.
265
266 For public PPA builds, subsequent builds for a given ppa and
267 architecture will not be returned until the current build for
268 the ppa and architecture is finished.
265 """269 """
266270
267 def dispatchBuildCandidate(candidate):271 def dispatchBuildCandidate(candidate):
268272
=== modified file 'lib/lp/soyuz/model/builder.py'
--- lib/lp/soyuz/model/builder.py 2009-10-12 15:56:25 +0000
+++ lib/lp/soyuz/model/builder.py 2009-10-14 14:28:13 +0000
@@ -570,8 +570,8 @@
570 """ % sqlvalues(self.virtualized))570 """ % sqlvalues(self.virtualized))
571571
572 # Ensure that if a currently-building build exists for the same572 # Ensure that if a currently-building build exists for the same
573 # ppa archive and architecture currently building then we don't573 # public ppa archive and architecture currently building then
574 # consider another as a candidate.574 # we don't consider another as a candidate.
575 clauses.append("""575 clauses.append("""
576 NOT EXISTS (576 NOT EXISTS (
577 SELECT Build.id577 SELECT Build.id
@@ -579,6 +579,7 @@
579 WHERE579 WHERE
580 build2.archive = build.archive AND580 build2.archive = build.archive AND
581 archive.purpose = %s AND581 archive.purpose = %s AND
582 archive.private IS FALSE AND
582 build2.distroarchseries = distroarchseries2.id AND583 build2.distroarchseries = distroarchseries2.id AND
583 distroarchseries2.processorfamily = %s AND584 distroarchseries2.processorfamily = %s AND
584 build2.buildstate = %s)585 build2.buildstate = %s)
@@ -586,7 +587,6 @@
586 ArchivePurpose.PPA, self.processor.family, BuildStatus.BUILDING))587 ArchivePurpose.PPA, self.processor.family, BuildStatus.BUILDING))
587588
588 query = " AND ".join(clauses)589 query = " AND ".join(clauses)
589
590 candidate = BuildQueue.selectFirst(590 candidate = BuildQueue.selectFirst(
591 query, clauseTables=clauseTables, prejoins=['build'],591 query, clauseTables=clauseTables, prejoins=['build'],
592 orderBy=['-buildqueue.lastscore', 'build.id'])592 orderBy=['-buildqueue.lastscore', 'build.id'])
593593
=== added file 'lib/lp/soyuz/tests/test_builder.py'
--- lib/lp/soyuz/tests/test_builder.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/tests/test_builder.py 2009-10-14 14:28:13 +0000
@@ -0,0 +1,127 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Test Builder features."""
5
6import unittest
7
8from zope.component import getUtility
9
10from canonical.testing import LaunchpadZopelessLayer
11from lp.soyuz.interfaces.archive import ArchivePurpose
12from lp.soyuz.interfaces.builder import IBuilderSet
13from lp.soyuz.interfaces.build import BuildStatus
14from lp.soyuz.interfaces.publishing import PackagePublishingStatus
15from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
16from lp.testing import TestCaseWithFactory
17
18
19class TestFindBuildCandidateBase(TestCaseWithFactory):
20 """Setup the test publisher and some builders."""
21
22 layer = LaunchpadZopelessLayer
23
24 def setUp(self):
25 super(TestFindBuildCandidateBase, self).setUp()
26 self.publisher = SoyuzTestPublisher()
27 self.publisher.prepareBreezyAutotest()
28
29 # Create two i386 builders ready to build PPA builds.
30 builder_set = getUtility(IBuilderSet)
31 self.builder1 = self.factory.makeBuilder(name='bob2')
32 self.builder2 = self.factory.makeBuilder(name='frog2')
33
34
35class TestFindBuildCandidatePPA(TestFindBuildCandidateBase):
36
37 def setUp(self):
38 """Publish some builds for the test archive."""
39 super(TestFindBuildCandidatePPA, self).setUp()
40
41 # Create two PPAs and add some builds to each.
42 self.ppa_joe = self.factory.makeArchive(name="joesppa")
43 self.ppa_jim = self.factory.makeArchive(name="jimsppa")
44
45 self.publisher.getPubSource(
46 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
47 archive=self.ppa_joe).createMissingBuilds()
48 self.publisher.getPubSource(
49 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
50 archive=self.ppa_joe).createMissingBuilds()
51
52 self.publisher.getPubSource(
53 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
54 archive=self.ppa_jim).createMissingBuilds()
55 self.publisher.getPubSource(
56 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
57 archive=self.ppa_jim).createMissingBuilds()
58
59 # Grab the first build, ensure that it is what we expect
60 # (ie. the first build from joesppa) and set it building.
61 self.first_job = self.builder1.findBuildCandidate()
62 self.failUnlessEqual('joesppa', self.first_job.build.archive.name)
63 self.failUnlessEqual(
64 u'i386 build of gedit 666 in ubuntutest breezy-autotest RELEASE',
65 self.first_job.build.title)
66 self.first_job.build.buildstate = BuildStatus.BUILDING
67 self.first_job.build.builder = self.builder1
68
69 def test_findBuildCandidate_first_build_started(self):
70 # Once a build for an ppa+arch has started, a second one for the
71 # same ppa+arch will not be a candidate.
72 next_job = self.builder2.findBuildCandidate()
73 self.failIfEqual('joesppa', next_job.build.archive.name)
74
75 def test_findBuildCandidate_first_build_finished(self):
76 # When joe's first ppa build finishes, his second i386 build
77 # will be the next build candidate.
78 self.first_job.build.buildstate = BuildStatus.FAILEDTOBUILD
79 next_job = self.builder2.findBuildCandidate()
80 self.failUnlessEqual('joesppa', next_job.build.archive.name)
81
82 def test_findBuildCandidate_for_private_ppa(self):
83 # If a ppa is private it will be able to have parallel builds
84 # for the one architecture.
85 self.ppa_joe.private = True
86 self.ppa_joe.buildd_secret = 'sekrit'
87 next_job = self.builder2.findBuildCandidate()
88 self.failUnlessEqual('joesppa', next_job.build.archive.name)
89
90
91class TestFindBuildCandidateDistroArchive(TestFindBuildCandidateBase):
92
93 def setUp(self):
94 """Publish some builds for the test archive."""
95 super(TestFindBuildCandidateDistroArchive, self).setUp()
96 # Create a primary archive and publish some builds for the
97 # queue.
98 non_ppa = self.factory.makeArchive(
99 name="primary", purpose=ArchivePurpose.PRIMARY)
100
101 gedit_build = self.publisher.getPubSource(
102 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
103 archive=non_ppa).createMissingBuilds()[0]
104 firefox_build = self.publisher.getPubSource(
105 sourcename="firefox", status=PackagePublishingStatus.PUBLISHED,
106 archive=non_ppa).createMissingBuilds()[0]
107
108 def test_findBuildCandidate_for_non_ppa(self):
109 # Normal archives are not restricted to serial builds per
110 # arch.
111
112 next_job = self.builder2.findBuildCandidate()
113 self.failUnlessEqual('primary', next_job.build.archive.name)
114 self.failUnlessEqual(
115 'gedit', next_job.build.sourcepackagerelease.name)
116
117 # Now even if we set the build building, we'll still get the
118 # second non-ppa build for the same archive as the next candidate.
119 next_job.build.buildstate = BuildStatus.BUILDING
120 next_job.build.builder = self.builder2
121 next_job = self.builder2.findBuildCandidate()
122 self.failUnlessEqual('primary', next_job.build.archive.name)
123 self.failUnlessEqual(
124 'firefox', next_job.build.sourcepackagerelease.name)
125
126def test_suite():
127 return unittest.TestLoader().loadTestsFromName(__name__)
0128
=== modified file 'lib/lp/testing/factory.py'
--- lib/lp/testing/factory.py 2009-10-06 00:01:08 +0000
+++ lib/lp/testing/factory.py 2009-10-14 14:28:13 +0000
@@ -106,6 +106,7 @@
106 ISourcePackageNameSet)106 ISourcePackageNameSet)
107from lp.registry.interfaces.ssh import ISSHKeySet, SSHKeyType107from lp.registry.interfaces.ssh import ISSHKeySet, SSHKeyType
108from lp.services.worlddata.interfaces.language import ILanguageSet108from lp.services.worlddata.interfaces.language import ILanguageSet
109from lp.soyuz.interfaces.builder import IBuilderSet
109from lp.soyuz.interfaces.component import IComponentSet110from lp.soyuz.interfaces.component import IComponentSet
110from lp.soyuz.interfaces.packageset import IPackagesetSet111from lp.soyuz.interfaces.packageset import IPackagesetSet
111from lp.testing import run_with_login, time_counter112from lp.testing import run_with_login, time_counter
@@ -1485,6 +1486,34 @@
1485 owner=owner, purpose=purpose,1486 owner=owner, purpose=purpose,
1486 distribution=distribution, name=name)1487 distribution=distribution, name=name)
14871488
1489 def makeBuilder(self, processor=None, url=None, name=None, title=None,
1490 description=None, owner=None, active=True,
1491 virtualized=True, vm_host=None):
1492 """Make a new builder for i386 virtualized builds by default.
1493
1494 Note: the builder returned will not be able to actually build -
1495 we currently have a build slave setup for 'bob' only in the
1496 test environment.
1497 See lib/canonical/buildd/tests/buildd-slave-test.conf
1498 """
1499 if processor is None:
1500 processor_fam = ProcessorFamilySet().getByName('x86')
1501 processor = processor_fam.processors[0]
1502 if url is None:
1503 url = 'http://%s:8221/' % self.getUniqueString()
1504 if name is None:
1505 name = self.getUniqueString()
1506 if title is None:
1507 title = self.getUniqueString()
1508 if description is None:
1509 description = self.getUniqueString()
1510 if owner is None:
1511 owner = self.makePerson()
1512
1513 return getUtility(IBuilderSet).new(
1514 processor, url, name, title, description, owner, active,
1515 virtualized, vm_host)
1516
1488 def makePOTemplate(self, productseries=None, distroseries=None,1517 def makePOTemplate(self, productseries=None, distroseries=None,
1489 sourcepackagename=None, owner=None, name=None,1518 sourcepackagename=None, owner=None, name=None,
1490 translation_domain=None, path=None):1519 translation_domain=None, path=None):