Launchpad itself

Merge lp:~wgrant/launchpad/more-buildmaster-cleanup into lp:launchpad

  1. more-buildmaster-cleanup
  2. Merge into devel
Proposed by William Grant
Status: Merged
Approved by: Michael Hudson-Doyle
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~wgrant/launchpad/more-buildmaster-cleanup
Merge into: lp:launchpad
Prerequisite: lp:~wgrant/launchpad/amputate-buildergroup
Diff against target: 270 lines (+66/-76)
7 files modified
lib/lp/buildmaster/doc/buildqueue.txt (+5/-5)
lib/lp/buildmaster/interfaces/buildqueue.py (+0/-17)
lib/lp/buildmaster/model/buildqueue.py (+0/-48)
lib/lp/soyuz/browser/build.py (+2/-3)
lib/lp/soyuz/interfaces/build.py (+17/-0)
lib/lp/soyuz/model/build.py (+41/-0)
lib/lp/soyuz/scripts/buildd.py (+1/-3)
To merge this branch: bzr merge lp:~wgrant/launchpad/more-buildmaster-cleanup
Reviewer Review Type Date Requested Status
Michael Nelson (community) code Approve
Aaron Bentley (community) Needs Information
Review via email: mp+22735@code.launchpad.net

Commit message

Finish the BuildQueue generalisation by moving the last two Soyuz-specific methods from BuildQueueSet to BuildSet.

Description of the change

This branch moves the last two Soyuz-specific methods from BuildQueueSet to BuildSet, completing the BuildQueue generalisation.

To post a comment you must log in.
Revision history for this message
Aaron Bentley (abentley) wrote :

This looks reasonable. Was there a preimplementation call? Was there lint?

review: Needs Information
Revision history for this message
Michael Nelson (michael.nelson) wrote :

Thanks William. As noted, please verify/clean any lint, but otherwise r=me.

review: Approve (code)
Revision history for this message
William Grant (wgrant) wrote :

There is no lint, beyond the external import complaints that appear to have become accepted fixtures.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/buildmaster/doc/buildqueue.txt'
--- lib/lp/buildmaster/doc/buildqueue.txt 2010-03-05 13:52:32 +0000
+++ lib/lp/buildmaster/doc/buildqueue.txt 2010-04-03 04:16:21 +0000
@@ -255,7 +255,7 @@
255Note that the results are ordered by descending BuildQueue.lastscore255Note that the results are ordered by descending BuildQueue.lastscore
256and restricted256and restricted
257257
258 >>> for bq in bqset.calculateCandidates(archseries):258 >>> for bq in getUtility(IBuildSet).calculateCandidates(archseries):
259 ... build = getUtility(IBuildSet).getByQueueEntry(bq)259 ... build = getUtility(IBuildSet).getByQueueEntry(bq)
260 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)260 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)
261 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)261 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)
@@ -287,7 +287,7 @@
287The fake language pack will be queued behind the other source packages287The fake language pack will be queued behind the other source packages
288as intended.288as intended.
289289
290 >>> for bq in bqset.calculateCandidates(archseries):290 >>> for bq in getUtility(IBuildSet).calculateCandidates(archseries):
291 ... build = getUtility(IBuildSet).getByQueueEntry(bq)291 ... build = getUtility(IBuildSet).getByQueueEntry(bq)
292 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)292 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)
293 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)293 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)
@@ -299,7 +299,7 @@
299Restricting the domain to only hoary/hppa:299Restricting the domain to only hoary/hppa:
300300
301 >>> archseries = [hoary['hppa']]301 >>> archseries = [hoary['hppa']]
302 >>> for bq in bqset.calculateCandidates(archseries):302 >>> for bq in getUtility(IBuildSet).calculateCandidates(archseries):
303 ... build = getUtility(IBuildSet).getByQueueEntry(bq)303 ... build = getUtility(IBuildSet).getByQueueEntry(bq)
304 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)304 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)
305 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)305 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)
@@ -308,12 +308,12 @@
308This method asserts a valid 'archseries' argument, None or an empty308This method asserts a valid 'archseries' argument, None or an empty
309list will result in a AssertionFailure.309list will result in a AssertionFailure.
310310
311 >>> bqset.calculateCandidates(None).count()311 >>> getUtility(IBuildSet).calculateCandidates(None).count()
312 Traceback (most recent call last):312 Traceback (most recent call last):
313 ...313 ...
314 AssertionError: Given 'archseries' cannot be None/empty.314 AssertionError: Given 'archseries' cannot be None/empty.
315315
316 >>> bqset.calculateCandidates([]).count()316 >>> getUtility(IBuildSet).calculateCandidates([]).count()
317 Traceback (most recent call last):317 Traceback (most recent call last):
318 ...318 ...
319 AssertionError: Given 'archseries' cannot be None/empty.319 AssertionError: Given 'archseries' cannot be None/empty.
320320
=== modified file 'lib/lp/buildmaster/interfaces/buildqueue.py'
--- lib/lp/buildmaster/interfaces/buildqueue.py 2010-03-16 05:08:47 +0000
+++ lib/lp/buildmaster/interfaces/buildqueue.py 2010-04-03 04:16:21 +0000
@@ -148,20 +148,3 @@
148148
149 def getActiveBuildJobs():149 def getActiveBuildJobs():
150 """Return All active Build Jobs."""150 """Return All active Build Jobs."""
151
152 def calculateCandidates(archseries):
153 """Return the BuildQueue records for the given archseries.
154
155 Returns a selectRelease of BuildQueue items sorted by descending
156 'lastscore' within the given archseries.
157
158 'archseries' argument should be a list of DistroArchSeries and it is
159 asserted to not be None/empty.
160 """
161
162 def getForBuilds(build_ids):
163 """Return the IBuildQueue instance for the IBuild IDs at hand.
164
165 Retrieve the build queue and related builder rows associated with the
166 builds in question where they exist.
167 """
168151
=== modified file 'lib/lp/buildmaster/model/buildqueue.py'
--- lib/lp/buildmaster/model/buildqueue.py 2010-03-17 05:58:22 +0000
+++ lib/lp/buildmaster/model/buildqueue.py 2010-04-03 04:16:21 +0000
@@ -18,7 +18,6 @@
1818
19from sqlobject import (19from sqlobject import (
20 StringCol, ForeignKey, BoolCol, IntCol, IntervalCol, SQLObjectNotFound)20 StringCol, ForeignKey, BoolCol, IntCol, IntervalCol, SQLObjectNotFound)
21from storm.expr import In, Join, LeftJoin
22from storm.store import Store21from storm.store import Store
23from zope.component import getSiteManager, getUtility22from zope.component import getSiteManager, getUtility
24from zope.interface import implements23from zope.interface import implements
@@ -27,7 +26,6 @@
27from canonical.database.sqlbase import SQLBase, sqlvalues26from canonical.database.sqlbase import SQLBase, sqlvalues
28from canonical.launchpad.webapp.interfaces import (27from canonical.launchpad.webapp.interfaces import (
29 DEFAULT_FLAVOR, IStoreSelector, MAIN_STORE, NotFoundError)28 DEFAULT_FLAVOR, IStoreSelector, MAIN_STORE, NotFoundError)
30from lp.buildmaster.interfaces.buildbase import BuildStatus
31from lp.buildmaster.interfaces.buildfarmjob import (29from lp.buildmaster.interfaces.buildfarmjob import (
32 BuildFarmJobType, IBuildFarmJob)30 BuildFarmJobType, IBuildFarmJob)
33from lp.buildmaster.interfaces.buildfarmjobbehavior import (31from lp.buildmaster.interfaces.buildfarmjobbehavior import (
@@ -35,7 +33,6 @@
35from lp.buildmaster.interfaces.buildqueue import IBuildQueue, IBuildQueueSet33from lp.buildmaster.interfaces.buildqueue import IBuildQueue, IBuildQueueSet
36from lp.services.job.interfaces.job import JobStatus34from lp.services.job.interfaces.job import JobStatus
37from lp.services.job.model.job import Job35from lp.services.job.model.job import Job
38from lp.soyuz.model.buildpackagejob import BuildPackageJob
3936
4037
41def normalize_virtualization(virtualized):38def normalize_virtualization(virtualized):
@@ -526,48 +523,3 @@
526 Job._status == JobStatus.RUNNING,523 Job._status == JobStatus.RUNNING,
527 Job.date_started != None)524 Job.date_started != None)
528 return result_set525 return result_set
529
530 def calculateCandidates(self, archseries):
531 """See `IBuildQueueSet`."""
532 if not archseries:
533 raise AssertionError("Given 'archseries' cannot be None/empty.")
534
535 arch_ids = [d.id for d in archseries]
536
537 query = """
538 Build.distroarchseries IN %s AND
539 Build.buildstate = %s AND
540 BuildQueue.job_type = %s AND
541 BuildQueue.job = BuildPackageJob.job AND
542 BuildPackageJob.build = build.id AND
543 BuildQueue.builder IS NULL
544 """ % sqlvalues(
545 arch_ids, BuildStatus.NEEDSBUILD, BuildFarmJobType.PACKAGEBUILD)
546
547 candidates = BuildQueue.select(
548 query, clauseTables=['Build', 'BuildPackageJob'],
549 orderBy=['-BuildQueue.lastscore'])
550
551 return candidates
552
553 def getForBuilds(self, build_ids):
554 """See `IBuildQueueSet`."""
555 # Avoid circular import problem.
556 from lp.soyuz.model.build import Build
557 from lp.buildmaster.model.builder import Builder
558
559 store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
560
561 origin = (
562 BuildPackageJob,
563 Join(BuildQueue, BuildPackageJob.job == BuildQueue.jobID),
564 Join(Build, BuildPackageJob.build == Build.id),
565 LeftJoin(
566 Builder,
567 BuildQueue.builderID == Builder.id),
568 )
569 result_set = store.using(*origin).find(
570 (BuildQueue, Builder, BuildPackageJob),
571 In(Build.id, build_ids))
572
573 return result_set
574526
=== modified file 'lib/lp/soyuz/browser/build.py'
--- lib/lp/soyuz/browser/build.py 2010-03-16 05:09:55 +0000
+++ lib/lp/soyuz/browser/build.py 2010-04-03 04:16:21 +0000
@@ -33,9 +33,8 @@
33from canonical.launchpad.webapp.interfaces import ICanonicalUrlData33from canonical.launchpad.webapp.interfaces import ICanonicalUrlData
34from canonical.lazr.utils import safe_hasattr34from canonical.lazr.utils import safe_hasattr
35from lp.buildmaster.interfaces.buildbase import BuildStatus35from lp.buildmaster.interfaces.buildbase import BuildStatus
36from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
37from lp.services.job.interfaces.job import JobStatus36from lp.services.job.interfaces.job import JobStatus
38from lp.soyuz.interfaces.build import IBuild, IBuildRescoreForm37from lp.soyuz.interfaces.build import IBuild, IBuildRescoreForm, IBuildSet
39from lp.soyuz.interfaces.buildrecords import IHasBuildRecords38from lp.soyuz.interfaces.buildrecords import IHasBuildRecords
40from canonical.launchpad.interfaces.launchpad import UnexpectedFormData39from canonical.launchpad.interfaces.launchpad import UnexpectedFormData
41from lp.soyuz.interfaces.queue import PackageUploadStatus40from lp.soyuz.interfaces.queue import PackageUploadStatus
@@ -321,7 +320,7 @@
321320
322 prefetched_data = dict()321 prefetched_data = dict()
323 build_ids = [build.id for build in builds]322 build_ids = [build.id for build in builds]
324 results = getUtility(IBuildQueueSet).getForBuilds(build_ids)323 results = getUtility(IBuildSet).getQueueEntriesForBuildIDs(build_ids)
325 for (buildqueue, _builder, build_job) in results:324 for (buildqueue, _builder, build_job) in results:
326 # Get the build's id, 'buildqueue', 'sourcepackagerelease' and325 # Get the build's id, 'buildqueue', 'sourcepackagerelease' and
327 # 'buildlog' (from the result set) respectively.326 # 'buildlog' (from the result set) respectively.
328327
=== modified file 'lib/lp/soyuz/interfaces/build.py'
--- lib/lp/soyuz/interfaces/build.py 2010-03-25 14:38:25 +0000
+++ lib/lp/soyuz/interfaces/build.py 2010-04-03 04:16:21 +0000
@@ -311,6 +311,23 @@
311 build queue entry. If not found, return None.311 build queue entry. If not found, return None.
312 """312 """
313313
314 def getQueueEntriesForBuildIDs(build_ids):
315 """Return the IBuildQueue instances for the IBuild IDs at hand.
316
317 Retrieve the build queue and related builder rows associated with the
318 builds in question where they exist.
319 """
320
321 def calculateCandidates(archseries):
322 """Return the BuildQueue records for the given archseries's Builds.
323
324 Returns a selectRelease of BuildQueue items for sorted by descending
325 'lastscore' for Builds within the given archseries.
326
327 'archseries' argument should be a list of DistroArchSeries and it is
328 asserted to not be None/empty.
329 """
330
314331
315class IBuildRescoreForm(Interface):332class IBuildRescoreForm(Interface):
316 """Form for rescoring a build."""333 """Form for rescoring a build."""
317334
=== modified file 'lib/lp/soyuz/model/build.py'
--- lib/lp/soyuz/model/build.py 2010-03-06 04:57:40 +0000
+++ lib/lp/soyuz/model/build.py 2010-04-03 04:16:21 +0000
@@ -1062,3 +1062,44 @@
1062 BuildQueue.job == queue_entry.job)1062 BuildQueue.job == queue_entry.job)
10631063
1064 return result_set.one()1064 return result_set.one()
1065
1066 def getQueueEntriesForBuildIDs(self, build_ids):
1067 """See `IBuildSet`."""
1068 store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
1069
1070 origin = (
1071 BuildPackageJob,
1072 Join(BuildQueue, BuildPackageJob.job == BuildQueue.jobID),
1073 Join(Build, BuildPackageJob.build == Build.id),
1074 LeftJoin(
1075 Builder,
1076 BuildQueue.builderID == Builder.id),
1077 )
1078 result_set = store.using(*origin).find(
1079 (BuildQueue, Builder, BuildPackageJob),
1080 In(Build.id, build_ids))
1081
1082 return result_set
1083
1084 def calculateCandidates(self, archseries):
1085 """See `IBuildSet`."""
1086 if not archseries:
1087 raise AssertionError("Given 'archseries' cannot be None/empty.")
1088
1089 arch_ids = [d.id for d in archseries]
1090
1091 query = """
1092 Build.distroarchseries IN %s AND
1093 Build.buildstate = %s AND
1094 BuildQueue.job_type = %s AND
1095 BuildQueue.job = BuildPackageJob.job AND
1096 BuildPackageJob.build = build.id AND
1097 BuildQueue.builder IS NULL
1098 """ % sqlvalues(
1099 arch_ids, BuildStatus.NEEDSBUILD, BuildFarmJobType.PACKAGEBUILD)
1100
1101 candidates = BuildQueue.select(
1102 query, clauseTables=['Build', 'BuildPackageJob'],
1103 orderBy=['-BuildQueue.lastscore'])
1104
1105 return candidates
10651106
=== modified file 'lib/lp/soyuz/scripts/buildd.py'
--- lib/lp/soyuz/scripts/buildd.py 2010-04-01 04:18:57 +0000
+++ lib/lp/soyuz/scripts/buildd.py 2010-04-03 04:16:21 +0000
@@ -19,7 +19,6 @@
19from lp.archivepublisher.utils import process_in_batches19from lp.archivepublisher.utils import process_in_batches
20from lp.buildmaster.interfaces.buildbase import BuildStatus20from lp.buildmaster.interfaces.buildbase import BuildStatus
21from lp.buildmaster.interfaces.builder import IBuilderSet21from lp.buildmaster.interfaces.builder import IBuilderSet
22from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
23from lp.registry.interfaces.distribution import IDistributionSet22from lp.registry.interfaces.distribution import IDistributionSet
24from lp.registry.interfaces.series import SeriesStatus23from lp.registry.interfaces.series import SeriesStatus
25from lp.services.scripts.base import (24from lp.services.scripts.base import (
@@ -187,8 +186,7 @@
187 return186 return
188187
189 # Get the current build job candidates.188 # Get the current build job candidates.
190 bqset = getUtility(IBuildQueueSet)189 candidates = getUtility(IBuildSet).calculateCandidates(archseries)
191 candidates = bqset.calculateCandidates(archseries)
192190
193 self.logger.info("Found %d build in NEEDSBUILD state. Rescoring"191 self.logger.info("Found %d build in NEEDSBUILD state. Rescoring"
194 % candidates.count())192 % candidates.count())