Launchpad itself

Merge lp:~wgrant/launchpad/more-buildmaster-cleanup into lp:launchpad

  1. more-buildmaster-cleanup
  2. Merge into devel
Proposed by William Grant
Status: Merged
Approved by: Michael Hudson-Doyle
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~wgrant/launchpad/more-buildmaster-cleanup
Merge into: lp:launchpad
Prerequisite: lp:~wgrant/launchpad/amputate-buildergroup
Diff against target: 270 lines (+66/-76)
7 files modified
lib/lp/buildmaster/doc/buildqueue.txt (+5/-5)
lib/lp/buildmaster/interfaces/buildqueue.py (+0/-17)
lib/lp/buildmaster/model/buildqueue.py (+0/-48)
lib/lp/soyuz/browser/build.py (+2/-3)
lib/lp/soyuz/interfaces/build.py (+17/-0)
lib/lp/soyuz/model/build.py (+41/-0)
lib/lp/soyuz/scripts/buildd.py (+1/-3)
To merge this branch: bzr merge lp:~wgrant/launchpad/more-buildmaster-cleanup
Reviewer Review Type Date Requested Status
Michael Nelson (community) code Approve
Aaron Bentley (community) Needs Information
Review via email: mp+22735@code.launchpad.net

Commit message

Finish the BuildQueue generalisation by moving the last two Soyuz-specific methods from BuildQueueSet to BuildSet.

Description of the change

This branch moves the last two Soyuz-specific methods from BuildQueueSet to BuildSet, completing the BuildQueue generalisation.

To post a comment you must log in.
Revision history for this message
Aaron Bentley (abentley) wrote :

This looks reasonable. Was there a preimplementation call? Was there lint?

review: Needs Information
Revision history for this message
Michael Nelson (michael.nelson) wrote :

Thanks William. As noted, please verify/clean any lint, but otherwise r=me.

review: Approve (code)
Revision history for this message
William Grant (wgrant) wrote :

There is no lint, beyond the external import complaints that appear to have become accepted fixtures.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/buildmaster/doc/buildqueue.txt'
2--- lib/lp/buildmaster/doc/buildqueue.txt 2010-03-05 13:52:32 +0000
3+++ lib/lp/buildmaster/doc/buildqueue.txt 2010-04-03 04:16:21 +0000
4@@ -255,7 +255,7 @@
5 Note that the results are ordered by descending BuildQueue.lastscore
6 and restricted
7
8- >>> for bq in bqset.calculateCandidates(archseries):
9+ >>> for bq in getUtility(IBuildSet).calculateCandidates(archseries):
10 ... build = getUtility(IBuildSet).getByQueueEntry(bq)
11 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)
12 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)
13@@ -287,7 +287,7 @@
14 The fake language pack will be queued behind the other source packages
15 as intended.
16
17- >>> for bq in bqset.calculateCandidates(archseries):
18+ >>> for bq in getUtility(IBuildSet).calculateCandidates(archseries):
19 ... build = getUtility(IBuildSet).getByQueueEntry(bq)
20 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)
21 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)
22@@ -299,7 +299,7 @@
23 Restricting the domain to only hoary/hppa:
24
25 >>> archseries = [hoary['hppa']]
26- >>> for bq in bqset.calculateCandidates(archseries):
27+ >>> for bq in getUtility(IBuildSet).calculateCandidates(archseries):
28 ... build = getUtility(IBuildSet).getByQueueEntry(bq)
29 ... print "%s (%s, %d)" % (build.title, bq.lastscore, bq.id)
30 hppa build of pmount 0.1-2 in ubuntu hoary RELEASE (1500, 4)
31@@ -308,12 +308,12 @@
32 This method asserts a valid 'archseries' argument, None or an empty
33 list will result in a AssertionFailure.
34
35- >>> bqset.calculateCandidates(None).count()
36+ >>> getUtility(IBuildSet).calculateCandidates(None).count()
37 Traceback (most recent call last):
38 ...
39 AssertionError: Given 'archseries' cannot be None/empty.
40
41- >>> bqset.calculateCandidates([]).count()
42+ >>> getUtility(IBuildSet).calculateCandidates([]).count()
43 Traceback (most recent call last):
44 ...
45 AssertionError: Given 'archseries' cannot be None/empty.
46
47=== modified file 'lib/lp/buildmaster/interfaces/buildqueue.py'
48--- lib/lp/buildmaster/interfaces/buildqueue.py 2010-03-16 05:08:47 +0000
49+++ lib/lp/buildmaster/interfaces/buildqueue.py 2010-04-03 04:16:21 +0000
50@@ -148,20 +148,3 @@
51
52 def getActiveBuildJobs():
53 """Return All active Build Jobs."""
54-
55- def calculateCandidates(archseries):
56- """Return the BuildQueue records for the given archseries.
57-
58- Returns a selectRelease of BuildQueue items sorted by descending
59- 'lastscore' within the given archseries.
60-
61- 'archseries' argument should be a list of DistroArchSeries and it is
62- asserted to not be None/empty.
63- """
64-
65- def getForBuilds(build_ids):
66- """Return the IBuildQueue instance for the IBuild IDs at hand.
67-
68- Retrieve the build queue and related builder rows associated with the
69- builds in question where they exist.
70- """
71
72=== modified file 'lib/lp/buildmaster/model/buildqueue.py'
73--- lib/lp/buildmaster/model/buildqueue.py 2010-03-17 05:58:22 +0000
74+++ lib/lp/buildmaster/model/buildqueue.py 2010-04-03 04:16:21 +0000
75@@ -18,7 +18,6 @@
76
77 from sqlobject import (
78 StringCol, ForeignKey, BoolCol, IntCol, IntervalCol, SQLObjectNotFound)
79-from storm.expr import In, Join, LeftJoin
80 from storm.store import Store
81 from zope.component import getSiteManager, getUtility
82 from zope.interface import implements
83@@ -27,7 +26,6 @@
84 from canonical.database.sqlbase import SQLBase, sqlvalues
85 from canonical.launchpad.webapp.interfaces import (
86 DEFAULT_FLAVOR, IStoreSelector, MAIN_STORE, NotFoundError)
87-from lp.buildmaster.interfaces.buildbase import BuildStatus
88 from lp.buildmaster.interfaces.buildfarmjob import (
89 BuildFarmJobType, IBuildFarmJob)
90 from lp.buildmaster.interfaces.buildfarmjobbehavior import (
91@@ -35,7 +33,6 @@
92 from lp.buildmaster.interfaces.buildqueue import IBuildQueue, IBuildQueueSet
93 from lp.services.job.interfaces.job import JobStatus
94 from lp.services.job.model.job import Job
95-from lp.soyuz.model.buildpackagejob import BuildPackageJob
96
97
98 def normalize_virtualization(virtualized):
99@@ -526,48 +523,3 @@
100 Job._status == JobStatus.RUNNING,
101 Job.date_started != None)
102 return result_set
103-
104- def calculateCandidates(self, archseries):
105- """See `IBuildQueueSet`."""
106- if not archseries:
107- raise AssertionError("Given 'archseries' cannot be None/empty.")
108-
109- arch_ids = [d.id for d in archseries]
110-
111- query = """
112- Build.distroarchseries IN %s AND
113- Build.buildstate = %s AND
114- BuildQueue.job_type = %s AND
115- BuildQueue.job = BuildPackageJob.job AND
116- BuildPackageJob.build = build.id AND
117- BuildQueue.builder IS NULL
118- """ % sqlvalues(
119- arch_ids, BuildStatus.NEEDSBUILD, BuildFarmJobType.PACKAGEBUILD)
120-
121- candidates = BuildQueue.select(
122- query, clauseTables=['Build', 'BuildPackageJob'],
123- orderBy=['-BuildQueue.lastscore'])
124-
125- return candidates
126-
127- def getForBuilds(self, build_ids):
128- """See `IBuildQueueSet`."""
129- # Avoid circular import problem.
130- from lp.soyuz.model.build import Build
131- from lp.buildmaster.model.builder import Builder
132-
133- store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
134-
135- origin = (
136- BuildPackageJob,
137- Join(BuildQueue, BuildPackageJob.job == BuildQueue.jobID),
138- Join(Build, BuildPackageJob.build == Build.id),
139- LeftJoin(
140- Builder,
141- BuildQueue.builderID == Builder.id),
142- )
143- result_set = store.using(*origin).find(
144- (BuildQueue, Builder, BuildPackageJob),
145- In(Build.id, build_ids))
146-
147- return result_set
148
149=== modified file 'lib/lp/soyuz/browser/build.py'
150--- lib/lp/soyuz/browser/build.py 2010-03-16 05:09:55 +0000
151+++ lib/lp/soyuz/browser/build.py 2010-04-03 04:16:21 +0000
152@@ -33,9 +33,8 @@
153 from canonical.launchpad.webapp.interfaces import ICanonicalUrlData
154 from canonical.lazr.utils import safe_hasattr
155 from lp.buildmaster.interfaces.buildbase import BuildStatus
156-from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
157 from lp.services.job.interfaces.job import JobStatus
158-from lp.soyuz.interfaces.build import IBuild, IBuildRescoreForm
159+from lp.soyuz.interfaces.build import IBuild, IBuildRescoreForm, IBuildSet
160 from lp.soyuz.interfaces.buildrecords import IHasBuildRecords
161 from canonical.launchpad.interfaces.launchpad import UnexpectedFormData
162 from lp.soyuz.interfaces.queue import PackageUploadStatus
163@@ -321,7 +320,7 @@
164
165 prefetched_data = dict()
166 build_ids = [build.id for build in builds]
167- results = getUtility(IBuildQueueSet).getForBuilds(build_ids)
168+ results = getUtility(IBuildSet).getQueueEntriesForBuildIDs(build_ids)
169 for (buildqueue, _builder, build_job) in results:
170 # Get the build's id, 'buildqueue', 'sourcepackagerelease' and
171 # 'buildlog' (from the result set) respectively.
172
173=== modified file 'lib/lp/soyuz/interfaces/build.py'
174--- lib/lp/soyuz/interfaces/build.py 2010-03-25 14:38:25 +0000
175+++ lib/lp/soyuz/interfaces/build.py 2010-04-03 04:16:21 +0000
176@@ -311,6 +311,23 @@
177 build queue entry. If not found, return None.
178 """
179
180+ def getQueueEntriesForBuildIDs(build_ids):
181+ """Return the IBuildQueue instances for the IBuild IDs at hand.
182+
183+ Retrieve the build queue and related builder rows associated with the
184+ builds in question where they exist.
185+ """
186+
187+ def calculateCandidates(archseries):
188+ """Return the BuildQueue records for the given archseries's Builds.
189+
190+ Returns a selectRelease of BuildQueue items for sorted by descending
191+ 'lastscore' for Builds within the given archseries.
192+
193+ 'archseries' argument should be a list of DistroArchSeries and it is
194+ asserted to not be None/empty.
195+ """
196+
197
198 class IBuildRescoreForm(Interface):
199 """Form for rescoring a build."""
200
201=== modified file 'lib/lp/soyuz/model/build.py'
202--- lib/lp/soyuz/model/build.py 2010-03-06 04:57:40 +0000
203+++ lib/lp/soyuz/model/build.py 2010-04-03 04:16:21 +0000
204@@ -1062,3 +1062,44 @@
205 BuildQueue.job == queue_entry.job)
206
207 return result_set.one()
208+
209+ def getQueueEntriesForBuildIDs(self, build_ids):
210+ """See `IBuildSet`."""
211+ store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
212+
213+ origin = (
214+ BuildPackageJob,
215+ Join(BuildQueue, BuildPackageJob.job == BuildQueue.jobID),
216+ Join(Build, BuildPackageJob.build == Build.id),
217+ LeftJoin(
218+ Builder,
219+ BuildQueue.builderID == Builder.id),
220+ )
221+ result_set = store.using(*origin).find(
222+ (BuildQueue, Builder, BuildPackageJob),
223+ In(Build.id, build_ids))
224+
225+ return result_set
226+
227+ def calculateCandidates(self, archseries):
228+ """See `IBuildSet`."""
229+ if not archseries:
230+ raise AssertionError("Given 'archseries' cannot be None/empty.")
231+
232+ arch_ids = [d.id for d in archseries]
233+
234+ query = """
235+ Build.distroarchseries IN %s AND
236+ Build.buildstate = %s AND
237+ BuildQueue.job_type = %s AND
238+ BuildQueue.job = BuildPackageJob.job AND
239+ BuildPackageJob.build = build.id AND
240+ BuildQueue.builder IS NULL
241+ """ % sqlvalues(
242+ arch_ids, BuildStatus.NEEDSBUILD, BuildFarmJobType.PACKAGEBUILD)
243+
244+ candidates = BuildQueue.select(
245+ query, clauseTables=['Build', 'BuildPackageJob'],
246+ orderBy=['-BuildQueue.lastscore'])
247+
248+ return candidates
249
250=== modified file 'lib/lp/soyuz/scripts/buildd.py'
251--- lib/lp/soyuz/scripts/buildd.py 2010-04-01 04:18:57 +0000
252+++ lib/lp/soyuz/scripts/buildd.py 2010-04-03 04:16:21 +0000
253@@ -19,7 +19,6 @@
254 from lp.archivepublisher.utils import process_in_batches
255 from lp.buildmaster.interfaces.buildbase import BuildStatus
256 from lp.buildmaster.interfaces.builder import IBuilderSet
257-from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
258 from lp.registry.interfaces.distribution import IDistributionSet
259 from lp.registry.interfaces.series import SeriesStatus
260 from lp.services.scripts.base import (
261@@ -187,8 +186,7 @@
262 return
263
264 # Get the current build job candidates.
265- bqset = getUtility(IBuildQueueSet)
266- candidates = bqset.calculateCandidates(archseries)
267+ candidates = getUtility(IBuildSet).calculateCandidates(archseries)
268
269 self.logger.info("Found %d build in NEEDSBUILD state. Rescoring"
270 % candidates.count())