Launchpad itself

Merge lp:~wgrant/launchpad/sprb-new-model-columns into lp:launchpad/db-devel

  1. sprb-new-model-columns
  2. Merge into db-devel
Proposed by William Grant
Status: Merged
Approved by: Tim Penhey
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~wgrant/launchpad/sprb-new-model-columns
Merge into: lp:launchpad/db-devel
Prerequisite: lp:~wgrant/launchpad/sprbu-columns-to-sprb
Diff against target: 355 lines (+89/-61)
9 files modified
.bzrignore (+1/-0)
database/sampledata/current-dev.sql (+0/-7)
database/sampledata/current.sql (+0/-7)
database/schema/security.cfg (+0/-2)
lib/lp/buildmaster/model/buildbase.py (+29/-1)
lib/lp/code/model/sourcepackagerecipebuild.py (+13/-19)
lib/lp/soyuz/doc/build.txt (+1/-1)
lib/lp/soyuz/model/build.py (+1/-24)
lib/lp/soyuz/tests/test_build.py (+44/-0)
To merge this branch: bzr merge lp:~wgrant/launchpad/sprb-new-model-columns
Reviewer Review Type Date Requested Status
Julian Edwards (community) Approve
Review via email: mp+19172@code.launchpad.net

Commit message

Drop SourcePackageRecipeBuildUpload, move its columns to SourcePackageRecipeBuild, and make storeUploadLog and storeBuildInfo function fully for SPRBs.

To post a comment you must log in.
Revision history for this message
William Grant (wgrant) wrote :

This branch extends the prerequisite DB branch by adding the new dependencies, upload_log and pocket columns to the SourcePackageRecipeBuild model class. It also moves Build.storeUploadLog onto BuildBase, where it can be used by SPRB.

I had to alter BuildBase.storeBuildInfo to explicitly turn the dependencies str into a unicode, since plain Storm is less forgiving than its SQLObject compatibility layer. Since the method didn't appear to be tested anywhere, I've added some unit tests for the dependencies behaviour.

Revision history for this message
Julian Edwards (julian-edwards) wrote :

Looks good, thanks for this. Just a couple of comments:

1. s/exist/exists/ in the assertion text message.
2. You get brownie points if you can remove the commit from the tests - we found out that they slow the tests down massively so now I am on a jihad against them. Try using store.flush() or if that fails, try store.commit() as it's a bit closer to the metal than transaction.commit()

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2010-01-13 22:16:04 +0000
3+++ .bzrignore 2010-02-15 21:43:48 +0000
4@@ -58,3 +58,4 @@
5 .subversion
6 lib/canonical/buildd/launchpad-files
7 .testrepository
8+./pipes
9
10=== modified file 'database/sampledata/current-dev.sql'
11--- database/sampledata/current-dev.sql 2010-02-04 11:12:20 +0000
12+++ database/sampledata/current-dev.sql 2010-02-15 21:43:48 +0000
13@@ -9062,13 +9062,6 @@
14 ALTER TABLE sourcepackagerecipebuildjob ENABLE TRIGGER ALL;
15
16
17-ALTER TABLE sourcepackagerecipebuildupload DISABLE TRIGGER ALL;
18-
19-
20-
21-ALTER TABLE sourcepackagerecipebuildupload ENABLE TRIGGER ALL;
22-
23-
24 ALTER TABLE sourcepackagerecipedata DISABLE TRIGGER ALL;
25
26
27
28=== modified file 'database/sampledata/current.sql'
29--- database/sampledata/current.sql 2010-02-04 11:12:20 +0000
30+++ database/sampledata/current.sql 2010-02-15 21:43:48 +0000
31@@ -8948,13 +8948,6 @@
32 ALTER TABLE sourcepackagerecipebuildjob ENABLE TRIGGER ALL;
33
34
35-ALTER TABLE sourcepackagerecipebuildupload DISABLE TRIGGER ALL;
36-
37-
38-
39-ALTER TABLE sourcepackagerecipebuildupload ENABLE TRIGGER ALL;
40-
41-
42 ALTER TABLE sourcepackagerecipedata DISABLE TRIGGER ALL;
43
44
45
46=== modified file 'database/schema/security.cfg'
47--- database/schema/security.cfg 2010-02-15 13:33:06 +0000
48+++ database/schema/security.cfg 2010-02-15 21:43:48 +0000
49@@ -281,7 +281,6 @@
50 public.sourcepackagerecipe = SELECT, INSERT, UPDATE, DELETE
51 public.sourcepackagerecipebuild = SELECT, INSERT, UPDATE, DELETE
52 public.sourcepackagerecipebuildjob = SELECT, INSERT, UPDATE
53-public.sourcepackagerecipebuildupload = SELECT, INSERT, UPDATE, DELETE
54 public.sourcepackagerecipedata = SELECT, INSERT, UPDATE, DELETE
55 public.sourcepackagerecipedatainstruction = SELECT, INSERT, UPDATE, DELETE
56 public.specificationbranch = SELECT, INSERT, UPDATE, DELETE
57@@ -413,7 +412,6 @@
58 public.sprint = SELECT
59 public.sourcepackagereleasefile = SELECT
60 public.sourcepackagerecipebuild = SELECT
61-public.sourcepackagerecipebuildupload = SELECT
62 public.temporaryblobstorage = SELECT, DELETE
63 public.translationimportqueueentry = SELECT
64
65
66=== modified file 'lib/lp/buildmaster/model/buildbase.py'
67--- lib/lp/buildmaster/model/buildbase.py 2010-02-10 15:02:42 +0000
68+++ lib/lp/buildmaster/model/buildbase.py 2010-02-15 21:43:48 +0000
69@@ -14,14 +14,18 @@
70 import os
71 import pytz
72 import subprocess
73+from cStringIO import StringIO
74
75 from storm.store import Store
76+from zope.component import getUtility
77 from zope.security.proxy import removeSecurityProxy
78
79 from canonical.config import config
80 from canonical.database.constants import UTC_NOW
81 from canonical.database.sqlbase import (
82 clear_current_connection_cache, cursor, flush_database_updates)
83+from canonical.launchpad.helpers import filenameToContentType
84+from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
85 from canonical.librarian.utils import copy_and_close
86 from lp.registry.interfaces.pocket import pocketsuffix
87 from lp.soyuz.interfaces.build import BuildStatus
88@@ -354,7 +358,31 @@
89 # the time operations for duration.
90 RIGHT_NOW = datetime.datetime.now(pytz.timezone('UTC'))
91 self.buildduration = RIGHT_NOW - self.buildqueue_record.date_started
92- self.dependencies = slave_status.get('dependencies')
93+ if slave_status.get('dependencies') is not None:
94+ self.dependencies = unicode(slave_status.get('dependencies'))
95+ else:
96+ self.dependencies = None
97+
98+ def storeUploadLog(self, content):
99+ """See `IBuildBase`."""
100+ # The given content is stored in the librarian, restricted as
101+ # necessary according to the targeted archive's privacy. The content
102+ # object's 'upload_log' attribute will point to the
103+ # `LibrarianFileAlias`.
104+
105+ assert self.upload_log is None, (
106+ "Upload log information already exists and cannot be overridden.")
107+
108+ filename = 'upload_%s_log.txt' % self.id
109+ contentType = filenameToContentType(filename)
110+ file_size = len(content)
111+ file_content = StringIO(content)
112+ restricted = self.is_private
113+
114+ library_file = getUtility(ILibraryFileAliasSet).create(
115+ filename, file_size, file_content, contentType=contentType,
116+ restricted=restricted)
117+ self.upload_log = library_file
118
119 def queueBuild(self, suspended=False):
120 """See `IBuildBase`"""
121
122=== modified file 'lib/lp/code/model/sourcepackagerecipebuild.py'
123--- lib/lp/code/model/sourcepackagerecipebuild.py 2010-02-05 15:06:28 +0000
124+++ lib/lp/code/model/sourcepackagerecipebuild.py 2010-02-15 21:43:48 +0000
125@@ -12,10 +12,10 @@
126
127 from canonical.database.constants import UTC_NOW
128 from canonical.database.datetimecol import UtcDateTimeCol
129-from canonical.database.enumcol import EnumCol
130+from canonical.database.enumcol import DBEnum
131 from canonical.launchpad.interfaces.lpstorm import IMasterStore
132
133-from storm.locals import Int, Reference, Storm, TimeDelta
134+from storm.locals import Int, Reference, Storm, TimeDelta, Unicode
135 from storm.store import Store
136
137 from zope.component import getUtility
138@@ -62,8 +62,12 @@
139 buildlog_id = Int(name='build_log', allow_none=True)
140 buildlog = Reference(buildlog_id, 'LibraryFileAlias.id')
141
142- buildstate = EnumCol(
143- dbName='build_state', notNull=True, schema=BuildStatus)
144+ buildstate = DBEnum(enum=BuildStatus, name='build_state')
145+
146+ dependencies = Unicode(allow_none=True)
147+
148+ upload_log_id = Int(name='upload_log', allow_none=True)
149+ upload_log = Reference(upload_log_id, 'LibraryFileAlias.id')
150
151 @property
152 def current_component(self):
153@@ -76,9 +80,6 @@
154 distroseries_id = Int(name='distroseries', allow_none=True)
155 distroseries = Reference(distroseries_id, 'DistroSeries.id')
156
157- # XXX wgrant 2010-01-15 bug=507751: Need a DB field for this.
158- dependencies = None
159-
160 sourcepackagename_id = Int(name='sourcepackagename', allow_none=True)
161 sourcepackagename = Reference(
162 sourcepackagename_id, 'SourcePackageName.id')
163@@ -88,12 +89,7 @@
164 """See `IBuildBase`."""
165 return self.distroseries.distribution
166
167- @property
168- def pocket(self):
169- # XXX: JRV 2010-01-15 bug=507307: The database table really should
170- # have a pocket column, although this is not a big problem at the
171- # moment as recipe builds only happen for PPA's (so far).
172- return PackagePublishingPocket.RELEASE
173+ pocket = DBEnum(enum=PackagePublishingPocket)
174
175 recipe_id = Int(name='recipe', allow_none=False)
176 recipe = Reference(recipe_id, 'SourcePackageRecipe.id')
177@@ -112,13 +108,14 @@
178 return results.one()
179
180 def __init__(self, distroseries, sourcepackagename, recipe, requester,
181- archive, date_created=None, date_first_dispatched=None,
182+ archive, pocket, date_created=None, date_first_dispatched=None,
183 date_built=None, builder=None,
184 build_state=BuildStatus.NEEDSBUILD, build_log=None,
185 build_duration=None):
186 """Construct a SourcePackageRecipeBuild."""
187 super(SourcePackageRecipeBuild, self).__init__()
188 self.archive = archive
189+ self.pocket = pocket
190 self.buildduration = build_duration
191 self.buildlog = build_log
192 self.builder = builder
193@@ -133,6 +130,7 @@
194
195 @classmethod
196 def new(cls, sourcepackage, recipe, requester, archive,
197+ pocket=PackagePublishingPocket.RELEASE,
198 date_created=None):
199 """See `ISourcePackageRecipeBuildSource`."""
200 store = IMasterStore(SourcePackageRecipeBuild)
201@@ -144,6 +142,7 @@
202 recipe,
203 requester,
204 archive,
205+ pocket,
206 date_created=date_created)
207 store.add(spbuild)
208 return spbuild
209@@ -168,11 +167,6 @@
210 # XXX: wgrant 2010-01-19 bug=507764: Need proper implementation.
211 return datetime.timedelta(minutes=2)
212
213- def storeUploadLog(self, content):
214- """See `IBuildBase`."""
215- # XXX: wgrant 2010-01-20 bug=509892: Store in the DB.
216- return
217-
218 def notify(self, extra_info=None):
219 """See `IBuildBase`."""
220 # XXX: wgrant 2010-01-20 bug=509893: Implement this.
221
222=== modified file 'lib/lp/soyuz/doc/build.txt'
223--- lib/lp/soyuz/doc/build.txt 2010-02-15 12:59:55 +0000
224+++ lib/lp/soyuz/doc/build.txt 2010-02-15 21:43:48 +0000
225@@ -966,7 +966,7 @@
226 >>> failedtoupload_build.storeUploadLog('something')
227 Traceback (most recent call last):
228 ...
229- AssertionError: Upload log information already exist and cannot be
230+ AssertionError: Upload log information already exists and cannot be
231 overridden.
232
233 It's only possible to store another 'upload_log' content once the
234
235=== modified file 'lib/lp/soyuz/model/build.py'
236--- lib/lp/soyuz/model/build.py 2010-02-10 10:31:15 +0000
237+++ lib/lp/soyuz/model/build.py 2010-02-15 21:43:48 +0000
238@@ -7,7 +7,6 @@
239 __all__ = ['Build', 'BuildSet']
240
241 import apt_pkg
242-from cStringIO import StringIO
243 import datetime
244 import logging
245 import operator
246@@ -32,10 +31,9 @@
247 from canonical.launchpad.database.librarian import (
248 LibraryFileAlias, LibraryFileContent)
249 from canonical.launchpad.helpers import (
250- get_contact_email_addresses, filenameToContentType, get_email_template)
251+ get_contact_email_addresses, get_email_template)
252 from canonical.launchpad.interfaces.launchpad import (
253 NotFoundError, ILaunchpadCelebrities)
254-from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
255 from canonical.launchpad.mail import (
256 simple_sendmail, format_address)
257 from canonical.launchpad.webapp import canonical_url
258@@ -668,27 +666,6 @@
259 fromaddress, toaddress, subject, message,
260 headers=extra_headers)
261
262- def storeUploadLog(self, content):
263- """See `IBuildBase`."""
264- # The given content is stored in the librarian, restricted as
265- # necessary according to the targeted archive's privacy. The content
266- # object's 'upload_log' attribute will point to the
267- # `LibrarianFileAlias`.
268-
269- assert self.upload_log is None, (
270- "Upload log information already exist and cannot be overridden.")
271-
272- filename = 'upload_%s_log.txt' % self.id
273- contentType = filenameToContentType(filename)
274- file_size = len(content)
275- file_content = StringIO(content)
276- restricted = self.archive.private
277-
278- library_file = getUtility(ILibraryFileAliasSet).create(
279- filename, file_size, file_content, contentType=contentType,
280- restricted=restricted)
281- self.upload_log = library_file
282-
283 def _getDebByFileName(self, filename):
284 """Helper function to get a .deb LFA in the context of this build."""
285 store = Store.of(self)
286
287=== modified file 'lib/lp/soyuz/tests/test_build.py'
288--- lib/lp/soyuz/tests/test_build.py 2010-02-03 16:21:57 +0000
289+++ lib/lp/soyuz/tests/test_build.py 2010-02-15 21:43:48 +0000
290@@ -8,8 +8,10 @@
291 import unittest
292
293 from storm.store import Store
294+import transaction
295 from zope.component import getUtility
296
297+from canonical.database.constants import UTC_NOW
298 from canonical.testing import LaunchpadZopelessLayer
299 from lp.services.job.model.job import Job
300 from lp.buildmaster.interfaces.buildbase import IBuildBase
301@@ -21,6 +23,7 @@
302 from lp.soyuz.model.buildpackagejob import BuildPackageJob
303 from lp.soyuz.model.processor import ProcessorFamilySet
304 from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
305+from lp.soyuz.tests.soyuzbuilddhelpers import WaitingSlave
306 from lp.testing import TestCaseWithFactory
307
308
309@@ -248,5 +251,46 @@
310 self.assertContentEqual(builds, i386_builds)
311
312
313+class TestStoreBuildInfo(TestCaseWithFactory):
314+
315+ layer = LaunchpadZopelessLayer
316+
317+ def setUp(self):
318+ super(TestStoreBuildInfo, self).setUp()
319+ self.publisher = SoyuzTestPublisher()
320+ self.publisher.prepareBreezyAutotest()
321+
322+ gedit_src_hist = self.publisher.getPubSource(
323+ sourcename="gedit", status=PackagePublishingStatus.PUBLISHED)
324+ self.build = gedit_src_hist.createMissingBuilds()[0]
325+
326+ self.builder = self.factory.makeBuilder()
327+ self.builder.setSlaveForTesting(WaitingSlave('BuildStatus.OK'))
328+ self.build.buildqueue_record.builder = self.builder
329+ self.build.buildqueue_record.setDateStarted(UTC_NOW)
330+
331+ def testDependencies(self):
332+ """Verify that storeBuildInfo sets any dependencies."""
333+ self.build.storeBuildInfo(None, {'dependencies': 'somepackage'})
334+ self.assertIsNot(None, self.build.buildlog)
335+ self.assertEqual(self.builder, self.build.builder)
336+ self.assertEqual(u'somepackage', self.build.dependencies)
337+ self.assertIsNot(None, self.build.datebuilt)
338+ self.assertIsNot(None, self.build.buildduration)
339+
340+ def testWithoutDependencies(self):
341+ """Verify that storeBuildInfo clears the build's dependencies."""
342+ # Set something just to make sure that storeBuildInfo actually
343+ # empties it.
344+ self.build.dependencies = u'something'
345+
346+ self.build.storeBuildInfo(None, {})
347+ self.assertIsNot(None, self.build.buildlog)
348+ self.assertEqual(self.builder, self.build.builder)
349+ self.assertIs(None, self.build.dependencies)
350+ self.assertIsNot(None, self.build.datebuilt)
351+ self.assertIsNot(None, self.build.buildduration)
352+
353+
354 def test_suite():
355 return unittest.TestLoader().loadTestsFromName(__name__)

Subscribers

People subscribed via source and target branches

to status/vote changes: