Launchpad itself

Merge lp:~wgrant/launchpad/sprb-new-model-columns into lp:launchpad/db-devel

  1. sprb-new-model-columns
  2. Merge into db-devel
Proposed by William Grant
Status: Merged
Approved by: Tim Penhey
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~wgrant/launchpad/sprb-new-model-columns
Merge into: lp:launchpad/db-devel
Prerequisite: lp:~wgrant/launchpad/sprbu-columns-to-sprb
Diff against target: 355 lines (+89/-61)
9 files modified
.bzrignore (+1/-0)
database/sampledata/current-dev.sql (+0/-7)
database/sampledata/current.sql (+0/-7)
database/schema/security.cfg (+0/-2)
lib/lp/buildmaster/model/buildbase.py (+29/-1)
lib/lp/code/model/sourcepackagerecipebuild.py (+13/-19)
lib/lp/soyuz/doc/build.txt (+1/-1)
lib/lp/soyuz/model/build.py (+1/-24)
lib/lp/soyuz/tests/test_build.py (+44/-0)
To merge this branch: bzr merge lp:~wgrant/launchpad/sprb-new-model-columns
Reviewer Review Type Date Requested Status
Julian Edwards (community) Approve
Review via email: mp+19172@code.launchpad.net

Commit message

Drop SourcePackageRecipeBuildUpload, move its columns to SourcePackageRecipeBuild, and make storeUploadLog and storeBuildInfo function fully for SPRBs.

To post a comment you must log in.
Revision history for this message
William Grant (wgrant) wrote :

This branch extends the prerequisite DB branch by adding the new dependencies, upload_log and pocket columns to the SourcePackageRecipeBuild model class. It also moves Build.storeUploadLog onto BuildBase, where it can be used by SPRB.

I had to alter BuildBase.storeBuildInfo to explicitly turn the dependencies str into a unicode, since plain Storm is less forgiving than its SQLObject compatibility layer. Since the method didn't appear to be tested anywhere, I've added some unit tests for the dependencies behaviour.

Revision history for this message
Julian Edwards (julian-edwards) wrote :

Looks good, thanks for this. Just a couple of comments:

1. s/exist/exists/ in the assertion text message.
2. You get brownie points if you can remove the commit from the tests - we found out that they slow the tests down massively so now I am on a jihad against them. Try using store.flush() or if that fails, try store.commit() as it's a bit closer to the metal than transaction.commit()

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file '.bzrignore'
--- .bzrignore 2010-01-13 22:16:04 +0000
+++ .bzrignore 2010-02-15 21:43:48 +0000
@@ -58,3 +58,4 @@
58.subversion58.subversion
59lib/canonical/buildd/launchpad-files59lib/canonical/buildd/launchpad-files
60.testrepository60.testrepository
61./pipes
6162
=== modified file 'database/sampledata/current-dev.sql'
--- database/sampledata/current-dev.sql 2010-02-04 11:12:20 +0000
+++ database/sampledata/current-dev.sql 2010-02-15 21:43:48 +0000
@@ -9062,13 +9062,6 @@
9062ALTER TABLE sourcepackagerecipebuildjob ENABLE TRIGGER ALL;9062ALTER TABLE sourcepackagerecipebuildjob ENABLE TRIGGER ALL;
90639063
90649064
9065ALTER TABLE sourcepackagerecipebuildupload DISABLE TRIGGER ALL;
9066
9067
9068
9069ALTER TABLE sourcepackagerecipebuildupload ENABLE TRIGGER ALL;
9070
9071
9072ALTER TABLE sourcepackagerecipedata DISABLE TRIGGER ALL;9065ALTER TABLE sourcepackagerecipedata DISABLE TRIGGER ALL;
90739066
90749067
90759068
=== modified file 'database/sampledata/current.sql'
--- database/sampledata/current.sql 2010-02-04 11:12:20 +0000
+++ database/sampledata/current.sql 2010-02-15 21:43:48 +0000
@@ -8948,13 +8948,6 @@
8948ALTER TABLE sourcepackagerecipebuildjob ENABLE TRIGGER ALL;8948ALTER TABLE sourcepackagerecipebuildjob ENABLE TRIGGER ALL;
89498949
89508950
8951ALTER TABLE sourcepackagerecipebuildupload DISABLE TRIGGER ALL;
8952
8953
8954
8955ALTER TABLE sourcepackagerecipebuildupload ENABLE TRIGGER ALL;
8956
8957
8958ALTER TABLE sourcepackagerecipedata DISABLE TRIGGER ALL;8951ALTER TABLE sourcepackagerecipedata DISABLE TRIGGER ALL;
89598952
89608953
89618954
=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg 2010-02-15 13:33:06 +0000
+++ database/schema/security.cfg 2010-02-15 21:43:48 +0000
@@ -281,7 +281,6 @@
281public.sourcepackagerecipe = SELECT, INSERT, UPDATE, DELETE281public.sourcepackagerecipe = SELECT, INSERT, UPDATE, DELETE
282public.sourcepackagerecipebuild = SELECT, INSERT, UPDATE, DELETE282public.sourcepackagerecipebuild = SELECT, INSERT, UPDATE, DELETE
283public.sourcepackagerecipebuildjob = SELECT, INSERT, UPDATE283public.sourcepackagerecipebuildjob = SELECT, INSERT, UPDATE
284public.sourcepackagerecipebuildupload = SELECT, INSERT, UPDATE, DELETE
285public.sourcepackagerecipedata = SELECT, INSERT, UPDATE, DELETE284public.sourcepackagerecipedata = SELECT, INSERT, UPDATE, DELETE
286public.sourcepackagerecipedatainstruction = SELECT, INSERT, UPDATE, DELETE285public.sourcepackagerecipedatainstruction = SELECT, INSERT, UPDATE, DELETE
287public.specificationbranch = SELECT, INSERT, UPDATE, DELETE286public.specificationbranch = SELECT, INSERT, UPDATE, DELETE
@@ -413,7 +412,6 @@
413public.sprint = SELECT412public.sprint = SELECT
414public.sourcepackagereleasefile = SELECT413public.sourcepackagereleasefile = SELECT
415public.sourcepackagerecipebuild = SELECT414public.sourcepackagerecipebuild = SELECT
416public.sourcepackagerecipebuildupload = SELECT
417public.temporaryblobstorage = SELECT, DELETE415public.temporaryblobstorage = SELECT, DELETE
418public.translationimportqueueentry = SELECT416public.translationimportqueueentry = SELECT
419417
420418
=== modified file 'lib/lp/buildmaster/model/buildbase.py'
--- lib/lp/buildmaster/model/buildbase.py 2010-02-10 15:02:42 +0000
+++ lib/lp/buildmaster/model/buildbase.py 2010-02-15 21:43:48 +0000
@@ -14,14 +14,18 @@
14import os14import os
15import pytz15import pytz
16import subprocess16import subprocess
17from cStringIO import StringIO
1718
18from storm.store import Store19from storm.store import Store
20from zope.component import getUtility
19from zope.security.proxy import removeSecurityProxy21from zope.security.proxy import removeSecurityProxy
2022
21from canonical.config import config23from canonical.config import config
22from canonical.database.constants import UTC_NOW24from canonical.database.constants import UTC_NOW
23from canonical.database.sqlbase import (25from canonical.database.sqlbase import (
24 clear_current_connection_cache, cursor, flush_database_updates)26 clear_current_connection_cache, cursor, flush_database_updates)
27from canonical.launchpad.helpers import filenameToContentType
28from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
25from canonical.librarian.utils import copy_and_close29from canonical.librarian.utils import copy_and_close
26from lp.registry.interfaces.pocket import pocketsuffix30from lp.registry.interfaces.pocket import pocketsuffix
27from lp.soyuz.interfaces.build import BuildStatus31from lp.soyuz.interfaces.build import BuildStatus
@@ -354,7 +358,31 @@
354 # the time operations for duration.358 # the time operations for duration.
355 RIGHT_NOW = datetime.datetime.now(pytz.timezone('UTC'))359 RIGHT_NOW = datetime.datetime.now(pytz.timezone('UTC'))
356 self.buildduration = RIGHT_NOW - self.buildqueue_record.date_started360 self.buildduration = RIGHT_NOW - self.buildqueue_record.date_started
357 self.dependencies = slave_status.get('dependencies')361 if slave_status.get('dependencies') is not None:
362 self.dependencies = unicode(slave_status.get('dependencies'))
363 else:
364 self.dependencies = None
365
366 def storeUploadLog(self, content):
367 """See `IBuildBase`."""
368 # The given content is stored in the librarian, restricted as
369 # necessary according to the targeted archive's privacy. The content
370 # object's 'upload_log' attribute will point to the
371 # `LibrarianFileAlias`.
372
373 assert self.upload_log is None, (
374 "Upload log information already exists and cannot be overridden.")
375
376 filename = 'upload_%s_log.txt' % self.id
377 contentType = filenameToContentType(filename)
378 file_size = len(content)
379 file_content = StringIO(content)
380 restricted = self.is_private
381
382 library_file = getUtility(ILibraryFileAliasSet).create(
383 filename, file_size, file_content, contentType=contentType,
384 restricted=restricted)
385 self.upload_log = library_file
358386
359 def queueBuild(self, suspended=False):387 def queueBuild(self, suspended=False):
360 """See `IBuildBase`"""388 """See `IBuildBase`"""
361389
=== modified file 'lib/lp/code/model/sourcepackagerecipebuild.py'
--- lib/lp/code/model/sourcepackagerecipebuild.py 2010-02-05 15:06:28 +0000
+++ lib/lp/code/model/sourcepackagerecipebuild.py 2010-02-15 21:43:48 +0000
@@ -12,10 +12,10 @@
1212
13from canonical.database.constants import UTC_NOW13from canonical.database.constants import UTC_NOW
14from canonical.database.datetimecol import UtcDateTimeCol14from canonical.database.datetimecol import UtcDateTimeCol
15from canonical.database.enumcol import EnumCol15from canonical.database.enumcol import DBEnum
16from canonical.launchpad.interfaces.lpstorm import IMasterStore16from canonical.launchpad.interfaces.lpstorm import IMasterStore
1717
18from storm.locals import Int, Reference, Storm, TimeDelta18from storm.locals import Int, Reference, Storm, TimeDelta, Unicode
19from storm.store import Store19from storm.store import Store
2020
21from zope.component import getUtility21from zope.component import getUtility
@@ -62,8 +62,12 @@
62 buildlog_id = Int(name='build_log', allow_none=True)62 buildlog_id = Int(name='build_log', allow_none=True)
63 buildlog = Reference(buildlog_id, 'LibraryFileAlias.id')63 buildlog = Reference(buildlog_id, 'LibraryFileAlias.id')
6464
65 buildstate = EnumCol(65 buildstate = DBEnum(enum=BuildStatus, name='build_state')
66 dbName='build_state', notNull=True, schema=BuildStatus)66
67 dependencies = Unicode(allow_none=True)
68
69 upload_log_id = Int(name='upload_log', allow_none=True)
70 upload_log = Reference(upload_log_id, 'LibraryFileAlias.id')
6771
68 @property72 @property
69 def current_component(self):73 def current_component(self):
@@ -76,9 +80,6 @@
76 distroseries_id = Int(name='distroseries', allow_none=True)80 distroseries_id = Int(name='distroseries', allow_none=True)
77 distroseries = Reference(distroseries_id, 'DistroSeries.id')81 distroseries = Reference(distroseries_id, 'DistroSeries.id')
7882
79 # XXX wgrant 2010-01-15 bug=507751: Need a DB field for this.
80 dependencies = None
81
82 sourcepackagename_id = Int(name='sourcepackagename', allow_none=True)83 sourcepackagename_id = Int(name='sourcepackagename', allow_none=True)
83 sourcepackagename = Reference(84 sourcepackagename = Reference(
84 sourcepackagename_id, 'SourcePackageName.id')85 sourcepackagename_id, 'SourcePackageName.id')
@@ -88,12 +89,7 @@
88 """See `IBuildBase`."""89 """See `IBuildBase`."""
89 return self.distroseries.distribution90 return self.distroseries.distribution
9091
91 @property92 pocket = DBEnum(enum=PackagePublishingPocket)
92 def pocket(self):
93 # XXX: JRV 2010-01-15 bug=507307: The database table really should
94 # have a pocket column, although this is not a big problem at the
95 # moment as recipe builds only happen for PPA's (so far).
96 return PackagePublishingPocket.RELEASE
9793
98 recipe_id = Int(name='recipe', allow_none=False)94 recipe_id = Int(name='recipe', allow_none=False)
99 recipe = Reference(recipe_id, 'SourcePackageRecipe.id')95 recipe = Reference(recipe_id, 'SourcePackageRecipe.id')
@@ -112,13 +108,14 @@
112 return results.one()108 return results.one()
113109
114 def __init__(self, distroseries, sourcepackagename, recipe, requester,110 def __init__(self, distroseries, sourcepackagename, recipe, requester,
115 archive, date_created=None, date_first_dispatched=None,111 archive, pocket, date_created=None, date_first_dispatched=None,
116 date_built=None, builder=None,112 date_built=None, builder=None,
117 build_state=BuildStatus.NEEDSBUILD, build_log=None,113 build_state=BuildStatus.NEEDSBUILD, build_log=None,
118 build_duration=None):114 build_duration=None):
119 """Construct a SourcePackageRecipeBuild."""115 """Construct a SourcePackageRecipeBuild."""
120 super(SourcePackageRecipeBuild, self).__init__()116 super(SourcePackageRecipeBuild, self).__init__()
121 self.archive = archive117 self.archive = archive
118 self.pocket = pocket
122 self.buildduration = build_duration119 self.buildduration = build_duration
123 self.buildlog = build_log120 self.buildlog = build_log
124 self.builder = builder121 self.builder = builder
@@ -133,6 +130,7 @@
133130
134 @classmethod131 @classmethod
135 def new(cls, sourcepackage, recipe, requester, archive,132 def new(cls, sourcepackage, recipe, requester, archive,
133 pocket=PackagePublishingPocket.RELEASE,
136 date_created=None):134 date_created=None):
137 """See `ISourcePackageRecipeBuildSource`."""135 """See `ISourcePackageRecipeBuildSource`."""
138 store = IMasterStore(SourcePackageRecipeBuild)136 store = IMasterStore(SourcePackageRecipeBuild)
@@ -144,6 +142,7 @@
144 recipe,142 recipe,
145 requester,143 requester,
146 archive,144 archive,
145 pocket,
147 date_created=date_created)146 date_created=date_created)
148 store.add(spbuild)147 store.add(spbuild)
149 return spbuild148 return spbuild
@@ -168,11 +167,6 @@
168 # XXX: wgrant 2010-01-19 bug=507764: Need proper implementation.167 # XXX: wgrant 2010-01-19 bug=507764: Need proper implementation.
169 return datetime.timedelta(minutes=2)168 return datetime.timedelta(minutes=2)
170169
171 def storeUploadLog(self, content):
172 """See `IBuildBase`."""
173 # XXX: wgrant 2010-01-20 bug=509892: Store in the DB.
174 return
175
176 def notify(self, extra_info=None):170 def notify(self, extra_info=None):
177 """See `IBuildBase`."""171 """See `IBuildBase`."""
178 # XXX: wgrant 2010-01-20 bug=509893: Implement this.172 # XXX: wgrant 2010-01-20 bug=509893: Implement this.
179173
=== modified file 'lib/lp/soyuz/doc/build.txt'
--- lib/lp/soyuz/doc/build.txt 2010-02-15 12:59:55 +0000
+++ lib/lp/soyuz/doc/build.txt 2010-02-15 21:43:48 +0000
@@ -966,7 +966,7 @@
966 >>> failedtoupload_build.storeUploadLog('something')966 >>> failedtoupload_build.storeUploadLog('something')
967 Traceback (most recent call last):967 Traceback (most recent call last):
968 ...968 ...
969 AssertionError: Upload log information already exist and cannot be969 AssertionError: Upload log information already exists and cannot be
970 overridden.970 overridden.
971971
972It's only possible to store another 'upload_log' content once the972It's only possible to store another 'upload_log' content once the
973973
=== modified file 'lib/lp/soyuz/model/build.py'
--- lib/lp/soyuz/model/build.py 2010-02-10 10:31:15 +0000
+++ lib/lp/soyuz/model/build.py 2010-02-15 21:43:48 +0000
@@ -7,7 +7,6 @@
7__all__ = ['Build', 'BuildSet']7__all__ = ['Build', 'BuildSet']
88
9import apt_pkg9import apt_pkg
10from cStringIO import StringIO
11import datetime10import datetime
12import logging11import logging
13import operator12import operator
@@ -32,10 +31,9 @@
32from canonical.launchpad.database.librarian import (31from canonical.launchpad.database.librarian import (
33 LibraryFileAlias, LibraryFileContent)32 LibraryFileAlias, LibraryFileContent)
34from canonical.launchpad.helpers import (33from canonical.launchpad.helpers import (
35 get_contact_email_addresses, filenameToContentType, get_email_template)34 get_contact_email_addresses, get_email_template)
36from canonical.launchpad.interfaces.launchpad import (35from canonical.launchpad.interfaces.launchpad import (
37 NotFoundError, ILaunchpadCelebrities)36 NotFoundError, ILaunchpadCelebrities)
38from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
39from canonical.launchpad.mail import (37from canonical.launchpad.mail import (
40 simple_sendmail, format_address)38 simple_sendmail, format_address)
41from canonical.launchpad.webapp import canonical_url39from canonical.launchpad.webapp import canonical_url
@@ -668,27 +666,6 @@
668 fromaddress, toaddress, subject, message,666 fromaddress, toaddress, subject, message,
669 headers=extra_headers)667 headers=extra_headers)
670668
671 def storeUploadLog(self, content):
672 """See `IBuildBase`."""
673 # The given content is stored in the librarian, restricted as
674 # necessary according to the targeted archive's privacy. The content
675 # object's 'upload_log' attribute will point to the
676 # `LibrarianFileAlias`.
677
678 assert self.upload_log is None, (
679 "Upload log information already exist and cannot be overridden.")
680
681 filename = 'upload_%s_log.txt' % self.id
682 contentType = filenameToContentType(filename)
683 file_size = len(content)
684 file_content = StringIO(content)
685 restricted = self.archive.private
686
687 library_file = getUtility(ILibraryFileAliasSet).create(
688 filename, file_size, file_content, contentType=contentType,
689 restricted=restricted)
690 self.upload_log = library_file
691
692 def _getDebByFileName(self, filename):669 def _getDebByFileName(self, filename):
693 """Helper function to get a .deb LFA in the context of this build."""670 """Helper function to get a .deb LFA in the context of this build."""
694 store = Store.of(self)671 store = Store.of(self)
695672
=== modified file 'lib/lp/soyuz/tests/test_build.py'
--- lib/lp/soyuz/tests/test_build.py 2010-02-03 16:21:57 +0000
+++ lib/lp/soyuz/tests/test_build.py 2010-02-15 21:43:48 +0000
@@ -8,8 +8,10 @@
8import unittest8import unittest
99
10from storm.store import Store10from storm.store import Store
11import transaction
11from zope.component import getUtility12from zope.component import getUtility
1213
14from canonical.database.constants import UTC_NOW
13from canonical.testing import LaunchpadZopelessLayer15from canonical.testing import LaunchpadZopelessLayer
14from lp.services.job.model.job import Job16from lp.services.job.model.job import Job
15from lp.buildmaster.interfaces.buildbase import IBuildBase17from lp.buildmaster.interfaces.buildbase import IBuildBase
@@ -21,6 +23,7 @@
21from lp.soyuz.model.buildpackagejob import BuildPackageJob23from lp.soyuz.model.buildpackagejob import BuildPackageJob
22from lp.soyuz.model.processor import ProcessorFamilySet24from lp.soyuz.model.processor import ProcessorFamilySet
23from lp.soyuz.tests.test_publishing import SoyuzTestPublisher25from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
26from lp.soyuz.tests.soyuzbuilddhelpers import WaitingSlave
24from lp.testing import TestCaseWithFactory27from lp.testing import TestCaseWithFactory
2528
2629
@@ -248,5 +251,46 @@
248 self.assertContentEqual(builds, i386_builds)251 self.assertContentEqual(builds, i386_builds)
249252
250253
254class TestStoreBuildInfo(TestCaseWithFactory):
255
256 layer = LaunchpadZopelessLayer
257
258 def setUp(self):
259 super(TestStoreBuildInfo, self).setUp()
260 self.publisher = SoyuzTestPublisher()
261 self.publisher.prepareBreezyAutotest()
262
263 gedit_src_hist = self.publisher.getPubSource(
264 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED)
265 self.build = gedit_src_hist.createMissingBuilds()[0]
266
267 self.builder = self.factory.makeBuilder()
268 self.builder.setSlaveForTesting(WaitingSlave('BuildStatus.OK'))
269 self.build.buildqueue_record.builder = self.builder
270 self.build.buildqueue_record.setDateStarted(UTC_NOW)
271
272 def testDependencies(self):
273 """Verify that storeBuildInfo sets any dependencies."""
274 self.build.storeBuildInfo(None, {'dependencies': 'somepackage'})
275 self.assertIsNot(None, self.build.buildlog)
276 self.assertEqual(self.builder, self.build.builder)
277 self.assertEqual(u'somepackage', self.build.dependencies)
278 self.assertIsNot(None, self.build.datebuilt)
279 self.assertIsNot(None, self.build.buildduration)
280
281 def testWithoutDependencies(self):
282 """Verify that storeBuildInfo clears the build's dependencies."""
283 # Set something just to make sure that storeBuildInfo actually
284 # empties it.
285 self.build.dependencies = u'something'
286
287 self.build.storeBuildInfo(None, {})
288 self.assertIsNot(None, self.build.buildlog)
289 self.assertEqual(self.builder, self.build.builder)
290 self.assertIs(None, self.build.dependencies)
291 self.assertIsNot(None, self.build.datebuilt)
292 self.assertIsNot(None, self.build.buildduration)
293
294
251def test_suite():295def test_suite():
252 return unittest.TestLoader().loadTestsFromName(__name__)296 return unittest.TestLoader().loadTestsFromName(__name__)

Subscribers

People subscribed via source and target branches

to status/vote changes: