Launchpad itself

Merge lp:~michael.nelson/launchpad/475808-unembargoing-oops into lp:launchpad/db-devel

  1. 475808-unembargoing-oops
  2. Merge into db-devel
Proposed by Michael Nelson
Status: Merged
Approved by: Aaron Bentley
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~michael.nelson/launchpad/475808-unembargoing-oops
Merge into: lp:launchpad/db-devel
Diff against target: 405 lines (+208/-63)
4 files modified
lib/lp/soyuz/interfaces/publishing.py (+28/-0)
lib/lp/soyuz/model/publishing.py (+29/-22)
lib/lp/soyuz/model/queue.py (+41/-6)
lib/lp/soyuz/scripts/tests/test_copypackage.py (+110/-35)
To merge this branch: bzr merge lp:~michael.nelson/launchpad/475808-unembargoing-oops
Reviewer Review Type Date Requested Status
Данило Шеган (community) release-critical Approve
Aaron Bentley (community) Approve
Review via email: mp+16141@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Michael Nelson (michael.nelson) wrote :

= Summary =

We need to check which component a delayed copy *will* be overridden to, when accepting it, rather than relying on the spr.component. This is a fix for bug 475808.

== Proposed fix ==

This branch ensures that when the component of a delayed_copy is checked before accepting it, the ancestry component is taken into account (as we can't actually do the component override until we publish the source package publishing history).

== Pre-implementation notes ==

See the pre-imp notes on bug 475808

== Implementation details ==

== Tests ==

bin/test -vvt DoDelayedCopyTestCase

== Demo and Q/A ==

We will need to Q/A this on dogfood.

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/soyuz/model/queue.py
  lib/lp/soyuz/scripts/tests/test_copypackage.py

== Pylint notices ==

lib/lp/soyuz/model/queue.py
    21: [F0401] Unable to import 'email.MIMEMultipart' (No module named MIMEMultipart)
    22: [F0401] Unable to import 'email.MIMEText' (No module named MIMEText)

--
Michael

Revision history for this message
Aaron Bentley (abentley) wrote :

Looks fine.

review: Approve
Revision history for this message
Данило Шеган (danilo) :
review: Approve (release-critical)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/soyuz/interfaces/publishing.py'
2--- lib/lp/soyuz/interfaces/publishing.py 2009-12-12 03:25:27 +0000
3+++ lib/lp/soyuz/interfaces/publishing.py 2009-12-21 09:19:13 +0000
4@@ -1139,6 +1139,34 @@
5 the source_package_pub, allowing the use of the cached results.
6 """
7
8+ def getNearestAncestor(
9+ package_name, archive, distroseries, pocket=None, status=None,
10+ binary=False):
11+ """Return the ancestor of the given parkage in a particular archive.
12+
13+ :param package_name: The package name for which we are checking for
14+ an ancestor.
15+ :type package_name: ``string``
16+ :param archive: The archive in which we are looking for an ancestor.
17+ :type archive: `IArchive`
18+ :param distroseries: The particular series in which we are looking for
19+ an ancestor.
20+ :type distroseries: `IDistroSeries`
21+ :param pocket: An optional pocket to restrict the search.
22+ :type pocket: `PackagePublishingPocket`.
23+ :param status: An optional status defaulting to PUBLISHED if not
24+ provided.
25+ :type status: `PackagePublishingStatus`
26+ :param binary: An optional argument to look for a binary ancestor
27+ instead of the default source.
28+ :type binary: ``Boolean``
29+
30+ :return: The most recent publishing history for the given
31+ arguments.
32+ :rtype: `ISourcePackagePublishingHistory` or
33+ `IBinaryPackagePublishingHistory` or None.
34+ """
35+
36 active_publishing_status = (
37 PackagePublishingStatus.PENDING,
38 PackagePublishingStatus.PUBLISHED,
39
40=== modified file 'lib/lp/soyuz/model/publishing.py'
41--- lib/lp/soyuz/model/publishing.py 2009-12-14 13:49:03 +0000
42+++ lib/lp/soyuz/model/publishing.py 2009-12-21 09:19:13 +0000
43@@ -835,17 +835,10 @@
44 archive = self.archive
45 if distroseries is None:
46 distroseries = self.distroseries
47- if status is None:
48- status = PackagePublishingStatus.PUBLISHED
49-
50- ancestries = archive.getPublishedSources(
51- name=self.source_package_name, exact_match=True,
52- status=status, distroseries=distroseries, pocket=pocket)
53-
54- if ancestries.count() > 0:
55- return ancestries[0]
56-
57- return None
58+
59+ return getUtility(IPublishingSet).getNearestAncestor(
60+ self.source_package_name, archive, distroseries, pocket,
61+ status)
62
63 def overrideFromAncestry(self):
64 """See `ISourcePackagePublishingHistory`."""
65@@ -1101,17 +1094,10 @@
66 archive = self.archive
67 if distroseries is None:
68 distroseries = self.distroarchseries.distroseries
69- if status is None:
70- status = PackagePublishingStatus.PUBLISHED
71-
72- ancestries = archive.getAllPublishedBinaries(
73- name=self.binary_package_name, exact_match=True, pocket=pocket,
74- status=status, distroarchseries=distroseries.architectures)
75-
76- if ancestries.count() > 0:
77- return ancestries[0]
78-
79- return None
80+
81+ return getUtility(IPublishingSet).getNearestAncestor(
82+ self.binary_package_name, archive, distroseries, pocket,
83+ status, binary=True)
84
85 def overrideFromAncestry(self):
86 """See `IBinaryPackagePublishingHistory`."""
87@@ -1699,3 +1685,24 @@
88 store.execute(query)
89
90 return sources + binary_packages
91+
92+ def getNearestAncestor(
93+ self, package_name, archive, distroseries, pocket=None,
94+ status=None, binary=False):
95+ """See `IPublishingSet`."""
96+ if status is None:
97+ status = PackagePublishingStatus.PUBLISHED
98+
99+ if binary:
100+ ancestries = archive.getAllPublishedBinaries(
101+ name=package_name, exact_match=True, pocket=pocket,
102+ status=status, distroarchseries=distroseries.architectures)
103+ else:
104+ ancestries = archive.getPublishedSources(
105+ name=package_name, exact_match=True, pocket=pocket,
106+ status=status, distroseries=distroseries)
107+
108+ if ancestries.count() > 0:
109+ return ancestries[0]
110+
111+ return None
112
113=== modified file 'lib/lp/soyuz/model/queue.py'
114--- lib/lp/soyuz/model/queue.py 2009-12-14 13:49:03 +0000
115+++ lib/lp/soyuz/model/queue.py 2009-12-21 09:19:13 +0000
116@@ -18,8 +18,8 @@
117 import StringIO
118 import tempfile
119
120-from email.MIMEMultipart import MIMEMultipart
121-from email.MIMEText import MIMEText
122+from email.mime.multipart import MIMEMultipart
123+from email.mime.text import MIMEText
124
125 from storm.locals import Desc, In, Join
126 from storm.store import Store
127@@ -546,7 +546,8 @@
128 # it's available.
129 changes_file = None
130 if ISourcePackagePublishingHistory.providedBy(pub_record):
131- changes_file = pub_record.sourcepackagerelease.package_upload.changesfile
132+ release = pub_record.sourcepackagerelease
133+ changes_file = release.package_upload.changesfile
134
135 for new_file in update_files_privacy(pub_record):
136 debug(logger,
137@@ -561,7 +562,8 @@
138 debug(
139 logger,
140 "sending email to %s" % self.distroseries.changeslist)
141- changes_file_object = StringIO.StringIO(changes_file.read())
142+ changes_file_object = StringIO.StringIO(
143+ changes_file.read())
144 self.notify(
145 announce_list=self.distroseries.changeslist,
146 changes_file_object=changes_file_object,
147@@ -1303,9 +1305,28 @@
148 def checkComponentAndSection(self):
149 """See `IPackageUploadBuild`."""
150 distroseries = self.packageupload.distroseries
151+ is_ppa = self.packageupload.archive.is_ppa
152+ is_delayed_copy = self.packageupload.is_delayed_copy
153+
154 for binary in self.build.binarypackages:
155- if (not self.packageupload.archive.is_ppa and
156- binary.component not in distroseries.upload_components):
157+ component = binary.component
158+
159+ if is_delayed_copy:
160+ # For a delayed copy the component will not yet have
161+ # had the chance to be overridden, so we'll check the value
162+ # that will be overridden by querying the ancestor in
163+ # the destination archive - if one is available.
164+ binary_name = binary.name
165+ ancestry = getUtility(IPublishingSet).getNearestAncestor(
166+ package_name=binary_name,
167+ archive=self.packageupload.archive,
168+ distroseries=self.packageupload.distroseries, binary=True)
169+
170+ if ancestry is not None:
171+ component = ancestry.component
172+
173+ if (not is_ppa and component not in
174+ distroseries.upload_components):
175 # Only complain about non-PPA uploads.
176 raise QueueBuildAcceptError(
177 'Component "%s" is not allowed in %s'
178@@ -1491,6 +1512,20 @@
179 component = self.sourcepackagerelease.component
180 section = self.sourcepackagerelease.section
181
182+ if self.packageupload.is_delayed_copy:
183+ # For a delayed copy the component will not yet have
184+ # had the chance to be overridden, so we'll check the value
185+ # that will be overridden by querying the ancestor in
186+ # the destination archive - if one is available.
187+ source_name = self.sourcepackagerelease.name
188+ ancestry = getUtility(IPublishingSet).getNearestAncestor(
189+ package_name=source_name,
190+ archive=self.packageupload.archive,
191+ distroseries=self.packageupload.distroseries)
192+
193+ if ancestry is not None:
194+ component = ancestry.component
195+
196 if (not self.packageupload.archive.is_ppa and
197 component not in distroseries.upload_components):
198 # Only complain about non-PPA uploads.
199
200=== modified file 'lib/lp/soyuz/scripts/tests/test_copypackage.py'
201--- lib/lp/soyuz/scripts/tests/test_copypackage.py 2009-12-14 13:49:03 +0000
202+++ lib/lp/soyuz/scripts/tests/test_copypackage.py 2009-12-21 09:19:13 +0000
203@@ -36,7 +36,8 @@
204 IBinaryPackagePublishingHistory, ISourcePackagePublishingHistory,
205 PackagePublishingStatus, active_publishing_status)
206 from lp.soyuz.interfaces.queue import (
207- PackageUploadCustomFormat, PackageUploadStatus)
208+ PackageUploadCustomFormat, PackageUploadStatus,
209+ QueueInconsistentStateError)
210 from lp.soyuz.interfaces.sourcepackageformat import (
211 ISourcePackageFormatSelectionSet, SourcePackageFormat)
212 from lp.soyuz.model.publishing import (
213@@ -937,14 +938,23 @@
214 super(DoDelayedCopyTestCase, self).setUp()
215 self.test_publisher = SoyuzTestPublisher()
216
217+ # Setup to copy into the main archive security pocket
218+ self.test_publisher.prepareBreezyAutotest()
219+ self.copy_archive = self.test_publisher.ubuntutest.main_archive
220+ self.copy_series = self.test_publisher.distroseries
221+ self.copy_pocket = PackagePublishingPocket.SECURITY
222+
223+ # Make ubuntutest/breezy-autotest CURRENT so uploads to SECURITY
224+ # pocket can be accepted.
225+ self.test_publisher.breezy_autotest.status = (
226+ DistroSeriesStatus.CURRENT)
227+
228 def createDelayedCopyContext(self):
229 """Create a context to allow delayed-copies test.
230
231 The returned source publication in a private archive with
232 binaries and a custom upload.
233 """
234- self.test_publisher.prepareBreezyAutotest()
235-
236 ppa = self.factory.makeArchive(
237 distribution=self.test_publisher.ubuntutest,
238 purpose=ArchivePurpose.PPA)
239@@ -959,34 +969,32 @@
240 build.package_upload.addCustom(
241 custom_file, PackageUploadCustomFormat.DIST_UPGRADER)
242
243+ # Commit for making the just-create library files available.
244+ self.layer.txn.commit()
245+
246 return source
247
248+ def do_delayed_copy(self, source):
249+ """Execute and return the delayed copy."""
250+
251+ self.layer.switchDbUser(self.dbuser)
252+
253+ delayed_copy = _do_delayed_copy(
254+ source, self.copy_archive, self.copy_series, self.copy_pocket,
255+ True)
256+
257+ self.layer.txn.commit()
258+ self.layer.switchDbUser('launchpad')
259+ return delayed_copy
260+
261 def test_do_delayed_copy_simple(self):
262 # _do_delayed_copy() return an `IPackageUpload` record configured
263 # as a delayed-copy and with the expected contents (source,
264 # binaries and custom uploads) in ACCEPTED state.
265-
266 source = self.createDelayedCopyContext()
267
268- # Make ubuntutest/breezy-autotest CURRENT so uploads to SECURITY
269- # pocket can be accepted.
270- self.test_publisher.breezy_autotest.status = (
271- DistroSeriesStatus.CURRENT)
272-
273 # Setup and execute the delayed copy procedure.
274- copy_archive = self.test_publisher.ubuntutest.main_archive
275- copy_series = source.distroseries
276- copy_pocket = PackagePublishingPocket.SECURITY
277-
278- # Commit for making the just-create library files available.
279- self.layer.txn.commit()
280- self.layer.switchDbUser(self.dbuser)
281-
282- delayed_copy = _do_delayed_copy(
283- source, copy_archive, copy_series, copy_pocket, True)
284-
285- self.layer.txn.commit()
286- self.layer.switchDbUser('launchpad')
287+ delayed_copy = self.do_delayed_copy(source)
288
289 # A delayed-copy `IPackageUpload` record is returned.
290 self.assertTrue(delayed_copy.is_delayed_copy)
291@@ -1001,9 +1009,82 @@
292 delayed_copy.displayname)
293
294 # It is targeted to the right publishing context.
295- self.assertEquals(copy_archive, delayed_copy.archive)
296- self.assertEquals(copy_series, delayed_copy.distroseries)
297- self.assertEquals(copy_pocket, delayed_copy.pocket)
298+ self.assertEquals(self.copy_archive, delayed_copy.archive)
299+ self.assertEquals(self.copy_series, delayed_copy.distroseries)
300+ self.assertEquals(self.copy_pocket, delayed_copy.pocket)
301+
302+ # And it contains the source, build and custom files.
303+ self.assertEquals(
304+ [source.sourcepackagerelease],
305+ [pus.sourcepackagerelease for pus in delayed_copy.sources])
306+
307+ [build] = source.getBuilds()
308+ self.assertEquals(
309+ [build],
310+ [pub.build for pub in delayed_copy.builds])
311+
312+ [custom_file] = [
313+ custom.libraryfilealias
314+ for custom in build.package_upload.customfiles]
315+ self.assertEquals(
316+ [custom_file],
317+ [custom.libraryfilealias for custom in delayed_copy.customfiles])
318+
319+ def test_do_delayed_copy_wrong_component_no_ancestry(self):
320+ """An original PPA upload for an invalid component will have been
321+ overridden when uploaded to the PPA, but when copying it to another
322+ archive, only the ancestry in the destination archive can be used.
323+ If that ancestry doesn't exist, an exception is raised."""
324+ # We'll simulate an upload that was overridden to main in the
325+ # ppa, by explicitly setting the spr's and bpr's component to
326+ # something else.
327+ source = self.createDelayedCopyContext()
328+ contrib = getUtility(IComponentSet).new('contrib')
329+ source.sourcepackagerelease.component = contrib
330+ [build] = source.getBuilds()
331+ [binary] = build.binarypackages
332+ binary.override(component=contrib)
333+ self.layer.txn.commit()
334+
335+ # Setup and execute the delayed copy procedure. This should
336+ # raise an exception, as it won't be able to find an ancestor
337+ # whose component can be used for overriding.
338+ do_delayed_copy_method = self.do_delayed_copy
339+ self.assertRaises(
340+ QueueInconsistentStateError, do_delayed_copy_method, source)
341+
342+ def test_do_delayed_copy_wrong_component_with_ancestry(self):
343+ """An original PPA upload for an invalid component will have been
344+ overridden when uploaded to the PPA, but when copying it to another
345+ archive, only the ancestry in the destination archive can be used.
346+ If an ancestor is found in the destination archive, its component
347+ is assumed for this package upload."""
348+ # We'll simulate an upload that was overridden to main in the
349+ # ppa, by explicitly setting the spr's and bpr's component to
350+ # something else.
351+ source = self.createDelayedCopyContext()
352+ contrib = getUtility(IComponentSet).new('contrib')
353+ source.sourcepackagerelease.component = contrib
354+ [build] = source.getBuilds()
355+ [binary] = build.binarypackages
356+ binary.override(component=contrib)
357+
358+ # This time, we'll ensure that there is already an ancestor for
359+ # foocom in the destination archive with binaries.
360+ ancestor = self.test_publisher.getPubSource(
361+ 'foocomm', '0.9', component='multiverse',
362+ archive=self.copy_archive,
363+ status=PackagePublishingStatus.PUBLISHED)
364+ ancestor_bins = self.test_publisher.getPubBinaries(
365+ binaryname='foo-bin', archive=self.copy_archive,
366+ status=PackagePublishingStatus.PUBLISHED, pub_source=ancestor)
367+ self.layer.txn.commit()
368+
369+ # Setup and execute the delayed copy procedure. This should
370+ # now result in an accepted delayed upload.
371+ delayed_copy = self.do_delayed_copy(source)
372+ self.assertEquals(
373+ PackageUploadStatus.ACCEPTED, delayed_copy.status)
374
375 # And it contains the source, build and custom files.
376 self.assertEquals(
377@@ -1055,6 +1136,9 @@
378 changes_file_name=changes_file_name)
379 package_upload.addBuild(build_i386)
380
381+ # Commit for making the just-create library files available.
382+ self.layer.txn.commit()
383+
384 return source
385
386 def test_do_delayed_copy_of_partially_built_sources(self):
387@@ -1062,17 +1146,8 @@
388 # the FULLYBUILT builds are copied.
389 source = self.createPartiallyBuiltDelayedCopyContext()
390
391- # Setup and execute the delayed copy procedure.
392- copy_archive = self.test_publisher.ubuntutest.main_archive
393- copy_series = source.distroseries
394- copy_pocket = PackagePublishingPocket.RELEASE
395-
396- # Make new libraryfiles available by committing the transaction.
397- self.layer.txn.commit()
398-
399 # Perform the delayed-copy including binaries.
400- delayed_copy = _do_delayed_copy(
401- source, copy_archive, copy_series, copy_pocket, True)
402+ delayed_copy = self.do_delayed_copy(source)
403
404 # Only the i386 build is included in the delayed-copy.
405 # For the record, later on, when the delayed-copy gets processed,

Subscribers

People subscribed via source and target branches

to status/vote changes: