Launchpad itself

Merge lp:~michael.nelson/launchpad/475808-unembargoing-oops into lp:launchpad/db-devel

  1. 475808-unembargoing-oops
  2. Merge into db-devel
Proposed by Michael Nelson
Status: Merged
Approved by: Aaron Bentley
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~michael.nelson/launchpad/475808-unembargoing-oops
Merge into: lp:launchpad/db-devel
Diff against target: 405 lines (+208/-63)
4 files modified
lib/lp/soyuz/interfaces/publishing.py (+28/-0)
lib/lp/soyuz/model/publishing.py (+29/-22)
lib/lp/soyuz/model/queue.py (+41/-6)
lib/lp/soyuz/scripts/tests/test_copypackage.py (+110/-35)
To merge this branch: bzr merge lp:~michael.nelson/launchpad/475808-unembargoing-oops
Reviewer Review Type Date Requested Status
Данило Шеган (community) release-critical Approve
Aaron Bentley (community) Approve
Review via email: mp+16141@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Michael Nelson (michael.nelson) wrote :

= Summary =

We need to check which component a delayed copy *will* be overridden to, when accepting it, rather than relying on the spr.component. This is a fix for bug 475808.

== Proposed fix ==

This branch ensures that when the component of a delayed_copy is checked before accepting it, the ancestry component is taken into account (as we can't actually do the component override until we publish the source package publishing history).

== Pre-implementation notes ==

See the pre-imp notes on bug 475808

== Implementation details ==

== Tests ==

bin/test -vvt DoDelayedCopyTestCase

== Demo and Q/A ==

We will need to Q/A this on dogfood.

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/soyuz/model/queue.py
  lib/lp/soyuz/scripts/tests/test_copypackage.py

== Pylint notices ==

lib/lp/soyuz/model/queue.py
    21: [F0401] Unable to import 'email.MIMEMultipart' (No module named MIMEMultipart)
    22: [F0401] Unable to import 'email.MIMEText' (No module named MIMEText)

--
Michael

Revision history for this message
Aaron Bentley (abentley) wrote :

Looks fine.

review: Approve
Revision history for this message
Данило Шеган (danilo) :
review: Approve (release-critical)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/soyuz/interfaces/publishing.py'
--- lib/lp/soyuz/interfaces/publishing.py 2009-12-12 03:25:27 +0000
+++ lib/lp/soyuz/interfaces/publishing.py 2009-12-21 09:19:13 +0000
@@ -1139,6 +1139,34 @@
1139 the source_package_pub, allowing the use of the cached results.1139 the source_package_pub, allowing the use of the cached results.
1140 """1140 """
11411141
1142 def getNearestAncestor(
1143 package_name, archive, distroseries, pocket=None, status=None,
1144 binary=False):
1145 """Return the ancestor of the given parkage in a particular archive.
1146
1147 :param package_name: The package name for which we are checking for
1148 an ancestor.
1149 :type package_name: ``string``
1150 :param archive: The archive in which we are looking for an ancestor.
1151 :type archive: `IArchive`
1152 :param distroseries: The particular series in which we are looking for
1153 an ancestor.
1154 :type distroseries: `IDistroSeries`
1155 :param pocket: An optional pocket to restrict the search.
1156 :type pocket: `PackagePublishingPocket`.
1157 :param status: An optional status defaulting to PUBLISHED if not
1158 provided.
1159 :type status: `PackagePublishingStatus`
1160 :param binary: An optional argument to look for a binary ancestor
1161 instead of the default source.
1162 :type binary: ``Boolean``
1163
1164 :return: The most recent publishing history for the given
1165 arguments.
1166 :rtype: `ISourcePackagePublishingHistory` or
1167 `IBinaryPackagePublishingHistory` or None.
1168 """
1169
1142active_publishing_status = (1170active_publishing_status = (
1143 PackagePublishingStatus.PENDING,1171 PackagePublishingStatus.PENDING,
1144 PackagePublishingStatus.PUBLISHED,1172 PackagePublishingStatus.PUBLISHED,
11451173
=== modified file 'lib/lp/soyuz/model/publishing.py'
--- lib/lp/soyuz/model/publishing.py 2009-12-14 13:49:03 +0000
+++ lib/lp/soyuz/model/publishing.py 2009-12-21 09:19:13 +0000
@@ -835,17 +835,10 @@
835 archive = self.archive835 archive = self.archive
836 if distroseries is None:836 if distroseries is None:
837 distroseries = self.distroseries837 distroseries = self.distroseries
838 if status is None:838
839 status = PackagePublishingStatus.PUBLISHED839 return getUtility(IPublishingSet).getNearestAncestor(
840840 self.source_package_name, archive, distroseries, pocket,
841 ancestries = archive.getPublishedSources(841 status)
842 name=self.source_package_name, exact_match=True,
843 status=status, distroseries=distroseries, pocket=pocket)
844
845 if ancestries.count() > 0:
846 return ancestries[0]
847
848 return None
849842
850 def overrideFromAncestry(self):843 def overrideFromAncestry(self):
851 """See `ISourcePackagePublishingHistory`."""844 """See `ISourcePackagePublishingHistory`."""
@@ -1101,17 +1094,10 @@
1101 archive = self.archive1094 archive = self.archive
1102 if distroseries is None:1095 if distroseries is None:
1103 distroseries = self.distroarchseries.distroseries1096 distroseries = self.distroarchseries.distroseries
1104 if status is None:1097
1105 status = PackagePublishingStatus.PUBLISHED1098 return getUtility(IPublishingSet).getNearestAncestor(
11061099 self.binary_package_name, archive, distroseries, pocket,
1107 ancestries = archive.getAllPublishedBinaries(1100 status, binary=True)
1108 name=self.binary_package_name, exact_match=True, pocket=pocket,
1109 status=status, distroarchseries=distroseries.architectures)
1110
1111 if ancestries.count() > 0:
1112 return ancestries[0]
1113
1114 return None
11151101
1116 def overrideFromAncestry(self):1102 def overrideFromAncestry(self):
1117 """See `IBinaryPackagePublishingHistory`."""1103 """See `IBinaryPackagePublishingHistory`."""
@@ -1699,3 +1685,24 @@
1699 store.execute(query)1685 store.execute(query)
17001686
1701 return sources + binary_packages1687 return sources + binary_packages
1688
1689 def getNearestAncestor(
1690 self, package_name, archive, distroseries, pocket=None,
1691 status=None, binary=False):
1692 """See `IPublishingSet`."""
1693 if status is None:
1694 status = PackagePublishingStatus.PUBLISHED
1695
1696 if binary:
1697 ancestries = archive.getAllPublishedBinaries(
1698 name=package_name, exact_match=True, pocket=pocket,
1699 status=status, distroarchseries=distroseries.architectures)
1700 else:
1701 ancestries = archive.getPublishedSources(
1702 name=package_name, exact_match=True, pocket=pocket,
1703 status=status, distroseries=distroseries)
1704
1705 if ancestries.count() > 0:
1706 return ancestries[0]
1707
1708 return None
17021709
=== modified file 'lib/lp/soyuz/model/queue.py'
--- lib/lp/soyuz/model/queue.py 2009-12-14 13:49:03 +0000
+++ lib/lp/soyuz/model/queue.py 2009-12-21 09:19:13 +0000
@@ -18,8 +18,8 @@
18import StringIO18import StringIO
19import tempfile19import tempfile
2020
21from email.MIMEMultipart import MIMEMultipart21from email.mime.multipart import MIMEMultipart
22from email.MIMEText import MIMEText22from email.mime.text import MIMEText
2323
24from storm.locals import Desc, In, Join24from storm.locals import Desc, In, Join
25from storm.store import Store25from storm.store import Store
@@ -546,7 +546,8 @@
546 # it's available.546 # it's available.
547 changes_file = None547 changes_file = None
548 if ISourcePackagePublishingHistory.providedBy(pub_record):548 if ISourcePackagePublishingHistory.providedBy(pub_record):
549 changes_file = pub_record.sourcepackagerelease.package_upload.changesfile549 release = pub_record.sourcepackagerelease
550 changes_file = release.package_upload.changesfile
550551
551 for new_file in update_files_privacy(pub_record):552 for new_file in update_files_privacy(pub_record):
552 debug(logger,553 debug(logger,
@@ -561,7 +562,8 @@
561 debug(562 debug(
562 logger,563 logger,
563 "sending email to %s" % self.distroseries.changeslist)564 "sending email to %s" % self.distroseries.changeslist)
564 changes_file_object = StringIO.StringIO(changes_file.read())565 changes_file_object = StringIO.StringIO(
566 changes_file.read())
565 self.notify(567 self.notify(
566 announce_list=self.distroseries.changeslist,568 announce_list=self.distroseries.changeslist,
567 changes_file_object=changes_file_object,569 changes_file_object=changes_file_object,
@@ -1303,9 +1305,28 @@
1303 def checkComponentAndSection(self):1305 def checkComponentAndSection(self):
1304 """See `IPackageUploadBuild`."""1306 """See `IPackageUploadBuild`."""
1305 distroseries = self.packageupload.distroseries1307 distroseries = self.packageupload.distroseries
1308 is_ppa = self.packageupload.archive.is_ppa
1309 is_delayed_copy = self.packageupload.is_delayed_copy
1310
1306 for binary in self.build.binarypackages:1311 for binary in self.build.binarypackages:
1307 if (not self.packageupload.archive.is_ppa and1312 component = binary.component
1308 binary.component not in distroseries.upload_components):1313
1314 if is_delayed_copy:
1315 # For a delayed copy the component will not yet have
1316 # had the chance to be overridden, so we'll check the value
1317 # that will be overridden by querying the ancestor in
1318 # the destination archive - if one is available.
1319 binary_name = binary.name
1320 ancestry = getUtility(IPublishingSet).getNearestAncestor(
1321 package_name=binary_name,
1322 archive=self.packageupload.archive,
1323 distroseries=self.packageupload.distroseries, binary=True)
1324
1325 if ancestry is not None:
1326 component = ancestry.component
1327
1328 if (not is_ppa and component not in
1329 distroseries.upload_components):
1309 # Only complain about non-PPA uploads.1330 # Only complain about non-PPA uploads.
1310 raise QueueBuildAcceptError(1331 raise QueueBuildAcceptError(
1311 'Component "%s" is not allowed in %s'1332 'Component "%s" is not allowed in %s'
@@ -1491,6 +1512,20 @@
1491 component = self.sourcepackagerelease.component1512 component = self.sourcepackagerelease.component
1492 section = self.sourcepackagerelease.section1513 section = self.sourcepackagerelease.section
14931514
1515 if self.packageupload.is_delayed_copy:
1516 # For a delayed copy the component will not yet have
1517 # had the chance to be overridden, so we'll check the value
1518 # that will be overridden by querying the ancestor in
1519 # the destination archive - if one is available.
1520 source_name = self.sourcepackagerelease.name
1521 ancestry = getUtility(IPublishingSet).getNearestAncestor(
1522 package_name=source_name,
1523 archive=self.packageupload.archive,
1524 distroseries=self.packageupload.distroseries)
1525
1526 if ancestry is not None:
1527 component = ancestry.component
1528
1494 if (not self.packageupload.archive.is_ppa and1529 if (not self.packageupload.archive.is_ppa and
1495 component not in distroseries.upload_components):1530 component not in distroseries.upload_components):
1496 # Only complain about non-PPA uploads.1531 # Only complain about non-PPA uploads.
14971532
=== modified file 'lib/lp/soyuz/scripts/tests/test_copypackage.py'
--- lib/lp/soyuz/scripts/tests/test_copypackage.py 2009-12-14 13:49:03 +0000
+++ lib/lp/soyuz/scripts/tests/test_copypackage.py 2009-12-21 09:19:13 +0000
@@ -36,7 +36,8 @@
36 IBinaryPackagePublishingHistory, ISourcePackagePublishingHistory,36 IBinaryPackagePublishingHistory, ISourcePackagePublishingHistory,
37 PackagePublishingStatus, active_publishing_status)37 PackagePublishingStatus, active_publishing_status)
38from lp.soyuz.interfaces.queue import (38from lp.soyuz.interfaces.queue import (
39 PackageUploadCustomFormat, PackageUploadStatus)39 PackageUploadCustomFormat, PackageUploadStatus,
40 QueueInconsistentStateError)
40from lp.soyuz.interfaces.sourcepackageformat import (41from lp.soyuz.interfaces.sourcepackageformat import (
41 ISourcePackageFormatSelectionSet, SourcePackageFormat)42 ISourcePackageFormatSelectionSet, SourcePackageFormat)
42from lp.soyuz.model.publishing import (43from lp.soyuz.model.publishing import (
@@ -937,14 +938,23 @@
937 super(DoDelayedCopyTestCase, self).setUp()938 super(DoDelayedCopyTestCase, self).setUp()
938 self.test_publisher = SoyuzTestPublisher()939 self.test_publisher = SoyuzTestPublisher()
939940
941 # Setup to copy into the main archive security pocket
942 self.test_publisher.prepareBreezyAutotest()
943 self.copy_archive = self.test_publisher.ubuntutest.main_archive
944 self.copy_series = self.test_publisher.distroseries
945 self.copy_pocket = PackagePublishingPocket.SECURITY
946
947 # Make ubuntutest/breezy-autotest CURRENT so uploads to SECURITY
948 # pocket can be accepted.
949 self.test_publisher.breezy_autotest.status = (
950 DistroSeriesStatus.CURRENT)
951
940 def createDelayedCopyContext(self):952 def createDelayedCopyContext(self):
941 """Create a context to allow delayed-copies test.953 """Create a context to allow delayed-copies test.
942954
943 The returned source publication in a private archive with955 The returned source publication in a private archive with
944 binaries and a custom upload.956 binaries and a custom upload.
945 """957 """
946 self.test_publisher.prepareBreezyAutotest()
947
948 ppa = self.factory.makeArchive(958 ppa = self.factory.makeArchive(
949 distribution=self.test_publisher.ubuntutest,959 distribution=self.test_publisher.ubuntutest,
950 purpose=ArchivePurpose.PPA)960 purpose=ArchivePurpose.PPA)
@@ -959,34 +969,32 @@
959 build.package_upload.addCustom(969 build.package_upload.addCustom(
960 custom_file, PackageUploadCustomFormat.DIST_UPGRADER)970 custom_file, PackageUploadCustomFormat.DIST_UPGRADER)
961971
972 # Commit for making the just-create library files available.
973 self.layer.txn.commit()
974
962 return source975 return source
963976
977 def do_delayed_copy(self, source):
978 """Execute and return the delayed copy."""
979
980 self.layer.switchDbUser(self.dbuser)
981
982 delayed_copy = _do_delayed_copy(
983 source, self.copy_archive, self.copy_series, self.copy_pocket,
984 True)
985
986 self.layer.txn.commit()
987 self.layer.switchDbUser('launchpad')
988 return delayed_copy
989
964 def test_do_delayed_copy_simple(self):990 def test_do_delayed_copy_simple(self):
965 # _do_delayed_copy() return an `IPackageUpload` record configured991 # _do_delayed_copy() return an `IPackageUpload` record configured
966 # as a delayed-copy and with the expected contents (source,992 # as a delayed-copy and with the expected contents (source,
967 # binaries and custom uploads) in ACCEPTED state.993 # binaries and custom uploads) in ACCEPTED state.
968
969 source = self.createDelayedCopyContext()994 source = self.createDelayedCopyContext()
970995
971 # Make ubuntutest/breezy-autotest CURRENT so uploads to SECURITY
972 # pocket can be accepted.
973 self.test_publisher.breezy_autotest.status = (
974 DistroSeriesStatus.CURRENT)
975
976 # Setup and execute the delayed copy procedure.996 # Setup and execute the delayed copy procedure.
977 copy_archive = self.test_publisher.ubuntutest.main_archive997 delayed_copy = self.do_delayed_copy(source)
978 copy_series = source.distroseries
979 copy_pocket = PackagePublishingPocket.SECURITY
980
981 # Commit for making the just-create library files available.
982 self.layer.txn.commit()
983 self.layer.switchDbUser(self.dbuser)
984
985 delayed_copy = _do_delayed_copy(
986 source, copy_archive, copy_series, copy_pocket, True)
987
988 self.layer.txn.commit()
989 self.layer.switchDbUser('launchpad')
990998
991 # A delayed-copy `IPackageUpload` record is returned.999 # A delayed-copy `IPackageUpload` record is returned.
992 self.assertTrue(delayed_copy.is_delayed_copy)1000 self.assertTrue(delayed_copy.is_delayed_copy)
@@ -1001,9 +1009,82 @@
1001 delayed_copy.displayname)1009 delayed_copy.displayname)
10021010
1003 # It is targeted to the right publishing context.1011 # It is targeted to the right publishing context.
1004 self.assertEquals(copy_archive, delayed_copy.archive)1012 self.assertEquals(self.copy_archive, delayed_copy.archive)
1005 self.assertEquals(copy_series, delayed_copy.distroseries)1013 self.assertEquals(self.copy_series, delayed_copy.distroseries)
1006 self.assertEquals(copy_pocket, delayed_copy.pocket)1014 self.assertEquals(self.copy_pocket, delayed_copy.pocket)
1015
1016 # And it contains the source, build and custom files.
1017 self.assertEquals(
1018 [source.sourcepackagerelease],
1019 [pus.sourcepackagerelease for pus in delayed_copy.sources])
1020
1021 [build] = source.getBuilds()
1022 self.assertEquals(
1023 [build],
1024 [pub.build for pub in delayed_copy.builds])
1025
1026 [custom_file] = [
1027 custom.libraryfilealias
1028 for custom in build.package_upload.customfiles]
1029 self.assertEquals(
1030 [custom_file],
1031 [custom.libraryfilealias for custom in delayed_copy.customfiles])
1032
1033 def test_do_delayed_copy_wrong_component_no_ancestry(self):
1034 """An original PPA upload for an invalid component will have been
1035 overridden when uploaded to the PPA, but when copying it to another
1036 archive, only the ancestry in the destination archive can be used.
1037 If that ancestry doesn't exist, an exception is raised."""
1038 # We'll simulate an upload that was overridden to main in the
1039 # ppa, by explicitly setting the spr's and bpr's component to
1040 # something else.
1041 source = self.createDelayedCopyContext()
1042 contrib = getUtility(IComponentSet).new('contrib')
1043 source.sourcepackagerelease.component = contrib
1044 [build] = source.getBuilds()
1045 [binary] = build.binarypackages
1046 binary.override(component=contrib)
1047 self.layer.txn.commit()
1048
1049 # Setup and execute the delayed copy procedure. This should
1050 # raise an exception, as it won't be able to find an ancestor
1051 # whose component can be used for overriding.
1052 do_delayed_copy_method = self.do_delayed_copy
1053 self.assertRaises(
1054 QueueInconsistentStateError, do_delayed_copy_method, source)
1055
1056 def test_do_delayed_copy_wrong_component_with_ancestry(self):
1057 """An original PPA upload for an invalid component will have been
1058 overridden when uploaded to the PPA, but when copying it to another
1059 archive, only the ancestry in the destination archive can be used.
1060 If an ancestor is found in the destination archive, its component
1061 is assumed for this package upload."""
1062 # We'll simulate an upload that was overridden to main in the
1063 # ppa, by explicitly setting the spr's and bpr's component to
1064 # something else.
1065 source = self.createDelayedCopyContext()
1066 contrib = getUtility(IComponentSet).new('contrib')
1067 source.sourcepackagerelease.component = contrib
1068 [build] = source.getBuilds()
1069 [binary] = build.binarypackages
1070 binary.override(component=contrib)
1071
1072 # This time, we'll ensure that there is already an ancestor for
1073 # foocom in the destination archive with binaries.
1074 ancestor = self.test_publisher.getPubSource(
1075 'foocomm', '0.9', component='multiverse',
1076 archive=self.copy_archive,
1077 status=PackagePublishingStatus.PUBLISHED)
1078 ancestor_bins = self.test_publisher.getPubBinaries(
1079 binaryname='foo-bin', archive=self.copy_archive,
1080 status=PackagePublishingStatus.PUBLISHED, pub_source=ancestor)
1081 self.layer.txn.commit()
1082
1083 # Setup and execute the delayed copy procedure. This should
1084 # now result in an accepted delayed upload.
1085 delayed_copy = self.do_delayed_copy(source)
1086 self.assertEquals(
1087 PackageUploadStatus.ACCEPTED, delayed_copy.status)
10071088
1008 # And it contains the source, build and custom files.1089 # And it contains the source, build and custom files.
1009 self.assertEquals(1090 self.assertEquals(
@@ -1055,6 +1136,9 @@
1055 changes_file_name=changes_file_name)1136 changes_file_name=changes_file_name)
1056 package_upload.addBuild(build_i386)1137 package_upload.addBuild(build_i386)
10571138
1139 # Commit for making the just-create library files available.
1140 self.layer.txn.commit()
1141
1058 return source1142 return source
10591143
1060 def test_do_delayed_copy_of_partially_built_sources(self):1144 def test_do_delayed_copy_of_partially_built_sources(self):
@@ -1062,17 +1146,8 @@
1062 # the FULLYBUILT builds are copied.1146 # the FULLYBUILT builds are copied.
1063 source = self.createPartiallyBuiltDelayedCopyContext()1147 source = self.createPartiallyBuiltDelayedCopyContext()
10641148
1065 # Setup and execute the delayed copy procedure.
1066 copy_archive = self.test_publisher.ubuntutest.main_archive
1067 copy_series = source.distroseries
1068 copy_pocket = PackagePublishingPocket.RELEASE
1069
1070 # Make new libraryfiles available by committing the transaction.
1071 self.layer.txn.commit()
1072
1073 # Perform the delayed-copy including binaries.1149 # Perform the delayed-copy including binaries.
1074 delayed_copy = _do_delayed_copy(1150 delayed_copy = self.do_delayed_copy(source)
1075 source, copy_archive, copy_series, copy_pocket, True)
10761151
1077 # Only the i386 build is included in the delayed-copy.1152 # Only the i386 build is included in the delayed-copy.
1078 # For the record, later on, when the delayed-copy gets processed,1153 # For the record, later on, when the delayed-copy gets processed,

Subscribers

People subscribed via source and target branches

to status/vote changes: