Merge lp:~wgrant/launchpad/link-uploaded-ddebs into lp:launchpad/db-devel
- link-uploaded-ddebs
- Merge into db-devel
Status: | Merged |
---|---|
Approved by: | Jelmer Vernooij |
Approved revision: | no longer in the source branch. |
Merged at revision: | 9567 |
Proposed branch: | lp:~wgrant/launchpad/link-uploaded-ddebs |
Merge into: | lp:launchpad/db-devel |
Diff against target: |
674 lines (+269/-81) 16 files modified
database/schema/comments.sql (+1/-0) database/schema/patch-2207-73-0.sql (+10/-0) lib/lp/archiveuploader/nascentupload.py (+46/-3) lib/lp/archiveuploader/nascentuploadfile.py (+11/-19) lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.dsc (+13/-0) lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_i386.changes (+31/-0) lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_source.changes (+26/-0) lib/lp/archiveuploader/tests/nascentupload-ddebs.txt (+80/-41) lib/lp/soyuz/doc/archive-files.txt (+8/-0) lib/lp/soyuz/doc/binarypackagebuild.txt (+3/-2) lib/lp/soyuz/doc/distroarchseriesbinarypackage.txt (+4/-2) lib/lp/soyuz/interfaces/binarypackagebuild.py (+9/-1) lib/lp/soyuz/interfaces/binarypackagerelease.py (+5/-1) lib/lp/soyuz/model/binarypackagebuild.py (+17/-10) lib/lp/soyuz/model/binarypackagerelease.py (+2/-0) lib/lp/soyuz/tests/test_publishing.py (+3/-2) |
To merge this branch: | bzr merge lp:~wgrant/launchpad/link-uploaded-ddebs |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jelmer Vernooij (community) | code | Approve | |
Julian Edwards (community) | direction | Approve | |
Stuart Bishop (community) | db | Approve | |
Robert Collins (community) | db | Approve | |
Review via email: mp+29669@code.launchpad.net |
Commit message
DDEBs are now linked to and respect the overrides of their corresponding DEBs.
Description of the change
This branch alters the upload processor to link DDEBs and their DEBs, a piece critical to the pleasant DDEB experience which will be provided by this series of branches.
DDEBs contain debug symbols for their corresponding DEBs, so they need to be very tightly tied and always have matching changes performed. The only way to determine the link between two uploaded files is by comparing names within a build, so it is most efficient and simplest to store this link explicitly as the upload is processed. BinaryPackageRe
In addition to being linked to the corresponding DEB, DDEBs will now also inherit the DEB's overrides. This fixes bug #430025
Julian Edwards (julian-edwards) wrote : | # |
<bigjools> wgrant: unit tests please, not tests in doctests
<bigjools> wgrant: also consider factoring out your new code so you can write unit tests for it
<bigjools> Running the upload processor in tests is something I want to reduce, not increase ;)
<bigjools> wgrant: finally, my alarm bells go off very strongly when I see removeSecurityProxy being used in code
<wgrant> bigjools: Fair point.
<wgrant> In my defense, it's a reasonably large test to rewrite, and I revived this branch from October last year. I'll probably replace nascentupload-
<bigjools> ok
<wgrant> And yes, I strongly considered factoring that out.
<bigjools> you should XXX it and file a bug plz
<wgrant> But it means a significant restructure, so... maybe for another branch.
<bigjools> that's fine
<bigjools> thanks for fixing ddebs!
William Grant (wgrant) wrote : | # |
The removeSecurityProxy call is gone. DDEBs are created in the DB first, and BaseBinaryUploa
Stuart Bishop (stub) wrote : | # |
We can make it easier to clean out unwanted binarypackages:
ALTER TABLE BinaryPackageRe
ADD COLUMN debug_package integer REFERENCES BinaryPackageRe
Otherwise fine. patch-2207-73-0.sql
Julian Edwards (julian-edwards) wrote : | # |
I'm happy with the direction this is going now, although I don't have the spare time to do a thorough review right now. Provided someone else blesses this then that's ok with me too.
<bigjools> wgrant: don't forget to file a bug about fixing the tests though
<bigjools> the basic doc tests are fine, we just don't want unit tests in them
<wgrant> Hmm.
<bigjools> and also I would prefer unit tests instead of uploading whole new packages where possible
<wgrant> Oh, certainly.
<bigjools> if you add unit tests to doctests, various people like jml will hunt you down :)
<wgrant> But archiveuploader isn't awesome for that at the moment.
<bigjools> you need to factor out your new code
<bigjools> then it becomes testable :)
<wgrant> I need to move around some other stuff before I can do that.
<bigjools> ok
<bigjools> like I said, feel free to file a bug about it
Jelmer Vernooij (jelmer) wrote : | # |
> === added file 'database/
> --- database/
> +++ database/
> @@ -0,0 +1,10 @@
> +-- Copyright 2009 Canonical Ltd. This software is licensed under the
^^^ 2010 ? :-)
> @@ -561,8 +590,17 @@
> uploaded file targeted to an architecture not present in the
> distroseries in context. So callsites needs to be aware.
> """
> + # If we are dealing with a DDEB, use the DEB's overrides.
> + # If there's no deb_file set, don't worry about it. Rejection is
> + # already guaranteed.
> + if (isinstance(
> + and uploaded_
> + ancestry_name = uploaded_
> + else:
> + ancestry_name = uploaded_
> +
> binary_name = getUtility(
> - IBinaryPackageN
> + IBinaryPackageN
Ugh, "deb_file" is a bit confusing here as I keep reading it as
"debug_file". Not really sure what to do about that though.
> === modified file 'lib/lp/
> --- lib/lp/
> +++ lib/lp/
> @@ -152,6 +153,13 @@
> :return the corresponding `ILibraryFileAlias` if the file was found.
> """
>
> + def getBinaryPackag
> + """Return the corresponding `IBinaryPackage
> +
> + :param filename: the filename to look up.
> + :return: the correpsonding `IBinaryPackage
> + """
> +
corresponding ?
> class IBinaryPackageB
> """A Build interface for items requiring launchpad.Edit."""
>
> === modified file 'lib/lp/
> --- lib/lp/
> +++ lib/lp/
> @@ -18,7 +18,7 @@
> from lazr.enum import DBEnumeratedType, DBItem
> from lazr.restful.
> from lazr.restful.fields import Reference, ReferenceChoice
> -from zope.schema import Bool, Date, Int, Text, TextLine, Datetime
> +from zope.schema import Bool, Date, Datetime, Int, Object, Text, TextLine
> from zope.interface import Interface, Attribute
>
> from canonical.launchpad import _
Yay for drive-by fixes.
> === modified file 'lib/lp/
> --- lib/lp/
> +++ lib/lp/
> @@ -450,7 +450,7 @@
> binpackageformat, component,section, priority, shlibdeps,
> depends, recommends, suggests, conflicts, replaces, provides,
> pre_depends, enhances, breaks, essential, installedsize,
> - architecturespe
> + architecturespe
> """See...
Jelmer Vernooij (jelmer) : | # |
Preview Diff
1 | === modified file 'database/schema/comments.sql' |
2 | --- database/schema/comments.sql 2010-07-15 15:29:28 +0000 |
3 | +++ database/schema/comments.sql 2010-07-22 11:40:00 +0000 |
4 | @@ -1483,6 +1483,7 @@ |
5 | COMMENT ON COLUMN BinaryPackageRelease.pre_depends IS 'The list of packages this binary requires to be installed beforehand in apt/dpkg format, as it is in control file "Pre-Depends:" field.'; |
6 | COMMENT ON COLUMN BinaryPackageRelease.enhances IS 'The list of packages pointed as "enhanced" after the installation of this package, as it is in control file "Enhances:" field.'; |
7 | COMMENT ON COLUMN BinaryPackageRelease.breaks IS 'The list of packages which will be broken by the installtion of this package, as it is in the control file "Breaks:" field.'; |
8 | +COMMENT ON COLUMN BinaryPackageRelease.debug_package IS 'The corresponding binary package release containing debug symbols for this binary, if any.'; |
9 | |
10 | |
11 | -- BinaryPackageFile |
12 | |
13 | === added file 'database/schema/patch-2207-73-0.sql' |
14 | --- database/schema/patch-2207-73-0.sql 1970-01-01 00:00:00 +0000 |
15 | +++ database/schema/patch-2207-73-0.sql 2010-07-22 11:40:00 +0000 |
16 | @@ -0,0 +1,10 @@ |
17 | +-- Copyright 2010 Canonical Ltd. This software is licensed under the |
18 | +-- GNU Affero General Public License version 3 (see the file LICENSE). |
19 | + |
20 | +ALTER TABLE BinaryPackageRelease |
21 | + ADD COLUMN debug_package integer REFERENCES BinaryPackageRelease; |
22 | + |
23 | +CREATE UNIQUE INDEX binarypackagerelease__debug_package__key |
24 | + ON BinaryPackageRelease(debug_package); |
25 | + |
26 | +INSERT INTO LaunchpadDatabaseRevision VALUES (2207, 73, 0); |
27 | |
28 | === modified file 'lib/lp/archiveuploader/nascentupload.py' |
29 | --- lib/lp/archiveuploader/nascentupload.py 2010-07-15 09:42:28 +0000 |
30 | +++ lib/lp/archiveuploader/nascentupload.py 2010-07-22 11:40:00 +0000 |
31 | @@ -27,7 +27,7 @@ |
32 | from lp.archiveuploader.dscfile import DSCFile |
33 | from lp.archiveuploader.nascentuploadfile import ( |
34 | UploadError, UploadWarning, CustomUploadFile, SourceUploadFile, |
35 | - BaseBinaryUploadFile) |
36 | + BaseBinaryUploadFile, DdebBinaryUploadFile, DebBinaryUploadFile) |
37 | from lp.archiveuploader.utils import determine_source_file_type |
38 | from lp.registry.interfaces.pocket import PackagePublishingPocket |
39 | from lp.registry.interfaces.sourcepackage import SourcePackageFileType |
40 | @@ -158,6 +158,35 @@ |
41 | for uploaded_file in self.changes.files: |
42 | self.run_and_collect_errors(uploaded_file.verify) |
43 | |
44 | + unmatched_ddebs = {} |
45 | + for uploaded_file in self.changes.files: |
46 | + if isinstance(uploaded_file, DdebBinaryUploadFile): |
47 | + ddeb_key = (uploaded_file.package, uploaded_file.version, |
48 | + uploaded_file.architecture) |
49 | + if ddeb_key in unmatched_ddebs: |
50 | + self.reject("Duplicated debug packages: %s %s (%s)" % |
51 | + ddeb_key) |
52 | + else: |
53 | + unmatched_ddebs[ddeb_key] = uploaded_file |
54 | + |
55 | + for uploaded_file in self.changes.files: |
56 | + # We need exactly a DEB, not a DDEB. |
57 | + if (isinstance(uploaded_file, DebBinaryUploadFile) and |
58 | + not isinstance(uploaded_file, DdebBinaryUploadFile)): |
59 | + try: |
60 | + matching_ddeb = unmatched_ddebs.pop( |
61 | + (uploaded_file.package + '-dbgsym', |
62 | + uploaded_file.version, |
63 | + uploaded_file.architecture)) |
64 | + except KeyError: |
65 | + continue |
66 | + uploaded_file.ddeb_file = matching_ddeb |
67 | + matching_ddeb.deb_file = uploaded_file |
68 | + |
69 | + if len(unmatched_ddebs) > 0: |
70 | + self.reject("Orphaned debug packages: %s" % ', '.join('%s %s (%s)' % d |
71 | + for d in unmatched_ddebs)) |
72 | + |
73 | if (len(self.changes.files) == 1 and |
74 | isinstance(self.changes.files[0], CustomUploadFile)): |
75 | self.logger.debug("Single Custom Upload detected.") |
76 | @@ -561,8 +590,17 @@ |
77 | uploaded file targeted to an architecture not present in the |
78 | distroseries in context. So callsites needs to be aware. |
79 | """ |
80 | + # If we are dealing with a DDEB, use the DEB's overrides. |
81 | + # If there's no deb_file set, don't worry about it. Rejection is |
82 | + # already guaranteed. |
83 | + if (isinstance(uploaded_file, DdebBinaryUploadFile) |
84 | + and uploaded_file.deb_file): |
85 | + ancestry_name = uploaded_file.deb_file.package |
86 | + else: |
87 | + ancestry_name = uploaded_file.package |
88 | + |
89 | binary_name = getUtility( |
90 | - IBinaryPackageNameSet).queryByName(uploaded_file.package) |
91 | + IBinaryPackageNameSet).queryByName(ancestry_name) |
92 | |
93 | if binary_name is None: |
94 | return None |
95 | @@ -939,7 +977,12 @@ |
96 | # Container for the build that will be processed. |
97 | processed_builds = [] |
98 | |
99 | - for binary_package_file in self.changes.binary_package_files: |
100 | + # Create the BPFs referencing DDEBs last. This lets |
101 | + # storeInDatabase find and link DDEBs when creating DEBs. |
102 | + bpfs_to_create = sorted( |
103 | + self.changes.binary_package_files, |
104 | + key=lambda file: file.ddeb_file is not None) |
105 | + for binary_package_file in bpfs_to_create: |
106 | if self.sourceful: |
107 | # The reason we need to do this verification |
108 | # so late in the game is that in the |
109 | |
110 | === modified file 'lib/lp/archiveuploader/nascentuploadfile.py' |
111 | --- lib/lp/archiveuploader/nascentuploadfile.py 2010-06-25 16:26:20 +0000 |
112 | +++ lib/lp/archiveuploader/nascentuploadfile.py 2010-07-22 11:40:00 +0000 |
113 | @@ -372,6 +372,7 @@ |
114 | """Base methods for binary upload modeling.""" |
115 | |
116 | format = None |
117 | + ddeb_file = None |
118 | |
119 | # Capitalised because we extract these directly from the control file. |
120 | mandatory_fields = set(["Package", "Architecture", "Version"]) |
121 | @@ -864,6 +865,12 @@ |
122 | binary_name = getUtility( |
123 | IBinaryPackageNameSet).getOrCreateByName(self.package) |
124 | |
125 | + if self.ddeb_file: |
126 | + debug_package = build.getBinaryPackageFileByName( |
127 | + self.ddeb_file.filename).binarypackagerelease |
128 | + else: |
129 | + debug_package = None |
130 | + |
131 | binary = build.createBinaryPackageRelease( |
132 | binarypackagename=binary_name, |
133 | version=self.control_version, |
134 | @@ -885,7 +892,8 @@ |
135 | breaks=encoded.get('Breaks', ''), |
136 | essential=is_essential, |
137 | installedsize=installedsize, |
138 | - architecturespecific=architecturespecific) |
139 | + architecturespecific=architecturespecific, |
140 | + debug_package=debug_package) |
141 | |
142 | library_file = self.librarian.create(self.filename, |
143 | self.size, open(self.filepath, "rb"), self.content_type, |
144 | @@ -932,22 +940,6 @@ |
145 | |
146 | |
147 | class DdebBinaryUploadFile(DebBinaryUploadFile): |
148 | - """Represents an uploaded binary package file in ddeb format. |
149 | - |
150 | - DDEBs are never considered 'NEW', they don't require review since |
151 | - they are automatically generated. |
152 | - """ |
153 | + """Represents an uploaded binary package file in ddeb format.""" |
154 | format = BinaryPackageFormat.DDEB |
155 | - |
156 | - # Override the 'new' flag in a way any values set are ignored and |
157 | - # it always return False. |
158 | - def _get_new(self): |
159 | - """DDEBs are never considered NEW.""" |
160 | - return False |
161 | - |
162 | - def _set_new(self, value): |
163 | - """DDEBs cannot be made NEW.""" |
164 | - pass |
165 | - |
166 | - new = property( |
167 | - _get_new, _set_new, doc="DDEBs are never flagged as NEW.") |
168 | + deb_file = None |
169 | |
170 | === added directory 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken' |
171 | === added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin-dbgsym_1.0-1_i386.ddeb' |
172 | Binary files lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin-dbgsym_1.0-1_i386.ddeb 1970-01-01 00:00:00 +0000 and lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin-dbgsym_1.0-1_i386.ddeb 2010-07-22 11:40:00 +0000 differ |
173 | === added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin_1.0-1_i386.deb' |
174 | Binary files lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin_1.0-1_i386.deb 1970-01-01 00:00:00 +0000 and lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug-bin_1.0-1_i386.deb 2010-07-22 11:40:00 +0000 differ |
175 | === added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.dsc' |
176 | --- lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.dsc 1970-01-01 00:00:00 +0000 |
177 | +++ lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.dsc 2010-07-22 11:40:00 +0000 |
178 | @@ -0,0 +1,13 @@ |
179 | +Format: 1.0 |
180 | +Source: debug |
181 | +Binary: debug-bin, debug-bin-dbgsym |
182 | +Architecture: any |
183 | +Version: 1.0-1 |
184 | +Maintainer: Launchpad team <launchpad@lists.canonical.com> |
185 | +Standards-Version: 3.6.2 |
186 | +Checksums-Sha1: |
187 | + 6851f625a32f6c83a1af4c972b5bead14fbee06c 740 debug_1.0-1.tar.gz |
188 | +Checksums-Sha256: |
189 | + 1a1ed198644169f16eeee93c843ab8b7d9eef25cd91b83f55f497dbe744d3c95 740 debug_1.0-1.tar.gz |
190 | +Files: |
191 | + 8225b515f30dd8c0252847f17f4b7069 740 debug_1.0-1.tar.gz |
192 | |
193 | === added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.tar.gz' |
194 | Binary files lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.tar.gz 1970-01-01 00:00:00 +0000 and lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1.tar.gz 2010-07-22 11:40:00 +0000 differ |
195 | === added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_i386.changes' |
196 | --- lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_i386.changes 1970-01-01 00:00:00 +0000 |
197 | +++ lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_i386.changes 2010-07-22 11:40:00 +0000 |
198 | @@ -0,0 +1,31 @@ |
199 | +Format: 1.8 |
200 | +Date: Thu, 15 Apr 2009 15:14:09 +0100 |
201 | +Source: debug |
202 | +Binary: debug-bin |
203 | +Architecture: i386 |
204 | +Version: 1.0-1 |
205 | +Distribution: hoary |
206 | +Urgency: low |
207 | +Maintainer: Launchpad team <launchpad@lists.canonical.com> |
208 | +Changed-By: Celso Providelo <cprov@canonical.com> |
209 | +Description: |
210 | + debug-bin - Stuff for testing |
211 | +Changes: |
212 | + debug (1.0-1) hoary; urgency=low |
213 | + . |
214 | + * Testing debug binaries |
215 | +Checksums-Sha1: |
216 | + 91556113ad38eb35d2fe03d27ae646e0ed487a3d 654 debug-bin_1.0-1_i386.deb |
217 | + d120be63a4535e31ec08657cc4d3018cc8b62d54 668 debug-bin-dbgsym_1.0-1_i386.ddeb |
218 | + d120be63a4535e31ec08657cc4d3018cc8b62d54 668 debug-bin-dbgsym_1.0-1_i386.ddeb |
219 | + 473bafe5b391c52f9a527cc550bb1fd6d5256031 692 not-debug-bin-dbgsym_1.0-1_i386.ddeb |
220 | +Checksums-Sha256: |
221 | + 5ca0ce3d3bfde3cc698d9ad4b027663abecb958bea641d29ac3bf8333fd3ebad 654 debug-bin_1.0-1_i386.deb |
222 | + 9c0013badb1bb78fa1ccbf1508398cbf51904f7242d8931f4994c1aa18519b63 668 debug-bin-dbgsym_1.0-1_i386.ddeb |
223 | + 9c0013badb1bb78fa1ccbf1508398cbf51904f7242d8931f4994c1aa18519b63 668 debug-bin-dbgsym_1.0-1_i386.ddeb |
224 | + c474f06a1481111b4af58c583df0a2c4504a18c0d2d960db3a0416aafd1bfd0e 692 not-debug-bin-dbgsym_1.0-1_i386.ddeb |
225 | +Files: |
226 | + 48b4eed5980c754cc74a06ffc4372f64 654 devel optional debug-bin_1.0-1_i386.deb |
227 | + ac5c44025ffc026485b22722e0f13804 668 devel optional debug-bin-dbgsym_1.0-1_i386.ddeb |
228 | + ac5c44025ffc026485b22722e0f13804 668 devel optional debug-bin-dbgsym_1.0-1_i386.ddeb |
229 | + e6cb038fdfed5f1150814f719a618698 692 devel optional not-debug-bin-dbgsym_1.0-1_i386.ddeb |
230 | |
231 | === added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_source.changes' |
232 | --- lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_source.changes 1970-01-01 00:00:00 +0000 |
233 | +++ lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/debug_1.0-1_source.changes 2010-07-22 11:40:00 +0000 |
234 | @@ -0,0 +1,26 @@ |
235 | +Format: 1.8 |
236 | +Date: Thu, 15 Apr 2009 15:14:09 +0100 |
237 | +Source: debug |
238 | +Binary: debug-bin debug-bin-dbgsym |
239 | +Architecture: source |
240 | +Version: 1.0-1 |
241 | +Distribution: hoary |
242 | +Urgency: low |
243 | +Maintainer: Launchpad team <launchpad@lists.canonical.com> |
244 | +Changed-By: Celso Providelo <cprov@canonical.com> |
245 | +Description: |
246 | + debug-bin - Stuff for testing |
247 | + debug-bin-dbgsym - Stuff for testing DDEBs |
248 | +Changes: |
249 | + debug (1.0-1) hoary; urgency=low |
250 | + . |
251 | + * Testing debug binaries |
252 | +Checksums-Sha1: |
253 | + f30c2462e897e07534a5c9960a8f653d5a51b066 434 debug_1.0-1.dsc |
254 | + 6851f625a32f6c83a1af4c972b5bead14fbee06c 740 debug_1.0-1.tar.gz |
255 | +Checksums-Sha256: |
256 | + ab5e1eea2551828761d8734224d1a024be401ac05e83010494a0d9c69b8982c3 434 debug_1.0-1.dsc |
257 | + 1a1ed198644169f16eeee93c843ab8b7d9eef25cd91b83f55f497dbe744d3c95 740 debug_1.0-1.tar.gz |
258 | +Files: |
259 | + 81d19cae151472dfd2f797a69a2c1f54 434 devel optional debug_1.0-1.dsc |
260 | + 8225b515f30dd8c0252847f17f4b7069 740 devel optional debug_1.0-1.tar.gz |
261 | |
262 | === added file 'lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/not-debug-bin-dbgsym_1.0-1_i386.ddeb' |
263 | Binary files lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/not-debug-bin-dbgsym_1.0-1_i386.ddeb 1970-01-01 00:00:00 +0000 and lib/lp/archiveuploader/tests/data/suite/debug_1.0-1_broken/not-debug-bin-dbgsym_1.0-1_i386.ddeb 2010-07-22 11:40:00 +0000 differ |
264 | === modified file 'lib/lp/archiveuploader/tests/nascentupload-ddebs.txt' |
265 | --- lib/lp/archiveuploader/tests/nascentupload-ddebs.txt 2009-10-11 05:25:10 +0000 |
266 | +++ lib/lp/archiveuploader/tests/nascentupload-ddebs.txt 2010-07-22 11:40:00 +0000 |
267 | @@ -1,4 +1,5 @@ |
268 | -== Uploading DDEBs == |
269 | +Uploading DDEBs |
270 | +=============== |
271 | |
272 | DDEBs ('.ddeb' extension) are essentially a Debian binary packages, |
273 | they only use a different extension as a convenience to identify their |
274 | @@ -9,10 +10,8 @@ |
275 | for obtaining extra information about crashes in the corresponding |
276 | feature. |
277 | |
278 | -First we will create a source and a corresponding binary publication |
279 | -in the database, 'debug' and 'debug-bin' version '0.9', which will be |
280 | -an ancestor for the subsequent upload containing a 'debug-bin-dbgsym' |
281 | -DDEB. This is done using `SoyuzTestPublisher`. |
282 | + >>> from canonical.config import config |
283 | + >>> from canonical.testing import LaunchpadZopelessLayer |
284 | |
285 | >>> from lp.soyuz.tests.test_publishing import ( |
286 | ... SoyuzTestPublisher) |
287 | @@ -26,31 +25,22 @@ |
288 | |
289 | >>> unused = test_publisher.setUpDefaultDistroSeries(hoary) |
290 | |
291 | - >>> source_ancestry = test_publisher.getPubSource( |
292 | - ... sourcename='debug', version='0.99') |
293 | - |
294 | -We need a slightly different database setup for creating binaries. |
295 | - |
296 | - >>> from canonical.config import config |
297 | - >>> from canonical.testing import LaunchpadZopelessLayer |
298 | - |
299 | - >>> LaunchpadZopelessLayer.txn.commit() |
300 | - >>> LaunchpadZopelessLayer.switchDbUser('launchpad') |
301 | - |
302 | - >>> binary_ancestry = test_publisher.getPubBinaries( |
303 | - ... binaryname='debug-bin', pub_source=source_ancestry) |
304 | - |
305 | - >>> LaunchpadZopelessLayer.txn.commit() |
306 | - >>> LaunchpadZopelessLayer.switchDbUser('uploader') |
307 | - |
308 | -With that done, we now upload a new source, 'debug - 1.0-1', which is |
309 | -immediately accepted and stands pending publication. |
310 | +We first upload a new source, 'debug - 1.0-1'. Because this is NEW, |
311 | +it is in the queue awaiting overrides and acceptance. |
312 | |
313 | >>> src = getUploadForSource( |
314 | ... 'suite/debug_1.0-1/debug_1.0-1_source.changes') |
315 | >>> src.process() |
316 | >>> result = src.do_accept() |
317 | >>> print src.queue_root.status.name |
318 | + NEW |
319 | + >>> LaunchpadZopelessLayer.txn.commit() |
320 | + |
321 | +We don't really care where the source ends up, so we just accept the |
322 | +default overrides. It is now pending publication. |
323 | + |
324 | + >>> src.queue_root.acceptFromQueue("announce@example.com") |
325 | + >>> print src.queue_root.status.name |
326 | DONE |
327 | |
328 | >>> [src_pub] = src.queue_root.archive.getPublishedSources( |
329 | @@ -66,17 +56,46 @@ |
330 | >>> bin = getUploadForBinary( |
331 | ... 'suite/debug_1.0-1/debug_1.0-1_i386.changes') |
332 | |
333 | -DDEBs undergo exactly the same checks as ordinary DEBs. These are |
334 | -never considered NEW, so the binary upload has a 'never-seen-before' |
335 | -DDEB even though it isn't parked in queue for review. |
336 | +Because the DEB is new, the binary upload is held in NEW. |
337 | |
338 | >>> bin.process() |
339 | >>> result = bin.do_accept() |
340 | >>> print bin.queue_root.status.name |
341 | + NEW |
342 | + |
343 | +This upload has one build with two binaries: a DEB and its corresponding |
344 | +DDEB. |
345 | + |
346 | + >>> build = bin.queue_root.builds[0].build |
347 | + >>> build.binarypackages.count() |
348 | + 2 |
349 | + >>> print build.binarypackages[0].binpackageformat.name |
350 | + DEB |
351 | + >>> print build.binarypackages[1].binpackageformat.name |
352 | + DDEB |
353 | + >>> build.binarypackages[0].debug_package == build.binarypackages[1] |
354 | + True |
355 | + >>> build.binarypackages[1].debug_package is None |
356 | + True |
357 | + |
358 | +We override the binary to main/devel, and accept it into the archive. |
359 | + |
360 | + >>> from lp.soyuz.interfaces.component import IComponentSet |
361 | + >>> from lp.soyuz.interfaces.section import ISectionSet |
362 | + >>> main = getUtility(IComponentSet)['main'] |
363 | + >>> universe = getUtility(IComponentSet)['universe'] |
364 | + >>> devel = getUtility(ISectionSet)['devel'] |
365 | + |
366 | + >>> LaunchpadZopelessLayer.txn.commit() |
367 | + >>> LaunchpadZopelessLayer.switchDbUser('launchpad') |
368 | + |
369 | + >>> bin.queue_root.overrideBinaries(main, devel, None, [main, universe]) |
370 | + True |
371 | + >>> bin.queue_root.acceptFromQueue('announce@example.com') |
372 | + |
373 | + >>> print bin.queue_root.status.name |
374 | ACCEPTED |
375 | |
376 | -Again, binary publications need an extra database setup. |
377 | - |
378 | >>> LaunchpadZopelessLayer.txn.commit() |
379 | >>> LaunchpadZopelessLayer.switchDbUser(config.uploadqueue.dbuser) |
380 | |
381 | @@ -88,11 +107,12 @@ |
382 | Now, both, binary and debug-symbol packages are pending publication. |
383 | |
384 | >>> for bin_pub in bin_pubs: |
385 | - ... print '%s %s %s' % ( |
386 | + ... print '%s %s %s %s %s' % ( |
387 | ... bin_pub.displayname, bin_pub.status.name, |
388 | + ... bin_pub.component.name, bin_pub.section.name, |
389 | ... bin_pub.archive.purpose.name) |
390 | - debug-bin 1.0-1 in hoary i386 PENDING PRIMARY |
391 | - debug-bin-dbgsym 1.0-1 in hoary i386 PENDING DEBUG |
392 | + debug-bin 1.0-1 in hoary i386 PENDING main devel PRIMARY |
393 | + debug-bin-dbgsym 1.0-1 in hoary i386 PENDING main devel DEBUG |
394 | |
395 | DEBs and DDEBs are uploaded to separate archives, because the size |
396 | impact of uploading them to a single archive on mirrors would be |
397 | @@ -118,7 +138,8 @@ |
398 | debug-bin-dbgsym_1.0-1_i386.ddeb DDEB |
399 | |
400 | |
401 | -== DDEBs in other archives == |
402 | +DDEBs in other archives |
403 | +----------------------- |
404 | |
405 | DDEBs can be produced in other contexts than the primary archive. For |
406 | instance, a COPY (test-rebuild) archive. In this case, they will be |
407 | @@ -134,14 +155,14 @@ |
408 | ... owner=ubuntu.owner, purpose=ArchivePurpose.COPY, |
409 | ... name='test-rebuild') |
410 | |
411 | - >>> copied_source = source_ancestry.copyTo( |
412 | - ... hoary, source_ancestry.pocket, copy) |
413 | - |
414 | >>> LaunchpadZopelessLayer.txn.commit() |
415 | >>> LaunchpadZopelessLayer.switchDbUser('launchpad') |
416 | |
417 | + >>> initial_source = test_publisher.getPubSource( |
418 | + ... sourcename='debug', version='0.9', archive=copy) |
419 | + |
420 | >>> binary_ancestry2 = test_publisher.getPubBinaries( |
421 | - ... binaryname='debug-bin', pub_source=copied_source, archive=copy) |
422 | + ... binaryname='debug-bin', pub_source=initial_source, archive=copy) |
423 | |
424 | >>> src_pub = test_publisher.getPubSource( |
425 | ... sourcename='debug', version='1.0-1', archive=copy) |
426 | @@ -172,9 +193,27 @@ |
427 | >>> LaunchpadZopelessLayer.switchDbUser('uploader') |
428 | |
429 | >>> for bin_pub in bin_pubs: |
430 | - ... print '%s %s %s' % ( |
431 | + ... print '%s %s %s %s %s' % ( |
432 | ... bin_pub.displayname, bin_pub.status.name, |
433 | + ... bin_pub.component.name, bin_pub.section.name, |
434 | ... bin_pub.archive.purpose.name) |
435 | - debug-bin 1.0-1 in hoary i386 PENDING COPY |
436 | - debug-bin-dbgsym 1.0-1 in hoary i386 PENDING COPY |
437 | - |
438 | + debug-bin 1.0-1 in hoary i386 PENDING main devel COPY |
439 | + debug-bin-dbgsym 1.0-1 in hoary i386 PENDING main devel COPY |
440 | + |
441 | + |
442 | +Mismatched DDEBs |
443 | +---------------- |
444 | + |
445 | +Each uploaded DDEB must be associated with a normal DEB. Any duplicated |
446 | +DDEBs or DDEBs without matching DEBs will cause the upload to be |
447 | +rejected. |
448 | + |
449 | + >>> bin = getUploadForBinary( |
450 | + ... 'suite/debug_1.0-1_broken/debug_1.0-1_i386.changes') |
451 | + >>> bin.process() |
452 | + >>> bin.is_rejected |
453 | + True |
454 | + >>> print bin.rejection_message |
455 | + Duplicated debug packages: debug-bin-dbgsym 1.0-1 (i386) |
456 | + Orphaned debug packages: not-debug-bin-dbgsym 1.0-1 (i386) |
457 | + |
458 | |
459 | === modified file 'lib/lp/soyuz/doc/archive-files.txt' |
460 | --- lib/lp/soyuz/doc/archive-files.txt 2010-05-11 07:09:18 +0000 |
461 | +++ lib/lp/soyuz/doc/archive-files.txt 2010-07-22 11:40:00 +0000 |
462 | @@ -209,3 +209,11 @@ |
463 | |
464 | >>> deb == build.getFileByName('test_1.0_all.deb') |
465 | True |
466 | + |
467 | +We can also retrieve the corresponding BinaryPackageFile: |
468 | + |
469 | + >>> bpf = build.getBinaryPackageFileByName('test_1.0_all.deb') |
470 | + >>> bpf |
471 | + <BinaryPackageFile ...> |
472 | + >>> bpf.libraryfile == deb |
473 | + True |
474 | |
475 | === modified file 'lib/lp/soyuz/doc/binarypackagebuild.txt' |
476 | --- lib/lp/soyuz/doc/binarypackagebuild.txt 2010-07-17 12:56:41 +0000 |
477 | +++ lib/lp/soyuz/doc/binarypackagebuild.txt 2010-07-22 11:40:00 +0000 |
478 | @@ -323,6 +323,7 @@ |
479 | >>> pre_depends = None |
480 | >>> enhances = None |
481 | >>> breaks = None |
482 | + >>> debug_package = None |
483 | |
484 | Invoke createBinaryPackageRelease with all required arguments. |
485 | |
486 | @@ -334,7 +335,7 @@ |
487 | ... binpackageformat, component, section, priority, shlibdeps, |
488 | ... depends, recommends, suggests, conflicts, replaces, provides, |
489 | ... pre_depends, enhances, breaks, essential, installedsize, |
490 | - ... architecturespecific) |
491 | + ... architecturespecific, debug_package) |
492 | |
493 | >>> from canonical.launchpad.interfaces import IBinaryPackageRelease |
494 | >>> from canonical.launchpad.webapp.testing import verifyObject |
495 | @@ -367,7 +368,7 @@ |
496 | ... binpackageformat, component, section, priority, shlibdeps, |
497 | ... depends, recommends, suggests, conflicts, replaces, provides, |
498 | ... pre_depends, enhances, breaks, essential, installedsize, |
499 | - ... architecturespecific) |
500 | + ... architecturespecific, debug_package) |
501 | |
502 | |
503 | Check if the property is still working: |
504 | |
505 | === modified file 'lib/lp/soyuz/doc/distroarchseriesbinarypackage.txt' |
506 | --- lib/lp/soyuz/doc/distroarchseriesbinarypackage.txt 2010-04-12 08:29:02 +0000 |
507 | +++ lib/lp/soyuz/doc/distroarchseriesbinarypackage.txt 2010-07-22 11:40:00 +0000 |
508 | @@ -66,7 +66,8 @@ |
509 | ... breaks=None, |
510 | ... essential=False, |
511 | ... installedsize=0, |
512 | - ... architecturespecific=False) |
513 | + ... architecturespecific=False, |
514 | + ... debug_package=None) |
515 | |
516 | >>> pe = BinaryPackagePublishingHistory( |
517 | ... binarypackagerelease=bpr.id, |
518 | @@ -107,7 +108,8 @@ |
519 | ... breaks=None, |
520 | ... essential=False, |
521 | ... installedsize=0, |
522 | - ... architecturespecific=False) |
523 | + ... architecturespecific=False, |
524 | + ... debug_package=None) |
525 | |
526 | >>> pe = BinaryPackagePublishingHistory( |
527 | ... binarypackagerelease=bpr.id, |
528 | |
529 | === modified file 'lib/lp/soyuz/interfaces/binarypackagebuild.py' |
530 | --- lib/lp/soyuz/interfaces/binarypackagebuild.py 2010-06-14 09:31:26 +0000 |
531 | +++ lib/lp/soyuz/interfaces/binarypackagebuild.py 2010-07-22 11:40:00 +0000 |
532 | @@ -127,7 +127,8 @@ |
533 | binarypackagename, version, summary, description, binpackageformat, |
534 | component, section, priority, shlibdeps, depends, recommends, |
535 | suggests, conflicts, replaces, provides, pre_depends, enhances, |
536 | - breaks, essential, installedsize, architecturespecific): |
537 | + breaks, essential, installedsize, architecturespecific, |
538 | + debug_package): |
539 | """Create and return a `BinaryPackageRelease`. |
540 | |
541 | The binarypackagerelease will be attached to this specific build. |
542 | @@ -152,6 +153,13 @@ |
543 | :return the corresponding `ILibraryFileAlias` if the file was found. |
544 | """ |
545 | |
546 | + def getBinaryPackageFileByName(filename): |
547 | + """Return the corresponding `IBinaryPackageFile` in this context. |
548 | + |
549 | + :param filename: the filename to look up. |
550 | + :return: the corresponding `IBinaryPackageFile` if it was found. |
551 | + """ |
552 | + |
553 | |
554 | class IBinaryPackageBuildEdit(Interface): |
555 | """A Build interface for items requiring launchpad.Edit.""" |
556 | |
557 | === modified file 'lib/lp/soyuz/interfaces/binarypackagerelease.py' |
558 | --- lib/lp/soyuz/interfaces/binarypackagerelease.py 2010-03-17 10:07:19 +0000 |
559 | +++ lib/lp/soyuz/interfaces/binarypackagerelease.py 2010-07-22 11:40:00 +0000 |
560 | @@ -18,7 +18,7 @@ |
561 | from lazr.enum import DBEnumeratedType, DBItem |
562 | from lazr.restful.declarations import exported, export_as_webservice_entry |
563 | from lazr.restful.fields import Reference, ReferenceChoice |
564 | -from zope.schema import Bool, Date, Int, Text, TextLine, Datetime |
565 | +from zope.schema import Bool, Date, Datetime, Int, Object, Text, TextLine |
566 | from zope.interface import Interface, Attribute |
567 | |
568 | from canonical.launchpad import _ |
569 | @@ -52,6 +52,10 @@ |
570 | installedsize = Int(required=False) |
571 | architecturespecific = Bool(required=True) |
572 | datecreated = Datetime(required=True, readonly=True) |
573 | + debug_package = Object( |
574 | + title=_("Debug package"), schema=Interface, required=False, |
575 | + description=_("The corresponding package containing debug symbols " |
576 | + "for this binary.")) |
577 | |
578 | files = Attribute("Related list of IBinaryPackageFile entries") |
579 | |
580 | |
581 | === modified file 'lib/lp/soyuz/model/binarypackagebuild.py' |
582 | --- lib/lp/soyuz/model/binarypackagebuild.py 2010-07-20 19:12:33 +0000 |
583 | +++ lib/lp/soyuz/model/binarypackagebuild.py 2010-07-22 11:40:00 +0000 |
584 | @@ -451,7 +451,7 @@ |
585 | binpackageformat, component,section, priority, shlibdeps, |
586 | depends, recommends, suggests, conflicts, replaces, provides, |
587 | pre_depends, enhances, breaks, essential, installedsize, |
588 | - architecturespecific): |
589 | + architecturespecific, debug_package): |
590 | """See IBuild.""" |
591 | return BinaryPackageRelease( |
592 | build=self, binarypackagename=binarypackagename, version=version, |
593 | @@ -462,7 +462,8 @@ |
594 | suggests=suggests, conflicts=conflicts, replaces=replaces, |
595 | provides=provides, pre_depends=pre_depends, enhances=enhances, |
596 | breaks=breaks, essential=essential, installedsize=installedsize, |
597 | - architecturespecific=architecturespecific) |
598 | + architecturespecific=architecturespecific, |
599 | + debug_package=debug_package) |
600 | |
601 | def estimateDuration(self): |
602 | """See `IBuildBase`.""" |
603 | @@ -678,14 +679,11 @@ |
604 | |
605 | def _getDebByFileName(self, filename): |
606 | """Helper function to get a .deb LFA in the context of this build.""" |
607 | - store = Store.of(self) |
608 | - return store.find( |
609 | - LibraryFileAlias, |
610 | - BinaryPackageRelease.build == self.id, |
611 | - BinaryPackageFile.binarypackagerelease == BinaryPackageRelease.id, |
612 | - LibraryFileAlias.id == BinaryPackageFile.libraryfileID, |
613 | - LibraryFileAlias.filename == filename |
614 | - ).one() |
615 | + bpf = self.getBinaryPackageFileByName(filename) |
616 | + if bpf is not None: |
617 | + return bpf.libraryfile |
618 | + else: |
619 | + return None |
620 | |
621 | def getFileByName(self, filename): |
622 | """See `IBuild`.""" |
623 | @@ -705,6 +703,15 @@ |
624 | |
625 | raise NotFoundError(filename) |
626 | |
627 | + def getBinaryPackageFileByName(self, filename): |
628 | + """See `IBuild`.""" |
629 | + return Store.of(self).find( |
630 | + BinaryPackageFile, |
631 | + BinaryPackageRelease.build == self.id, |
632 | + BinaryPackageFile.binarypackagerelease == BinaryPackageRelease.id, |
633 | + LibraryFileAlias.id == BinaryPackageFile.libraryfileID, |
634 | + LibraryFileAlias.filename == filename).one() |
635 | + |
636 | def getSpecificJob(self): |
637 | """See `IBuildFarmJob`.""" |
638 | # If we are asked to adapt an object that is already a binary |
639 | |
640 | === modified file 'lib/lp/soyuz/model/binarypackagerelease.py' |
641 | --- lib/lp/soyuz/model/binarypackagerelease.py 2010-05-11 09:35:33 +0000 |
642 | +++ lib/lp/soyuz/model/binarypackagerelease.py 2010-07-22 11:40:00 +0000 |
643 | @@ -62,6 +62,8 @@ |
644 | architecturespecific = BoolCol(dbName='architecturespecific', |
645 | notNull=True) |
646 | datecreated = UtcDateTimeCol(notNull=True, default=UTC_NOW) |
647 | + debug_package = ForeignKey(dbName='debug_package', |
648 | + foreignKey='BinaryPackageRelease') |
649 | |
650 | files = SQLMultipleJoin('BinaryPackageFile', |
651 | joinColumn='binarypackagerelease', orderBy="libraryfile") |
652 | |
653 | === modified file 'lib/lp/soyuz/tests/test_publishing.py' |
654 | --- lib/lp/soyuz/tests/test_publishing.py 2010-07-21 14:14:54 +0000 |
655 | +++ lib/lp/soyuz/tests/test_publishing.py 2010-07-22 11:40:00 +0000 |
656 | @@ -321,7 +321,7 @@ |
657 | summary="summary", description="description", shlibdep=None, |
658 | depends=None, recommends=None, suggests=None, conflicts=None, |
659 | replaces=None, provides=None, pre_depends=None, enhances=None, |
660 | - breaks=None, format=BinaryPackageFormat.DEB): |
661 | + breaks=None, format=BinaryPackageFormat.DEB, debug_package=None): |
662 | """Return the corresponding `BinaryPackageRelease`.""" |
663 | sourcepackagerelease = build.source_package_release |
664 | distroarchseries = build.distro_arch_series |
665 | @@ -352,7 +352,8 @@ |
666 | installedsize=100, |
667 | architecturespecific=architecturespecific, |
668 | binpackageformat=format, |
669 | - priority=PackagePublishingPriority.STANDARD) |
670 | + priority=PackagePublishingPriority.STANDARD, |
671 | + debug_package=debug_package) |
672 | |
673 | # Create the corresponding binary file. |
674 | if architecturespecific: |
Ok from a schema pov, I haven't done a code review.