Launchpad itself

Merge lp:~wgrant/launchpad/bug-598345-restrict-dep-contexts into lp:launchpad

  1. bug-598345-restrict-dep-contexts
  2. Merge into devel
Proposed by William Grant
Status: Merged
Approved by: Robert Collins
Approved revision: no longer in the source branch.
Merged at revision: 11175
Proposed branch: lp:~wgrant/launchpad/bug-598345-restrict-dep-contexts
Merge into: lp:launchpad
Diff against target: 770 lines (+326/-182)
10 files modified
lib/lp/soyuz/adapters/archivedependencies.py (+83/-55)
lib/lp/soyuz/doc/archive-dependencies.txt (+3/-3)
lib/lp/soyuz/doc/archive.txt (+1/-67)
lib/lp/soyuz/doc/binarypackagebuild.txt (+5/-3)
lib/lp/soyuz/interfaces/archive.py (+18/-5)
lib/lp/soyuz/model/archive.py (+19/-15)
lib/lp/soyuz/model/binarypackagebuild.py (+16/-25)
lib/lp/soyuz/tests/test_archive.py (+138/-3)
lib/lp/soyuz/tests/test_binarypackagebuild.py (+39/-4)
lib/lp/soyuz/tests/test_publishing.py (+4/-2)
To merge this branch: bzr merge lp:~wgrant/launchpad/bug-598345-restrict-dep-contexts
Reviewer Review Type Date Requested Status
Robert Collins (community) Approve
Review via email: mp+30203@code.launchpad.net

Commit message

Only retry depwait builds if the candidate is in an available pocket and component.

Description of the change

This branches fixes bug #52698 and bug #598345. At present some depwait builds are retried prematurely, as the candidate query in Archive.findDepCandidateByName constrains just the archive and distroarchseries context -- not the pocket or component, which are also included in the sources.list entries and restrict the available packages. A notable case is that of Hardy PPA packages dependent on debhelper 7: while a sufficient version is present in hardy-backports, most PPAs are not configured to use -backports. This results in such builds being retried every time the script runs, using vast amounts of buildd time to no effect.

A related (and also fixed here) issue is bug #606789: findDepCandidateByName() returned just the latest publication (by BPPH.id), and BinaryPackageBuild._isDependencySatisfied() checked the version constraint against that. But this ignores the fact that apt will look at *all* available versions across all archives and pockets, so _isDependencySatisfied needs to check that at least one of *any* of the available versions satisfies the version constraint.

I've fixed the first problem by refactoring the archive dependency expansion logic in lp.soyuz.adapters.archivedependencies, creating a new expand_dependencies() which returns a list of archives with the available pockets and components. The existing get_sources_list_for_building() wraps that, converting each dependency into a sources.list line, and adding any custom external dependencies as before. findDepCandidateByName() then uses expand_dependencies() to create the context restriction clause (archive, pocket and component). This is the same logic used to generate sources.list itself -- so it results in a completely accurate query.

For the second issue, findDepCandidateByName() has been replaced with findDepCandidates(), which returns a sequence of all matching binaries, latest first. _isDependencySatisfied() has been tweaked to look through all of them, returning True when a satisfying candidate is identified.

I've also removed some manual ogre-model logic from _isDependencySatisfied(), since that's now handled by findDepCandidates().

The findDepCandidateByName() doctests made me sad, so they've been replaced with expanded unit tests in test_archive. test_binarypackagebuild has two new tests: one checking that versioned dependencies work (a hole in the existing suite), and another verifying that more than just the latest binary is considered (new behaviour).

To post a comment you must log in.
Revision history for this message
Robert Collins (lifeless) wrote :

Please say /why/ here:
+ # We want the external dependency lines to show up second: after the
174 + # archive itself, but before any other dependencies.
175 + return [sources_list_lines[0]] + external_dep_lines + \

e.g. 'for an unknown reason We want ...

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/soyuz/adapters/archivedependencies.py'
2--- lib/lp/soyuz/adapters/archivedependencies.py 2010-05-10 19:49:02 +0000
3+++ lib/lp/soyuz/adapters/archivedependencies.py 2010-07-20 09:26:10 +0000
4@@ -12,8 +12,9 @@
5
6 Auxiliary functions exposed for testing purposes:
7
8- * get_components_for_building: return the corresponding component
9- dependencies for a build, this result is known as 'ogre_components';
10+ * get_components_for_context: return the corresponding component
11+ dependencies for a component and pocket, this result is known as
12+ 'ogre_components';
13 * get_primary_current_component: return the component name where the
14 building source is published in the primary archive.
15
16@@ -30,7 +31,8 @@
17 'component_dependencies',
18 'default_component_dependency_name',
19 'default_pocket_dependency',
20- 'get_components_for_building',
21+ 'expand_dependencies',
22+ 'get_components_for_context',
23 'get_primary_current_component',
24 'get_sources_list_for_building',
25 'pocket_dependencies',
26@@ -86,19 +88,20 @@
27 default_component_dependency_name = 'multiverse'
28
29
30-def get_components_for_building(build):
31+def get_components_for_context(component, pocket):
32 """Return the components allowed to be used in the build context.
33
34- :param build: a context `IBuild`.
35+ :param component: the context `IComponent`.
36+ :param pocket: the context `IPocket`.
37 :return: a list of component names.
38 """
39 # BACKPORTS should be able to fetch build dependencies from any
40 # component in order to cope with component changes occurring
41- # accross distroseries. See bug #198936 for further information.
42- if build.pocket == PackagePublishingPocket.BACKPORTS:
43+ # across distroseries. See bug #198936 for further information.
44+ if pocket == PackagePublishingPocket.BACKPORTS:
45 return component_dependencies['multiverse']
46
47- return component_dependencies[build.current_component.name]
48+ return component_dependencies[component.name]
49
50
51 def get_primary_current_component(archive, distroseries, sourcepackagename):
52@@ -119,6 +122,54 @@
53 return 'universe'
54
55
56+def expand_dependencies(archive, distro_series, pocket, component,
57+ source_package_name):
58+ """Return the set of dependency archives, pockets and components.
59+
60+ :param archive: the context `IArchive`.
61+ :param distro_series: the context `IDistroSeries`.
62+ :param pocket: the context `PackagePublishingPocket`.
63+ :param component: the context `IComponent`.
64+ :param source_package_name: A source package name (as text)
65+ :return: a list of (archive, pocket, [component]), representing the
66+ dependencies defined by the given build context.
67+ """
68+ deps = []
69+
70+ # Add implicit self-dependency for non-primary contexts.
71+ if archive.purpose in ALLOW_RELEASE_BUILDS:
72+ deps.append((
73+ archive, PackagePublishingPocket.RELEASE,
74+ get_components_for_context(component, pocket)))
75+
76+ primary_component = get_primary_current_component(
77+ archive, distro_series, source_package_name)
78+ # Consider user-selected archive dependencies.
79+ for archive_dependency in archive.dependencies:
80+ # When the dependency component is undefined, we should use
81+ # the component where the source is published in the primary
82+ # archive.
83+ if archive_dependency.component is None:
84+ components = component_dependencies[primary_component]
85+ else:
86+ components = component_dependencies[
87+ archive_dependency.component.name]
88+ # Follow pocket dependencies.
89+ for pocket in pocket_dependencies[archive_dependency.pocket]:
90+ deps.append(
91+ (archive_dependency.dependency, pocket, components))
92+
93+ # Consider primary archive dependency override. Add the default
94+ # primary archive dependencies if it's not present.
95+ if archive.getArchiveDependency(
96+ archive.distribution.main_archive) is None:
97+ primary_dependencies = _get_default_primary_dependencies(
98+ archive, component, pocket)
99+ deps.extend(primary_dependencies)
100+
101+ return deps
102+
103+
104 def get_sources_list_for_building(build, distroarchseries, sourcepackagename):
105 """Return the sources_list entries required to build the given item.
106
107@@ -133,17 +184,13 @@
108 :param sourcepackagename: A source package name (as text)
109 :return: a deb sources_list entries (lines).
110 """
111- deps = []
112- sources_list_lines = []
113-
114- # Add implicit self-dependency for non-primary contexts.
115- if build.archive.purpose in ALLOW_RELEASE_BUILDS:
116- self_dep = [(
117- build.archive, PackagePublishingPocket.RELEASE,
118- get_components_for_building(build))]
119- sources_list_lines = _get_sources_list_for_dependencies(
120- self_dep, distroarchseries)
121-
122+ deps = expand_dependencies(
123+ build.archive, distroarchseries.distroseries, build.pocket,
124+ build.current_component, sourcepackagename)
125+ sources_list_lines = \
126+ _get_sources_list_for_dependencies(deps, distroarchseries)
127+
128+ external_dep_lines = []
129 # Append external sources_list lines for this archive if it's
130 # specified in the configuration.
131 try:
132@@ -152,7 +199,7 @@
133 for archive_dep in dependencies.splitlines():
134 line = archive_dep % (
135 {'series': distroarchseries.distroseries.name})
136- sources_list_lines.append(line)
137+ external_dep_lines.append(line)
138 except StandardError, e:
139 # Malformed external dependencies can incapacitate the build farm
140 # manager (lp:516169). That's obviously not acceptable.
141@@ -166,34 +213,13 @@
142 if build.archive.enabled == True:
143 build.archive.disable()
144
145- # Consider user-selected archive dependencies.
146- primary_component = get_primary_current_component(
147- build.archive, build.distro_series, sourcepackagename)
148- for archive_dependency in build.archive.dependencies:
149- # When the dependency component is undefined, we should use
150- # the component where the source is published in the primary
151- # archive.
152- if archive_dependency.component is None:
153- components = component_dependencies[primary_component]
154- else:
155- components = component_dependencies[
156- archive_dependency.component.name]
157- # Follow pocket dependencies.
158- for pocket in pocket_dependencies[archive_dependency.pocket]:
159- deps.append(
160- (archive_dependency.dependency, pocket, components)
161- )
162-
163- # Consider primary archive dependency override. Add the default
164- # primary archive dependencies if it's not present.
165- if build.archive.getArchiveDependency(
166- build.archive.distribution.main_archive) is None:
167- primary_dependencies = _get_default_primary_dependencies(build)
168- deps.extend(primary_dependencies)
169-
170- sources_list_lines.extend(
171- _get_sources_list_for_dependencies(deps, distroarchseries))
172- return sources_list_lines
173+ # For an unknown reason (perhaps because OEM has archives with
174+ # binaries that need to override primary binaries of the same
175+ # version), we want the external dependency lines to show up second:
176+ # after the archive itself, but before any other dependencies.
177+ return [sources_list_lines[0]] + external_dep_lines + \
178+ sources_list_lines[1:]
179+
180
181 def _has_published_binaries(archive, distroarchseries, pocket):
182 """Whether or not the archive dependency has published binaries."""
183@@ -252,27 +278,29 @@
184 return sources_list_lines
185
186
187-def _get_default_primary_dependencies(build):
188- """Return the default primary dependencies for a given build.
189+def _get_default_primary_dependencies(archive, component, pocket):
190+ """Return the default primary dependencies for a given context.
191
192- :param build: the `IBuild` context;
193+ :param archive: the context `IArchive`.
194+ :param component: the context `IComponent`.
195+ :param pocket: the context `PackagePublishingPocket`.
196
197 :return: a list containing the default dependencies to primary
198 archive.
199 """
200- if build.archive.purpose in ALLOW_RELEASE_BUILDS:
201+ if archive.purpose in ALLOW_RELEASE_BUILDS:
202 primary_pockets = pocket_dependencies[
203 default_pocket_dependency]
204 primary_components = component_dependencies[
205 default_component_dependency_name]
206 else:
207- primary_pockets = pocket_dependencies[build.pocket]
208- primary_components = get_components_for_building(build)
209+ primary_pockets = pocket_dependencies[pocket]
210+ primary_components = get_components_for_context(component, pocket)
211
212 primary_dependencies = []
213 for pocket in primary_pockets:
214 primary_dependencies.append(
215- (build.distro_series.distribution.main_archive, pocket,
216+ (archive.distribution.main_archive, pocket,
217 primary_components))
218
219 return primary_dependencies
220
221=== modified file 'lib/lp/soyuz/doc/archive-dependencies.txt'
222--- lib/lp/soyuz/doc/archive-dependencies.txt 2010-05-10 19:49:02 +0000
223+++ lib/lp/soyuz/doc/archive-dependencies.txt 2010-07-20 09:26:10 +0000
224@@ -97,7 +97,7 @@
225 ... 'main', 'restricted', 'universe', 'multiverse', 'partner']
226
227 >>> from lp.soyuz.adapters.archivedependencies import (
228- ... get_components_for_building)
229+ ... get_components_for_context)
230
231 >>> ogre_pub = test_publisher.getPubSource(sourcename='ogre')
232 >>> [ogre_build] = ogre_pub.createMissingBuilds()
233@@ -111,7 +111,8 @@
234 ... syncUpdate(ogre_pub)
235 ... flush_database_caches()
236 ... components_term = " ".join(
237- ... get_components_for_building(ogre_build))
238+ ... get_components_for_context(
239+ ... ogre_build.current_component, ogre_build.pocket))
240 ... print '%10s | %s' % (ogre_build.current_component.name,
241 ... components_term)
242
243@@ -566,4 +567,3 @@
244 main restricted universe multiverse
245 deb http://ftpmaster.internal/ubuntu hoary-updates
246 main restricted universe multiverse
247-
248
249=== modified file 'lib/lp/soyuz/doc/archive.txt'
250--- lib/lp/soyuz/doc/archive.txt 2010-07-02 21:25:36 +0000
251+++ lib/lp/soyuz/doc/archive.txt 2010-07-20 09:26:10 +0000
252@@ -1285,72 +1285,6 @@
253 ...
254 AssertionError: This dependency does not exist.
255
256-== Find binary package dependency candidates ==
257-
258-Archive allows a lookup on a single binary package dependency
259-candidate by its name, via the `PublishedPackage` table:
260-
261- >>> warty_i386 = warty['i386']
262-
263- >>> candidate = ubuntu.main_archive.findDepCandidateByName(
264- ... warty_i386, "pmount")
265- >>> print candidate.binarypackagerelease.binarypackagename.name
266- pmount
267-
268- >>> candidate = cprov.archive.findDepCandidateByName(
269- ... warty_i386, "pmount")
270- >>> print candidate.binarypackagerelease.binarypackagename.name
271- pmount
272-
273-Since 'python2.4' isn't available in our sampledata (not even
274-published), None is returned:
275-
276- >>> print ubuntu.main_archive.findDepCandidateByName(
277- ... warty_i386, "python2.4")
278- None
279-
280- >>> print cprov.archive.findDepCandidateByName(
281- ... warty_i386, "python2.4")
282- None
283-
284-This method is aware of the archive dependency tree. So, even when a
285-package is not published on the context PPA but is available somewhere
286-in the archive dependency domain it will be found.
287-
288-We also add another archive dependency here to exercise findDepCandidateByName
289-a little more.
290-
291- >>> joe = factory.makePerson(email='joe@example.com')
292- >>> second_ppa = factory.makeArchive(name="secondppa", owner=joe)
293- >>> second_archive_dep = cprov.archive.addArchiveDependency(
294- ... second_ppa, release_pocket, main_component)
295-
296-'at' binary package is not present in Celso's PPA.
297-
298- >>> cprov_archive.getAllPublishedBinaries(name='at').count()
299- 0
300-
301-But it is available in PRIMARY ubuntu archive.
302-
303- >>> primary_candidate = ubuntu.main_archive.findDepCandidateByName(
304- ... warty_i386, "at")
305- >>> primary_candidate is not None
306- True
307-
308-Then a lookup on Celso's PPA will find it.
309-
310- >>> ppa_candidate = cprov.archive.findDepCandidateByName(
311- ... warty_i386, "at")
312- >>> ppa_candidate is not None
313- True
314-
315- >>> primary_candidate == ppa_candidate
316- True
317-
318-And clean-up after ourselves:
319-
320- >>> ignore = cprov.archive.removeArchiveDependency(second_ppa)
321-
322 == Creating a package copy request from an IArchive ==
323
324 The IArchive interface includes a convenience method for creating a
325@@ -1529,7 +1463,7 @@
326
327 >>> archive_purposes = [archive.purpose.name for archive in archive_set]
328 >>> len(archive_purposes)
329- 19
330+ 18
331
332 >>> print sorted(set(archive_purposes))
333 ['COPY', 'DEBUG', 'PARTNER', 'PPA', 'PRIMARY']
334
335=== modified file 'lib/lp/soyuz/doc/binarypackagebuild.txt'
336--- lib/lp/soyuz/doc/binarypackagebuild.txt 2010-06-10 22:12:08 +0000
337+++ lib/lp/soyuz/doc/binarypackagebuild.txt 2010-07-20 09:26:10 +0000
338@@ -987,8 +987,9 @@
339 main
340
341 >>> from lp.soyuz.adapters.archivedependencies import (
342- ... get_components_for_building)
343- >>> print get_components_for_building(depwait_build)
344+ ... get_components_for_context)
345+ >>> print get_components_for_context(
346+ ... depwait_build.current_component, depwait_build.pocket)
347 ['main']
348
349 Thus the 'pmount' dependency remains unsatisfied.
350@@ -1036,7 +1037,8 @@
351 >>> flush_database_caches()
352 >>> login(ANONYMOUS)
353
354- >>> print get_components_for_building(depwait_build)
355+ >>> print get_components_for_context(
356+ ... depwait_build.current_component, depwait_build.pocket)
357 ['main']
358
359 >>> print pmount_pub.component.name
360
361=== modified file 'lib/lp/soyuz/interfaces/archive.py'
362--- lib/lp/soyuz/interfaces/archive.py 2010-07-12 08:45:32 +0000
363+++ lib/lp/soyuz/interfaces/archive.py 2010-07-20 09:26:10 +0000
364@@ -443,11 +443,24 @@
365 Person table indexes while searching.
366 """
367
368- def findDepCandidateByName(distroarchseries, name):
369- """Return the last published binarypackage by given name.
370-
371- Return the `BinaryPackagePublishingHistory` record by distroarchseries
372- and name, or None if not found.
373+ def findDepCandidates(distro_arch_series, pocket, component,
374+ source_package_name, dep_name):
375+ """Return matching binaries in this archive and its dependencies.
376+
377+ Return all published `IBinaryPackagePublishingHistory` records with
378+ the given name, in this archive and dependencies as specified by the
379+ given build context, using the usual archive dependency rules.
380+
381+ We can't just use the first, since there may be other versions
382+ published in other dependency archives.
383+
384+ :param distro_arch_series: the context `IDistroArchSeries`.
385+ :param pocket: the context `PackagePublishingPocket`.
386+ :param component: the context `IComponent`.
387+ :param source_package_name: the context source package name (as text).
388+ :param dep_name: the name of the binary package to look up.
389+ :return: a sequence of matching `IBinaryPackagePublishingHistory`
390+ records.
391 """
392
393 def removeArchiveDependency(dependency):
394
395=== modified file 'lib/lp/soyuz/model/archive.py'
396--- lib/lp/soyuz/model/archive.py 2010-07-12 08:45:32 +0000
397+++ lib/lp/soyuz/model/archive.py 2010-07-20 09:26:10 +0000
398@@ -37,6 +37,7 @@
399 from lp.buildmaster.model.buildfarmjob import BuildFarmJob
400 from lp.buildmaster.model.packagebuild import PackageBuild
401 from lp.services.job.interfaces.job import JobStatus
402+from lp.soyuz.adapters.archivedependencies import expand_dependencies
403 from lp.soyuz.adapters.packagelocation import PackageLocation
404 from canonical.launchpad.components.tokens import (
405 create_unique_token_for_table)
406@@ -47,6 +48,7 @@
407 from lp.soyuz.model.binarypackagerelease import (
408 BinaryPackageReleaseDownloadCount)
409 from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
410+from lp.soyuz.model.component import Component
411 from lp.soyuz.model.distributionsourcepackagecache import (
412 DistributionSourcePackageCache)
413 from lp.soyuz.model.distroseriespackagecache import DistroSeriesPackageCache
414@@ -807,30 +809,32 @@
415 self.sources_cached = sources_cached.count()
416 self.binaries_cached = binaries_cached.count()
417
418- def findDepCandidateByName(self, distroarchseries, name):
419+ def findDepCandidates(self, distro_arch_series, pocket, component,
420+ source_package_name, dep_name):
421 """See `IArchive`."""
422- archives = []
423- if self.is_ppa:
424- archives.append(self.distribution.main_archive.id)
425- archives.append(self.id)
426- archives.extend(
427- IResultSet(self.dependencies).values(
428- ArchiveDependency.dependencyID))
429+ deps = expand_dependencies(
430+ self, distro_arch_series.distroseries, pocket, component,
431+ source_package_name)
432+ archive_clause = Or([And(
433+ BinaryPackagePublishingHistory.archiveID == archive.id,
434+ BinaryPackagePublishingHistory.pocket == pocket,
435+ Component.name.is_in(components))
436+ for (archive, pocket, components) in deps])
437
438 store = ISlaveStore(BinaryPackagePublishingHistory)
439- candidate = store.find(
440+ return store.find(
441 BinaryPackagePublishingHistory,
442- BinaryPackageName.name == name,
443+ BinaryPackageName.name == dep_name,
444 BinaryPackageRelease.binarypackagename == BinaryPackageName.id,
445 BinaryPackagePublishingHistory.binarypackagerelease ==
446 BinaryPackageRelease.id,
447 BinaryPackagePublishingHistory.distroarchseries ==
448- distroarchseries,
449- In(BinaryPackagePublishingHistory.archiveID, archives),
450+ distro_arch_series,
451 BinaryPackagePublishingHistory.status ==
452- PackagePublishingStatus.PUBLISHED
453- ).order_by(Desc(BinaryPackagePublishingHistory.id))
454- return candidate.first()
455+ PackagePublishingStatus.PUBLISHED,
456+ BinaryPackagePublishingHistory.componentID == Component.id,
457+ archive_clause).order_by(
458+ Desc(BinaryPackagePublishingHistory.id))
459
460 def getArchiveDependency(self, dependency):
461 """See `IArchive`."""
462
463=== modified file 'lib/lp/soyuz/model/binarypackagebuild.py'
464--- lib/lp/soyuz/model/binarypackagebuild.py 2010-06-15 11:11:27 +0000
465+++ lib/lp/soyuz/model/binarypackagebuild.py 2010-07-20 09:26:10 +0000
466@@ -50,7 +50,6 @@
467 from lp.buildmaster.model.packagebuild import (
468 PackageBuild, PackageBuildDerived)
469 from lp.services.job.model.job import Job
470-from lp.soyuz.adapters.archivedependencies import get_components_for_building
471 from lp.soyuz.interfaces.archive import ArchivePurpose
472 from lp.soyuz.interfaces.binarypackagebuild import (
473 BuildSetStatus, CannotBeRescored, IBinaryPackageBuild,
474@@ -387,33 +386,25 @@
475 def _isDependencySatisfied(self, token):
476 """Check if the given dependency token is satisfied.
477
478- Check if the dependency exists, if its version constraint is
479- satisfied and if it is reachable in the build context.
480+ Check if the dependency exists and that its version constraint is
481+ satisfied.
482 """
483 name, version, relation = self._parseDependencyToken(token)
484
485- dep_candidate = self.archive.findDepCandidateByName(
486- self.distro_arch_series, name)
487-
488- if not dep_candidate:
489- return False
490-
491- if not self._checkDependencyVersion(
492- dep_candidate.binarypackagerelease.version, version, relation):
493- return False
494-
495- # Only PRIMARY archive build dependencies should be restricted
496- # to the ogre_components. Both PARTNER and PPA can reach
497- # dependencies from all components in the PRIMARY archive.
498- # Moreover, PARTNER and PPA component domain is single, i.e,
499- # PARTNER only contains packages in 'partner' component and PPAs
500- # only contains packages in 'main' component.
501- ogre_components = get_components_for_building(self)
502- if (self.archive.purpose == ArchivePurpose.PRIMARY and
503- dep_candidate.component.name not in ogre_components):
504- return False
505-
506- return True
507+ # There may be several published versions in the available
508+ # archives and pockets. If any one of them satisifies our
509+ # constraints, the dependency is satisfied.
510+ dep_candidates = self.archive.findDepCandidates(
511+ self.distro_arch_series, self.pocket, self.current_component,
512+ self.source_package_release.sourcepackagename.name, name)
513+
514+ for dep_candidate in dep_candidates:
515+ if self._checkDependencyVersion(
516+ dep_candidate.binarypackagerelease.version, version,
517+ relation):
518+ return True
519+
520+ return False
521
522 def _toAptFormat(self, token):
523 """Rebuild dependencies line in apt format."""
524
525=== modified file 'lib/lp/soyuz/tests/test_archive.py'
526--- lib/lp/soyuz/tests/test_archive.py 2010-07-12 13:32:53 +0000
527+++ lib/lp/soyuz/tests/test_archive.py 2010-07-20 09:26:10 +0000
528@@ -4,9 +4,10 @@
529 """Test Archive features."""
530
531 from datetime import date, datetime, timedelta
532+import unittest
533+
534 import pytz
535-import unittest
536-
537+import transaction
538 from zope.component import getUtility
539 from zope.security.interfaces import Unauthorized
540 from zope.security.proxy import removeSecurityProxy
541@@ -1099,7 +1100,141 @@
542 login("commercial-member@canonical.com")
543 self.setCommercial(self.archive, True)
544 self.assertTrue(self.archive.commercial)
545-
546+
547+
548+class TestFindDepCandidates(TestCaseWithFactory):
549+ """Tests for Archive.findDepCandidates."""
550+
551+ layer = LaunchpadZopelessLayer
552+
553+ def setUp(self):
554+ super(TestFindDepCandidates, self).setUp()
555+ self.archive = self.factory.makeArchive()
556+ self.publisher = SoyuzTestPublisher()
557+ login('admin@canonical.com')
558+ self.publisher.prepareBreezyAutotest()
559+
560+ def assertDep(self, arch_tag, name, expected, archive=None,
561+ pocket=PackagePublishingPocket.RELEASE, component=None,
562+ source_package_name='something-new'):
563+ """Helper to check that findDepCandidates works.
564+
565+ Searches for the given dependency name in the given architecture and
566+ archive, and compares it to the given expected value.
567+ The archive defaults to self.archive.
568+
569+ Also commits, since findDepCandidates uses the slave store.
570+ """
571+ transaction.commit()
572+
573+ if component is None:
574+ component = getUtility(IComponentSet)['main']
575+ if archive is None:
576+ archive = self.archive
577+
578+ self.assertEquals(
579+ list(
580+ archive.findDepCandidates(
581+ self.publisher.distroseries[arch_tag], pocket, component,
582+ source_package_name, name)),
583+ expected)
584+
585+ def test_finds_candidate_in_same_archive(self):
586+ # A published candidate in the same archive should be found.
587+ bins = self.publisher.getPubBinaries(
588+ binaryname='foo', archive=self.archive,
589+ status=PackagePublishingStatus.PUBLISHED)
590+ self.assertDep('i386', 'foo', [bins[0]])
591+ self.assertDep('hppa', 'foo', [bins[1]])
592+
593+ def test_does_not_find_pending_publication(self):
594+ # A pending candidate in the same archive should not be found.
595+ bins = self.publisher.getPubBinaries(
596+ binaryname='foo', archive=self.archive)
597+ self.assertDep('i386', 'foo', [])
598+
599+ def test_ppa_searches_primary_archive(self):
600+ # PPA searches implicitly look in the primary archive too.
601+ self.assertEquals(self.archive.purpose, ArchivePurpose.PPA)
602+ self.assertDep('i386', 'foo', [])
603+
604+ bins = self.publisher.getPubBinaries(
605+ binaryname='foo', archive=self.archive.distribution.main_archive,
606+ status=PackagePublishingStatus.PUBLISHED)
607+
608+ self.assertDep('i386', 'foo', [bins[0]])
609+
610+ def test_searches_dependencies(self):
611+ # Candidates from archives on which the target explicitly depends
612+ # should be found.
613+ bins = self.publisher.getPubBinaries(
614+ binaryname='foo', archive=self.archive,
615+ status=PackagePublishingStatus.PUBLISHED)
616+ other_archive = self.factory.makeArchive()
617+ self.assertDep('i386', 'foo', [], archive=other_archive)
618+
619+ other_archive.addArchiveDependency(
620+ self.archive, PackagePublishingPocket.RELEASE)
621+ self.assertDep('i386', 'foo', [bins[0]], archive=other_archive)
622+
623+ def test_obeys_dependency_pockets(self):
624+ # Only packages published in a pocket matching the dependency should
625+ # be found.
626+ release_bins = self.publisher.getPubBinaries(
627+ binaryname='foo-release', archive=self.archive,
628+ status=PackagePublishingStatus.PUBLISHED)
629+ updates_bins = self.publisher.getPubBinaries(
630+ binaryname='foo-updates', archive=self.archive,
631+ status=PackagePublishingStatus.PUBLISHED,
632+ pocket=PackagePublishingPocket.UPDATES)
633+ proposed_bins = self.publisher.getPubBinaries(
634+ binaryname='foo-proposed', archive=self.archive,
635+ status=PackagePublishingStatus.PUBLISHED,
636+ pocket=PackagePublishingPocket.PROPOSED)
637+
638+ # Temporarily turn our test PPA into a copy archive, so we can
639+ # add non-RELEASE dependencies on it.
640+ removeSecurityProxy(self.archive).purpose = ArchivePurpose.COPY
641+
642+ other_archive = self.factory.makeArchive()
643+ other_archive.addArchiveDependency(
644+ self.archive, PackagePublishingPocket.UPDATES)
645+ self.assertDep(
646+ 'i386', 'foo-release', [release_bins[0]], archive=other_archive)
647+ self.assertDep(
648+ 'i386', 'foo-updates', [updates_bins[0]], archive=other_archive)
649+ self.assertDep('i386', 'foo-proposed', [], archive=other_archive)
650+
651+ other_archive.removeArchiveDependency(self.archive)
652+ other_archive.addArchiveDependency(
653+ self.archive, PackagePublishingPocket.PROPOSED)
654+ self.assertDep(
655+ 'i386', 'foo-proposed', [proposed_bins[0]], archive=other_archive)
656+
657+ def test_obeys_dependency_components(self):
658+ # Only packages published in a component matching the dependency
659+ # should be found.
660+ primary = self.archive.distribution.main_archive
661+ main_bins = self.publisher.getPubBinaries(
662+ binaryname='foo-main', archive=primary, component='main',
663+ status=PackagePublishingStatus.PUBLISHED)
664+ universe_bins = self.publisher.getPubBinaries(
665+ binaryname='foo-universe', archive=primary,
666+ component='universe',
667+ status=PackagePublishingStatus.PUBLISHED)
668+
669+ self.archive.addArchiveDependency(
670+ primary, PackagePublishingPocket.RELEASE,
671+ component=getUtility(IComponentSet)['main'])
672+ self.assertDep('i386', 'foo-main', [main_bins[0]])
673+ self.assertDep('i386', 'foo-universe', [])
674+
675+ self.archive.removeArchiveDependency(primary)
676+ self.archive.addArchiveDependency(
677+ primary, PackagePublishingPocket.RELEASE,
678+ component=getUtility(IComponentSet)['universe'])
679+ self.assertDep('i386', 'foo-main', [main_bins[0]])
680+ self.assertDep('i386', 'foo-universe', [universe_bins[0]])
681
682
683 def test_suite():
684
685=== modified file 'lib/lp/soyuz/tests/test_binarypackagebuild.py'
686--- lib/lp/soyuz/tests/test_binarypackagebuild.py 2010-06-21 07:26:51 +0000
687+++ lib/lp/soyuz/tests/test_binarypackagebuild.py 2010-07-20 09:26:10 +0000
688@@ -164,13 +164,13 @@
689 Return an `IBinaryPackageBuild` in MANUALDEWAIT state and depending on a
690 binary that exists and is reachable.
691 """
692- test_publisher = SoyuzTestPublisher()
693- test_publisher.prepareBreezyAutotest()
694+ self.publisher = SoyuzTestPublisher()
695+ self.publisher.prepareBreezyAutotest()
696
697- depwait_source = test_publisher.getPubSource(
698+ depwait_source = self.publisher.getPubSource(
699 sourcename='depwait-source')
700
701- test_publisher.getPubBinaries(
702+ self.publisher.getPubBinaries(
703 binaryname='dep-bin',
704 status=PackagePublishingStatus.PUBLISHED)
705
706@@ -273,6 +273,41 @@
707 depwait_build.updateDependencies()
708 self.assertEquals(depwait_build.dependencies, '')
709
710+ def testVersionedDependencies(self):
711+ # `IBinaryPackageBuild.updateDependencies` supports versioned
712+ # dependencies. A build will not be retried unless the candidate
713+ # complies with the version restriction.
714+ # In this case, dep-bin 666 is available. >> 666 isn't
715+ # satisified, but >= 666 is.
716+ depwait_build = self._setupSimpleDepwaitContext()
717+ self.layer.txn.commit()
718+
719+ depwait_build.dependencies = u'dep-bin (>> 666)'
720+ depwait_build.updateDependencies()
721+ self.assertEquals(depwait_build.dependencies, u'dep-bin (>> 666)')
722+ depwait_build.dependencies = u'dep-bin (>= 666)'
723+ depwait_build.updateDependencies()
724+ self.assertEquals(depwait_build.dependencies, u'')
725+
726+ def testVersionedDependencyOnOldPublication(self):
727+ # `IBinaryPackageBuild.updateDependencies` doesn't just consider
728+ # the latest publication. There may be older publications which
729+ # satisfy the version constraints (in other archives or pockets).
730+ # In this case, dep-bin 666 and 999 are available, so both = 666
731+ # and = 999 are satisfied.
732+ depwait_build = self._setupSimpleDepwaitContext()
733+ self.publisher.getPubBinaries(
734+ binaryname='dep-bin', version='999',
735+ status=PackagePublishingStatus.PUBLISHED)
736+ self.layer.txn.commit()
737+
738+ depwait_build.dependencies = u'dep-bin (= 666)'
739+ depwait_build.updateDependencies()
740+ self.assertEquals(depwait_build.dependencies, u'')
741+ depwait_build.dependencies = u'dep-bin (= 999)'
742+ depwait_build.updateDependencies()
743+ self.assertEquals(depwait_build.dependencies, u'')
744+
745
746 class BaseTestCaseWithThreeBuilds(TestCaseWithFactory):
747
748
749=== modified file 'lib/lp/soyuz/tests/test_publishing.py'
750--- lib/lp/soyuz/tests/test_publishing.py 2010-07-20 09:16:14 +0000
751+++ lib/lp/soyuz/tests/test_publishing.py 2010-07-20 09:26:10 +0000
752@@ -267,7 +267,8 @@
753 pub_source=None,
754 version='666',
755 architecturespecific=False,
756- builder=None):
757+ builder=None,
758+ component='main'):
759 """Return a list of binary publishing records."""
760 if distroseries is None:
761 distroseries = self.distroseries
762@@ -285,7 +286,8 @@
763 pub_source = self.getPubSource(
764 sourcename=sourcename, status=status, pocket=pocket,
765 archive=archive, distroseries=distroseries,
766- version=version, architecturehintlist=architecturehintlist)
767+ version=version, architecturehintlist=architecturehintlist,
768+ component=component)
769 else:
770 archive = pub_source.archive
771