launchpad-buildd

Merge lp:~cjwatson/launchpad-buildd/apt-lists into lp:launchpad-buildd

  1. apt-lists
  2. Merge into trunk
Proposed by Colin Watson
Status: Rejected
Rejected by: Colin Watson
Proposed branch: lp:~cjwatson/launchpad-buildd/apt-lists
Merge into: lp:launchpad-buildd
Diff against target: 95 lines (+46/-21)
2 files modified
debian/changelog (+3/-0)
lpbuildd/binarypackage.py (+43/-21)
To merge this branch: bzr merge lp:~cjwatson/launchpad-buildd/apt-lists
Reviewer Review Type Date Requested Status
Launchpad code reviewers Pending
Review via email: mp+286751@code.launchpad.net

Commit message

lpbuildd/binarypackage.py: Use "apt-get indextargets" and "apt-helper cat-file" where they exist to read Packages files, rather than looking in /var/lib/apt/lists/ directly.

Description of the change

The apt maintainers don't want other code poking around in /var/lib/apt/lists/ directly, not least because they might e.g. want to start lz4-compressing the package lists there in the near future. Use the recommended helpers where available to read Packages files for dep-wait analysis. We still need the old paths because these helpers are only available in rather recent versions of apt.

To post a comment you must log in.
Revision history for this message
Colin Watson (cjwatson) wrote :

Superseded by https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383060.

Unmerged revisions

194. By Colin Watson

lpbuildd/binarypackage.py: Use "apt-get indextargets" and "apt-helper
cat-file" where they exist to read Packages files, rather than looking
in /var/lib/apt/lists/ directly.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'debian/changelog'
2--- debian/changelog 2016-02-18 14:44:09 +0000
3+++ debian/changelog 2016-02-21 18:26:03 +0000
4@@ -5,6 +5,9 @@
5 qemu-user-static.
6 * Always raise exception instances rather than using the two-argument form
7 of raise.
8+ * lpbuildd/binarypackage.py: Use "apt-get indextargets" and "apt-helper
9+ cat-file" where they exist to read Packages files, rather than looking
10+ in /var/lib/apt/lists/ directly.
11
12 [ Kit Randel ]
13 * Add http/s proxy support for snap builds.
14
15=== modified file 'lpbuildd/binarypackage.py'
16--- lpbuildd/binarypackage.py 2016-02-18 14:44:09 +0000
17+++ lpbuildd/binarypackage.py 2016-02-21 18:26:03 +0000
18@@ -147,6 +147,34 @@
19 args.append(self._dscfile)
20 self.runSubProcess(self._sbuildpath, args)
21
22+ def getAptLists(self):
23+ """Yield each of apt's Packages files in turn as a file object."""
24+ apt_helper = "/usr/lib/apt/apt-helper"
25+ if os.path.exists(os.path.join(self.chroot_path, apt_helper[1:])):
26+ paths = subprocess.check_output(
27+ ["sudo", "chroot", self.chroot_path,
28+ "apt-get", "indextargets", "--format", "$(FILENAME)",
29+ "Created-By: Packages"],
30+ universal_newlines=True).splitlines()
31+ for path in paths:
32+ helper = subprocess.Popen(
33+ ["sudo", "chroot", self.chroot_path,
34+ apt_helper, "cat-file", path],
35+ stdout=subprocess.PIPE)
36+ try:
37+ yield helper.stdout
38+ finally:
39+ helper.stdout.read()
40+ helper.wait()
41+ else:
42+ apt_lists = os.path.join(
43+ self.chroot_path, "var", "lib", "apt", "lists")
44+ for name in sorted(os.listdir(apt_lists)):
45+ if name.endswith("_Packages"):
46+ path = os.path.join(apt_lists, name)
47+ with open(path, "rb") as packages_file:
48+ yield packages_file
49+
50 def getAvailablePackages(self):
51 """Return the available binary packages in the chroot.
52
53@@ -154,27 +182,21 @@
54 available versions of each package.
55 """
56 available = defaultdict(set)
57- apt_lists = os.path.join(
58- self.chroot_path, "var", "lib", "apt", "lists")
59- for name in sorted(os.listdir(apt_lists)):
60- if name.endswith("_Packages"):
61- path = os.path.join(apt_lists, name)
62- with open(path, "rb") as packages_file:
63- for section in apt_pkg.TagFile(packages_file):
64- available[section["package"]].add(section["version"])
65- if "provides" in section:
66- provides = apt_pkg.parse_depends(
67- section["provides"])
68- for provide in provides:
69- # Disjunctions are currently undefined here.
70- if len(provide) > 1:
71- continue
72- # Virtual packages may only provide an exact
73- # version or none.
74- if provide[0][1] and provide[0][2] != "=":
75- continue
76- available[provide[0][0]].add(
77- provide[0][1] if provide[0][1] else None)
78+ for packages_file in self.getAptLists():
79+ for section in apt_pkg.TagFile(packages_file):
80+ available[section["package"]].add(section["version"])
81+ if "provides" in section:
82+ provides = apt_pkg.parse_depends(section["provides"])
83+ for provide in provides:
84+ # Disjunctions are currently undefined here.
85+ if len(provide) > 1:
86+ continue
87+ # Virtual packages may only provide an exact version
88+ # or none.
89+ if provide[0][1] and provide[0][2] != "=":
90+ continue
91+ available[provide[0][0]].add(
92+ provide[0][1] if provide[0][1] else None)
93 return available
94
95 def getBuildDepends(self, dscpath, arch_indep):

Subscribers

People subscribed via source and target branches

to all changes: