launchpad-buildd

Merge lp:~cjwatson/launchpad-buildd/apt-lists into lp:launchpad-buildd

  1. apt-lists
  2. Merge into trunk
Proposed by Colin Watson
Status: Rejected
Rejected by: Colin Watson
Proposed branch: lp:~cjwatson/launchpad-buildd/apt-lists
Merge into: lp:launchpad-buildd
Diff against target: 95 lines (+46/-21)
2 files modified
debian/changelog (+3/-0)
lpbuildd/binarypackage.py (+43/-21)
To merge this branch: bzr merge lp:~cjwatson/launchpad-buildd/apt-lists
Reviewer Review Type Date Requested Status
Launchpad code reviewers Pending
Review via email: mp+286751@code.launchpad.net

Commit message

lpbuildd/binarypackage.py: Use "apt-get indextargets" and "apt-helper cat-file" where they exist to read Packages files, rather than looking in /var/lib/apt/lists/ directly.

Description of the change

The apt maintainers don't want other code poking around in /var/lib/apt/lists/ directly, not least because they might e.g. want to start lz4-compressing the package lists there in the near future. Use the recommended helpers where available to read Packages files for dep-wait analysis. We still need the old paths because these helpers are only available in rather recent versions of apt.

To post a comment you must log in.
Revision history for this message
Colin Watson (cjwatson) wrote :

Superseded by https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383060.

Unmerged revisions

194. By Colin Watson

lpbuildd/binarypackage.py: Use "apt-get indextargets" and "apt-helper
cat-file" where they exist to read Packages files, rather than looking
in /var/lib/apt/lists/ directly.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'debian/changelog'
--- debian/changelog 2016-02-18 14:44:09 +0000
+++ debian/changelog 2016-02-21 18:26:03 +0000
@@ -5,6 +5,9 @@
5 qemu-user-static.5 qemu-user-static.
6 * Always raise exception instances rather than using the two-argument form6 * Always raise exception instances rather than using the two-argument form
7 of raise.7 of raise.
8 * lpbuildd/binarypackage.py: Use "apt-get indextargets" and "apt-helper
9 cat-file" where they exist to read Packages files, rather than looking
10 in /var/lib/apt/lists/ directly.
811
9 [ Kit Randel ]12 [ Kit Randel ]
10 * Add http/s proxy support for snap builds.13 * Add http/s proxy support for snap builds.
1114
=== modified file 'lpbuildd/binarypackage.py'
--- lpbuildd/binarypackage.py 2016-02-18 14:44:09 +0000
+++ lpbuildd/binarypackage.py 2016-02-21 18:26:03 +0000
@@ -147,6 +147,34 @@
147 args.append(self._dscfile)147 args.append(self._dscfile)
148 self.runSubProcess(self._sbuildpath, args)148 self.runSubProcess(self._sbuildpath, args)
149149
150 def getAptLists(self):
151 """Yield each of apt's Packages files in turn as a file object."""
152 apt_helper = "/usr/lib/apt/apt-helper"
153 if os.path.exists(os.path.join(self.chroot_path, apt_helper[1:])):
154 paths = subprocess.check_output(
155 ["sudo", "chroot", self.chroot_path,
156 "apt-get", "indextargets", "--format", "$(FILENAME)",
157 "Created-By: Packages"],
158 universal_newlines=True).splitlines()
159 for path in paths:
160 helper = subprocess.Popen(
161 ["sudo", "chroot", self.chroot_path,
162 apt_helper, "cat-file", path],
163 stdout=subprocess.PIPE)
164 try:
165 yield helper.stdout
166 finally:
167 helper.stdout.read()
168 helper.wait()
169 else:
170 apt_lists = os.path.join(
171 self.chroot_path, "var", "lib", "apt", "lists")
172 for name in sorted(os.listdir(apt_lists)):
173 if name.endswith("_Packages"):
174 path = os.path.join(apt_lists, name)
175 with open(path, "rb") as packages_file:
176 yield packages_file
177
150 def getAvailablePackages(self):178 def getAvailablePackages(self):
151 """Return the available binary packages in the chroot.179 """Return the available binary packages in the chroot.
152180
@@ -154,27 +182,21 @@
154 available versions of each package.182 available versions of each package.
155 """183 """
156 available = defaultdict(set)184 available = defaultdict(set)
157 apt_lists = os.path.join(185 for packages_file in self.getAptLists():
158 self.chroot_path, "var", "lib", "apt", "lists")186 for section in apt_pkg.TagFile(packages_file):
159 for name in sorted(os.listdir(apt_lists)):187 available[section["package"]].add(section["version"])
160 if name.endswith("_Packages"):188 if "provides" in section:
161 path = os.path.join(apt_lists, name)189 provides = apt_pkg.parse_depends(section["provides"])
162 with open(path, "rb") as packages_file:190 for provide in provides:
163 for section in apt_pkg.TagFile(packages_file):191 # Disjunctions are currently undefined here.
164 available[section["package"]].add(section["version"])192 if len(provide) > 1:
165 if "provides" in section:193 continue
166 provides = apt_pkg.parse_depends(194 # Virtual packages may only provide an exact version
167 section["provides"])195 # or none.
168 for provide in provides:196 if provide[0][1] and provide[0][2] != "=":
169 # Disjunctions are currently undefined here.197 continue
170 if len(provide) > 1:198 available[provide[0][0]].add(
171 continue199 provide[0][1] if provide[0][1] else None)
172 # Virtual packages may only provide an exact
173 # version or none.
174 if provide[0][1] and provide[0][2] != "=":
175 continue
176 available[provide[0][0]].add(
177 provide[0][1] if provide[0][1] else None)
178 return available200 return available
179201
180 def getBuildDepends(self, dscpath, arch_indep):202 def getBuildDepends(self, dscpath, arch_indep):

Subscribers

People subscribed via source and target branches

to all changes: