pkgme-devportal

Merge lp:~jml/pkgme-devportal/talk-to-libdep into lp:pkgme-devportal

  1. talk-to-libdep
  2. Merge into trunk
Proposed by Jonathan Lange
Status: Merged
Approved by: Jonathan Lange
Approved revision: 145
Merged at revision: 133
Proposed branch: lp:~jml/pkgme-devportal/talk-to-libdep
Merge into: lp:pkgme-devportal
Diff against target: 315 lines (+103/-18)
9 files modified
.bzrignore (+1/-0)
buildout.cfg (+1/-0)
devportalbinary/aptfile.py (+7/-0)
devportalbinary/binary.py (+7/-6)
devportalbinary/configuration.py (+3/-0)
devportalbinary/database.py (+33/-10)
devportalbinary/tests/test_database.py (+31/-2)
setup.py (+1/-0)
versions.cfg (+19/-0)
To merge this branch: bzr merge lp:~jml/pkgme-devportal/talk-to-libdep
Reviewer Review Type Date Requested Status
James Westby Approve
Review via email: mp+129205@code.launchpad.net

Commit message

Allow libdep-service to be used

Description of the change

* Lots of dependencies added in order to accommodate the test double.

 * Ditched PackageDatabase.create in favour of a top-level function that
   does the dispatch based on a dict.

 * Handled it in config as a new db_type, with base_url being the only
   parameter.

To post a comment you must log in.
Revision history for this message
James Westby (james-w) wrote :

Looks great.

Thanks,

James

review: Approve
Revision history for this message
ISD Branch Mangler (isd-branches-mangler) wrote :

The attempt to merge lp:~jml/pkgme-devportal/talk-to-libdep into lp:pkgme-devportal failed. Below is the output from the failed tests.

Tree is up to date at revision 26 of branch bzr+ssh://bazaar.launchpad.net/+branch/ca-download-cache
Downloading file:///tmp/tmp8qpWrC/download-cache/dist/distribute-0.6.29DEV.tar.gz
Extracting in /tmp/tmp0tkCr7
Now working in /tmp/tmp0tkCr7/distribute-0.6.29DEV
Building a Distribute egg in /tmp/tmp8qpWrC/eggs
/tmp/tmp8qpWrC/eggs/distribute-0.6.29DEV-py2.7.egg
warning: no files found matching '*.png' under directory 'docs'
warning: no files found matching '*.css' under directory 'docs'
warning: no files found matching '*.html' under directory 'docs'
warning: no files found matching '*.js' under directory 'docs'
zip_safe flag not set; analyzing archive contents...
warning: no previously-included files found matching '.bzrignore'
warning: no previously-included files found matching 'aclocal.m4'
no previously-included directories found matching 'autom4te.cache'
no previously-included directories found matching 'c'
no previously-included directories found matching 'c++'
no previously-included directories found matching 'compile'
warning: no previously-included files found matching 'configure*'
warning: no previously-included files found matching 'depcomp'
warning: no previously-included files found matching 'INSTALL'
warning: no previously-included files found matching 'install-sh'
warning: no previously-included files found matching 'lib*'
warning: no previously-included files found matching 'ltmain.sh'
no previously-included directories found matching 'm4'
warning: no previously-included files found matching 'Makefile*'
warning: no previously-included files found matching 'missing'
no previously-included directories found matching 'perl'
warning: no previously-included files found matching 'py-compile'
no previously-included directories found matching 'shell'
no previously-included directories found matching 'python/iso8601'
warning: no previously-included files found matching 'stamp-h1'
zip_safe flag not set; analyzing archive contents...
subunit.__init__: module references __file__
no previously-included directories found matching '_build'
zip_safe flag not set; analyzing archive contents...
pkgme.tests.test_vala_backend: module references __file__
pkgme.tests.test_python_backend: module references __file__
Couldn't find index page for 'libdep-service-python' (maybe misspelled?)
While:
  Installing scripts.
  Getting distribution for 'libdep-service-python[testing]==0.0.2'.
Error: Couldn't find a distribution for 'libdep-service-python[testing]==0.0.2'.
make: *** [bin/py] Error 1

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2012-09-12 18:17:49 +0000
3+++ .bzrignore 2012-10-11 14:17:27 +0000
4@@ -12,3 +12,4 @@
5 bin/buildout
6 bin/py
7 bin/pkgme
8+bin/dump-apt-file-db
9
10=== modified file 'buildout.cfg'
11--- buildout.cfg 2012-09-12 18:17:49 +0000
12+++ buildout.cfg 2012-10-11 14:17:27 +0000
13@@ -35,5 +35,6 @@
14 mock
15 python-subunit
16 pkgme
17+ libdep-service-python[testing]
18 include-site-packages = false
19 interpreter = py
20
21=== modified file 'devportalbinary/aptfile.py'
22--- devportalbinary/aptfile.py 2012-09-17 15:29:29 +0000
23+++ devportalbinary/aptfile.py 2012-10-11 14:17:27 +0000
24@@ -87,6 +87,9 @@
25
26 DISTROSERIES = "oneiric"
27
28+ # If db_type is set to this in the config, that means use this database.
29+ DB_TYPE = 'aptfile'
30+
31 CONTENTS_FILE_URL_LOCATION = (
32 "http://archive.ubuntu.com/ubuntu/dists/%(distroseries)s/"
33 "Contents-%(arch)s.gz")
34@@ -97,6 +100,10 @@
35 self.cachedir = os.path.expanduser(cachedir)
36 self._distroseries_arch_cache = {}
37
38+ @classmethod
39+ def from_options(cls, options):
40+ return cls(options.database_aptfile_cachedir)
41+
42 def _get_lib_to_pkgs_mapping(self, distroseries, arch):
43 """Returns a dict of { library-name : set([pkg1,pkg2])
44
45
46=== modified file 'devportalbinary/binary.py'
47--- devportalbinary/binary.py 2012-09-27 21:50:03 +0000
48+++ devportalbinary/binary.py 2012-10-11 14:17:27 +0000
49@@ -43,7 +43,7 @@
50 from pkgme import trace
51
52 from devportalbinary.configuration import load_configuration
53-from devportalbinary.database import PackageDatabase
54+from devportalbinary.database import get_dependency_database
55 from devportalbinary.metadata import (
56 make_all_info_fn,
57 make_want_fn,
58@@ -278,7 +278,7 @@
59 def get_packages_for_libraries(library_names, arch):
60 conf = load_configuration()
61 lib_overrides = conf.libraries_overrides
62- with closing(PackageDatabase.create()) as db:
63+ with closing(get_dependency_database()) as db:
64 deps = set()
65 for lib in library_names:
66 # ensure that overrides always "trump" the existing set if they
67@@ -287,12 +287,13 @@
68 # -- james_w
69 if lib in lib_overrides:
70 new_deps = set([lib_overrides[lib]])
71- trace.log("found dependencies '%s' for lib '%s' via override" % (
72- new_deps, lib))
73+ trace.log(
74+ "found dependencies '%s' for lib '%s' via override" % (
75+ new_deps, lib))
76 else:
77 new_deps = db.get_dependencies(lib, arch)
78- trace.log("found dependencies '%s' for lib '%s'" % (
79- new_deps, lib))
80+ trace.log(
81+ "found dependencies '%s' for lib '%s'" % (new_deps, lib))
82
83 if not new_deps:
84 raise UnknownDependency('Can\'t find dependency for "%s".' % lib)
85
86=== modified file 'devportalbinary/configuration.py'
87--- devportalbinary/configuration.py 2012-09-27 21:50:03 +0000
88+++ devportalbinary/configuration.py 2012-10-11 14:17:27 +0000
89@@ -46,6 +46,9 @@
90 help='The database name (for postgres)')
91 database.aptfile_cachedir = StringOption(default="~/.cache/pkgme-devportal",
92 help='The cache directory for the aptfile backend')
93+ database.base_url = StringOption(
94+ default='https://libdep-service.ubuntu.com/',
95+ help='The base URL for libdep-service')
96
97 scan_mode = StringOption(
98 help='Deprecated option, only binary is supported..',
99
100=== modified file 'devportalbinary/database.py'
101--- devportalbinary/database.py 2012-09-27 21:50:03 +0000
102+++ devportalbinary/database.py 2012-10-11 14:17:27 +0000
103@@ -31,6 +31,8 @@
104 )
105 from .utils import download_file
106
107+from libdep_service_client.client import Client
108+
109
110 # XXX: Historic name of this package. Update to 'pkgme-devportal' and
111 # re-authorize.
112@@ -381,10 +383,8 @@
113
114 class PackageDatabase(object):
115
116- # the various db backends, aptfile is a bit special
117 SQLITE = 'sqlite'
118 POSTGRES = 'postgres'
119- APTFILE = 'aptfile'
120
121 def __init__(self, store):
122 self._store = store
123@@ -439,14 +439,8 @@
124 return Store(database)
125
126 @classmethod
127- def create(cls, store=None):
128- if store is None:
129- options = load_configuration()
130- # XXX: not elegant
131- if options.database_db_type == cls.APTFILE:
132- return AptFilePackageDatabase(options.database_aptfile_cachedir)
133- store = cls.get_store_from_config(options)
134- return cls(store)
135+ def from_options(cls, options):
136+ return cls(cls.get_store_from_config(options))
137
138 def get_dependencies(self, library_name, arch='i386'):
139 """Get the binary packages that provide 'library_name'."""
140@@ -493,6 +487,35 @@
141 self._store.close()
142
143
144+class LibdepServiceClient(object):
145+ """Implements the read part of PackageDatabase's interface."""
146+
147+ DB_TYPE = 'libdep-service'
148+
149+ def __init__(self, client):
150+ self._client = client
151+
152+ @classmethod
153+ def from_options(cls, options):
154+ return cls(Client(options.database_base_url))
155+
156+ def get_dependencies(self, library_name, arch='i386'):
157+ return self._client.get_binaries_for_libraries(library_name)
158+
159+
160+def get_dependency_database():
161+ """Return an object that can get dependencies."""
162+ # TODO: Change this to return LibdepServiceClient sometimes
163+ databases = {
164+ AptFilePackageDatabase.DB_TYPE: AptFilePackageDatabase.from_options,
165+ PackageDatabase.POSTGRES: PackageDatabase.from_options,
166+ PackageDatabase.SQLITE: PackageDatabase.from_options,
167+ LibdepServiceClient.DB_TYPE: LibdepServiceClient.from_options,
168+ }
169+ options = load_configuration()
170+ return databases[options.database_db_type](options)
171+
172+
173 def dict_add(*dicts):
174 """Add dicts, with later dicts taking precedence."""
175 result = dict()
176
177=== modified file 'devportalbinary/tests/test_database.py'
178--- devportalbinary/tests/test_database.py 2012-09-27 21:50:03 +0000
179+++ devportalbinary/tests/test_database.py 2012-10-11 14:17:27 +0000
180@@ -20,9 +20,11 @@
181 deb_file_url_for_publication,
182 dict_add,
183 find_file_under_dir,
184+ get_dependency_database,
185 get_file_contents,
186 get_package_info_from_publication,
187 is_library_package,
188+ LibdepServiceClient,
189 libdep_mapping_from_symbols,
190 load_configuration,
191 NoSharedObject,
192@@ -39,6 +41,9 @@
193 postgres_db_resource,
194 )
195
196+from djlibdep.test_double import LibdepServiceDouble
197+from libdep_service_client.client import Client
198+
199
200 class ResultsIn(Matcher):
201
202@@ -158,12 +163,19 @@
203 nonexistent = self.getUniqueString()
204 self.useFixture(ConfigFileFixture(nonexistent))
205 self.assertIsInstance(
206- PackageDatabase.create(), AptFilePackageDatabase)
207+ get_dependency_database(), AptFilePackageDatabase)
208
209 def test_default_create_empty_config(self):
210 self.useFixture(ConfigSettings())
211 self.assertIsInstance(
212- PackageDatabase.create(), AptFilePackageDatabase)
213+ get_dependency_database(), AptFilePackageDatabase)
214+
215+ def test_remote_service(self):
216+ base_url = 'http://example.com/libdep-service/'
217+ self.use_database_config(db_type='libdep-service', base_url=base_url)
218+ db = get_dependency_database()
219+ self.assertIsInstance(db, LibdepServiceClient)
220+ self.assertEqual(base_url, db._client.base_url)
221
222 def test_get_db_info_from_config_postgres(self):
223 expected_username = self.getUniqueString()
224@@ -436,3 +448,20 @@
225 def test_includes_mapping(self):
226 self.assertEqual({'libfoo.so.1': 'libfoo'},
227 libdep_mapping_from_symbols('libfoo.so.1 libfoo #MINVER#\n'))
228+
229+
230+class TestLibdepServiceClient(TestCase):
231+
232+ TEST_DATA = {
233+ 'v1/get_binaries_for_libraries': [
234+ ({'libs': ['libfoo']}, '{"libfoo": ["libfoo-bin"]}'),
235+ ],
236+ }
237+
238+ def test_wraps_libdep_service(self):
239+ double = self.useFixture(LibdepServiceDouble(self.TEST_DATA))
240+ client = Client(double.base_url)
241+ wrapper = LibdepServiceClient(client)
242+ self.assertEqual(
243+ client.get_binaries_for_libraries(['libfoo']),
244+ wrapper.get_dependencies(['libfoo'], 'i386'))
245
246=== modified file 'setup.py'
247--- setup.py 2012-09-17 15:26:18 +0000
248+++ setup.py 2012-10-11 14:17:27 +0000
249@@ -37,6 +37,7 @@
250 'bzr',
251 'configglue',
252 'launchpadlib',
253+ 'libdep-service-python',
254 'PIL',
255 'pkgme>=0.4.1',
256 'fixtures',
257
258=== modified file 'versions.cfg'
259--- versions.cfg 2012-09-13 22:15:41 +0000
260+++ versions.cfg 2012-10-11 14:17:27 +0000
261@@ -9,6 +9,7 @@
262 # Alphabetical, case-insensitive, please! :-)
263 [versions]
264 argparse = 1.2.1
265+bson = 0.3.3
266 bzr = 2.5.1
267 Cheetah = 2.4.4
268 configglue = 1.0.3
269@@ -16,28 +17,46 @@
270 # DEFAULT_VERSION in distribute_setup.py must match the distribute version here
271 # If you change it here, change it there.
272 distribute = 0.6.29DEV
273+django = 1.3.1
274+django-configglue = 0.6.1
275+django-openid-auth = 0.4
276+django-pgtools = 0.1
277+django-preflight = 0.1.4
278 fixtures = 0.3.9
279 httplib2 = 0.7.4
280+iso8601 = 0.1.4
281 keyring = 0.9.2
282 launchpadlib = 1.10.2
283 lazr.authentication = 0.1.2
284 lazr.restfulclient = 0.13.0
285 lazr.uri = 1.0.3
286+libdep-service = 0.1
287+libdep-service-python = 0.0.2
288 Markdown = 2.2.0
289 mock = 0.8.0
290 oauth = 1.0.1
291+oops = 0.0.10
292+oops-datedir-repo = 0.0.20
293+oops-dictconfig = 0.0.1
294+oops-wsgi = 0.0.10
295+paste = 1.7.5.1
296 PIL = 1.1.7
297 postgresfixture = 0.1.2
298 pkgme = 0.4.1
299 psycopg2 = 2.4.5
300 python-debian = 0.1.16
301+python-openid = 2.2.5
302 python-subunit = 0.0.8
303 pyxdg = 0.23
304+pytz = 2012f
305 simplejson = 2.6.1
306+south = 0.7.3
307 storm = 0.19
308 testresources = 0.2.5
309 testtools = 0.9.16
310 treeshape = 0.2.1
311+Twisted = 12.2.0
312+txstatsd = 1.0.0
313 wadllib = 1.3.1
314 wsgi-intercept = 0.5.1
315 z3c.recipe.scripts = 1.0.1

Subscribers

People subscribed via source and target branches