Merge lp:~lifeless/launchpad/databasefixture into lp:launchpad
- databasefixture
- Merge into devel
Status: | Merged |
---|---|
Approved by: | Jonathan Lange |
Approved revision: | no longer in the source branch. |
Merged at revision: | 12019 |
Proposed branch: | lp:~lifeless/launchpad/databasefixture |
Merge into: | lp:launchpad |
Prerequisite: | lp:~lifeless/launchpad/uniqueconfig |
Diff against target: |
1324 lines (+659/-204) 18 files modified
database/schema/fti.py (+1/-1) lib/canonical/config/fixture.py (+8/-4) lib/canonical/database/sqlbase.py (+1/-1) lib/canonical/ftests/pgsql.py (+22/-2) lib/canonical/ftests/test_pgsql.py (+25/-2) lib/canonical/launchpad/doc/canonical-config.txt (+26/-21) lib/canonical/launchpad/doc/scripts-and-zcml.txt (+1/-4) lib/canonical/launchpad/scripts/__init__.py (+6/-6) lib/canonical/lp/__init__.py (+14/-6) lib/canonical/testing/ftests/test_layers.py (+12/-7) lib/canonical/testing/layers.py (+161/-138) lib/lp/code/interfaces/branchmergequeue.py (+129/-0) lib/lp/code/model/branchmergequeue.py (+88/-0) lib/lp/code/model/tests/test_branchmergequeue.py (+155/-0) lib/lp/registry/tests/test_mlists.py (+2/-1) lib/lp/services/memcache/doc/restful-cache.txt (+3/-3) lib/lp/services/scripts/doc/script-monitoring.txt (+4/-7) scripts/gina.py (+1/-1) |
To merge this branch: | bzr merge lp:~lifeless/launchpad/databasefixture |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jonathan Lange (community) | Approve | ||
Review via email: mp+38694@code.launchpad.net |
Commit message
Using unique DB names for test running.
Description of the change
Start doing unique db names by default, permitting parallel testing of layers upto and including DatabaseLayer, as long as the appserver isn't started (because that has statically configured ports)
Currently has a test failure on ec2 setting up the db. The legacy canonical.
Robert Collins (lifeless) wrote : | # |
Thanks jml; turns out its got furrther (shallow) changes needed to not
break things (ProcessLayerCo
help (or hinder) your point 2. I think that wanting a composite is
fair, but we'd want to be able to inspect the innards anyway which
somewhat defeats having a composite. For now, I think treating
BaseLayer as that composite itself isn't unreasonable.
Jonathan Lange (jml) wrote : | # |
Well, even then, we could have an API for asking BaseLayer what its configs are, rather than assuming we know. Even so, it's a small point. The real problem with the configs runs deeper.
Jonathan Lange (jml) wrote : | # |
It's hard to see if anything much changed in the LibrarianLayer, since diff completely removes and then adds it.
The rest of the changes look sane to me, but I haven't gone through them with a tooth comb looking for correctness issues.
Robert Collins (lifeless) wrote : | # |
On Wed, Oct 20, 2010 at 4:48 AM, Jonathan Lange <email address hidden> wrote:
> It's hard to see if anything much changed in the LibrarianLayer, since diff completely removes and then adds it.
Yeah, it would be nice to show that better. Nothing did.
> The rest of the changes look sane to me, but I haven't gone through them with a tooth comb looking for correctness issues.
Thats fine, thanks.
-Rob
Preview Diff
1 | === modified file 'database/schema/fti.py' |
2 | --- database/schema/fti.py 2010-06-10 09:32:34 +0000 |
3 | +++ database/schema/fti.py 2010-11-28 00:57:58 +0000 |
4 | @@ -314,7 +314,7 @@ |
5 | """ |
6 | |
7 | log.debug('Installing tsearch2') |
8 | - cmd = 'psql -f - -d %s' % lp.dbname |
9 | + cmd = 'psql -f - -d %s' % lp.get_dbname() |
10 | if lp.dbhost: |
11 | cmd += ' -h %s' % lp.dbhost |
12 | if options.dbuser: |
13 | |
14 | === modified file 'lib/canonical/config/fixture.py' |
15 | --- lib/canonical/config/fixture.py 2010-10-26 15:47:24 +0000 |
16 | +++ lib/canonical/config/fixture.py 2010-11-28 00:57:58 +0000 |
17 | @@ -40,17 +40,21 @@ |
18 | self.instance_name = instance_name |
19 | self.copy_from_instance = copy_from_instance |
20 | |
21 | + def add_section(self, sectioncontent): |
22 | + """Add sectioncontent to the lazy config.""" |
23 | + with open(self.absroot + '/launchpad-lazr.conf', 'ab') as out: |
24 | + out.write(sectioncontent) |
25 | + |
26 | def setUp(self): |
27 | super(ConfigFixture, self).setUp() |
28 | root = 'configs/' + self.instance_name |
29 | os.mkdir(root) |
30 | - absroot = os.path.abspath(root) |
31 | - self.addCleanup(shutil.rmtree, absroot) |
32 | + self.absroot = os.path.abspath(root) |
33 | + self.addCleanup(shutil.rmtree, self.absroot) |
34 | source = 'configs/' + self.copy_from_instance |
35 | for basename in os.listdir(source): |
36 | if basename == 'launchpad-lazr.conf': |
37 | - with open(root + '/launchpad-lazr.conf', 'wb') as out: |
38 | - out.write(self._extend_str % self.copy_from_instance) |
39 | + self.add_section(self._extend_str % self.copy_from_instance) |
40 | continue |
41 | with open(source + '/' + basename, 'rb') as input: |
42 | with open(root + '/' + basename, 'wb') as out: |
43 | |
44 | === modified file 'lib/canonical/database/sqlbase.py' |
45 | --- lib/canonical/database/sqlbase.py 2010-11-08 12:52:43 +0000 |
46 | +++ lib/canonical/database/sqlbase.py 2010-11-28 00:57:58 +0000 |
47 | @@ -820,7 +820,7 @@ |
48 | con_str = re.sub(r'host=\S*', '', con_str) # Remove stanza if exists. |
49 | con_str_overrides.append('host=%s' % lp.dbhost) |
50 | if dbname is None: |
51 | - dbname = lp.dbname # Note that lp.dbname may be None. |
52 | + dbname = lp.get_dbname() # Note that lp.dbname may be None. |
53 | if dbname is not None: |
54 | con_str = re.sub(r'dbname=\S*', '', con_str) # Remove if exists. |
55 | con_str_overrides.append('dbname=%s' % dbname) |
56 | |
57 | === modified file 'lib/canonical/ftests/pgsql.py' |
58 | --- lib/canonical/ftests/pgsql.py 2010-10-17 09:51:08 +0000 |
59 | +++ lib/canonical/ftests/pgsql.py 2010-11-28 00:57:58 +0000 |
60 | @@ -8,9 +8,12 @@ |
61 | __metaclass__ = type |
62 | |
63 | import os |
64 | +import random |
65 | import time |
66 | |
67 | import psycopg2 |
68 | + |
69 | +from canonical.config import config |
70 | from canonical.database.postgresql import ( |
71 | generateResetSequencesSQL, resetSequences) |
72 | |
73 | @@ -159,7 +162,7 @@ |
74 | # Class attribute. True if we should destroy the DB because changes made. |
75 | _reset_db = True |
76 | |
77 | - def __init__(self, template=None, dbname=None, dbuser=None, |
78 | + def __init__(self, template=None, dbname=dynamic, dbuser=None, |
79 | host=None, port=None, reset_sequences_sql=None): |
80 | '''Construct the PgTestSetup |
81 | |
82 | @@ -169,9 +172,25 @@ |
83 | if template is not None: |
84 | self.template = template |
85 | if dbname is PgTestSetup.dynamic: |
86 | + from canonical.testing.layers import BaseLayer |
87 | if os.environ.get('LP_TEST_INSTANCE'): |
88 | self.dbname = "%s_%s" % ( |
89 | self.__class__.dbname, os.environ.get('LP_TEST_INSTANCE')) |
90 | + # Stash the name we use in the config if a writable config is |
91 | + # available. |
92 | + # Avoid circular imports |
93 | + section = """[database] |
94 | +rw_main_master: dbname=%s |
95 | +rw_main_slave: dbname=%s |
96 | + |
97 | +""" % (self.dbname, self.dbname) |
98 | + if BaseLayer.config_fixture is not None: |
99 | + BaseLayer.config_fixture.add_section(section) |
100 | + if BaseLayer.appserver_config_fixture is not None: |
101 | + BaseLayer.appserver_config_fixture.add_section(section) |
102 | + if config.instance_name in ( |
103 | + BaseLayer.config_name, BaseLayer.appserver_config_name): |
104 | + config.reloadConfig() |
105 | else: |
106 | # Fallback to the class name. |
107 | self.dbname = self.__class__.dbname |
108 | @@ -257,7 +276,8 @@ |
109 | raise |
110 | finally: |
111 | con.close() |
112 | - time.sleep(1) |
113 | + # Let the other user complete their copying of the template DB. |
114 | + time.sleep(random.random()) |
115 | ConnectionWrapper.committed = False |
116 | ConnectionWrapper.dirty = False |
117 | PgTestSetup._last_db = (self.template, self.dbname) |
118 | |
119 | === modified file 'lib/canonical/ftests/test_pgsql.py' |
120 | --- lib/canonical/ftests/test_pgsql.py 2010-10-17 09:51:08 +0000 |
121 | +++ lib/canonical/ftests/test_pgsql.py 2010-11-28 00:57:58 +0000 |
122 | @@ -9,10 +9,13 @@ |
123 | ) |
124 | import testtools |
125 | |
126 | +from canonical.config import config, dbconfig |
127 | +from canonical.config.fixture import ConfigUseFixture |
128 | from canonical.ftests.pgsql import ( |
129 | ConnectionWrapper, |
130 | PgTestSetup, |
131 | ) |
132 | +from canonical.testing.layers import BaseLayer |
133 | |
134 | |
135 | class TestPgTestSetup(testtools.TestCase, TestWithFixtures): |
136 | @@ -30,9 +33,10 @@ |
137 | |
138 | def test_db_naming_LP_TEST_INSTANCE_set(self): |
139 | # when LP_TEST_INSTANCE is set, it is used for dynamic db naming. |
140 | - self.useFixture(EnvironmentVariableFixture('LP_TEST_INSTANCE', 'xx')) |
141 | + BaseLayer.setUp() |
142 | + self.addCleanup(BaseLayer.tearDown) |
143 | fixture = PgTestSetup(dbname=PgTestSetup.dynamic) |
144 | - expected_name = "%s_xx" % (PgTestSetup.dbname,) |
145 | + expected_name = "%s_%d" % (PgTestSetup.dbname, os.getpid()) |
146 | self.assertDBName(expected_name, fixture) |
147 | |
148 | def test_db_naming_without_LP_TEST_INSTANCE_is_static(self): |
149 | @@ -41,6 +45,25 @@ |
150 | expected_name = PgTestSetup.dbname |
151 | self.assertDBName(expected_name, fixture) |
152 | |
153 | + def test_db_naming_stored_in_BaseLayer_configs(self): |
154 | + BaseLayer.setUp() |
155 | + self.addCleanup(BaseLayer.tearDown) |
156 | + fixture = PgTestSetup(dbname=PgTestSetup.dynamic) |
157 | + fixture.setUp() |
158 | + self.addCleanup(fixture.dropDb) |
159 | + self.addCleanup(fixture.tearDown) |
160 | + expected_value = 'dbname=%s' % fixture.dbname |
161 | + self.assertEqual(expected_value, dbconfig.rw_main_master) |
162 | + self.assertEqual(expected_value, dbconfig.rw_main_slave) |
163 | + with ConfigUseFixture(BaseLayer.appserver_config_name): |
164 | + self.assertEqual(expected_value, dbconfig.rw_main_master) |
165 | + self.assertEqual(expected_value, dbconfig.rw_main_slave) |
166 | + |
167 | + |
168 | +class TestPgTestSetupTuning(testtools.TestCase, TestWithFixtures): |
169 | + |
170 | + layer = BaseLayer |
171 | + |
172 | def testOptimization(self): |
173 | # Test to ensure that the database is destroyed only when necessary |
174 | |
175 | |
176 | === modified file 'lib/canonical/launchpad/doc/canonical-config.txt' |
177 | --- lib/canonical/launchpad/doc/canonical-config.txt 2010-10-21 02:00:15 +0000 |
178 | +++ lib/canonical/launchpad/doc/canonical-config.txt 2010-11-28 00:57:58 +0000 |
179 | @@ -121,59 +121,65 @@ |
180 | Setting just the instance_name will change the directory from which the |
181 | conf file is loaded. |
182 | |
183 | - >>> config.setInstance('development') |
184 | - >>> config.instance_name |
185 | + >>> from canonical.config import CanonicalConfig |
186 | + >>> test_config = CanonicalConfig('testrunner', 'test') |
187 | + >>> test_config.setInstance('development') |
188 | + >>> test_config.instance_name |
189 | 'development' |
190 | |
191 | - >>> config.filename |
192 | + >>> test_config.filename |
193 | '.../configs/development/launchpad-lazr.conf' |
194 | - >>> config.extends.filename |
195 | + >>> test_config.extends.filename |
196 | '.../config/schema-lazr.conf' |
197 | |
198 | - >>> config.answertracker.days_before_expiration |
199 | + >>> test_config.answertracker.days_before_expiration |
200 | 15 |
201 | |
202 | Changing the instance_name and process_name changes the directory and |
203 | conf file name that is loaded. |
204 | |
205 | - >>> config.setInstance('testrunner') |
206 | - >>> config.instance_name |
207 | + >>> test_config.setInstance('testrunner') |
208 | + >>> test_config.instance_name |
209 | 'testrunner' |
210 | |
211 | - >>> config.answertracker.days_before_expiration |
212 | + >>> test_config.answertracker.days_before_expiration |
213 | 15 |
214 | |
215 | - >>> config.setProcess('test-process') |
216 | - >>> config.process_name |
217 | + >>> test_config.setProcess('test-process') |
218 | + >>> test_config.process_name |
219 | 'test-process' |
220 | |
221 | - >>> config.filename |
222 | + >>> test_config.filename |
223 | '.../configs/testrunner/test-process-lazr.conf' |
224 | - >>> config.extends.filename |
225 | + >>> test_config.extends.filename |
226 | '.../configs/testrunner/launchpad-lazr.conf' |
227 | |
228 | - >>> config.answertracker.days_before_expiration |
229 | + >>> test_config.answertracker.days_before_expiration |
230 | 300 |
231 | |
232 | The default 'launchpad-lazr.conf' is loaded if no conf files match |
233 | the process's name. |
234 | |
235 | - >>> config.setInstance('testrunner') |
236 | - >>> config.instance_name |
237 | + >>> test_config.setInstance('testrunner') |
238 | + >>> test_config.instance_name |
239 | 'testrunner' |
240 | |
241 | - >>> config.setProcess('test_no_conf') |
242 | - >>> config.process_name |
243 | + >>> test_config.setProcess('test_no_conf') |
244 | + >>> test_config.process_name |
245 | 'test_no_conf' |
246 | |
247 | - >>> config.filename |
248 | + >>> test_config.filename |
249 | '.../configs/testrunner/launchpad-lazr.conf' |
250 | - >>> config.extends.filename |
251 | + >>> test_config.extends.filename |
252 | '.../configs/development/launchpad-lazr.conf' |
253 | |
254 | - >>> config.answertracker.days_before_expiration |
255 | + >>> test_config.answertracker.days_before_expiration |
256 | 15 |
257 | |
258 | + >>> # setInstance sets the LPCONFIG environment variable, so set it |
259 | + >>> # back to the real value. |
260 | + >>> config.setInstance(config.instance_name) |
261 | + |
262 | The initial instance_name is set via the LPCONFIG environment variable. |
263 | Because Config is designed to failover to the default development |
264 | environment, and the testrunner overrides the environment and config, |
265 | @@ -183,7 +189,6 @@ |
266 | Alternatively, the instance name and process name can be specified as |
267 | argument to the constructor. |
268 | |
269 | - >>> from canonical.config import CanonicalConfig |
270 | >>> dev_config = CanonicalConfig('development', 'authserver') |
271 | >>> dev_config.instance_name |
272 | 'development' |
273 | |
274 | === modified file 'lib/canonical/launchpad/doc/scripts-and-zcml.txt' |
275 | --- lib/canonical/launchpad/doc/scripts-and-zcml.txt 2010-10-09 16:36:22 +0000 |
276 | +++ lib/canonical/launchpad/doc/scripts-and-zcml.txt 2010-11-28 00:57:58 +0000 |
277 | @@ -23,13 +23,10 @@ |
278 | |
279 | Run the script (making sure it uses the testrunner configuration). |
280 | |
281 | - >>> env = dict(os.environ) |
282 | - >>> env['LPCONFIG'] = 'testrunner' |
283 | >>> from canonical.config import config |
284 | >>> bin_py = os.path.join(config.root, 'bin', 'py') |
285 | >>> proc = subprocess.Popen([bin_py, script_file.name], |
286 | - ... stdout=subprocess.PIPE, stderr=None, |
287 | - ... env=env) |
288 | + ... stdout=subprocess.PIPE, stderr=None) |
289 | |
290 | Check that we get the expected output. |
291 | |
292 | |
293 | === modified file 'lib/canonical/launchpad/scripts/__init__.py' |
294 | --- lib/canonical/launchpad/scripts/__init__.py 2010-10-20 01:23:52 +0000 |
295 | +++ lib/canonical/launchpad/scripts/__init__.py 2010-11-28 00:57:58 +0000 |
296 | @@ -132,13 +132,13 @@ |
297 | |
298 | dbname and dbhost are also propagated to config.database.dbname and |
299 | config.database.dbhost. dbname, dbhost and dbuser are also propagated to |
300 | - lp.dbname, lp.dbhost and lp.dbuser. This ensures that all systems will |
301 | + lp.get_dbname(), lp.dbhost and lp.dbuser. This ensures that all systems will |
302 | be using the requested connection details. |
303 | |
304 | To test, we first need to store the current values so we can reset them |
305 | later. |
306 | |
307 | - >>> dbname, dbhost, dbuser = lp.dbname, lp.dbhost, lp.dbuser |
308 | + >>> dbname, dbhost, dbuser = lp.get_dbname(), lp.dbhost, lp.dbuser |
309 | |
310 | Ensure that command line options propagate to where we say they do |
311 | |
312 | @@ -146,7 +146,7 @@ |
313 | >>> db_options(parser) |
314 | >>> options, args = parser.parse_args( |
315 | ... ['--dbname=foo', '--host=bar', '--user=baz']) |
316 | - >>> options.dbname, lp.dbname, config.database.dbname |
317 | + >>> options.dbname, lp.get_dbname(), config.database.dbname |
318 | ('foo', 'foo', 'foo') |
319 | >>> (options.dbhost, lp.dbhost, config.database.dbhost) |
320 | ('bar', 'bar', 'bar') |
321 | @@ -163,7 +163,7 @@ |
322 | |
323 | Reset config |
324 | |
325 | - >>> lp.dbname, lp.dbhost, lp.dbuser = dbname, dbhost, dbuser |
326 | + >>> lp.dbhost, lp.dbuser = dbhost, dbuser |
327 | """ |
328 | def dbname_callback(option, opt_str, value, parser): |
329 | parser.values.dbname = value |
330 | @@ -172,11 +172,11 @@ |
331 | dbname: %s |
332 | """ % value) |
333 | config.push('dbname_callback', config_data) |
334 | - lp.dbname = value |
335 | + lp.dbname_override = value |
336 | |
337 | parser.add_option( |
338 | "-d", "--dbname", action="callback", callback=dbname_callback, |
339 | - type="string", dest="dbname", default=lp.dbname, |
340 | + type="string", dest="dbname", default=config.database.rw_main_master, |
341 | help="PostgreSQL database to connect to." |
342 | ) |
343 | |
344 | |
345 | === modified file 'lib/canonical/lp/__init__.py' |
346 | --- lib/canonical/lp/__init__.py 2009-06-25 05:39:50 +0000 |
347 | +++ lib/canonical/lp/__init__.py 2010-11-28 00:57:58 +0000 |
348 | @@ -20,7 +20,7 @@ |
349 | |
350 | |
351 | __all__ = [ |
352 | - 'dbname', 'dbhost', 'dbuser', 'isZopeless', 'initZopeless', |
353 | + 'get_dbname', 'dbhost', 'dbuser', 'isZopeless', 'initZopeless', |
354 | ] |
355 | |
356 | # SQLObject compatibility - dbname, dbhost and dbuser are DEPRECATED. |
357 | @@ -34,14 +34,22 @@ |
358 | # if the host is empty it can be overridden by the standard PostgreSQL |
359 | # environment variables, this feature currently required by Async's |
360 | # office environment. |
361 | -dbname = os.environ.get('LP_DBNAME', None) |
362 | dbhost = os.environ.get('LP_DBHOST', None) |
363 | dbuser = os.environ.get('LP_DBUSER', None) |
364 | - |
365 | -if dbname is None: |
366 | +dbname_override = os.environ.get('LP_DBNAME', None) |
367 | + |
368 | + |
369 | +def get_dbname(): |
370 | + """Get the DB Name for scripts: deprecated. |
371 | + |
372 | + :return: The dbname for scripts. |
373 | + """ |
374 | + if dbname_override is not None: |
375 | + return dbname_override |
376 | match = re.search(r'dbname=(\S*)', dbconfig.main_master) |
377 | assert match is not None, 'Invalid main_master connection string' |
378 | - dbname = match.group(1) |
379 | + return match.group(1) |
380 | + |
381 | |
382 | if dbhost is None: |
383 | match = re.search(r'host=(\S*)', dbconfig.main_master) |
384 | @@ -74,7 +82,7 @@ |
385 | # ) |
386 | pass # Disabled. Bug#3050 |
387 | if dbname is None: |
388 | - dbname = globals()['dbname'] |
389 | + dbname = get_dbname() |
390 | if dbhost is None: |
391 | dbhost = globals()['dbhost'] |
392 | if dbuser is None: |
393 | |
394 | === modified file 'lib/canonical/testing/ftests/test_layers.py' |
395 | --- lib/canonical/testing/ftests/test_layers.py 2010-10-22 09:49:44 +0000 |
396 | +++ lib/canonical/testing/ftests/test_layers.py 2010-11-28 00:57:58 +0000 |
397 | @@ -232,19 +232,18 @@ |
398 | class LibrarianTestCase(BaseTestCase): |
399 | layer = LibrarianLayer |
400 | |
401 | + want_launchpad_database = True |
402 | want_librarian_running = True |
403 | |
404 | - def testUploadsFail(self): |
405 | - # This layer is not particularly useful by itself, as the Librarian |
406 | - # cannot function correctly as there is no database setup. |
407 | + def testUploadsSucceed(self): |
408 | + # This layer is able to be used on its own as it depends on |
409 | + # DatabaseLayer. |
410 | # We can test this using remoteAddFile (it does not need the CA |
411 | # loaded) |
412 | client = LibrarianClient() |
413 | data = 'This is a test' |
414 | - self.failUnlessRaises( |
415 | - UploadFailed, client.remoteAddFile, |
416 | - 'foo.txt', len(data), StringIO(data), 'text/plain' |
417 | - ) |
418 | + client.remoteAddFile( |
419 | + 'foo.txt', len(data), StringIO(data), 'text/plain') |
420 | |
421 | |
422 | class LibrarianNoResetTestCase(testtools.TestCase): |
423 | @@ -328,6 +327,12 @@ |
424 | num = cur.fetchone()[0] |
425 | return num |
426 | |
427 | + # XXX: Parallel-fail: because layers are not cleanly integrated with |
428 | + # unittest, what should be one test is expressed as three distinct |
429 | + # tests here. We need to either write enough glue to push/pop the |
430 | + # global state of zope.testing.runner or we need to stop using layers, |
431 | + # before these tests will pass in a parallel run. Robert Collins |
432 | + # 2010-11-01 |
433 | def testNoReset1(self): |
434 | # Ensure that we can switch off database resets between tests |
435 | # if necessary, such as used by the page tests |
436 | |
437 | === modified file 'lib/canonical/testing/layers.py' |
438 | --- lib/canonical/testing/layers.py 2010-11-24 19:50:35 +0000 |
439 | +++ lib/canonical/testing/layers.py 2010-11-28 00:57:58 +0000 |
440 | @@ -261,16 +261,24 @@ |
441 | # Things we need to cleanup. |
442 | fixture = None |
443 | |
444 | - # The config names that are generated for this layer |
445 | + # ConfigFixtures for the configs generated for this layer. Set to None |
446 | + # if the layer is not setUp, or if persistent tests services are in use. |
447 | + config_fixture = None |
448 | + appserver_config_fixture = None |
449 | + |
450 | + # The config names that are generated for this layer. Set to None when |
451 | + # the layer is not setUp. |
452 | config_name = None |
453 | appserver_config_name = None |
454 | |
455 | @classmethod |
456 | - def make_config(cls, config_name, clone_from): |
457 | + def make_config(cls, config_name, clone_from, attr_name): |
458 | """Create a temporary config and link it into the layer cleanup.""" |
459 | cfg_fixture = ConfigFixture(config_name, clone_from) |
460 | cls.fixture.addCleanup(cfg_fixture.cleanUp) |
461 | cfg_fixture.setUp() |
462 | + cls.fixture.addCleanup(setattr, cls, attr_name, None) |
463 | + setattr(cls, attr_name, cfg_fixture) |
464 | |
465 | @classmethod |
466 | @profiled |
467 | @@ -296,9 +304,12 @@ |
468 | # about killing memcached - just do it quickly. |
469 | kill_by_pidfile(MemcachedLayer.getPidFile(), num_polls=0) |
470 | config_name = 'testrunner_%s' % test_instance |
471 | - cls.make_config(config_name, 'testrunner') |
472 | + cls.make_config(config_name, 'testrunner', 'config_fixture') |
473 | app_config_name = 'testrunner-appserver_%s' % test_instance |
474 | - cls.make_config(app_config_name, 'testrunner-appserver') |
475 | + cls.make_config( |
476 | + app_config_name, 'testrunner-appserver', |
477 | + 'appserver_config_fixture') |
478 | + cls.appserver_config_name = app_config_name |
479 | else: |
480 | config_name = 'testrunner' |
481 | app_config_name = 'testrunner-appserver' |
482 | @@ -622,122 +633,6 @@ |
483 | MemcachedLayer.client.flush_all() # Only do this in tests! |
484 | |
485 | |
486 | -class LibrarianLayer(BaseLayer): |
487 | - """Provides tests access to a Librarian instance. |
488 | - |
489 | - Calls to the Librarian will fail unless there is also a Launchpad |
490 | - database available. |
491 | - """ |
492 | - _reset_between_tests = True |
493 | - |
494 | - librarian_fixture = None |
495 | - |
496 | - @classmethod |
497 | - @profiled |
498 | - def setUp(cls): |
499 | - if not cls._reset_between_tests: |
500 | - raise LayerInvariantError( |
501 | - "_reset_between_tests changed before LibrarianLayer " |
502 | - "was actually used." |
503 | - ) |
504 | - cls.librarian_fixture = LibrarianTestSetup() |
505 | - cls.librarian_fixture.setUp() |
506 | - cls._check_and_reset() |
507 | - |
508 | - @classmethod |
509 | - @profiled |
510 | - def tearDown(cls): |
511 | - if cls.librarian_fixture is None: |
512 | - return |
513 | - try: |
514 | - cls._check_and_reset() |
515 | - finally: |
516 | - librarian = cls.librarian_fixture |
517 | - cls.librarian_fixture = None |
518 | - try: |
519 | - if not cls._reset_between_tests: |
520 | - raise LayerInvariantError( |
521 | - "_reset_between_tests not reset before LibrarianLayer " |
522 | - "shutdown" |
523 | - ) |
524 | - finally: |
525 | - librarian.cleanUp() |
526 | - |
527 | - @classmethod |
528 | - @profiled |
529 | - def _check_and_reset(cls): |
530 | - """Raise an exception if the Librarian has been killed. |
531 | - Reset the storage unless this has been disabled. |
532 | - """ |
533 | - try: |
534 | - f = urlopen(config.librarian.download_url) |
535 | - f.read() |
536 | - except Exception, e: |
537 | - raise LayerIsolationError( |
538 | - "Librarian has been killed or has hung." |
539 | - "Tests should use LibrarianLayer.hide() and " |
540 | - "LibrarianLayer.reveal() where possible, and ensure " |
541 | - "the Librarian is restarted if it absolutely must be " |
542 | - "shutdown: " + str(e) |
543 | - ) |
544 | - if cls._reset_between_tests: |
545 | - cls.librarian_fixture.clear() |
546 | - |
547 | - @classmethod |
548 | - @profiled |
549 | - def testSetUp(cls): |
550 | - cls._check_and_reset() |
551 | - |
552 | - @classmethod |
553 | - @profiled |
554 | - def testTearDown(cls): |
555 | - if cls._hidden: |
556 | - cls.reveal() |
557 | - cls._check_and_reset() |
558 | - |
559 | - # Flag maintaining state of hide()/reveal() calls |
560 | - _hidden = False |
561 | - |
562 | - # Fake upload socket used when the librarian is hidden |
563 | - _fake_upload_socket = None |
564 | - |
565 | - @classmethod |
566 | - @profiled |
567 | - def hide(cls): |
568 | - """Hide the Librarian so nothing can find it. We don't want to |
569 | - actually shut it down because starting it up again is expensive. |
570 | - |
571 | - We do this by altering the configuration so the Librarian client |
572 | - looks for the Librarian server on the wrong port. |
573 | - """ |
574 | - cls._hidden = True |
575 | - if cls._fake_upload_socket is None: |
576 | - # Bind to a socket, but don't listen to it. This way we |
577 | - # guarantee that connections to the given port will fail. |
578 | - cls._fake_upload_socket = socket.socket( |
579 | - socket.AF_INET, socket.SOCK_STREAM) |
580 | - assert config.librarian.upload_host == 'localhost', ( |
581 | - 'Can only hide librarian if it is running locally') |
582 | - cls._fake_upload_socket.bind(('127.0.0.1', 0)) |
583 | - |
584 | - host, port = cls._fake_upload_socket.getsockname() |
585 | - librarian_data = dedent(""" |
586 | - [librarian] |
587 | - upload_port: %s |
588 | - """ % port) |
589 | - config.push('hide_librarian', librarian_data) |
590 | - |
591 | - @classmethod |
592 | - @profiled |
593 | - def reveal(cls): |
594 | - """Reveal a hidden Librarian. |
595 | - |
596 | - This just involves restoring the config to the original value. |
597 | - """ |
598 | - cls._hidden = False |
599 | - config.pop('hide_librarian') |
600 | - |
601 | - |
602 | # We store a reference to the DB-API connect method here when we |
603 | # put a proxy in its place. |
604 | _org_connect = None |
605 | @@ -764,7 +659,20 @@ |
606 | cls._db_fixture = LaunchpadTestSetup( |
607 | reset_sequences_sql=reset_sequences_sql) |
608 | cls.force_dirty_database() |
609 | - cls._db_fixture.tearDown() |
610 | + # Nuke any existing DB (for persistent-test-services) [though they |
611 | + # prevent this !?] |
612 | + cls._db_fixture.tearDown() |
613 | + # Force a db creation for unique db names - needed at layer init |
614 | + # because appserver using layers run things at layer setup, not |
615 | + # test setup. |
616 | + cls._db_fixture.setUp() |
617 | + # And take it 'down' again to be in the right state for testSetUp |
618 | + # - note that this conflicts in principle with layers whose setUp |
619 | + # needs the db working, but this is a conceptually cleaner starting |
620 | + # point for addressing that mismatch. |
621 | + cls._db_fixture.tearDown() |
622 | + # Bring up the db, so that it is available for other layers. |
623 | + cls._ensure_db() |
624 | |
625 | @classmethod |
626 | @profiled |
627 | @@ -782,6 +690,13 @@ |
628 | @classmethod |
629 | @profiled |
630 | def testSetUp(cls): |
631 | + # The DB is already available - setUp and testTearDown both make it |
632 | + # available. |
633 | + if cls.use_mockdb is True: |
634 | + cls.installMockDb() |
635 | + |
636 | + @classmethod |
637 | + def _ensure_db(cls): |
638 | if cls._reset_between_tests: |
639 | cls._db_fixture.setUp() |
640 | # Ensure that the database is connectable. Because we might have |
641 | @@ -798,9 +713,6 @@ |
642 | else: |
643 | break |
644 | |
645 | - if cls.use_mockdb is True: |
646 | - cls.installMockDb() |
647 | - |
648 | @classmethod |
649 | @profiled |
650 | def testTearDown(cls): |
651 | @@ -819,6 +731,10 @@ |
652 | BaseLayer.flagTestIsolationFailure( |
653 | "Database policy %s still installed" |
654 | % repr(StoreSelector.pop())) |
655 | + # Reset/bring up the db - makes it available for either the next test, |
656 | + # or a subordinate layer which builds on the db. This wastes one setup |
657 | + # per db layer teardown per run, but thats tolerable. |
658 | + cls._ensure_db() |
659 | |
660 | use_mockdb = False |
661 | mockdb_mode = None |
662 | @@ -886,12 +802,128 @@ |
663 | return cls._db_fixture.dropDb() |
664 | |
665 | |
666 | +class LibrarianLayer(DatabaseLayer): |
667 | + """Provides tests access to a Librarian instance. |
668 | + |
669 | + Calls to the Librarian will fail unless there is also a Launchpad |
670 | + database available. |
671 | + """ |
672 | + _reset_between_tests = True |
673 | + |
674 | + librarian_fixture = None |
675 | + |
676 | + @classmethod |
677 | + @profiled |
678 | + def setUp(cls): |
679 | + if not cls._reset_between_tests: |
680 | + raise LayerInvariantError( |
681 | + "_reset_between_tests changed before LibrarianLayer " |
682 | + "was actually used." |
683 | + ) |
684 | + cls.librarian_fixture = LibrarianTestSetup() |
685 | + cls.librarian_fixture.setUp() |
686 | + cls._check_and_reset() |
687 | + |
688 | + @classmethod |
689 | + @profiled |
690 | + def tearDown(cls): |
691 | + if cls.librarian_fixture is None: |
692 | + return |
693 | + try: |
694 | + cls._check_and_reset() |
695 | + finally: |
696 | + librarian = cls.librarian_fixture |
697 | + cls.librarian_fixture = None |
698 | + try: |
699 | + if not cls._reset_between_tests: |
700 | + raise LayerInvariantError( |
701 | + "_reset_between_tests not reset before LibrarianLayer " |
702 | + "shutdown" |
703 | + ) |
704 | + finally: |
705 | + librarian.cleanUp() |
706 | + |
707 | + @classmethod |
708 | + @profiled |
709 | + def _check_and_reset(cls): |
710 | + """Raise an exception if the Librarian has been killed. |
711 | + Reset the storage unless this has been disabled. |
712 | + """ |
713 | + try: |
714 | + f = urlopen(config.librarian.download_url) |
715 | + f.read() |
716 | + except Exception, e: |
717 | + raise LayerIsolationError( |
718 | + "Librarian has been killed or has hung." |
719 | + "Tests should use LibrarianLayer.hide() and " |
720 | + "LibrarianLayer.reveal() where possible, and ensure " |
721 | + "the Librarian is restarted if it absolutely must be " |
722 | + "shutdown: " + str(e) |
723 | + ) |
724 | + if cls._reset_between_tests: |
725 | + cls.librarian_fixture.clear() |
726 | + |
727 | + @classmethod |
728 | + @profiled |
729 | + def testSetUp(cls): |
730 | + cls._check_and_reset() |
731 | + |
732 | + @classmethod |
733 | + @profiled |
734 | + def testTearDown(cls): |
735 | + if cls._hidden: |
736 | + cls.reveal() |
737 | + cls._check_and_reset() |
738 | + |
739 | + # Flag maintaining state of hide()/reveal() calls |
740 | + _hidden = False |
741 | + |
742 | + # Fake upload socket used when the librarian is hidden |
743 | + _fake_upload_socket = None |
744 | + |
745 | + @classmethod |
746 | + @profiled |
747 | + def hide(cls): |
748 | + """Hide the Librarian so nothing can find it. We don't want to |
749 | + actually shut it down because starting it up again is expensive. |
750 | + |
751 | + We do this by altering the configuration so the Librarian client |
752 | + looks for the Librarian server on the wrong port. |
753 | + """ |
754 | + cls._hidden = True |
755 | + if cls._fake_upload_socket is None: |
756 | + # Bind to a socket, but don't listen to it. This way we |
757 | + # guarantee that connections to the given port will fail. |
758 | + cls._fake_upload_socket = socket.socket( |
759 | + socket.AF_INET, socket.SOCK_STREAM) |
760 | + assert config.librarian.upload_host == 'localhost', ( |
761 | + 'Can only hide librarian if it is running locally') |
762 | + cls._fake_upload_socket.bind(('127.0.0.1', 0)) |
763 | + |
764 | + host, port = cls._fake_upload_socket.getsockname() |
765 | + librarian_data = dedent(""" |
766 | + [librarian] |
767 | + upload_port: %s |
768 | + """ % port) |
769 | + config.push('hide_librarian', librarian_data) |
770 | + |
771 | + @classmethod |
772 | + @profiled |
773 | + def reveal(cls): |
774 | + """Reveal a hidden Librarian. |
775 | + |
776 | + This just involves restoring the config to the original value. |
777 | + """ |
778 | + cls._hidden = False |
779 | + config.pop('hide_librarian') |
780 | + |
781 | + |
782 | def test_default_timeout(): |
783 | """Don't timeout by default in tests.""" |
784 | return None |
785 | |
786 | |
787 | -class LaunchpadLayer(DatabaseLayer, LibrarianLayer, MemcachedLayer): |
788 | +class LaunchpadLayer(LibrarianLayer, MemcachedLayer): |
789 | """Provides access to the Launchpad database and daemons. |
790 | |
791 | We need to ensure that the database setup runs before the daemon |
792 | @@ -1659,7 +1691,7 @@ |
793 | appserver = None |
794 | |
795 | # The config used by the spawned app server. |
796 | - appserver_config = CanonicalConfig('testrunner-appserver', 'runlaunchpad') |
797 | + appserver_config = None |
798 | |
799 | # The SMTP server for layer tests. See |
800 | # configs/testrunner-appserver/mail-configure.zcml |
801 | @@ -1798,12 +1830,6 @@ |
802 | @classmethod |
803 | def _runAppServer(cls): |
804 | """Start the app server using runlaunchpad.py""" |
805 | - # The database must be available for the app server to start. |
806 | - cls._db_fixture = LaunchpadTestSetup() |
807 | - # This is not torn down properly: rather the singleton nature is abused |
808 | - # and the fixture is simply marked as being dirty. |
809 | - # XXX: Robert Collins 2010-10-17 bug=661967 |
810 | - cls._db_fixture.setUp() |
811 | # The app server will not start at all if the database hasn't been |
812 | # correctly patched. The app server will make exactly this check, |
813 | # doing it here makes the error more obvious. |
814 | @@ -1860,8 +1886,7 @@ |
815 | @classmethod |
816 | @profiled |
817 | def setUp(cls): |
818 | - LayerProcessController.startSMTPServer() |
819 | - LayerProcessController.startAppServer() |
820 | + LayerProcessController.setUp() |
821 | |
822 | @classmethod |
823 | @profiled |
824 | @@ -1887,8 +1912,7 @@ |
825 | @classmethod |
826 | @profiled |
827 | def setUp(cls): |
828 | - LayerProcessController.startSMTPServer() |
829 | - LayerProcessController.startAppServer() |
830 | + LayerProcessController.setUp() |
831 | |
832 | @classmethod |
833 | @profiled |
834 | @@ -1914,8 +1938,7 @@ |
835 | @classmethod |
836 | @profiled |
837 | def setUp(cls): |
838 | - LayerProcessController.startSMTPServer() |
839 | - LayerProcessController.startAppServer() |
840 | + LayerProcessController.setUp() |
841 | |
842 | @classmethod |
843 | @profiled |
844 | |
845 | === added file 'lib/lp/code/interfaces/branchmergequeue.py' |
846 | --- lib/lp/code/interfaces/branchmergequeue.py 1970-01-01 00:00:00 +0000 |
847 | +++ lib/lp/code/interfaces/branchmergequeue.py 2010-11-03 08:28:44 +0000 |
848 | @@ -0,0 +1,129 @@ |
849 | +# Copyright 2009-2010 Canonical Ltd. This software is licensed under the |
850 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
851 | + |
852 | +"""Branch merge queue interfaces.""" |
853 | + |
854 | +__metaclass__ = type |
855 | + |
856 | +__all__ = [ |
857 | + 'IBranchMergeQueue', |
858 | + 'IBranchMergeQueueSource', |
859 | + 'user_has_special_merge_queue_access', |
860 | + ] |
861 | + |
862 | +from lazr.restful.declarations import ( |
863 | + export_as_webservice_entry, |
864 | + export_write_operation, |
865 | + exported, |
866 | + mutator_for, |
867 | + operation_parameters, |
868 | + ) |
869 | +from lazr.restful.fields import ( |
870 | + CollectionField, |
871 | + Reference, |
872 | + ) |
873 | +from zope.component import getUtility |
874 | +from zope.interface import Interface |
875 | +from zope.schema import ( |
876 | + Datetime, |
877 | + Int, |
878 | + Text, |
879 | + TextLine, |
880 | + ) |
881 | + |
882 | +from canonical.launchpad import _ |
883 | +from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities |
884 | +from lp.services.fields import ( |
885 | + PersonChoice, |
886 | + PublicPersonChoice, |
887 | + ) |
888 | + |
889 | + |
890 | +class IBranchMergeQueue(Interface): |
891 | + """An interface for managing branch merges.""" |
892 | + |
893 | + export_as_webservice_entry() |
894 | + |
895 | + id = Int(title=_('ID'), readonly=True, required=True) |
896 | + |
897 | + registrant = exported( |
898 | + PublicPersonChoice( |
899 | + title=_("The user that registered the branch."), |
900 | + required=True, readonly=True, |
901 | + vocabulary='ValidPersonOrTeam')) |
902 | + |
903 | + owner = exported( |
904 | + PersonChoice( |
905 | + title=_('Owner'), |
906 | + required=True, readonly=True, |
907 | + vocabulary='UserTeamsParticipationPlusSelf', |
908 | + description=_("The owner of the merge queue."))) |
909 | + |
910 | + name = exported( |
911 | + TextLine( |
912 | + title=_('Name'), required=True, |
913 | + description=_( |
914 | + "Keep very short, unique, and descriptive, because it will " |
915 | + "be used in URLs. " |
916 | + "Examples: main, devel, release-1.0, gnome-vfs."))) |
917 | + |
918 | + description = exported( |
919 | + Text( |
920 | + title=_('Description'), required=False, |
921 | + description=_( |
922 | + 'A short description of the purpose of this merge queue.'))) |
923 | + |
924 | + configuration = exported( |
925 | + TextLine( |
926 | + title=_('Configuration'), required=False, readonly=True, |
927 | + description=_( |
928 | + "A JSON string of configuration values."))) |
929 | + |
930 | + date_created = exported( |
931 | + Datetime( |
932 | + title=_('Date Created'), |
933 | + required=True, |
934 | + readonly=True)) |
935 | + |
936 | + branches = exported( |
937 | + CollectionField( |
938 | + title=_('Dependent Branches'), |
939 | + description=_( |
940 | + 'A collection of branches that this queue manages.'), |
941 | + readonly=True, |
942 | + value_type=Reference(Interface))) |
943 | + |
944 | + @mutator_for(configuration) |
945 | + @operation_parameters( |
946 | + config=TextLine(title=_("A JSON string of configuration values."))) |
947 | + @export_write_operation() |
948 | + def setMergeQueueConfig(config): |
949 | + """Set the JSON string configuration of the merge queue. |
950 | + |
951 | + :param config: A JSON string of configuration values. |
952 | + """ |
953 | + |
954 | + |
955 | +class IBranchMergeQueueSource(Interface): |
956 | + |
957 | + def new(name, owner, registrant, description, configuration, branches): |
958 | + """Create a new IBranchMergeQueue object. |
959 | + |
960 | + :param name: The name of the branch merge queue. |
961 | + :param description: A description of queue. |
962 | + :param configuration: A JSON string of configuration values. |
963 | + :param owner: The owner of the queue. |
964 | + :param registrant: The registrant of the queue. |
965 | + :param branches: A list of branches to add to the queue. |
966 | + """ |
967 | + |
968 | + |
969 | +def user_has_special_merge_queue_access(user): |
970 | + """Admins and bazaar experts have special access. |
971 | + |
972 | + :param user: A 'Person' or None. |
973 | + """ |
974 | + if user is None: |
975 | + return False |
976 | + celebs = getUtility(ILaunchpadCelebrities) |
977 | + return user.inTeam(celebs.admin) or user.inTeam(celebs.bazaar_experts) |
978 | |
979 | === added file 'lib/lp/code/model/branchmergequeue.py' |
980 | --- lib/lp/code/model/branchmergequeue.py 1970-01-01 00:00:00 +0000 |
981 | +++ lib/lp/code/model/branchmergequeue.py 2010-11-01 12:35:07 +0000 |
982 | @@ -0,0 +1,88 @@ |
983 | +# Copyright 2010 Canonical Ltd. This software is licensed under the |
984 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
985 | + |
986 | +"""Implementation classes for IBranchMergeQueue, etc.""" |
987 | + |
988 | +__metaclass__ = type |
989 | +__all__ = ['BranchMergeQueue'] |
990 | + |
991 | +import simplejson |
992 | +from storm.locals import ( |
993 | + Int, |
994 | + Reference, |
995 | + Store, |
996 | + Storm, |
997 | + Unicode, |
998 | + ) |
999 | +from zope.interface import ( |
1000 | + classProvides, |
1001 | + implements, |
1002 | + ) |
1003 | + |
1004 | +from canonical.database.datetimecol import UtcDateTimeCol |
1005 | +from canonical.launchpad.interfaces.lpstorm import IMasterStore |
1006 | +from lp.code.errors import InvalidMergeQueueConfig |
1007 | +from lp.code.interfaces.branchmergequeue import ( |
1008 | + IBranchMergeQueue, |
1009 | + IBranchMergeQueueSource, |
1010 | + ) |
1011 | +from lp.code.model.branch import Branch |
1012 | + |
1013 | + |
1014 | +class BranchMergeQueue(Storm): |
1015 | + """See `IBranchMergeQueue`.""" |
1016 | + |
1017 | + __storm_table__ = 'BranchMergeQueue' |
1018 | + implements(IBranchMergeQueue) |
1019 | + classProvides(IBranchMergeQueueSource) |
1020 | + |
1021 | + id = Int(primary=True) |
1022 | + |
1023 | + registrant_id = Int(name='registrant', allow_none=True) |
1024 | + registrant = Reference(registrant_id, 'Person.id') |
1025 | + |
1026 | + owner_id = Int(name='owner', allow_none=True) |
1027 | + owner = Reference(owner_id, 'Person.id') |
1028 | + |
1029 | + name = Unicode(allow_none=False) |
1030 | + description = Unicode(allow_none=False) |
1031 | + configuration = Unicode(allow_none=False) |
1032 | + |
1033 | + date_created = UtcDateTimeCol(notNull=True) |
1034 | + |
1035 | + @property |
1036 | + def branches(self): |
1037 | + """See `IBranchMergeQueue`.""" |
1038 | + return Store.of(self).find( |
1039 | + Branch, |
1040 | + Branch.merge_queue_id == self.id) |
1041 | + |
1042 | + def setMergeQueueConfig(self, config): |
1043 | + """See `IBranchMergeQueue`.""" |
1044 | + try: |
1045 | + simplejson.loads(config) |
1046 | + self.configuration = config |
1047 | + except ValueError: # The config string is not valid JSON |
1048 | + raise InvalidMergeQueueConfig |
1049 | + |
1050 | + @classmethod |
1051 | + def new(cls, name, owner, registrant, description=None, |
1052 | + configuration=None, branches=None): |
1053 | + """See `IBranchMergeQueueSource`.""" |
1054 | + store = IMasterStore(BranchMergeQueue) |
1055 | + |
1056 | + if configuration is None: |
1057 | + configuration = unicode(simplejson.dumps({})) |
1058 | + |
1059 | + queue = cls() |
1060 | + queue.name = name |
1061 | + queue.owner = owner |
1062 | + queue.registrant = registrant |
1063 | + queue.description = description |
1064 | + queue.configuration = configuration |
1065 | + if branches is not None: |
1066 | + for branch in branches: |
1067 | + branch.addToQueue(queue) |
1068 | + |
1069 | + store.add(queue) |
1070 | + return queue |
1071 | |
1072 | === added file 'lib/lp/code/model/tests/test_branchmergequeue.py' |
1073 | --- lib/lp/code/model/tests/test_branchmergequeue.py 1970-01-01 00:00:00 +0000 |
1074 | +++ lib/lp/code/model/tests/test_branchmergequeue.py 2010-10-25 14:51:56 +0000 |
1075 | @@ -0,0 +1,155 @@ |
1076 | +# Copyright 2010 Canonical Ltd. This software is licensed under the |
1077 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
1078 | + |
1079 | +"""Unit tests for methods of BranchMergeQueue.""" |
1080 | + |
1081 | +from __future__ import with_statement |
1082 | + |
1083 | +import simplejson |
1084 | + |
1085 | +from canonical.launchpad.interfaces.lpstorm import IStore |
1086 | +from canonical.launchpad.webapp.testing import verifyObject |
1087 | +from canonical.testing.layers import ( |
1088 | + AppServerLayer, |
1089 | + DatabaseFunctionalLayer, |
1090 | + ) |
1091 | +from lp.code.errors import InvalidMergeQueueConfig |
1092 | +from lp.code.interfaces.branchmergequeue import IBranchMergeQueue |
1093 | +from lp.code.model.branchmergequeue import BranchMergeQueue |
1094 | +from lp.testing import ( |
1095 | + ANONYMOUS, |
1096 | + person_logged_in, |
1097 | + launchpadlib_for, |
1098 | + TestCaseWithFactory, |
1099 | + ws_object, |
1100 | + ) |
1101 | + |
1102 | + |
1103 | +class TestBranchMergeQueueInterface(TestCaseWithFactory): |
1104 | + """Test IBranchMergeQueue interface.""" |
1105 | + |
1106 | + layer = DatabaseFunctionalLayer |
1107 | + |
1108 | + def test_implements_interface(self): |
1109 | + queue = self.factory.makeBranchMergeQueue() |
1110 | + IStore(BranchMergeQueue).add(queue) |
1111 | + verifyObject(IBranchMergeQueue, queue) |
1112 | + |
1113 | + |
1114 | +class TestBranchMergeQueueSource(TestCaseWithFactory): |
1115 | + """Test the methods of IBranchMergeQueueSource.""" |
1116 | + |
1117 | + layer = DatabaseFunctionalLayer |
1118 | + |
1119 | + def test_new(self): |
1120 | + owner = self.factory.makePerson() |
1121 | + name = u'SooperQueue' |
1122 | + description = u'This is Sooper Queue' |
1123 | + config = unicode(simplejson.dumps({'test': 'make check'})) |
1124 | + |
1125 | + queue = BranchMergeQueue.new( |
1126 | + name, owner, owner, description, config) |
1127 | + |
1128 | + self.assertEqual(queue.name, name) |
1129 | + self.assertEqual(queue.owner, owner) |
1130 | + self.assertEqual(queue.registrant, owner) |
1131 | + self.assertEqual(queue.description, description) |
1132 | + self.assertEqual(queue.configuration, config) |
1133 | + |
1134 | + |
1135 | +class TestBranchMergeQueue(TestCaseWithFactory): |
1136 | + """Test the functions of the BranchMergeQueue.""" |
1137 | + |
1138 | + layer = DatabaseFunctionalLayer |
1139 | + |
1140 | + def test_branches(self): |
1141 | + """Test that a merge queue can get all its managed branches.""" |
1142 | + store = IStore(BranchMergeQueue) |
1143 | + |
1144 | + queue = self.factory.makeBranchMergeQueue() |
1145 | + store.add(queue) |
1146 | + |
1147 | + branch = self.factory.makeBranch() |
1148 | + store.add(branch) |
1149 | + with person_logged_in(branch.owner): |
1150 | + branch.addToQueue(queue) |
1151 | + |
1152 | + self.assertEqual( |
1153 | + list(queue.branches), |
1154 | + [branch]) |
1155 | + |
1156 | + def test_setMergeQueueConfig(self): |
1157 | + """Test that the configuration is set properly.""" |
1158 | + queue = self.factory.makeBranchMergeQueue() |
1159 | + config = unicode(simplejson.dumps({ |
1160 | + 'test': 'make test'})) |
1161 | + |
1162 | + with person_logged_in(queue.owner): |
1163 | + queue.setMergeQueueConfig(config) |
1164 | + |
1165 | + self.assertEqual(queue.configuration, config) |
1166 | + |
1167 | + def test_setMergeQueueConfig_invalid_json(self): |
1168 | + """Test that invalid json can't be set as the config.""" |
1169 | + queue = self.factory.makeBranchMergeQueue() |
1170 | + |
1171 | + with person_logged_in(queue.owner): |
1172 | + self.assertRaises( |
1173 | + InvalidMergeQueueConfig, |
1174 | + queue.setMergeQueueConfig, |
1175 | + 'abc') |
1176 | + |
1177 | + |
1178 | +class TestWebservice(TestCaseWithFactory): |
1179 | + |
1180 | + layer = AppServerLayer |
1181 | + |
1182 | + def test_properties(self): |
1183 | + """Test that the correct properties are exposed.""" |
1184 | + with person_logged_in(ANONYMOUS): |
1185 | + name = u'teh-queue' |
1186 | + description = u'Oh hai! I are a queues' |
1187 | + configuration = unicode(simplejson.dumps({'test': 'make check'})) |
1188 | + |
1189 | + queuer = self.factory.makePerson() |
1190 | + db_queue = self.factory.makeBranchMergeQueue( |
1191 | + registrant=queuer, owner=queuer, name=name, |
1192 | + description=description, |
1193 | + configuration=configuration) |
1194 | + branch1 = self.factory.makeBranch() |
1195 | + with person_logged_in(branch1.owner): |
1196 | + branch1.addToQueue(db_queue) |
1197 | + branch2 = self.factory.makeBranch() |
1198 | + with person_logged_in(branch2.owner): |
1199 | + branch2.addToQueue(db_queue) |
1200 | + launchpad = launchpadlib_for('test', db_queue.owner, |
1201 | + service_root="http://api.launchpad.dev:8085") |
1202 | + |
1203 | + queuer = ws_object(launchpad, queuer) |
1204 | + queue = ws_object(launchpad, db_queue) |
1205 | + branch1 = ws_object(launchpad, branch1) |
1206 | + branch2 = ws_object(launchpad, branch2) |
1207 | + |
1208 | + self.assertEqual(queue.registrant, queuer) |
1209 | + self.assertEqual(queue.owner, queuer) |
1210 | + self.assertEqual(queue.name, name) |
1211 | + self.assertEqual(queue.description, description) |
1212 | + self.assertEqual(queue.configuration, configuration) |
1213 | + self.assertEqual(queue.date_created, db_queue.date_created) |
1214 | + self.assertEqual(len(queue.branches), 2) |
1215 | + |
1216 | + def test_set_configuration(self): |
1217 | + """Test the mutator for setting configuration.""" |
1218 | + with person_logged_in(ANONYMOUS): |
1219 | + db_queue = self.factory.makeBranchMergeQueue() |
1220 | + launchpad = launchpadlib_for('test', db_queue.owner, |
1221 | + service_root="http://api.launchpad.dev:8085") |
1222 | + |
1223 | + configuration = simplejson.dumps({'test': 'make check'}) |
1224 | + |
1225 | + queue = ws_object(launchpad, db_queue) |
1226 | + queue.configuration = configuration |
1227 | + queue.lp_save() |
1228 | + |
1229 | + queue2 = ws_object(launchpad, db_queue) |
1230 | + self.assertEqual(queue2.configuration, configuration) |
1231 | |
1232 | === modified file 'lib/lp/registry/tests/test_mlists.py' |
1233 | --- lib/lp/registry/tests/test_mlists.py 2010-08-20 20:31:18 +0000 |
1234 | +++ lib/lp/registry/tests/test_mlists.py 2010-11-28 00:57:58 +0000 |
1235 | @@ -26,6 +26,7 @@ |
1236 | from canonical.launchpad.scripts.mlistimport import Importer |
1237 | from canonical.testing.layers import ( |
1238 | AppServerLayer, |
1239 | + BaseLayer, |
1240 | DatabaseFunctionalLayer, |
1241 | LayerProcessController, |
1242 | ) |
1243 | @@ -399,7 +400,7 @@ |
1244 | args.append(self.team.name) |
1245 | return Popen(args, stdout=PIPE, stderr=STDOUT, |
1246 | cwd=LayerProcessController.appserver_config.root, |
1247 | - env=dict(LPCONFIG='testrunner-appserver', |
1248 | + env=dict(LPCONFIG=BaseLayer.appserver_config_name, |
1249 | PATH=os.environ['PATH'])) |
1250 | |
1251 | def test_import(self): |
1252 | |
1253 | === modified file 'lib/lp/services/memcache/doc/restful-cache.txt' |
1254 | --- lib/lp/services/memcache/doc/restful-cache.txt 2010-07-16 09:06:21 +0000 |
1255 | +++ lib/lp/services/memcache/doc/restful-cache.txt 2010-11-28 00:57:58 +0000 |
1256 | @@ -23,7 +23,7 @@ |
1257 | >>> cache_key = cache.key_for( |
1258 | ... person, 'media/type', 'web-service-version') |
1259 | >>> print person.id, cache_key |
1260 | - 29 Person(29,),testrunner,media/type,web-service-version |
1261 | + 29 Person(29,),testrunner...,media/type,web-service-version |
1262 | |
1263 | >>> from operator import attrgetter |
1264 | >>> languages = sorted(person.languages, key=attrgetter('englishname')) |
1265 | @@ -31,8 +31,8 @@ |
1266 | ... cache_key = cache.key_for( |
1267 | ... language, 'media/type', 'web-service-version') |
1268 | ... print language.id, cache_key |
1269 | - 119 Language(119,),testrunner,media/type,web-service-version |
1270 | - 521 Language(521,),testrunner,media/type,web-service-version |
1271 | + 119 Language(119,),testrunner...,media/type,web-service-version |
1272 | + 521 Language(521,),testrunner...,media/type,web-service-version |
1273 | |
1274 | The cache starts out empty. |
1275 | |
1276 | |
1277 | === modified file 'lib/lp/services/scripts/doc/script-monitoring.txt' |
1278 | --- lib/lp/services/scripts/doc/script-monitoring.txt 2009-07-28 13:47:01 +0000 |
1279 | +++ lib/lp/services/scripts/doc/script-monitoring.txt 2010-11-28 00:57:58 +0000 |
1280 | @@ -97,9 +97,6 @@ |
1281 | |
1282 | Prepare an environment to run the testing script. |
1283 | |
1284 | - >>> env = dict(os.environ) |
1285 | - >>> env['LPCONFIG'] = 'testrunner' |
1286 | - |
1287 | >>> import canonical |
1288 | >>> lp_tree = os.path.join( |
1289 | ... os.path.dirname(canonical.__file__), os.pardir, os.pardir) |
1290 | @@ -108,8 +105,8 @@ |
1291 | We'll now run this script, telling it to fail: |
1292 | |
1293 | >>> proc = subprocess.Popen([lp_py, script_file.name, 'fail'], |
1294 | - ... env=env, stdin=subprocess.PIPE, |
1295 | - ... stdout=subprocess.PIPE, stderr=subprocess.PIPE) |
1296 | + ... stdin=subprocess.PIPE, stdout=subprocess.PIPE, |
1297 | + ... stderr=subprocess.PIPE) |
1298 | >>> (out, err) = proc.communicate() |
1299 | >>> transaction.abort() |
1300 | |
1301 | @@ -126,8 +123,8 @@ |
1302 | If we run it such that it succeeds, we will get an activity record: |
1303 | |
1304 | >>> proc = subprocess.Popen([lp_py, script_file.name, 'pass'], |
1305 | - ... env=env, stdin=subprocess.PIPE, |
1306 | - ... stdout=subprocess.PIPE, stderr=subprocess.PIPE) |
1307 | + ... stdin=subprocess.PIPE, stdout=subprocess.PIPE, |
1308 | + ... stderr=subprocess.PIPE) |
1309 | >>> (out, err) = proc.communicate() |
1310 | >>> transaction.abort() |
1311 | |
1312 | |
1313 | === modified file 'scripts/gina.py' |
1314 | --- scripts/gina.py 2010-11-25 02:59:54 +0000 |
1315 | +++ scripts/gina.py 2010-11-28 00:57:58 +0000 |
1316 | @@ -79,7 +79,7 @@ |
1317 | |
1318 | dry_run = options.dry_run |
1319 | |
1320 | - LPDB = lp.dbname |
1321 | + LPDB = lp.get_dbname() |
1322 | LPDB_HOST = lp.dbhost |
1323 | LPDB_USER = config.gina.dbuser |
1324 | KTDB = target_section.katie_dbname |
Hey Rob,
This looks good. Two concerns:
1. The branch adds a new 'print' to pgsql.py. Is this leftover debugging? If so, delete it. If not, you're going to have to print more information, otherwise no one else will know what's going on.
2. The way we add to the config here seems fragile: get the base layer, get two different configs, add the same thing twice. I wish there was a get_all_configs() or some kind of Composite object so that the knowledge about the -appserver business didn't have to be spread around so much.
I'm not going to block on the second point. Please address the first and then land.
jml