Launchpad itself

Merge lp:~lifeless/launchpad/databasefixture into lp:launchpad

  1. databasefixture
  2. Merge into devel
Proposed by Robert Collins
Status: Merged
Approved by: Jonathan Lange
Approved revision: no longer in the source branch.
Merged at revision: 12019
Proposed branch: lp:~lifeless/launchpad/databasefixture
Merge into: lp:launchpad
Prerequisite: lp:~lifeless/launchpad/uniqueconfig
Diff against target: 1324 lines (+659/-204)
18 files modified
database/schema/fti.py (+1/-1)
lib/canonical/config/fixture.py (+8/-4)
lib/canonical/database/sqlbase.py (+1/-1)
lib/canonical/ftests/pgsql.py (+22/-2)
lib/canonical/ftests/test_pgsql.py (+25/-2)
lib/canonical/launchpad/doc/canonical-config.txt (+26/-21)
lib/canonical/launchpad/doc/scripts-and-zcml.txt (+1/-4)
lib/canonical/launchpad/scripts/__init__.py (+6/-6)
lib/canonical/lp/__init__.py (+14/-6)
lib/canonical/testing/ftests/test_layers.py (+12/-7)
lib/canonical/testing/layers.py (+161/-138)
lib/lp/code/interfaces/branchmergequeue.py (+129/-0)
lib/lp/code/model/branchmergequeue.py (+88/-0)
lib/lp/code/model/tests/test_branchmergequeue.py (+155/-0)
lib/lp/registry/tests/test_mlists.py (+2/-1)
lib/lp/services/memcache/doc/restful-cache.txt (+3/-3)
lib/lp/services/scripts/doc/script-monitoring.txt (+4/-7)
scripts/gina.py (+1/-1)
To merge this branch: bzr merge lp:~lifeless/launchpad/databasefixture
Reviewer Review Type Date Requested Status
Jonathan Lange (community) Approve
Review via email: mp+38694@code.launchpad.net

Commit message

Using unique DB names for test running.

Description of the change

Start doing unique db names by default, permitting parallel testing of layers upto and including DatabaseLayer, as long as the appserver isn't started (because that has statically configured ports)

Currently has a test failure on ec2 setting up the db. The legacy canonical.lp.dbhost(sp?) which is meant to be better integrated in this branch fails to pickup the -d launchpad_ftest_template parameter and thus tries to open launchpad_dev instead - which doesn't exist.

To post a comment you must log in.
Revision history for this message
Jonathan Lange (jml) wrote :

Hey Rob,

This looks good. Two concerns:

 1. The branch adds a new 'print' to pgsql.py. Is this leftover debugging? If so, delete it. If not, you're going to have to print more information, otherwise no one else will know what's going on.

 2. The way we add to the config here seems fragile: get the base layer, get two different configs, add the same thing twice. I wish there was a get_all_configs() or some kind of Composite object so that the knowledge about the -appserver business didn't have to be spread around so much.

I'm not going to block on the second point. Please address the first and then land.

jml

review: Approve
Revision history for this message
Robert Collins (lifeless) wrote :

Thanks jml; turns out its got furrther (shallow) changes needed to not
break things (ProcessLayerController needed some love). This doesn't
help (or hinder) your point 2. I think that wanting a composite is
fair, but we'd want to be able to inspect the innards anyway which
somewhat defeats having a composite. For now, I think treating
BaseLayer as that composite itself isn't unreasonable.

Revision history for this message
Jonathan Lange (jml) wrote :

Well, even then, we could have an API for asking BaseLayer what its configs are, rather than assuming we know. Even so, it's a small point. The real problem with the configs runs deeper.

Revision history for this message
Jonathan Lange (jml) wrote :

It's hard to see if anything much changed in the LibrarianLayer, since diff completely removes and then adds it.

The rest of the changes look sane to me, but I haven't gone through them with a tooth comb looking for correctness issues.

Revision history for this message
Robert Collins (lifeless) wrote :

On Wed, Oct 20, 2010 at 4:48 AM, Jonathan Lange <email address hidden> wrote:
> It's hard to see if anything much changed in the LibrarianLayer, since diff completely removes and then adds it.

Yeah, it would be nice to show that better. Nothing did.

> The rest of the changes look sane to me, but I haven't gone through them with a tooth comb looking for correctness issues.

Thats fine, thanks.

-Rob

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'database/schema/fti.py'
--- database/schema/fti.py 2010-06-10 09:32:34 +0000
+++ database/schema/fti.py 2010-11-28 00:57:58 +0000
@@ -314,7 +314,7 @@
314 """314 """
315315
316 log.debug('Installing tsearch2')316 log.debug('Installing tsearch2')
317 cmd = 'psql -f - -d %s' % lp.dbname317 cmd = 'psql -f - -d %s' % lp.get_dbname()
318 if lp.dbhost:318 if lp.dbhost:
319 cmd += ' -h %s' % lp.dbhost319 cmd += ' -h %s' % lp.dbhost
320 if options.dbuser:320 if options.dbuser:
321321
=== modified file 'lib/canonical/config/fixture.py'
--- lib/canonical/config/fixture.py 2010-10-26 15:47:24 +0000
+++ lib/canonical/config/fixture.py 2010-11-28 00:57:58 +0000
@@ -40,17 +40,21 @@
40 self.instance_name = instance_name40 self.instance_name = instance_name
41 self.copy_from_instance = copy_from_instance41 self.copy_from_instance = copy_from_instance
4242
43 def add_section(self, sectioncontent):
44 """Add sectioncontent to the lazy config."""
45 with open(self.absroot + '/launchpad-lazr.conf', 'ab') as out:
46 out.write(sectioncontent)
47
43 def setUp(self):48 def setUp(self):
44 super(ConfigFixture, self).setUp()49 super(ConfigFixture, self).setUp()
45 root = 'configs/' + self.instance_name50 root = 'configs/' + self.instance_name
46 os.mkdir(root)51 os.mkdir(root)
47 absroot = os.path.abspath(root)52 self.absroot = os.path.abspath(root)
48 self.addCleanup(shutil.rmtree, absroot)53 self.addCleanup(shutil.rmtree, self.absroot)
49 source = 'configs/' + self.copy_from_instance54 source = 'configs/' + self.copy_from_instance
50 for basename in os.listdir(source):55 for basename in os.listdir(source):
51 if basename == 'launchpad-lazr.conf':56 if basename == 'launchpad-lazr.conf':
52 with open(root + '/launchpad-lazr.conf', 'wb') as out:57 self.add_section(self._extend_str % self.copy_from_instance)
53 out.write(self._extend_str % self.copy_from_instance)
54 continue58 continue
55 with open(source + '/' + basename, 'rb') as input:59 with open(source + '/' + basename, 'rb') as input:
56 with open(root + '/' + basename, 'wb') as out:60 with open(root + '/' + basename, 'wb') as out:
5761
=== modified file 'lib/canonical/database/sqlbase.py'
--- lib/canonical/database/sqlbase.py 2010-11-08 12:52:43 +0000
+++ lib/canonical/database/sqlbase.py 2010-11-28 00:57:58 +0000
@@ -820,7 +820,7 @@
820 con_str = re.sub(r'host=\S*', '', con_str) # Remove stanza if exists.820 con_str = re.sub(r'host=\S*', '', con_str) # Remove stanza if exists.
821 con_str_overrides.append('host=%s' % lp.dbhost)821 con_str_overrides.append('host=%s' % lp.dbhost)
822 if dbname is None:822 if dbname is None:
823 dbname = lp.dbname # Note that lp.dbname may be None.823 dbname = lp.get_dbname() # Note that lp.dbname may be None.
824 if dbname is not None:824 if dbname is not None:
825 con_str = re.sub(r'dbname=\S*', '', con_str) # Remove if exists.825 con_str = re.sub(r'dbname=\S*', '', con_str) # Remove if exists.
826 con_str_overrides.append('dbname=%s' % dbname)826 con_str_overrides.append('dbname=%s' % dbname)
827827
=== modified file 'lib/canonical/ftests/pgsql.py'
--- lib/canonical/ftests/pgsql.py 2010-10-17 09:51:08 +0000
+++ lib/canonical/ftests/pgsql.py 2010-11-28 00:57:58 +0000
@@ -8,9 +8,12 @@
8__metaclass__ = type8__metaclass__ = type
99
10import os10import os
11import random
11import time12import time
1213
13import psycopg214import psycopg2
15
16from canonical.config import config
14from canonical.database.postgresql import (17from canonical.database.postgresql import (
15 generateResetSequencesSQL, resetSequences)18 generateResetSequencesSQL, resetSequences)
1619
@@ -159,7 +162,7 @@
159 # Class attribute. True if we should destroy the DB because changes made.162 # Class attribute. True if we should destroy the DB because changes made.
160 _reset_db = True163 _reset_db = True
161164
162 def __init__(self, template=None, dbname=None, dbuser=None,165 def __init__(self, template=None, dbname=dynamic, dbuser=None,
163 host=None, port=None, reset_sequences_sql=None):166 host=None, port=None, reset_sequences_sql=None):
164 '''Construct the PgTestSetup167 '''Construct the PgTestSetup
165168
@@ -169,9 +172,25 @@
169 if template is not None:172 if template is not None:
170 self.template = template173 self.template = template
171 if dbname is PgTestSetup.dynamic:174 if dbname is PgTestSetup.dynamic:
175 from canonical.testing.layers import BaseLayer
172 if os.environ.get('LP_TEST_INSTANCE'):176 if os.environ.get('LP_TEST_INSTANCE'):
173 self.dbname = "%s_%s" % (177 self.dbname = "%s_%s" % (
174 self.__class__.dbname, os.environ.get('LP_TEST_INSTANCE'))178 self.__class__.dbname, os.environ.get('LP_TEST_INSTANCE'))
179 # Stash the name we use in the config if a writable config is
180 # available.
181 # Avoid circular imports
182 section = """[database]
183rw_main_master: dbname=%s
184rw_main_slave: dbname=%s
185
186""" % (self.dbname, self.dbname)
187 if BaseLayer.config_fixture is not None:
188 BaseLayer.config_fixture.add_section(section)
189 if BaseLayer.appserver_config_fixture is not None:
190 BaseLayer.appserver_config_fixture.add_section(section)
191 if config.instance_name in (
192 BaseLayer.config_name, BaseLayer.appserver_config_name):
193 config.reloadConfig()
175 else:194 else:
176 # Fallback to the class name.195 # Fallback to the class name.
177 self.dbname = self.__class__.dbname196 self.dbname = self.__class__.dbname
@@ -257,7 +276,8 @@
257 raise276 raise
258 finally:277 finally:
259 con.close()278 con.close()
260 time.sleep(1)279 # Let the other user complete their copying of the template DB.
280 time.sleep(random.random())
261 ConnectionWrapper.committed = False281 ConnectionWrapper.committed = False
262 ConnectionWrapper.dirty = False282 ConnectionWrapper.dirty = False
263 PgTestSetup._last_db = (self.template, self.dbname)283 PgTestSetup._last_db = (self.template, self.dbname)
264284
=== modified file 'lib/canonical/ftests/test_pgsql.py'
--- lib/canonical/ftests/test_pgsql.py 2010-10-17 09:51:08 +0000
+++ lib/canonical/ftests/test_pgsql.py 2010-11-28 00:57:58 +0000
@@ -9,10 +9,13 @@
9 )9 )
10import testtools10import testtools
1111
12from canonical.config import config, dbconfig
13from canonical.config.fixture import ConfigUseFixture
12from canonical.ftests.pgsql import (14from canonical.ftests.pgsql import (
13 ConnectionWrapper,15 ConnectionWrapper,
14 PgTestSetup,16 PgTestSetup,
15 )17 )
18from canonical.testing.layers import BaseLayer
1619
1720
18class TestPgTestSetup(testtools.TestCase, TestWithFixtures):21class TestPgTestSetup(testtools.TestCase, TestWithFixtures):
@@ -30,9 +33,10 @@
3033
31 def test_db_naming_LP_TEST_INSTANCE_set(self):34 def test_db_naming_LP_TEST_INSTANCE_set(self):
32 # when LP_TEST_INSTANCE is set, it is used for dynamic db naming.35 # when LP_TEST_INSTANCE is set, it is used for dynamic db naming.
33 self.useFixture(EnvironmentVariableFixture('LP_TEST_INSTANCE', 'xx'))36 BaseLayer.setUp()
37 self.addCleanup(BaseLayer.tearDown)
34 fixture = PgTestSetup(dbname=PgTestSetup.dynamic)38 fixture = PgTestSetup(dbname=PgTestSetup.dynamic)
35 expected_name = "%s_xx" % (PgTestSetup.dbname,)39 expected_name = "%s_%d" % (PgTestSetup.dbname, os.getpid())
36 self.assertDBName(expected_name, fixture)40 self.assertDBName(expected_name, fixture)
3741
38 def test_db_naming_without_LP_TEST_INSTANCE_is_static(self):42 def test_db_naming_without_LP_TEST_INSTANCE_is_static(self):
@@ -41,6 +45,25 @@
41 expected_name = PgTestSetup.dbname45 expected_name = PgTestSetup.dbname
42 self.assertDBName(expected_name, fixture)46 self.assertDBName(expected_name, fixture)
4347
48 def test_db_naming_stored_in_BaseLayer_configs(self):
49 BaseLayer.setUp()
50 self.addCleanup(BaseLayer.tearDown)
51 fixture = PgTestSetup(dbname=PgTestSetup.dynamic)
52 fixture.setUp()
53 self.addCleanup(fixture.dropDb)
54 self.addCleanup(fixture.tearDown)
55 expected_value = 'dbname=%s' % fixture.dbname
56 self.assertEqual(expected_value, dbconfig.rw_main_master)
57 self.assertEqual(expected_value, dbconfig.rw_main_slave)
58 with ConfigUseFixture(BaseLayer.appserver_config_name):
59 self.assertEqual(expected_value, dbconfig.rw_main_master)
60 self.assertEqual(expected_value, dbconfig.rw_main_slave)
61
62
63class TestPgTestSetupTuning(testtools.TestCase, TestWithFixtures):
64
65 layer = BaseLayer
66
44 def testOptimization(self):67 def testOptimization(self):
45 # Test to ensure that the database is destroyed only when necessary68 # Test to ensure that the database is destroyed only when necessary
4669
4770
=== modified file 'lib/canonical/launchpad/doc/canonical-config.txt'
--- lib/canonical/launchpad/doc/canonical-config.txt 2010-10-21 02:00:15 +0000
+++ lib/canonical/launchpad/doc/canonical-config.txt 2010-11-28 00:57:58 +0000
@@ -121,59 +121,65 @@
121Setting just the instance_name will change the directory from which the121Setting just the instance_name will change the directory from which the
122conf file is loaded.122conf file is loaded.
123123
124 >>> config.setInstance('development')124 >>> from canonical.config import CanonicalConfig
125 >>> config.instance_name125 >>> test_config = CanonicalConfig('testrunner', 'test')
126 >>> test_config.setInstance('development')
127 >>> test_config.instance_name
126 'development'128 'development'
127129
128 >>> config.filename130 >>> test_config.filename
129 '.../configs/development/launchpad-lazr.conf'131 '.../configs/development/launchpad-lazr.conf'
130 >>> config.extends.filename132 >>> test_config.extends.filename
131 '.../config/schema-lazr.conf'133 '.../config/schema-lazr.conf'
132134
133 >>> config.answertracker.days_before_expiration135 >>> test_config.answertracker.days_before_expiration
134 15136 15
135137
136Changing the instance_name and process_name changes the directory and138Changing the instance_name and process_name changes the directory and
137conf file name that is loaded.139conf file name that is loaded.
138140
139 >>> config.setInstance('testrunner')141 >>> test_config.setInstance('testrunner')
140 >>> config.instance_name142 >>> test_config.instance_name
141 'testrunner'143 'testrunner'
142144
143 >>> config.answertracker.days_before_expiration145 >>> test_config.answertracker.days_before_expiration
144 15146 15
145147
146 >>> config.setProcess('test-process')148 >>> test_config.setProcess('test-process')
147 >>> config.process_name149 >>> test_config.process_name
148 'test-process'150 'test-process'
149151
150 >>> config.filename152 >>> test_config.filename
151 '.../configs/testrunner/test-process-lazr.conf'153 '.../configs/testrunner/test-process-lazr.conf'
152 >>> config.extends.filename154 >>> test_config.extends.filename
153 '.../configs/testrunner/launchpad-lazr.conf'155 '.../configs/testrunner/launchpad-lazr.conf'
154156
155 >>> config.answertracker.days_before_expiration157 >>> test_config.answertracker.days_before_expiration
156 300158 300
157159
158The default 'launchpad-lazr.conf' is loaded if no conf files match160The default 'launchpad-lazr.conf' is loaded if no conf files match
159the process's name.161the process's name.
160162
161 >>> config.setInstance('testrunner')163 >>> test_config.setInstance('testrunner')
162 >>> config.instance_name164 >>> test_config.instance_name
163 'testrunner'165 'testrunner'
164166
165 >>> config.setProcess('test_no_conf')167 >>> test_config.setProcess('test_no_conf')
166 >>> config.process_name168 >>> test_config.process_name
167 'test_no_conf'169 'test_no_conf'
168170
169 >>> config.filename171 >>> test_config.filename
170 '.../configs/testrunner/launchpad-lazr.conf'172 '.../configs/testrunner/launchpad-lazr.conf'
171 >>> config.extends.filename173 >>> test_config.extends.filename
172 '.../configs/development/launchpad-lazr.conf'174 '.../configs/development/launchpad-lazr.conf'
173175
174 >>> config.answertracker.days_before_expiration176 >>> test_config.answertracker.days_before_expiration
175 15177 15
176178
179 >>> # setInstance sets the LPCONFIG environment variable, so set it
180 >>> # back to the real value.
181 >>> config.setInstance(config.instance_name)
182
177The initial instance_name is set via the LPCONFIG environment variable.183The initial instance_name is set via the LPCONFIG environment variable.
178Because Config is designed to failover to the default development184Because Config is designed to failover to the default development
179environment, and the testrunner overrides the environment and config,185environment, and the testrunner overrides the environment and config,
@@ -183,7 +189,6 @@
183Alternatively, the instance name and process name can be specified as189Alternatively, the instance name and process name can be specified as
184argument to the constructor.190argument to the constructor.
185191
186 >>> from canonical.config import CanonicalConfig
187 >>> dev_config = CanonicalConfig('development', 'authserver')192 >>> dev_config = CanonicalConfig('development', 'authserver')
188 >>> dev_config.instance_name193 >>> dev_config.instance_name
189 'development'194 'development'
190195
=== modified file 'lib/canonical/launchpad/doc/scripts-and-zcml.txt'
--- lib/canonical/launchpad/doc/scripts-and-zcml.txt 2010-10-09 16:36:22 +0000
+++ lib/canonical/launchpad/doc/scripts-and-zcml.txt 2010-11-28 00:57:58 +0000
@@ -23,13 +23,10 @@
2323
24Run the script (making sure it uses the testrunner configuration).24Run the script (making sure it uses the testrunner configuration).
2525
26 >>> env = dict(os.environ)
27 >>> env['LPCONFIG'] = 'testrunner'
28 >>> from canonical.config import config26 >>> from canonical.config import config
29 >>> bin_py = os.path.join(config.root, 'bin', 'py')27 >>> bin_py = os.path.join(config.root, 'bin', 'py')
30 >>> proc = subprocess.Popen([bin_py, script_file.name], 28 >>> proc = subprocess.Popen([bin_py, script_file.name],
31 ... stdout=subprocess.PIPE, stderr=None,29 ... stdout=subprocess.PIPE, stderr=None)
32 ... env=env)
3330
34Check that we get the expected output.31Check that we get the expected output.
3532
3633
=== modified file 'lib/canonical/launchpad/scripts/__init__.py'
--- lib/canonical/launchpad/scripts/__init__.py 2010-10-20 01:23:52 +0000
+++ lib/canonical/launchpad/scripts/__init__.py 2010-11-28 00:57:58 +0000
@@ -132,13 +132,13 @@
132132
133 dbname and dbhost are also propagated to config.database.dbname and133 dbname and dbhost are also propagated to config.database.dbname and
134 config.database.dbhost. dbname, dbhost and dbuser are also propagated to134 config.database.dbhost. dbname, dbhost and dbuser are also propagated to
135 lp.dbname, lp.dbhost and lp.dbuser. This ensures that all systems will135 lp.get_dbname(), lp.dbhost and lp.dbuser. This ensures that all systems will
136 be using the requested connection details.136 be using the requested connection details.
137137
138 To test, we first need to store the current values so we can reset them138 To test, we first need to store the current values so we can reset them
139 later.139 later.
140140
141 >>> dbname, dbhost, dbuser = lp.dbname, lp.dbhost, lp.dbuser141 >>> dbname, dbhost, dbuser = lp.get_dbname(), lp.dbhost, lp.dbuser
142142
143 Ensure that command line options propagate to where we say they do143 Ensure that command line options propagate to where we say they do
144144
@@ -146,7 +146,7 @@
146 >>> db_options(parser)146 >>> db_options(parser)
147 >>> options, args = parser.parse_args(147 >>> options, args = parser.parse_args(
148 ... ['--dbname=foo', '--host=bar', '--user=baz'])148 ... ['--dbname=foo', '--host=bar', '--user=baz'])
149 >>> options.dbname, lp.dbname, config.database.dbname149 >>> options.dbname, lp.get_dbname(), config.database.dbname
150 ('foo', 'foo', 'foo')150 ('foo', 'foo', 'foo')
151 >>> (options.dbhost, lp.dbhost, config.database.dbhost)151 >>> (options.dbhost, lp.dbhost, config.database.dbhost)
152 ('bar', 'bar', 'bar')152 ('bar', 'bar', 'bar')
@@ -163,7 +163,7 @@
163163
164 Reset config164 Reset config
165165
166 >>> lp.dbname, lp.dbhost, lp.dbuser = dbname, dbhost, dbuser166 >>> lp.dbhost, lp.dbuser = dbhost, dbuser
167 """167 """
168 def dbname_callback(option, opt_str, value, parser):168 def dbname_callback(option, opt_str, value, parser):
169 parser.values.dbname = value169 parser.values.dbname = value
@@ -172,11 +172,11 @@
172 dbname: %s172 dbname: %s
173 """ % value)173 """ % value)
174 config.push('dbname_callback', config_data)174 config.push('dbname_callback', config_data)
175 lp.dbname = value175 lp.dbname_override = value
176176
177 parser.add_option(177 parser.add_option(
178 "-d", "--dbname", action="callback", callback=dbname_callback,178 "-d", "--dbname", action="callback", callback=dbname_callback,
179 type="string", dest="dbname", default=lp.dbname,179 type="string", dest="dbname", default=config.database.rw_main_master,
180 help="PostgreSQL database to connect to."180 help="PostgreSQL database to connect to."
181 )181 )
182182
183183
=== modified file 'lib/canonical/lp/__init__.py'
--- lib/canonical/lp/__init__.py 2009-06-25 05:39:50 +0000
+++ lib/canonical/lp/__init__.py 2010-11-28 00:57:58 +0000
@@ -20,7 +20,7 @@
2020
2121
22__all__ = [22__all__ = [
23 'dbname', 'dbhost', 'dbuser', 'isZopeless', 'initZopeless',23 'get_dbname', 'dbhost', 'dbuser', 'isZopeless', 'initZopeless',
24 ]24 ]
2525
26# SQLObject compatibility - dbname, dbhost and dbuser are DEPRECATED.26# SQLObject compatibility - dbname, dbhost and dbuser are DEPRECATED.
@@ -34,14 +34,22 @@
34# if the host is empty it can be overridden by the standard PostgreSQL34# if the host is empty it can be overridden by the standard PostgreSQL
35# environment variables, this feature currently required by Async's35# environment variables, this feature currently required by Async's
36# office environment.36# office environment.
37dbname = os.environ.get('LP_DBNAME', None)
38dbhost = os.environ.get('LP_DBHOST', None)37dbhost = os.environ.get('LP_DBHOST', None)
39dbuser = os.environ.get('LP_DBUSER', None)38dbuser = os.environ.get('LP_DBUSER', None)
4039dbname_override = os.environ.get('LP_DBNAME', None)
41if dbname is None:40
41
42def get_dbname():
43 """Get the DB Name for scripts: deprecated.
44
45 :return: The dbname for scripts.
46 """
47 if dbname_override is not None:
48 return dbname_override
42 match = re.search(r'dbname=(\S*)', dbconfig.main_master)49 match = re.search(r'dbname=(\S*)', dbconfig.main_master)
43 assert match is not None, 'Invalid main_master connection string'50 assert match is not None, 'Invalid main_master connection string'
44 dbname = match.group(1)51 return match.group(1)
52
4553
46if dbhost is None:54if dbhost is None:
47 match = re.search(r'host=(\S*)', dbconfig.main_master)55 match = re.search(r'host=(\S*)', dbconfig.main_master)
@@ -74,7 +82,7 @@
74 # )82 # )
75 pass # Disabled. Bug#305083 pass # Disabled. Bug#3050
76 if dbname is None:84 if dbname is None:
77 dbname = globals()['dbname']85 dbname = get_dbname()
78 if dbhost is None:86 if dbhost is None:
79 dbhost = globals()['dbhost']87 dbhost = globals()['dbhost']
80 if dbuser is None:88 if dbuser is None:
8189
=== modified file 'lib/canonical/testing/ftests/test_layers.py'
--- lib/canonical/testing/ftests/test_layers.py 2010-10-22 09:49:44 +0000
+++ lib/canonical/testing/ftests/test_layers.py 2010-11-28 00:57:58 +0000
@@ -232,19 +232,18 @@
232class LibrarianTestCase(BaseTestCase):232class LibrarianTestCase(BaseTestCase):
233 layer = LibrarianLayer233 layer = LibrarianLayer
234234
235 want_launchpad_database = True
235 want_librarian_running = True236 want_librarian_running = True
236237
237 def testUploadsFail(self):238 def testUploadsSucceed(self):
238 # This layer is not particularly useful by itself, as the Librarian239 # This layer is able to be used on its own as it depends on
239 # cannot function correctly as there is no database setup.240 # DatabaseLayer.
240 # We can test this using remoteAddFile (it does not need the CA241 # We can test this using remoteAddFile (it does not need the CA
241 # loaded)242 # loaded)
242 client = LibrarianClient()243 client = LibrarianClient()
243 data = 'This is a test'244 data = 'This is a test'
244 self.failUnlessRaises(245 client.remoteAddFile(
245 UploadFailed, client.remoteAddFile,246 'foo.txt', len(data), StringIO(data), 'text/plain')
246 'foo.txt', len(data), StringIO(data), 'text/plain'
247 )
248247
249248
250class LibrarianNoResetTestCase(testtools.TestCase):249class LibrarianNoResetTestCase(testtools.TestCase):
@@ -328,6 +327,12 @@
328 num = cur.fetchone()[0]327 num = cur.fetchone()[0]
329 return num328 return num
330329
330 # XXX: Parallel-fail: because layers are not cleanly integrated with
331 # unittest, what should be one test is expressed as three distinct
332 # tests here. We need to either write enough glue to push/pop the
333 # global state of zope.testing.runner or we need to stop using layers,
334 # before these tests will pass in a parallel run. Robert Collins
335 # 2010-11-01
331 def testNoReset1(self):336 def testNoReset1(self):
332 # Ensure that we can switch off database resets between tests337 # Ensure that we can switch off database resets between tests
333 # if necessary, such as used by the page tests338 # if necessary, such as used by the page tests
334339
=== modified file 'lib/canonical/testing/layers.py'
--- lib/canonical/testing/layers.py 2010-11-24 19:50:35 +0000
+++ lib/canonical/testing/layers.py 2010-11-28 00:57:58 +0000
@@ -261,16 +261,24 @@
261 # Things we need to cleanup.261 # Things we need to cleanup.
262 fixture = None262 fixture = None
263263
264 # The config names that are generated for this layer264 # ConfigFixtures for the configs generated for this layer. Set to None
265 # if the layer is not setUp, or if persistent tests services are in use.
266 config_fixture = None
267 appserver_config_fixture = None
268
269 # The config names that are generated for this layer. Set to None when
270 # the layer is not setUp.
265 config_name = None271 config_name = None
266 appserver_config_name = None272 appserver_config_name = None
267273
268 @classmethod274 @classmethod
269 def make_config(cls, config_name, clone_from):275 def make_config(cls, config_name, clone_from, attr_name):
270 """Create a temporary config and link it into the layer cleanup."""276 """Create a temporary config and link it into the layer cleanup."""
271 cfg_fixture = ConfigFixture(config_name, clone_from)277 cfg_fixture = ConfigFixture(config_name, clone_from)
272 cls.fixture.addCleanup(cfg_fixture.cleanUp)278 cls.fixture.addCleanup(cfg_fixture.cleanUp)
273 cfg_fixture.setUp()279 cfg_fixture.setUp()
280 cls.fixture.addCleanup(setattr, cls, attr_name, None)
281 setattr(cls, attr_name, cfg_fixture)
274282
275 @classmethod283 @classmethod
276 @profiled284 @profiled
@@ -296,9 +304,12 @@
296 # about killing memcached - just do it quickly.304 # about killing memcached - just do it quickly.
297 kill_by_pidfile(MemcachedLayer.getPidFile(), num_polls=0)305 kill_by_pidfile(MemcachedLayer.getPidFile(), num_polls=0)
298 config_name = 'testrunner_%s' % test_instance306 config_name = 'testrunner_%s' % test_instance
299 cls.make_config(config_name, 'testrunner')307 cls.make_config(config_name, 'testrunner', 'config_fixture')
300 app_config_name = 'testrunner-appserver_%s' % test_instance308 app_config_name = 'testrunner-appserver_%s' % test_instance
301 cls.make_config(app_config_name, 'testrunner-appserver')309 cls.make_config(
310 app_config_name, 'testrunner-appserver',
311 'appserver_config_fixture')
312 cls.appserver_config_name = app_config_name
302 else:313 else:
303 config_name = 'testrunner'314 config_name = 'testrunner'
304 app_config_name = 'testrunner-appserver'315 app_config_name = 'testrunner-appserver'
@@ -622,122 +633,6 @@
622 MemcachedLayer.client.flush_all() # Only do this in tests!633 MemcachedLayer.client.flush_all() # Only do this in tests!
623634
624635
625class LibrarianLayer(BaseLayer):
626 """Provides tests access to a Librarian instance.
627
628 Calls to the Librarian will fail unless there is also a Launchpad
629 database available.
630 """
631 _reset_between_tests = True
632
633 librarian_fixture = None
634
635 @classmethod
636 @profiled
637 def setUp(cls):
638 if not cls._reset_between_tests:
639 raise LayerInvariantError(
640 "_reset_between_tests changed before LibrarianLayer "
641 "was actually used."
642 )
643 cls.librarian_fixture = LibrarianTestSetup()
644 cls.librarian_fixture.setUp()
645 cls._check_and_reset()
646
647 @classmethod
648 @profiled
649 def tearDown(cls):
650 if cls.librarian_fixture is None:
651 return
652 try:
653 cls._check_and_reset()
654 finally:
655 librarian = cls.librarian_fixture
656 cls.librarian_fixture = None
657 try:
658 if not cls._reset_between_tests:
659 raise LayerInvariantError(
660 "_reset_between_tests not reset before LibrarianLayer "
661 "shutdown"
662 )
663 finally:
664 librarian.cleanUp()
665
666 @classmethod
667 @profiled
668 def _check_and_reset(cls):
669 """Raise an exception if the Librarian has been killed.
670 Reset the storage unless this has been disabled.
671 """
672 try:
673 f = urlopen(config.librarian.download_url)
674 f.read()
675 except Exception, e:
676 raise LayerIsolationError(
677 "Librarian has been killed or has hung."
678 "Tests should use LibrarianLayer.hide() and "
679 "LibrarianLayer.reveal() where possible, and ensure "
680 "the Librarian is restarted if it absolutely must be "
681 "shutdown: " + str(e)
682 )
683 if cls._reset_between_tests:
684 cls.librarian_fixture.clear()
685
686 @classmethod
687 @profiled
688 def testSetUp(cls):
689 cls._check_and_reset()
690
691 @classmethod
692 @profiled
693 def testTearDown(cls):
694 if cls._hidden:
695 cls.reveal()
696 cls._check_and_reset()
697
698 # Flag maintaining state of hide()/reveal() calls
699 _hidden = False
700
701 # Fake upload socket used when the librarian is hidden
702 _fake_upload_socket = None
703
704 @classmethod
705 @profiled
706 def hide(cls):
707 """Hide the Librarian so nothing can find it. We don't want to
708 actually shut it down because starting it up again is expensive.
709
710 We do this by altering the configuration so the Librarian client
711 looks for the Librarian server on the wrong port.
712 """
713 cls._hidden = True
714 if cls._fake_upload_socket is None:
715 # Bind to a socket, but don't listen to it. This way we
716 # guarantee that connections to the given port will fail.
717 cls._fake_upload_socket = socket.socket(
718 socket.AF_INET, socket.SOCK_STREAM)
719 assert config.librarian.upload_host == 'localhost', (
720 'Can only hide librarian if it is running locally')
721 cls._fake_upload_socket.bind(('127.0.0.1', 0))
722
723 host, port = cls._fake_upload_socket.getsockname()
724 librarian_data = dedent("""
725 [librarian]
726 upload_port: %s
727 """ % port)
728 config.push('hide_librarian', librarian_data)
729
730 @classmethod
731 @profiled
732 def reveal(cls):
733 """Reveal a hidden Librarian.
734
735 This just involves restoring the config to the original value.
736 """
737 cls._hidden = False
738 config.pop('hide_librarian')
739
740
741# We store a reference to the DB-API connect method here when we636# We store a reference to the DB-API connect method here when we
742# put a proxy in its place.637# put a proxy in its place.
743_org_connect = None638_org_connect = None
@@ -764,7 +659,20 @@
764 cls._db_fixture = LaunchpadTestSetup(659 cls._db_fixture = LaunchpadTestSetup(
765 reset_sequences_sql=reset_sequences_sql)660 reset_sequences_sql=reset_sequences_sql)
766 cls.force_dirty_database()661 cls.force_dirty_database()
767 cls._db_fixture.tearDown()662 # Nuke any existing DB (for persistent-test-services) [though they
663 # prevent this !?]
664 cls._db_fixture.tearDown()
665 # Force a db creation for unique db names - needed at layer init
666 # because appserver using layers run things at layer setup, not
667 # test setup.
668 cls._db_fixture.setUp()
669 # And take it 'down' again to be in the right state for testSetUp
670 # - note that this conflicts in principle with layers whose setUp
671 # needs the db working, but this is a conceptually cleaner starting
672 # point for addressing that mismatch.
673 cls._db_fixture.tearDown()
674 # Bring up the db, so that it is available for other layers.
675 cls._ensure_db()
768676
769 @classmethod677 @classmethod
770 @profiled678 @profiled
@@ -782,6 +690,13 @@
782 @classmethod690 @classmethod
783 @profiled691 @profiled
784 def testSetUp(cls):692 def testSetUp(cls):
693 # The DB is already available - setUp and testTearDown both make it
694 # available.
695 if cls.use_mockdb is True:
696 cls.installMockDb()
697
698 @classmethod
699 def _ensure_db(cls):
785 if cls._reset_between_tests:700 if cls._reset_between_tests:
786 cls._db_fixture.setUp()701 cls._db_fixture.setUp()
787 # Ensure that the database is connectable. Because we might have702 # Ensure that the database is connectable. Because we might have
@@ -798,9 +713,6 @@
798 else:713 else:
799 break714 break
800715
801 if cls.use_mockdb is True:
802 cls.installMockDb()
803
804 @classmethod716 @classmethod
805 @profiled717 @profiled
806 def testTearDown(cls):718 def testTearDown(cls):
@@ -819,6 +731,10 @@
819 BaseLayer.flagTestIsolationFailure(731 BaseLayer.flagTestIsolationFailure(
820 "Database policy %s still installed"732 "Database policy %s still installed"
821 % repr(StoreSelector.pop()))733 % repr(StoreSelector.pop()))
734 # Reset/bring up the db - makes it available for either the next test,
735 # or a subordinate layer which builds on the db. This wastes one setup
736 # per db layer teardown per run, but thats tolerable.
737 cls._ensure_db()
822738
823 use_mockdb = False739 use_mockdb = False
824 mockdb_mode = None740 mockdb_mode = None
@@ -886,12 +802,128 @@
886 return cls._db_fixture.dropDb()802 return cls._db_fixture.dropDb()
887803
888804
805class LibrarianLayer(DatabaseLayer):
806 """Provides tests access to a Librarian instance.
807
808 Calls to the Librarian will fail unless there is also a Launchpad
809 database available.
810 """
811 _reset_between_tests = True
812
813 librarian_fixture = None
814
815 @classmethod
816 @profiled
817 def setUp(cls):
818 if not cls._reset_between_tests:
819 raise LayerInvariantError(
820 "_reset_between_tests changed before LibrarianLayer "
821 "was actually used."
822 )
823 cls.librarian_fixture = LibrarianTestSetup()
824 cls.librarian_fixture.setUp()
825 cls._check_and_reset()
826
827 @classmethod
828 @profiled
829 def tearDown(cls):
830 if cls.librarian_fixture is None:
831 return
832 try:
833 cls._check_and_reset()
834 finally:
835 librarian = cls.librarian_fixture
836 cls.librarian_fixture = None
837 try:
838 if not cls._reset_between_tests:
839 raise LayerInvariantError(
840 "_reset_between_tests not reset before LibrarianLayer "
841 "shutdown"
842 )
843 finally:
844 librarian.cleanUp()
845
846 @classmethod
847 @profiled
848 def _check_and_reset(cls):
849 """Raise an exception if the Librarian has been killed.
850 Reset the storage unless this has been disabled.
851 """
852 try:
853 f = urlopen(config.librarian.download_url)
854 f.read()
855 except Exception, e:
856 raise LayerIsolationError(
857 "Librarian has been killed or has hung."
858 "Tests should use LibrarianLayer.hide() and "
859 "LibrarianLayer.reveal() where possible, and ensure "
860 "the Librarian is restarted if it absolutely must be "
861 "shutdown: " + str(e)
862 )
863 if cls._reset_between_tests:
864 cls.librarian_fixture.clear()
865
866 @classmethod
867 @profiled
868 def testSetUp(cls):
869 cls._check_and_reset()
870
871 @classmethod
872 @profiled
873 def testTearDown(cls):
874 if cls._hidden:
875 cls.reveal()
876 cls._check_and_reset()
877
878 # Flag maintaining state of hide()/reveal() calls
879 _hidden = False
880
881 # Fake upload socket used when the librarian is hidden
882 _fake_upload_socket = None
883
884 @classmethod
885 @profiled
886 def hide(cls):
887 """Hide the Librarian so nothing can find it. We don't want to
888 actually shut it down because starting it up again is expensive.
889
890 We do this by altering the configuration so the Librarian client
891 looks for the Librarian server on the wrong port.
892 """
893 cls._hidden = True
894 if cls._fake_upload_socket is None:
895 # Bind to a socket, but don't listen to it. This way we
896 # guarantee that connections to the given port will fail.
897 cls._fake_upload_socket = socket.socket(
898 socket.AF_INET, socket.SOCK_STREAM)
899 assert config.librarian.upload_host == 'localhost', (
900 'Can only hide librarian if it is running locally')
901 cls._fake_upload_socket.bind(('127.0.0.1', 0))
902
903 host, port = cls._fake_upload_socket.getsockname()
904 librarian_data = dedent("""
905 [librarian]
906 upload_port: %s
907 """ % port)
908 config.push('hide_librarian', librarian_data)
909
910 @classmethod
911 @profiled
912 def reveal(cls):
913 """Reveal a hidden Librarian.
914
915 This just involves restoring the config to the original value.
916 """
917 cls._hidden = False
918 config.pop('hide_librarian')
919
920
889def test_default_timeout():921def test_default_timeout():
890 """Don't timeout by default in tests."""922 """Don't timeout by default in tests."""
891 return None923 return None
892924
893925
894class LaunchpadLayer(DatabaseLayer, LibrarianLayer, MemcachedLayer):926class LaunchpadLayer(LibrarianLayer, MemcachedLayer):
895 """Provides access to the Launchpad database and daemons.927 """Provides access to the Launchpad database and daemons.
896928
897 We need to ensure that the database setup runs before the daemon929 We need to ensure that the database setup runs before the daemon
@@ -1659,7 +1691,7 @@
1659 appserver = None1691 appserver = None
16601692
1661 # The config used by the spawned app server.1693 # The config used by the spawned app server.
1662 appserver_config = CanonicalConfig('testrunner-appserver', 'runlaunchpad')1694 appserver_config = None
16631695
1664 # The SMTP server for layer tests. See1696 # The SMTP server for layer tests. See
1665 # configs/testrunner-appserver/mail-configure.zcml1697 # configs/testrunner-appserver/mail-configure.zcml
@@ -1798,12 +1830,6 @@
1798 @classmethod1830 @classmethod
1799 def _runAppServer(cls):1831 def _runAppServer(cls):
1800 """Start the app server using runlaunchpad.py"""1832 """Start the app server using runlaunchpad.py"""
1801 # The database must be available for the app server to start.
1802 cls._db_fixture = LaunchpadTestSetup()
1803 # This is not torn down properly: rather the singleton nature is abused
1804 # and the fixture is simply marked as being dirty.
1805 # XXX: Robert Collins 2010-10-17 bug=661967
1806 cls._db_fixture.setUp()
1807 # The app server will not start at all if the database hasn't been1833 # The app server will not start at all if the database hasn't been
1808 # correctly patched. The app server will make exactly this check,1834 # correctly patched. The app server will make exactly this check,
1809 # doing it here makes the error more obvious.1835 # doing it here makes the error more obvious.
@@ -1860,8 +1886,7 @@
1860 @classmethod1886 @classmethod
1861 @profiled1887 @profiled
1862 def setUp(cls):1888 def setUp(cls):
1863 LayerProcessController.startSMTPServer()1889 LayerProcessController.setUp()
1864 LayerProcessController.startAppServer()
18651890
1866 @classmethod1891 @classmethod
1867 @profiled1892 @profiled
@@ -1887,8 +1912,7 @@
1887 @classmethod1912 @classmethod
1888 @profiled1913 @profiled
1889 def setUp(cls):1914 def setUp(cls):
1890 LayerProcessController.startSMTPServer()1915 LayerProcessController.setUp()
1891 LayerProcessController.startAppServer()
18921916
1893 @classmethod1917 @classmethod
1894 @profiled1918 @profiled
@@ -1914,8 +1938,7 @@
1914 @classmethod1938 @classmethod
1915 @profiled1939 @profiled
1916 def setUp(cls):1940 def setUp(cls):
1917 LayerProcessController.startSMTPServer()1941 LayerProcessController.setUp()
1918 LayerProcessController.startAppServer()
19191942
1920 @classmethod1943 @classmethod
1921 @profiled1944 @profiled
19221945
=== added file 'lib/lp/code/interfaces/branchmergequeue.py'
--- lib/lp/code/interfaces/branchmergequeue.py 1970-01-01 00:00:00 +0000
+++ lib/lp/code/interfaces/branchmergequeue.py 2010-11-03 08:28:44 +0000
@@ -0,0 +1,129 @@
1# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Branch merge queue interfaces."""
5
6__metaclass__ = type
7
8__all__ = [
9 'IBranchMergeQueue',
10 'IBranchMergeQueueSource',
11 'user_has_special_merge_queue_access',
12 ]
13
14from lazr.restful.declarations import (
15 export_as_webservice_entry,
16 export_write_operation,
17 exported,
18 mutator_for,
19 operation_parameters,
20 )
21from lazr.restful.fields import (
22 CollectionField,
23 Reference,
24 )
25from zope.component import getUtility
26from zope.interface import Interface
27from zope.schema import (
28 Datetime,
29 Int,
30 Text,
31 TextLine,
32 )
33
34from canonical.launchpad import _
35from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities
36from lp.services.fields import (
37 PersonChoice,
38 PublicPersonChoice,
39 )
40
41
42class IBranchMergeQueue(Interface):
43 """An interface for managing branch merges."""
44
45 export_as_webservice_entry()
46
47 id = Int(title=_('ID'), readonly=True, required=True)
48
49 registrant = exported(
50 PublicPersonChoice(
51 title=_("The user that registered the branch."),
52 required=True, readonly=True,
53 vocabulary='ValidPersonOrTeam'))
54
55 owner = exported(
56 PersonChoice(
57 title=_('Owner'),
58 required=True, readonly=True,
59 vocabulary='UserTeamsParticipationPlusSelf',
60 description=_("The owner of the merge queue.")))
61
62 name = exported(
63 TextLine(
64 title=_('Name'), required=True,
65 description=_(
66 "Keep very short, unique, and descriptive, because it will "
67 "be used in URLs. "
68 "Examples: main, devel, release-1.0, gnome-vfs.")))
69
70 description = exported(
71 Text(
72 title=_('Description'), required=False,
73 description=_(
74 'A short description of the purpose of this merge queue.')))
75
76 configuration = exported(
77 TextLine(
78 title=_('Configuration'), required=False, readonly=True,
79 description=_(
80 "A JSON string of configuration values.")))
81
82 date_created = exported(
83 Datetime(
84 title=_('Date Created'),
85 required=True,
86 readonly=True))
87
88 branches = exported(
89 CollectionField(
90 title=_('Dependent Branches'),
91 description=_(
92 'A collection of branches that this queue manages.'),
93 readonly=True,
94 value_type=Reference(Interface)))
95
96 @mutator_for(configuration)
97 @operation_parameters(
98 config=TextLine(title=_("A JSON string of configuration values.")))
99 @export_write_operation()
100 def setMergeQueueConfig(config):
101 """Set the JSON string configuration of the merge queue.
102
103 :param config: A JSON string of configuration values.
104 """
105
106
107class IBranchMergeQueueSource(Interface):
108
109 def new(name, owner, registrant, description, configuration, branches):
110 """Create a new IBranchMergeQueue object.
111
112 :param name: The name of the branch merge queue.
113 :param description: A description of queue.
114 :param configuration: A JSON string of configuration values.
115 :param owner: The owner of the queue.
116 :param registrant: The registrant of the queue.
117 :param branches: A list of branches to add to the queue.
118 """
119
120
121def user_has_special_merge_queue_access(user):
122 """Admins and bazaar experts have special access.
123
124 :param user: A 'Person' or None.
125 """
126 if user is None:
127 return False
128 celebs = getUtility(ILaunchpadCelebrities)
129 return user.inTeam(celebs.admin) or user.inTeam(celebs.bazaar_experts)
0130
=== added file 'lib/lp/code/model/branchmergequeue.py'
--- lib/lp/code/model/branchmergequeue.py 1970-01-01 00:00:00 +0000
+++ lib/lp/code/model/branchmergequeue.py 2010-11-01 12:35:07 +0000
@@ -0,0 +1,88 @@
1# Copyright 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Implementation classes for IBranchMergeQueue, etc."""
5
6__metaclass__ = type
7__all__ = ['BranchMergeQueue']
8
9import simplejson
10from storm.locals import (
11 Int,
12 Reference,
13 Store,
14 Storm,
15 Unicode,
16 )
17from zope.interface import (
18 classProvides,
19 implements,
20 )
21
22from canonical.database.datetimecol import UtcDateTimeCol
23from canonical.launchpad.interfaces.lpstorm import IMasterStore
24from lp.code.errors import InvalidMergeQueueConfig
25from lp.code.interfaces.branchmergequeue import (
26 IBranchMergeQueue,
27 IBranchMergeQueueSource,
28 )
29from lp.code.model.branch import Branch
30
31
32class BranchMergeQueue(Storm):
33 """See `IBranchMergeQueue`."""
34
35 __storm_table__ = 'BranchMergeQueue'
36 implements(IBranchMergeQueue)
37 classProvides(IBranchMergeQueueSource)
38
39 id = Int(primary=True)
40
41 registrant_id = Int(name='registrant', allow_none=True)
42 registrant = Reference(registrant_id, 'Person.id')
43
44 owner_id = Int(name='owner', allow_none=True)
45 owner = Reference(owner_id, 'Person.id')
46
47 name = Unicode(allow_none=False)
48 description = Unicode(allow_none=False)
49 configuration = Unicode(allow_none=False)
50
51 date_created = UtcDateTimeCol(notNull=True)
52
53 @property
54 def branches(self):
55 """See `IBranchMergeQueue`."""
56 return Store.of(self).find(
57 Branch,
58 Branch.merge_queue_id == self.id)
59
60 def setMergeQueueConfig(self, config):
61 """See `IBranchMergeQueue`."""
62 try:
63 simplejson.loads(config)
64 self.configuration = config
65 except ValueError: # The config string is not valid JSON
66 raise InvalidMergeQueueConfig
67
68 @classmethod
69 def new(cls, name, owner, registrant, description=None,
70 configuration=None, branches=None):
71 """See `IBranchMergeQueueSource`."""
72 store = IMasterStore(BranchMergeQueue)
73
74 if configuration is None:
75 configuration = unicode(simplejson.dumps({}))
76
77 queue = cls()
78 queue.name = name
79 queue.owner = owner
80 queue.registrant = registrant
81 queue.description = description
82 queue.configuration = configuration
83 if branches is not None:
84 for branch in branches:
85 branch.addToQueue(queue)
86
87 store.add(queue)
88 return queue
089
=== added file 'lib/lp/code/model/tests/test_branchmergequeue.py'
--- lib/lp/code/model/tests/test_branchmergequeue.py 1970-01-01 00:00:00 +0000
+++ lib/lp/code/model/tests/test_branchmergequeue.py 2010-10-25 14:51:56 +0000
@@ -0,0 +1,155 @@
1# Copyright 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Unit tests for methods of BranchMergeQueue."""
5
6from __future__ import with_statement
7
8import simplejson
9
10from canonical.launchpad.interfaces.lpstorm import IStore
11from canonical.launchpad.webapp.testing import verifyObject
12from canonical.testing.layers import (
13 AppServerLayer,
14 DatabaseFunctionalLayer,
15 )
16from lp.code.errors import InvalidMergeQueueConfig
17from lp.code.interfaces.branchmergequeue import IBranchMergeQueue
18from lp.code.model.branchmergequeue import BranchMergeQueue
19from lp.testing import (
20 ANONYMOUS,
21 person_logged_in,
22 launchpadlib_for,
23 TestCaseWithFactory,
24 ws_object,
25 )
26
27
28class TestBranchMergeQueueInterface(TestCaseWithFactory):
29 """Test IBranchMergeQueue interface."""
30
31 layer = DatabaseFunctionalLayer
32
33 def test_implements_interface(self):
34 queue = self.factory.makeBranchMergeQueue()
35 IStore(BranchMergeQueue).add(queue)
36 verifyObject(IBranchMergeQueue, queue)
37
38
39class TestBranchMergeQueueSource(TestCaseWithFactory):
40 """Test the methods of IBranchMergeQueueSource."""
41
42 layer = DatabaseFunctionalLayer
43
44 def test_new(self):
45 owner = self.factory.makePerson()
46 name = u'SooperQueue'
47 description = u'This is Sooper Queue'
48 config = unicode(simplejson.dumps({'test': 'make check'}))
49
50 queue = BranchMergeQueue.new(
51 name, owner, owner, description, config)
52
53 self.assertEqual(queue.name, name)
54 self.assertEqual(queue.owner, owner)
55 self.assertEqual(queue.registrant, owner)
56 self.assertEqual(queue.description, description)
57 self.assertEqual(queue.configuration, config)
58
59
60class TestBranchMergeQueue(TestCaseWithFactory):
61 """Test the functions of the BranchMergeQueue."""
62
63 layer = DatabaseFunctionalLayer
64
65 def test_branches(self):
66 """Test that a merge queue can get all its managed branches."""
67 store = IStore(BranchMergeQueue)
68
69 queue = self.factory.makeBranchMergeQueue()
70 store.add(queue)
71
72 branch = self.factory.makeBranch()
73 store.add(branch)
74 with person_logged_in(branch.owner):
75 branch.addToQueue(queue)
76
77 self.assertEqual(
78 list(queue.branches),
79 [branch])
80
81 def test_setMergeQueueConfig(self):
82 """Test that the configuration is set properly."""
83 queue = self.factory.makeBranchMergeQueue()
84 config = unicode(simplejson.dumps({
85 'test': 'make test'}))
86
87 with person_logged_in(queue.owner):
88 queue.setMergeQueueConfig(config)
89
90 self.assertEqual(queue.configuration, config)
91
92 def test_setMergeQueueConfig_invalid_json(self):
93 """Test that invalid json can't be set as the config."""
94 queue = self.factory.makeBranchMergeQueue()
95
96 with person_logged_in(queue.owner):
97 self.assertRaises(
98 InvalidMergeQueueConfig,
99 queue.setMergeQueueConfig,
100 'abc')
101
102
103class TestWebservice(TestCaseWithFactory):
104
105 layer = AppServerLayer
106
107 def test_properties(self):
108 """Test that the correct properties are exposed."""
109 with person_logged_in(ANONYMOUS):
110 name = u'teh-queue'
111 description = u'Oh hai! I are a queues'
112 configuration = unicode(simplejson.dumps({'test': 'make check'}))
113
114 queuer = self.factory.makePerson()
115 db_queue = self.factory.makeBranchMergeQueue(
116 registrant=queuer, owner=queuer, name=name,
117 description=description,
118 configuration=configuration)
119 branch1 = self.factory.makeBranch()
120 with person_logged_in(branch1.owner):
121 branch1.addToQueue(db_queue)
122 branch2 = self.factory.makeBranch()
123 with person_logged_in(branch2.owner):
124 branch2.addToQueue(db_queue)
125 launchpad = launchpadlib_for('test', db_queue.owner,
126 service_root="http://api.launchpad.dev:8085")
127
128 queuer = ws_object(launchpad, queuer)
129 queue = ws_object(launchpad, db_queue)
130 branch1 = ws_object(launchpad, branch1)
131 branch2 = ws_object(launchpad, branch2)
132
133 self.assertEqual(queue.registrant, queuer)
134 self.assertEqual(queue.owner, queuer)
135 self.assertEqual(queue.name, name)
136 self.assertEqual(queue.description, description)
137 self.assertEqual(queue.configuration, configuration)
138 self.assertEqual(queue.date_created, db_queue.date_created)
139 self.assertEqual(len(queue.branches), 2)
140
141 def test_set_configuration(self):
142 """Test the mutator for setting configuration."""
143 with person_logged_in(ANONYMOUS):
144 db_queue = self.factory.makeBranchMergeQueue()
145 launchpad = launchpadlib_for('test', db_queue.owner,
146 service_root="http://api.launchpad.dev:8085")
147
148 configuration = simplejson.dumps({'test': 'make check'})
149
150 queue = ws_object(launchpad, db_queue)
151 queue.configuration = configuration
152 queue.lp_save()
153
154 queue2 = ws_object(launchpad, db_queue)
155 self.assertEqual(queue2.configuration, configuration)
0156
=== modified file 'lib/lp/registry/tests/test_mlists.py'
--- lib/lp/registry/tests/test_mlists.py 2010-08-20 20:31:18 +0000
+++ lib/lp/registry/tests/test_mlists.py 2010-11-28 00:57:58 +0000
@@ -26,6 +26,7 @@
26from canonical.launchpad.scripts.mlistimport import Importer26from canonical.launchpad.scripts.mlistimport import Importer
27from canonical.testing.layers import (27from canonical.testing.layers import (
28 AppServerLayer,28 AppServerLayer,
29 BaseLayer,
29 DatabaseFunctionalLayer,30 DatabaseFunctionalLayer,
30 LayerProcessController,31 LayerProcessController,
31 )32 )
@@ -399,7 +400,7 @@
399 args.append(self.team.name)400 args.append(self.team.name)
400 return Popen(args, stdout=PIPE, stderr=STDOUT,401 return Popen(args, stdout=PIPE, stderr=STDOUT,
401 cwd=LayerProcessController.appserver_config.root,402 cwd=LayerProcessController.appserver_config.root,
402 env=dict(LPCONFIG='testrunner-appserver',403 env=dict(LPCONFIG=BaseLayer.appserver_config_name,
403 PATH=os.environ['PATH']))404 PATH=os.environ['PATH']))
404405
405 def test_import(self):406 def test_import(self):
406407
=== modified file 'lib/lp/services/memcache/doc/restful-cache.txt'
--- lib/lp/services/memcache/doc/restful-cache.txt 2010-07-16 09:06:21 +0000
+++ lib/lp/services/memcache/doc/restful-cache.txt 2010-11-28 00:57:58 +0000
@@ -23,7 +23,7 @@
23 >>> cache_key = cache.key_for(23 >>> cache_key = cache.key_for(
24 ... person, 'media/type', 'web-service-version')24 ... person, 'media/type', 'web-service-version')
25 >>> print person.id, cache_key25 >>> print person.id, cache_key
26 29 Person(29,),testrunner,media/type,web-service-version26 29 Person(29,),testrunner...,media/type,web-service-version
2727
28 >>> from operator import attrgetter28 >>> from operator import attrgetter
29 >>> languages = sorted(person.languages, key=attrgetter('englishname'))29 >>> languages = sorted(person.languages, key=attrgetter('englishname'))
@@ -31,8 +31,8 @@
31 ... cache_key = cache.key_for(31 ... cache_key = cache.key_for(
32 ... language, 'media/type', 'web-service-version')32 ... language, 'media/type', 'web-service-version')
33 ... print language.id, cache_key33 ... print language.id, cache_key
34 119 Language(119,),testrunner,media/type,web-service-version34 119 Language(119,),testrunner...,media/type,web-service-version
35 521 Language(521,),testrunner,media/type,web-service-version35 521 Language(521,),testrunner...,media/type,web-service-version
3636
37The cache starts out empty.37The cache starts out empty.
3838
3939
=== modified file 'lib/lp/services/scripts/doc/script-monitoring.txt'
--- lib/lp/services/scripts/doc/script-monitoring.txt 2009-07-28 13:47:01 +0000
+++ lib/lp/services/scripts/doc/script-monitoring.txt 2010-11-28 00:57:58 +0000
@@ -97,9 +97,6 @@
9797
98Prepare an environment to run the testing script.98Prepare an environment to run the testing script.
9999
100 >>> env = dict(os.environ)
101 >>> env['LPCONFIG'] = 'testrunner'
102
103 >>> import canonical100 >>> import canonical
104 >>> lp_tree = os.path.join(101 >>> lp_tree = os.path.join(
105 ... os.path.dirname(canonical.__file__), os.pardir, os.pardir)102 ... os.path.dirname(canonical.__file__), os.pardir, os.pardir)
@@ -108,8 +105,8 @@
108We'll now run this script, telling it to fail:105We'll now run this script, telling it to fail:
109106
110 >>> proc = subprocess.Popen([lp_py, script_file.name, 'fail'],107 >>> proc = subprocess.Popen([lp_py, script_file.name, 'fail'],
111 ... env=env, stdin=subprocess.PIPE,108 ... stdin=subprocess.PIPE, stdout=subprocess.PIPE,
112 ... stdout=subprocess.PIPE, stderr=subprocess.PIPE)109 ... stderr=subprocess.PIPE)
113 >>> (out, err) = proc.communicate()110 >>> (out, err) = proc.communicate()
114 >>> transaction.abort()111 >>> transaction.abort()
115112
@@ -126,8 +123,8 @@
126If we run it such that it succeeds, we will get an activity record:123If we run it such that it succeeds, we will get an activity record:
127124
128 >>> proc = subprocess.Popen([lp_py, script_file.name, 'pass'],125 >>> proc = subprocess.Popen([lp_py, script_file.name, 'pass'],
129 ... env=env, stdin=subprocess.PIPE,126 ... stdin=subprocess.PIPE, stdout=subprocess.PIPE,
130 ... stdout=subprocess.PIPE, stderr=subprocess.PIPE)127 ... stderr=subprocess.PIPE)
131 >>> (out, err) = proc.communicate()128 >>> (out, err) = proc.communicate()
132 >>> transaction.abort()129 >>> transaction.abort()
133130
134131
=== modified file 'scripts/gina.py'
--- scripts/gina.py 2010-11-25 02:59:54 +0000
+++ scripts/gina.py 2010-11-28 00:57:58 +0000
@@ -79,7 +79,7 @@
7979
80 dry_run = options.dry_run80 dry_run = options.dry_run
8181
82 LPDB = lp.dbname82 LPDB = lp.get_dbname()
83 LPDB_HOST = lp.dbhost83 LPDB_HOST = lp.dbhost
84 LPDB_USER = config.gina.dbuser84 LPDB_USER = config.gina.dbuser
85 KTDB = target_section.katie_dbname85 KTDB = target_section.katie_dbname