Launchpad itself

Merge lp:~lifeless/launchpad/paralleltests into lp:launchpad

  1. paralleltests
  2. Merge into devel
Proposed by Robert Collins
Status: Merged
Approved by: Robert Collins
Approved revision: no longer in the source branch.
Merged at revision: 11731
Proposed branch: lp:~lifeless/launchpad/paralleltests
Merge into: lp:launchpad
Diff against target: 1162 lines (+945/-5) (has conflicts)
14 files modified
buildout-templates/bin/test.in (+10/-2)
database/schema/security.cfg (+7/-0)
lib/canonical/testing/parallel.py (+174/-0)
lib/canonical/testing/tests/test_parallel.py (+117/-0)
lib/lp/archivepublisher/tests/test_publisher.py (+70/-0)
lib/lp/registry/doc/structural-subscriptions.txt (+62/-2)
lib/lp/registry/interfaces/structuralsubscription.py (+41/-0)
lib/lp/registry/model/structuralsubscription.py (+67/-0)
lib/lp/registry/tests/test_structuralsubscriptiontarget.py (+194/-0)
lib/lp/services/mail/tests/test_incoming.py (+5/-0)
lib/lp/soyuz/interfaces/distributionjob.py (+69/-0)
lib/lp/soyuz/model/initialisedistroseriesjob.py (+51/-0)
lib/lp/soyuz/tests/test_initialisedistroseriesjob.py (+77/-0)
versions.cfg (+1/-1)
Text conflict in database/schema/security.cfg
Text conflict in lib/lp/archivepublisher/tests/test_publisher.py
Text conflict in lib/lp/registry/doc/structural-subscriptions.txt
Text conflict in lib/lp/registry/interfaces/structuralsubscription.py
Text conflict in lib/lp/registry/model/structuralsubscription.py
Text conflict in lib/lp/registry/tests/test_structuralsubscriptiontarget.py
Text conflict in lib/lp/services/mail/tests/test_incoming.py
Text conflict in lib/lp/soyuz/interfaces/distributionjob.py
Text conflict in lib/lp/soyuz/model/initialisedistroseriesjob.py
Text conflict in lib/lp/soyuz/tests/test_initialisedistroseriesjob.py
To merge this branch: bzr merge lp:~lifeless/launchpad/paralleltests
Reviewer Review Type Date Requested Status
Jonathan Lange (community) Approve
Review via email: mp+38594@code.launchpad.net

Commit message

parallel testing facility that works with subunit.

Description of the change

Add a layer agnostic parallel testing facility.

The bin/test -j facility has a number of flaws; firstly it buffers without propogating timing data in subunit; secondly it groups by layers when we don't have a homogeneous distribution across layers.

This facility layers on the --load-list facility to run a simple partition of the requested tests in parallel. It works.

Its not as thoroughly tested as I normally would, but the glue is very shallow and some bits tests would be wasted on. I do think that more tests *may* be needed in future, but only if we're making further changes based on experience with the result of this patch.

To post a comment you must log in.
Revision history for this message
Robert Collins (lifeless) wrote :
Download full text (8.8 KiB)

Direct mail from jml:
"""Wow. Thanks for this Rob. While I lament the fact that we have to do hacks
to bin/test, I really like that you're starting on this, and that you've kept
the hack very clean.

Sending this to you directly since you didn't create an MP, and I
assume that you didn't for a reason :)

Lots of this stuff should just land now, regardless of whether the parallel
support is any good. I'm thinking particularly of all the boilerplate you
deleted.

Most of my comments below are asking for more or clearer documentation. I've
tried to supply docs where I know what's going on.

Thanks,
jml

> === modified file 'lib/canonical/launchpad/testing/tests/test_pages.py'
> --- lib/canonical/launchpad/testing/tests/test_pages.py 2010-10-04 19:50:45 +0000
> +++ lib/canonical/launchpad/testing/tests/test_pages.py 2010-10-15 11:52:20 +0000
> @@ -9,6 +9,8 @@ import shutil
> import tempfile
> import unittest
>
> +from bzrlib.tests import iter_suite_tests
> +

You can get this from testtools under a different name. Probably you should.

> === added file 'lib/canonical/testing/parallel.py'
> --- lib/canonical/testing/parallel.py 1970-01-01 00:00:00 +0000
> +++ lib/canonical/testing/parallel.py 2010-10-15 11:52:20 +0000
> @@ -0,0 +1,168 @@
> +# Copyright 2010 Canonical Ltd. This software is licensed under the
> +# GNU Affero General Public License version 3 (see the file LICENSE).
> +
> +from __future__ import with_statement
> +
> +"""Parallel test glue."""
> +
> +__metaclass__ = type
> +__all__ = ["main"]
> +
> +import itertools
> +import os
> +import subprocess
> +import sys
> +import tempfile
> +
> +from bzrlib.osutils import local_concurrency
> +from subunit import ProtocolTestCase
> +from subunit.run import SubunitTestRunner
> +from testtools.compat import unicode_output_stream
> +from testtools import (
> + ConcurrentTestSuite,
> + TestResult,
> + TextTestResult,
> + )
> +
> +
> +def prepare_argv(argv):
> + """Remove options from argv that ListTestCase will add in itself."""

This might seem pedantic, but I think the comment reads more clearly like
this:

 """Remove options from argv that would be added by ListTestCase."""

> +def find_load_list(args):

Add a docstring like:

 """Get the value passed in to --load-list=FOO."""

> +def find_tests(argv):
> + """Find tests to parallel run.
> +
> + :param argv: The argv given to the test runner, used to get the tests to
> + run.

Add:

 :return: A list of test IDs.

> + """
> + load_list = find_load_list(argv)
> + if load_list:
> + # just use the load_list
> + with open(load_list, 'rt') as list_file:
> + return [id for id in list_file.read().split('\n') if id]

I think you could also write that as:

 return filter(None, list_file.read().splitlines())

But better yet might be to do this:

 return [id for id in (line.strip() for line in list_file) if id]

or:

 return filter(None, (line.strip() for line in list_file))

But whatever. :)

> + # run in --list-tests mode
> + argv = prepare_argv(argv) + ['--list-tests', '--subunit']
> + process = subprocess.Popen(argv, stdin=subprocess.PIPE,
> + stdout=subproc...

Read more...

Revision history for this message
Robert Collins (lifeless) wrote :

Ah, I should have used a dependent branch, the stuff I deleted was from my approved 'doctests' branch, a necessary prereq.

Revision history for this message
Jonathan Lange (jml) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'buildout-templates/bin/test.in'
--- buildout-templates/bin/test.in 2010-08-09 16:26:45 +0000
+++ buildout-templates/bin/test.in 2010-10-16 18:22:51 +0000
@@ -45,7 +45,6 @@
45BUILD_DIR = ${buildout:directory|path-repr}45BUILD_DIR = ${buildout:directory|path-repr}
46CUSTOM_SITE_DIR = ${scripts:parts-directory|path-repr}46CUSTOM_SITE_DIR = ${scripts:parts-directory|path-repr}
4747
48
49# Make tests run in a timezone no launchpad developers live in.48# Make tests run in a timezone no launchpad developers live in.
50# Our tests need to run in any timezone.49# Our tests need to run in any timezone.
51# (This is no longer actually required, as PQM does this.)50# (This is no longer actually required, as PQM does this.)
@@ -79,7 +78,6 @@
79 sys.exit(-1 * signal.SIGTERM)78 sys.exit(-1 * signal.SIGTERM)
80signal.signal(signal.SIGTERM, exit_with_atexit_handlers)79signal.signal(signal.SIGTERM, exit_with_atexit_handlers)
8180
82
83# Tell canonical.config to use the testrunner config instance.81# Tell canonical.config to use the testrunner config instance.
84from canonical.config import config82from canonical.config import config
85config.setInstance('testrunner')83config.setInstance('testrunner')
@@ -212,10 +210,20 @@
212 else:210 else:
213 args = sys.argv211 args = sys.argv
214212
213 # thunk across to parallel support if needed.
214 if '--parallel' in sys.argv and '--list-tests' not in sys.argv:
215 # thunk over to parallel testing.
216 from canonical.testing.parallel import main
217 sys.exit(main(sys.argv))
218
215 def load_list(option, opt_str, list_name, parser):219 def load_list(option, opt_str, list_name, parser):
216 patch_find_tests(filter_tests(list_name))220 patch_find_tests(filter_tests(list_name))
217 options.parser.add_option(221 options.parser.add_option(
218 '--load-list', type=str, action='callback', callback=load_list)222 '--load-list', type=str, action='callback', callback=load_list)
223 options.parser.add_option(
224 '--parallel', action='store_true',
225 help='Run tests in parallel processes. '
226 'Poorly isolated tests will break.')
219227
220 # tests_pattern is a regexp, so the parsed value is hard to compare228 # tests_pattern is a regexp, so the parsed value is hard to compare
221 # with the default value in the loop below.229 # with the default value in the loop below.
222230
=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg 2010-10-08 10:06:11 +0000
+++ database/schema/security.cfg 2010-10-16 18:22:51 +0000
@@ -545,11 +545,18 @@
545public.bugnotification = SELECT, INSERT545public.bugnotification = SELECT, INSERT
546public.bugnotificationrecipient = SELECT, INSERT546public.bugnotificationrecipient = SELECT, INSERT
547public.bugsubscription = SELECT547public.bugsubscription = SELECT
548<<<<<<< TREE
548public.bugsubscriptionfilter = SELECT549public.bugsubscriptionfilter = SELECT
549public.bugsubscriptionfilterstatus = SELECT550public.bugsubscriptionfilterstatus = SELECT
550public.bugsubscriptionfilterimportance = SELECT551public.bugsubscriptionfilterimportance = SELECT
551public.bugsubscriptionfiltertag = SELECT552public.bugsubscriptionfiltertag = SELECT
552public.bugtag = SELECT553public.bugtag = SELECT
554=======
555public.bugsubscriptionfilter = SELECT
556public.bugsubscriptionfilterstatus = SELECT
557public.bugsubscriptionfilterimportance = SELECT
558public.bugsubscriptionfiltertag = SELECT
559>>>>>>> MERGE-SOURCE
553public.bugtask = SELECT, INSERT, UPDATE560public.bugtask = SELECT, INSERT, UPDATE
554public.bugtracker = SELECT, INSERT561public.bugtracker = SELECT, INSERT
555public.bugtrackercomponent = SELECT, INSERT, UPDATE, DELETE562public.bugtrackercomponent = SELECT, INSERT, UPDATE, DELETE
556563
=== modified file 'lib/canonical/config/schema-lazr.conf'
=== added file 'lib/canonical/testing/parallel.py'
--- lib/canonical/testing/parallel.py 1970-01-01 00:00:00 +0000
+++ lib/canonical/testing/parallel.py 2010-10-16 18:22:51 +0000
@@ -0,0 +1,174 @@
1# Copyright 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4from __future__ import with_statement
5
6"""Parallel test glue."""
7
8__metaclass__ = type
9__all__ = ["main"]
10
11import itertools
12import os
13import subprocess
14import sys
15import tempfile
16
17from bzrlib.osutils import local_concurrency
18from subunit import ProtocolTestCase
19from subunit.run import SubunitTestRunner
20from testtools.compat import unicode_output_stream
21from testtools import (
22 ConcurrentTestSuite,
23 TestResult,
24 TextTestResult,
25 )
26
27
28def prepare_argv(argv):
29 """Remove options from argv that would be added by ListTestCase."""
30 result = []
31 skipn = 0
32 for pos, arg in enumerate(argv):
33 if skipn:
34 skipn -= 1
35 continue
36 if arg in ('--subunit', '--parallel'):
37 continue
38 if arg.startswith('--load-list='):
39 continue
40 if arg == '--load-list':
41 skipn = 1
42 continue
43 result.append(arg)
44 return result
45
46
47def find_load_list(args):
48 """Get the value passed in to --load-list=FOO."""
49 load_list = None
50 for pos, arg in enumerate(args):
51 if arg.startswith('--load-list='):
52 load_list = arg[len('--load-list='):]
53 if arg == '--load-list':
54 load_list = args[pos+1]
55 return load_list
56
57
58class GatherIDs(TestResult):
59 """Gather test ids from a test run."""
60
61 def __init__(self):
62 super(GatherIDs, self).__init__()
63 self.ids = []
64
65 def startTest(self, test):
66 super(GatherIDs, self).startTest(test)
67 self.ids.append(test.id())
68
69
70def find_tests(argv):
71 """Find tests to parallel run.
72
73 :param argv: The argv given to the test runner, used to get the tests to
74 run.
75 :return: A list of test IDs.
76 """
77 load_list = find_load_list(argv)
78 if load_list:
79 # just use the load_list
80 with open(load_list, 'rt') as list_file:
81 return [id for id in list_file.read().split('\n') if id]
82 # run in --list-tests mode
83 argv = prepare_argv(argv) + ['--list-tests', '--subunit']
84 process = subprocess.Popen(argv, stdin=subprocess.PIPE,
85 stdout=subprocess.PIPE)
86 process.stdin.close()
87 test = ProtocolTestCase(process.stdout)
88 result = GatherIDs()
89 test.run(result)
90 process.wait()
91 if process.returncode:
92 raise Exception('error listing tests: %s' % err)
93 return result.ids
94
95
96class ListTestCase(ProtocolTestCase):
97
98 def __init__(self, test_ids, args):
99 """Create a ListTestCase.
100
101 :param test_ids: The ids of the tests to run.
102 :param args: The args to use to run the test runner (without
103 --load-list - that is added automatically).
104 """
105 self._test_ids = test_ids
106 self._args = args
107
108 def run(self, result):
109 with tempfile.NamedTemporaryFile() as test_list_file:
110 for test_id in self._test_ids:
111 test_list_file.write(test_id + '\n')
112 test_list_file.flush()
113 argv = self._args + ['--subunit', '--load-list', test_list_file.name]
114 process = subprocess.Popen(argv, stdin=subprocess.PIPE,
115 stdout=subprocess.PIPE, bufsize=1)
116 try:
117 # If it tries to read, give it EOF.
118 process.stdin.close()
119 ProtocolTestCase.__init__(self, process.stdout)
120 ProtocolTestCase.run(self, result)
121 finally:
122 process.wait()
123
124
125def concurrency():
126 """Return the number of current tests we should run on this machine.
127
128 Each test is run in its own process, and we assume that the optimal number
129 is one per core.
130 """
131 # TODO: limit by memory as well.
132 procs = local_concurrency()
133 return procs
134
135
136def partition_tests(test_ids, count):
137 """Partition suite into count lists of tests."""
138 # This just assigns tests in a round-robin fashion. On one hand this
139 # splits up blocks of related tests that might run faster if they shared
140 # resources, but on the other it avoids assigning blocks of slow tests to
141 # just one partition. So the slowest partition shouldn't be much slower
142 # than the fastest.
143 partitions = [list() for i in range(count)]
144 for partition, test_id in itertools.izip(itertools.cycle(partitions), test_ids):
145 partition.append(test_id)
146 return partitions
147
148
149def main(argv, prepare_args=prepare_argv, find_tests=find_tests):
150 """CLI entry point to adapt a test run to parallel testing."""
151 child_args = prepare_argv(argv)
152 test_ids = find_tests(argv)
153 # We could create a proxy object per test id if desired in future)
154 def parallelise_tests(suite):
155 test_ids = list(suite)[0]._test_ids
156 count = concurrency()
157 partitions = partition_tests(test_ids, count)
158 return [ListTestCase(partition, child_args) for partition in partitions]
159 suite = ConcurrentTestSuite(ListTestCase(test_ids, None), parallelise_tests)
160 if '--subunit' in argv:
161 runner = SubunitTestRunner(sys.stdout)
162 result = runner.run(suite)
163 else:
164 stream = unicode_output_stream(sys.stdout)
165 result = TextTestResult(stream)
166 result.startTestRun()
167 try:
168 suite.run(result)
169 finally:
170 result.stopTestRun()
171 if result.wasSuccessful():
172 return 0
173 return -1
174
0175
=== added file 'lib/canonical/testing/tests/test_parallel.py'
--- lib/canonical/testing/tests/test_parallel.py 1970-01-01 00:00:00 +0000
+++ lib/canonical/testing/tests/test_parallel.py 2010-10-16 18:22:51 +0000
@@ -0,0 +1,117 @@
1# Copyright 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4from __future__ import with_statement
5
6"""Parallel test glue."""
7
8__metaclass__ = type
9
10from StringIO import StringIO
11import subprocess
12import tempfile
13
14from testtools import (
15 TestCase,
16 TestResult,
17 )
18from fixtures import (
19 PopenFixture,
20 TestWithFixtures,
21 )
22
23from canonical.testing.parallel import (
24 find_load_list,
25 find_tests,
26 ListTestCase,
27 main,
28 prepare_argv,
29 )
30
31
32class TestListTestCase(TestCase, TestWithFixtures):
33
34 def test_run(self):
35 # ListTestCase.run should run bin/test adding in --subunit,
36 # --load-list, with a list file containing the supplied list ids.
37 def check_list_file(info):
38 """Callback from subprocess.Popen for testing run()."""
39 # A temp file should have been made.
40 args = info['args']
41 load_list = find_load_list(args)
42 self.assertNotEqual(None, load_list)
43 with open(load_list, 'rt') as testlist:
44 contents = testlist.readlines()
45 self.assertEqual(['foo\n', 'bar\n'], contents)
46 return {'stdout': StringIO(''), 'stdin': StringIO()}
47 popen = self.useFixture(PopenFixture(check_list_file))
48 case = ListTestCase(['foo', 'bar'], ['bin/test'])
49 self.assertEqual([], popen.procs)
50 result = TestResult()
51 case.run(result)
52 self.assertEqual(0, result.testsRun)
53 self.assertEqual(1, len(popen.procs))
54 self.assertEqual(['bin/test', '--subunit', '--load-list'],
55 popen.procs[0]._args['args'][:-1])
56
57
58class TestUtilities(TestCase, TestWithFixtures):
59
60 def test_prepare_argv_removes_subunit(self):
61 self.assertEqual(
62 ['bin/test', 'foo'],
63 prepare_argv(['bin/test', '--subunit', 'foo']))
64
65 def test_prepare_argv_removes_parallel(self):
66 self.assertEqual(
67 ['bin/test', 'foo'],
68 prepare_argv(['bin/test', '--parallel', 'foo']))
69
70 def test_prepare_argv_removes_load_list_with_equals(self):
71 self.assertEqual(
72 ['bin/test', 'foo'],
73 prepare_argv(['bin/test', '--load-list=Foo', 'foo']))
74
75 def test_prepare_argv_removes_load_2_arg_form(self):
76 self.assertEqual(
77 ['bin/test', 'foo'],
78 prepare_argv(['bin/test', '--load-list', 'Foo', 'foo']))
79
80 def test_find_tests_honours_list_list_equals(self):
81 with tempfile.NamedTemporaryFile() as listfile:
82 listfile.write('foo\nbar\n')
83 listfile.flush()
84 self.assertEqual(
85 ['foo', 'bar'],
86 find_tests(
87 ['bin/test', '--load-list=%s' % listfile.name, 'foo']))
88
89 def test_find_tests_honours_list_list_two_arg_form(self):
90 with tempfile.NamedTemporaryFile() as listfile:
91 listfile.write('foo\nbar\n')
92 listfile.flush()
93 self.assertEqual(
94 ['foo', 'bar'],
95 find_tests(
96 ['bin/test', '--load-list', listfile.name, 'foo']))
97
98 def test_find_tests_live(self):
99 # When --load-tests wasn't supplied, find_tests needs to run bin/test
100 # with --list-tests and --subunit, and parse the resulting subunit
101 # stream.
102 def inject_testlist(args):
103 self.assertEqual(subprocess.PIPE, args['stdin'])
104 self.assertEqual(subprocess.PIPE, args['stdout'])
105 self.assertEqual(
106 ['bin/test', '-vt', 'filter', '--list-tests', '--subunit'],
107 args['args'])
108 return {'stdin': StringIO(), 'stdout': StringIO(u"""
109test: quux
110successful: quux
111test: glom
112successful: glom
113""")}
114 popen = self.useFixture(PopenFixture(inject_testlist))
115 self.assertEqual(
116 ['quux', 'glom'],
117 find_tests(['bin/test', '-vt', 'filter', '--parallel']))
0118
=== modified file 'lib/lp/archivepublisher/tests/test_publisher.py'
--- lib/lp/archivepublisher/tests/test_publisher.py 2010-10-15 08:30:17 +0000
+++ lib/lp/archivepublisher/tests/test_publisher.py 2010-10-16 18:22:51 +0000
@@ -1081,6 +1081,7 @@
1081 # The Label: field should be set to the archive displayname1081 # The Label: field should be set to the archive displayname
1082 self.assertEqual(release_contents[1], 'Label: Partner archive')1082 self.assertEqual(release_contents[1], 'Label: Partner archive')
10831083
1084<<<<<<< TREE
10841085
1085class TestArchiveIndices(TestPublisherBase):1086class TestArchiveIndices(TestPublisherBase):
1086 """Tests for the native publisher's index generation.1087 """Tests for the native publisher's index generation.
@@ -1187,6 +1188,75 @@
1187 publisher, self.breezy_autotest, PackagePublishingPocket.RELEASE,1188 publisher, self.breezy_autotest, PackagePublishingPocket.RELEASE,
1188 present=['hppa'], absent=['i386'])1189 present=['hppa'], absent=['i386'])
11891190
1191=======
1192 def assertIndicesForArchitectures(self, publisher, present, absent):
1193 """Assert that the correct set of archs has indices.
1194
1195 Checks that the architecture tags in 'present' have Packages and
1196 Release files and are in the series' Release file, and confirms
1197 that those in 'absent' are not.
1198 """
1199
1200 self.checkAllRequestedReleaseFiles(
1201 publisher, architecturetags=present)
1202
1203 arch_template = os.path.join(
1204 publisher._config.distsroot, 'breezy-autotest/main/binary-%s')
1205 release_template = os.path.join(arch_template, 'Release')
1206 packages_template = os.path.join(arch_template, 'Packages')
1207 release_content = open(os.path.join(
1208 publisher._config.distsroot,
1209 'breezy-autotest/Release')).read()
1210
1211 for arch in present:
1212 self.assertTrue(os.path.exists(arch_template % arch))
1213 self.assertTrue(os.path.exists(release_template % arch))
1214 self.assertTrue(os.path.exists(packages_template % arch))
1215 self.assertTrue(arch in release_content)
1216
1217 for arch in absent:
1218 self.assertFalse(os.path.exists(arch_template % arch))
1219 self.assertFalse(arch in release_content)
1220
1221 def testNativeNoIndicesForDisabledArchitectures(self):
1222 """Test that no indices are created for disabled archs."""
1223 self.getPubBinaries()
1224
1225 ds = self.ubuntutest.getSeries('breezy-autotest')
1226 ds.getDistroArchSeries('i386').enabled = False
1227 self.config = Config(self.ubuntutest)
1228
1229 publisher = Publisher(
1230 self.logger, self.config, self.disk_pool,
1231 self.ubuntutest.main_archive)
1232
1233 publisher.A_publish(False)
1234 publisher.C_writeIndexes(False)
1235 publisher.D_writeReleaseFiles(False)
1236
1237 self.assertIndicesForArchitectures(
1238 publisher, present=['hppa'], absent=['i386'])
1239
1240 def testAptFtparchiveNoIndicesForDisabledArchitectures(self):
1241 """Test that no indices are created for disabled archs."""
1242 self.getPubBinaries()
1243
1244 ds = self.ubuntutest.getSeries('breezy-autotest')
1245 ds.getDistroArchSeries('i386').enabled = False
1246 self.config = Config(self.ubuntutest)
1247
1248 publisher = Publisher(
1249 self.logger, self.config, self.disk_pool,
1250 self.ubuntutest.main_archive)
1251
1252 publisher.A_publish(False)
1253 publisher.C_doFTPArchive(False)
1254 publisher.D_writeReleaseFiles(False)
1255
1256 self.assertIndicesForArchitectures(
1257 publisher, present=['hppa'], absent=['i386'])
1258
1259>>>>>>> MERGE-SOURCE
1190 def testWorldAndGroupReadablePackagesAndSources(self):1260 def testWorldAndGroupReadablePackagesAndSources(self):
1191 """Test Packages.gz and Sources.gz files are world readable."""1261 """Test Packages.gz and Sources.gz files are world readable."""
1192 publisher = Publisher(1262 publisher = Publisher(
11931263
=== modified file 'lib/lp/bugs/browser/configure.zcml'
=== modified file 'lib/lp/bugs/configure.zcml'
=== modified file 'lib/lp/bugs/doc/initial-bug-contacts.txt'
=== modified file 'lib/lp/bugs/stories/bug-tags/xx-official-bug-tags.txt'
=== modified file 'lib/lp/bugs/stories/webservice/xx-bug-target.txt'
=== modified file 'lib/lp/bugs/tests/has-bug-supervisor.txt'
=== modified file 'lib/lp/registry/browser/configure.zcml'
=== modified file 'lib/lp/registry/browser/tests/poll-views_0.txt'
=== modified file 'lib/lp/registry/doc/structural-subscriptions.txt'
--- lib/lp/registry/doc/structural-subscriptions.txt 2010-10-15 21:48:34 +0000
+++ lib/lp/registry/doc/structural-subscriptions.txt 2010-10-16 18:22:51 +0000
@@ -198,8 +198,68 @@
198 name16198 name16
199199
200200
201Target type display201<<<<<<< TREE
202===================202Target type display
203===================
204=======
205Retrieving structural subscribers of targets
206============================================
207
208Subscribers of one or more targets are retrieved by
209PersonSet.getSubscribersForTargets.
210
211XXX Abel Deuring 2008-07-11 We must remove the security proxy, because
212PersonSet.getSubscribersForTargets() accesses the private attribute
213firefox._targets_args. This attribute should be renamed. Bug #247525.
214
215 >>> from zope.security.proxy import removeSecurityProxy
216 >>> firefox_no_proxy = removeSecurityProxy(firefox)
217 >>> from lp.registry.interfaces.person import IPersonSet
218 >>> person_set = getUtility(IPersonSet)
219 >>> firefox_subscribers = person_set.getSubscribersForTargets(
220 ... [firefox_no_proxy])
221 >>> print_bug_subscribers(firefox_subscribers)
222 name12
223
224If PersonSet.getSubscribersForTargets() is passed a
225BugNotificationLevel in its `level` parameter, only structural
226subscribers with that notification level or higher will be returned.
227The subscription level of ff_sub, the only structural subscription
228to firefox, is NOTHING, hence we get an empty list if we pass any
229larger bug notification level.
230
231 >>> firefox_subscribers = person_set.getSubscribersForTargets(
232 ... [firefox_no_proxy], level=BugNotificationLevel.LIFECYCLE)
233 >>> print len(firefox_subscribers)
234 0
235
236If the bug notification level of ff_sub is increased, the subscription
237is included in the result of PersonSet.getSubscribersForTarget().
238
239 >>> ff_sub.level = BugNotificationLevel.LIFECYCLE
240 >>> firefox_subscribers = person_set.getSubscribersForTargets(
241 ... [firefox_no_proxy])
242 >>> print_bug_subscribers(firefox_subscribers)
243 name12
244 >>> ff_sub.level = BugNotificationLevel.METADATA
245 >>> firefox_subscribers = person_set.getSubscribersForTargets(
246 ... [firefox_no_proxy])
247 >>> print_bug_subscribers(firefox_subscribers)
248 name12
249
250More than one target can be passed to PersonSet.getSubscribersForTargets().
251
252 >>> ubuntu_no_proxy = removeSecurityProxy(ubuntu)
253 >>> ubuntu_or_firefox_subscribers = person_set.getSubscribersForTargets(
254 ... [firefox_no_proxy, ubuntu_no_proxy])
255 >>> print_bug_subscribers(ubuntu_or_firefox_subscribers)
256 name12
257 name16
258
259
260Target type display
261===================
262>>>>>>> MERGE-SOURCE
203263
204Structural subscription targets have a `target_type_display` attribute, which264Structural subscription targets have a `target_type_display` attribute, which
205can be used to refer to them in display.265can be used to refer to them in display.
206266
=== modified file 'lib/lp/registry/doc/teammembership-email-notification.txt'
=== modified file 'lib/lp/registry/interfaces/structuralsubscription.py'
--- lib/lp/registry/interfaces/structuralsubscription.py 2010-10-07 10:06:55 +0000
+++ lib/lp/registry/interfaces/structuralsubscription.py 2010-10-16 18:22:51 +0000
@@ -172,6 +172,7 @@
172 def userCanAlterSubscription(subscriber, subscribed_by):172 def userCanAlterSubscription(subscriber, subscribed_by):
173 """Check if a user can change a subscription for a person."""173 """Check if a user can change a subscription for a person."""
174174
175<<<<<<< TREE
175 def userCanAlterBugSubscription(subscriber, subscribed_by):176 def userCanAlterBugSubscription(subscriber, subscribed_by):
176 """Check if a user can change a bug subscription for a person."""177 """Check if a user can change a bug subscription for a person."""
177178
@@ -210,6 +211,46 @@
210 Modify-only parts.211 Modify-only parts.
211 """212 """
212213
214=======
215 def userCanAlterBugSubscription(subscriber, subscribed_by):
216 """Check if a user can change a bug subscription for a person."""
217
218 @operation_parameters(person=Reference(schema=IPerson))
219 @operation_returns_entry(IStructuralSubscription)
220 @export_read_operation()
221 def getSubscription(person):
222 """Return the subscription for `person`, if it exists."""
223
224 def getBugNotificationsRecipients(recipients=None, level=None):
225 """Return the set of bug subscribers to this target.
226
227 :param recipients: If recipients is not None, a rationale
228 is added for each subscriber.
229 :type recipients: `INotificationRecipientSet`
230 'param level: If level is not None, only strucutral
231 subscribers with a subscrition level greater or equal
232 to the given value are returned.
233 :type level: `BugNotificationLevel`
234 :return: An `INotificationRecipientSet` instance containing
235 the bug subscribers.
236 """
237
238 target_type_display = Attribute("The type of the target, for display.")
239
240 def userHasBugSubscriptions(user):
241 """Is `user` subscribed, directly or via a team, to bug mail?"""
242
243 def getSubscriptionsForBug(bug, level):
244 """Return subscriptions for a given `IBug` at `level`."""
245
246
247class IStructuralSubscriptionTargetWrite(Interface):
248 """A Launchpad Structure allowing users to subscribe to it.
249
250 Modify-only parts.
251 """
252
253>>>>>>> MERGE-SOURCE
213 def addSubscription(subscriber, subscribed_by):254 def addSubscription(subscriber, subscribed_by):
214 """Add a subscription for this structure.255 """Add a subscription for this structure.
215256
216257
=== modified file 'lib/lp/registry/model/distribution.py'
=== modified file 'lib/lp/registry/model/structuralsubscription.py'
--- lib/lp/registry/model/structuralsubscription.py 2010-10-07 10:06:55 +0000
+++ lib/lp/registry/model/structuralsubscription.py 2010-10-16 18:22:51 +0000
@@ -483,6 +483,7 @@
483 # The user has a bug subscription483 # The user has a bug subscription
484 return True484 return True
485 return False485 return False
486<<<<<<< TREE
486487
487 def getSubscriptionsForBugTask(self, bugtask, level):488 def getSubscriptionsForBugTask(self, bugtask, level):
488 """See `IStructuralSubscriptionTarget`."""489 """See `IStructuralSubscriptionTarget`."""
@@ -533,3 +534,69 @@
533534
534 return Store.of(self.__helper.pillar).using(*origin).find(535 return Store.of(self.__helper.pillar).using(*origin).find(
535 StructuralSubscription, self.__helper.join, *conditions)536 StructuralSubscription, self.__helper.join, *conditions)
537=======
538
539 def getSubscriptionsForBug(self, bug, level):
540 """See `IStructuralSubscriptionTarget`."""
541 if IProjectGroup.providedBy(self):
542 targets = set(self.products)
543 elif IMilestone.providedBy(self):
544 targets = [self.series_target]
545 else:
546 targets = [self]
547
548 bugtasks = [
549 bugtask for bugtask in bug.bugtasks
550 if bugtask.target in targets]
551
552 assert len(bugtasks) != 0, repr(self)
553
554 origin = [
555 StructuralSubscription,
556 LeftJoin(
557 BugSubscriptionFilter,
558 BugSubscriptionFilter.structural_subscription_id == (
559 StructuralSubscription.id)),
560 LeftJoin(
561 BugSubscriptionFilterStatus,
562 BugSubscriptionFilterStatus.filter_id == (
563 BugSubscriptionFilter.id)),
564 LeftJoin(
565 BugSubscriptionFilterImportance,
566 BugSubscriptionFilterImportance.filter_id == (
567 BugSubscriptionFilter.id)),
568 ]
569
570 if len(bug.tags) == 0:
571 tag_conditions = [
572 BugSubscriptionFilter.include_any_tags == False,
573 ]
574 else:
575 tag_conditions = [
576 BugSubscriptionFilter.exclude_any_tags == False,
577 ]
578
579 conditions = [
580 StructuralSubscription.bug_notification_level >= level,
581 Or(
582 # There's no filter or ...
583 BugSubscriptionFilter.id == None,
584 # There is a filter and ...
585 And(
586 # There's no status filter, or there is a status filter
587 # and and it matches.
588 Or(BugSubscriptionFilterStatus.id == None,
589 BugSubscriptionFilterStatus.status.is_in(
590 bugtask.status for bugtask in bugtasks)),
591 # There's no importance filter, or there is an importance
592 # filter and it matches.
593 Or(BugSubscriptionFilterImportance.id == None,
594 BugSubscriptionFilterImportance.importance.is_in(
595 bugtask.importance for bugtask in bugtasks)),
596 # Any number of conditions relating to tags.
597 *tag_conditions)),
598 ]
599
600 return Store.of(self.__helper.pillar).using(*origin).find(
601 StructuralSubscription, self.__helper.join, *conditions)
602>>>>>>> MERGE-SOURCE
536603
=== modified file 'lib/lp/registry/tests/structural-subscription-target.txt'
=== modified file 'lib/lp/registry/tests/test_structuralsubscriptiontarget.py'
--- lib/lp/registry/tests/test_structuralsubscriptiontarget.py 2010-10-07 10:07:51 +0000
+++ lib/lp/registry/tests/test_structuralsubscriptiontarget.py 2010-10-16 18:22:51 +0000
@@ -167,6 +167,7 @@
167 None)167 None)
168168
169169
170<<<<<<< TREE
170class FilteredStructuralSubscriptionTestBase(StructuralSubscriptionTestBase):171class FilteredStructuralSubscriptionTestBase(StructuralSubscriptionTestBase):
171 """Tests for filtered structural subscriptions."""172 """Tests for filtered structural subscriptions."""
172173
@@ -359,6 +360,199 @@
359360
360class TestStructuralSubscriptionForDistro(361class TestStructuralSubscriptionForDistro(
361 FilteredStructuralSubscriptionTestBase, TestCaseWithFactory):362 FilteredStructuralSubscriptionTestBase, TestCaseWithFactory):
363=======
364class FilteredStructuralSubscriptionTestBase(StructuralSubscriptionTestBase):
365 """Tests for filtered structural subscriptions."""
366
367 def makeBugTask(self):
368 return self.factory.makeBugTask(target=self.target)
369
370 def test_getSubscriptionsForBug(self):
371 # If no one has a filtered subscription for the given bug, the result
372 # of getSubscriptionsForBug() is the same as for getSubscriptions().
373 bugtask = self.makeBugTask()
374 subscriptions = self.target.getSubscriptions(
375 min_bug_notification_level=BugNotificationLevel.NOTHING)
376 subscriptions_for_bug = self.target.getSubscriptionsForBug(
377 bugtask.bug, BugNotificationLevel.NOTHING)
378 self.assertEqual(list(subscriptions), list(subscriptions_for_bug))
379
380 def test_getSubscriptionsForBug_with_filter_on_status(self):
381 # If a status filter exists for a subscription, the result of
382 # getSubscriptionsForBug() may be a subset of getSubscriptions().
383 bugtask = self.makeBugTask()
384
385 # Create a new subscription on self.target.
386 login_person(self.ordinary_subscriber)
387 subscription = self.target.addSubscription(
388 self.ordinary_subscriber, self.ordinary_subscriber)
389 subscription.bug_notification_level = BugNotificationLevel.COMMENTS
390
391 # Without any filters the subscription is found.
392 subscriptions_for_bug = self.target.getSubscriptionsForBug(
393 bugtask.bug, BugNotificationLevel.NOTHING)
394 self.assertEqual([subscription], list(subscriptions_for_bug))
395
396 # Filter the subscription to bugs in the CONFIRMED state.
397 subscription_filter = BugSubscriptionFilter()
398 subscription_filter.structural_subscription = subscription
399 subscription_filter.statuses = [BugTaskStatus.CONFIRMED]
400
401 # With the filter the subscription is not found.
402 subscriptions_for_bug = self.target.getSubscriptionsForBug(
403 bugtask.bug, BugNotificationLevel.NOTHING)
404 self.assertEqual([], list(subscriptions_for_bug))
405
406 # If the filter is adjusted, the subscription is found again.
407 subscription_filter.statuses = [bugtask.status]
408 subscriptions_for_bug = self.target.getSubscriptionsForBug(
409 bugtask.bug, BugNotificationLevel.NOTHING)
410 self.assertEqual([subscription], list(subscriptions_for_bug))
411
412 def test_getSubscriptionsForBug_with_filter_on_importance(self):
413 # If an importance filter exists for a subscription, the result of
414 # getSubscriptionsForBug() may be a subset of getSubscriptions().
415 bugtask = self.makeBugTask()
416
417 # Create a new subscription on self.target.
418 login_person(self.ordinary_subscriber)
419 subscription = self.target.addSubscription(
420 self.ordinary_subscriber, self.ordinary_subscriber)
421 subscription.bug_notification_level = BugNotificationLevel.COMMENTS
422
423 # Without any filters the subscription is found.
424 subscriptions_for_bug = self.target.getSubscriptionsForBug(
425 bugtask.bug, BugNotificationLevel.NOTHING)
426 self.assertEqual([subscription], list(subscriptions_for_bug))
427
428 # Filter the subscription to bugs in the CRITICAL state.
429 subscription_filter = BugSubscriptionFilter()
430 subscription_filter.structural_subscription = subscription
431 subscription_filter.importances = [BugTaskImportance.CRITICAL]
432
433 # With the filter the subscription is not found.
434 subscriptions_for_bug = self.target.getSubscriptionsForBug(
435 bugtask.bug, BugNotificationLevel.NOTHING)
436 self.assertEqual([], list(subscriptions_for_bug))
437
438 # If the filter is adjusted, the subscription is found again.
439 subscription_filter.importances = [bugtask.importance]
440 subscriptions_for_bug = self.target.getSubscriptionsForBug(
441 bugtask.bug, BugNotificationLevel.NOTHING)
442 self.assertEqual([subscription], list(subscriptions_for_bug))
443
444 def test_getSubscriptionsForBug_with_filter_on_level(self):
445 # All structural subscriptions have a level for bug notifications
446 # which getSubscriptionsForBug() observes.
447 bugtask = self.makeBugTask()
448
449 # Create a new METADATA level subscription on self.target.
450 login_person(self.ordinary_subscriber)
451 subscription = self.target.addSubscription(
452 self.ordinary_subscriber, self.ordinary_subscriber)
453 subscription.bug_notification_level = BugNotificationLevel.METADATA
454
455 # The subscription is found when looking for NOTHING or above.
456 subscriptions_for_bug = self.target.getSubscriptionsForBug(
457 bugtask.bug, BugNotificationLevel.NOTHING)
458 self.assertEqual([subscription], list(subscriptions_for_bug))
459 # The subscription is found when looking for METADATA or above.
460 subscriptions_for_bug = self.target.getSubscriptionsForBug(
461 bugtask.bug, BugNotificationLevel.METADATA)
462 self.assertEqual([subscription], list(subscriptions_for_bug))
463 # The subscription is not found when looking for COMMENTS or above.
464 subscriptions_for_bug = self.target.getSubscriptionsForBug(
465 bugtask.bug, BugNotificationLevel.COMMENTS)
466 self.assertEqual([], list(subscriptions_for_bug))
467
468 def test_getSubscriptionsForBug_with_filter_include_any_tags(self):
469 # If a subscription filter has include_any_tags, a bug with one or
470 # more tags is matched.
471 bugtask = self.makeBugTask()
472
473 # Create a new subscription on self.target.
474 login_person(self.ordinary_subscriber)
475 subscription = self.target.addSubscription(
476 self.ordinary_subscriber, self.ordinary_subscriber)
477 subscription.bug_notification_level = BugNotificationLevel.COMMENTS
478 subscription_filter = subscription.newBugFilter()
479 subscription_filter.include_any_tags = True
480
481 # Without any tags the subscription is not found.
482 subscriptions_for_bug = self.target.getSubscriptionsForBug(
483 bugtask.bug, BugNotificationLevel.NOTHING)
484 self.assertEqual([], list(subscriptions_for_bug))
485
486 # With any tag the subscription is found.
487 bugtask.bug.tags = ["foo"]
488 subscriptions_for_bug = self.target.getSubscriptionsForBug(
489 bugtask.bug, BugNotificationLevel.NOTHING)
490 self.assertEqual([subscription], list(subscriptions_for_bug))
491
492 def test_getSubscriptionsForBug_with_filter_exclude_any_tags(self):
493 # If a subscription filter has exclude_any_tags, only bugs with no
494 # tags are matched.
495 bugtask = self.makeBugTask()
496
497 # Create a new subscription on self.target.
498 login_person(self.ordinary_subscriber)
499 subscription = self.target.addSubscription(
500 self.ordinary_subscriber, self.ordinary_subscriber)
501 subscription.bug_notification_level = BugNotificationLevel.COMMENTS
502 subscription_filter = subscription.newBugFilter()
503 subscription_filter.exclude_any_tags = True
504
505 # Without any tags the subscription is found.
506 subscriptions_for_bug = self.target.getSubscriptionsForBug(
507 bugtask.bug, BugNotificationLevel.NOTHING)
508 self.assertEqual([subscription], list(subscriptions_for_bug))
509
510 # With any tag the subscription is not found.
511 bugtask.bug.tags = ["foo"]
512 subscriptions_for_bug = self.target.getSubscriptionsForBug(
513 bugtask.bug, BugNotificationLevel.NOTHING)
514 self.assertEqual([], list(subscriptions_for_bug))
515
516 def test_getSubscriptionsForBug_with_multiple_filters(self):
517 # If multiple filters exist for a subscription, all filters must
518 # match.
519 bugtask = self.makeBugTask()
520
521 # Create a new subscription on self.target.
522 login_person(self.ordinary_subscriber)
523 subscription = self.target.addSubscription(
524 self.ordinary_subscriber, self.ordinary_subscriber)
525 subscription.bug_notification_level = BugNotificationLevel.COMMENTS
526
527 # Filter the subscription to bugs in the CRITICAL state.
528 subscription_filter = BugSubscriptionFilter()
529 subscription_filter.structural_subscription = subscription
530 subscription_filter.statuses = [BugTaskStatus.CONFIRMED]
531 subscription_filter.importances = [BugTaskImportance.CRITICAL]
532
533 # With the filter the subscription is not found.
534 subscriptions_for_bug = self.target.getSubscriptionsForBug(
535 bugtask.bug, BugNotificationLevel.NOTHING)
536 self.assertEqual([], list(subscriptions_for_bug))
537
538 # If the filter is adjusted to match status but not importance, the
539 # subscription is still not found.
540 subscription_filter.statuses = [bugtask.status]
541 subscriptions_for_bug = self.target.getSubscriptionsForBug(
542 bugtask.bug, BugNotificationLevel.NOTHING)
543 self.assertEqual([], list(subscriptions_for_bug))
544
545 # If the filter is adjusted to also match importance, the subscription
546 # is found again.
547 subscription_filter.importances = [bugtask.importance]
548 subscriptions_for_bug = self.target.getSubscriptionsForBug(
549 bugtask.bug, BugNotificationLevel.NOTHING)
550 self.assertEqual([subscription], list(subscriptions_for_bug))
551
552
553class TestStructuralSubscriptionForDistro(
554 FilteredStructuralSubscriptionTestBase, TestCaseWithFactory):
555>>>>>>> MERGE-SOURCE
362556
363 layer = LaunchpadFunctionalLayer557 layer = LaunchpadFunctionalLayer
364558
365559
=== modified file 'lib/lp/services/mail/tests/test_incoming.py'
--- lib/lp/services/mail/tests/test_incoming.py 2010-10-15 20:44:53 +0000
+++ lib/lp/services/mail/tests/test_incoming.py 2010-10-16 18:22:51 +0000
@@ -92,6 +92,7 @@
9292
9393
94def test_suite():94def test_suite():
95<<<<<<< TREE
95 suite = unittest.TestLoader().loadTestsFromName(__name__)96 suite = unittest.TestLoader().loadTestsFromName(__name__)
96 suite.addTest(DocTestSuite('lp.services.mail.incoming'))97 suite.addTest(DocTestSuite('lp.services.mail.incoming'))
97 suite.addTest(98 suite.addTest(
@@ -101,4 +102,8 @@
101 tearDown=tearDown,102 tearDown=tearDown,
102 layer=LaunchpadZopelessLayer,103 layer=LaunchpadZopelessLayer,
103 stdout_logging_level=logging.WARNING))104 stdout_logging_level=logging.WARNING))
105=======
106 suite = unittest.TestLoader().loadTestsFromName(__name__)
107 suite.addTest(DocTestSuite('canonical.launchpad.mail.incoming'))
108>>>>>>> MERGE-SOURCE
104 return suite109 return suite
105110
=== modified file 'lib/lp/services/mailman/doc/postings.txt'
=== modified file 'lib/lp/soyuz/doc/distroarchseries.txt'
=== modified file 'lib/lp/soyuz/doc/package-arch-specific.txt'
=== modified file 'lib/lp/soyuz/interfaces/distributionjob.py'
--- lib/lp/soyuz/interfaces/distributionjob.py 2010-10-12 07:46:56 +0000
+++ lib/lp/soyuz/interfaces/distributionjob.py 2010-10-16 18:22:51 +0000
@@ -1,3 +1,4 @@
1<<<<<<< TREE
1# Copyright 2010 Canonical Ltd. This software is licensed under the2# Copyright 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).3# GNU Affero General Public License version 3 (see the file LICENSE).
34
@@ -64,3 +65,71 @@
6465
65class IInitialiseDistroSeriesJob(IRunnableJob):66class IInitialiseDistroSeriesJob(IRunnableJob):
66 """A Job that performs actions on a distribution."""67 """A Job that performs actions on a distribution."""
68=======
69# Copyright 2010 Canonical Ltd. This software is licensed under the
70# GNU Affero General Public License version 3 (see the file LICENSE).
71
72__metaclass__ = type
73
74__all__ = [
75 "DistributionJobType",
76 "IDistributionJob",
77 "IInitialiseDistroSeriesJob",
78 "IInitialiseDistroSeriesJobSource",
79]
80
81from lazr.enum import DBEnumeratedType, DBItem
82from zope.interface import Attribute, Interface
83from zope.schema import Int, Object
84
85from canonical.launchpad import _
86
87from lp.services.job.interfaces.job import IJob, IJobSource, IRunnableJob
88from lp.registry.interfaces.distribution import IDistribution
89from lp.registry.interfaces.distroseries import IDistroSeries
90
91
92class IDistributionJob(Interface):
93 """A Job that initialises acts on a distribution."""
94
95 id = Int(
96 title=_('DB ID'), required=True, readonly=True,
97 description=_("The tracking number for this job."))
98
99 distribution = Object(
100 title=_('The Distribution this job is about.'),
101 schema=IDistribution, required=True)
102
103 distroseries = Object(
104 title=_('The DistroSeries this job is about.'),
105 schema=IDistroSeries, required=False)
106
107 job = Object(
108 title=_('The common Job attributes'), schema=IJob, required=True)
109
110 metadata = Attribute('A dict of data about the job.')
111
112 def destroySelf():
113 """Destroy this object."""
114
115
116class DistributionJobType(DBEnumeratedType):
117
118 INITIALISE_SERIES = DBItem(1, """
119 Initialise a Distro Series.
120
121 This job initialises a given distro series, creating builds, and
122 populating the archive from the parent distroseries.
123 """)
124
125
126class IInitialiseDistroSeriesJobSource(IJobSource):
127 """An interface for acquiring IDistributionJobs."""
128
129 def create(distroseries):
130 """Create a new initialisation job for a distroseries."""
131
132
133class IInitialiseDistroSeriesJob(IRunnableJob):
134 """A Job that performs actions on a distribution."""
135>>>>>>> MERGE-SOURCE
67136
=== modified file 'lib/lp/soyuz/model/initialisedistroseriesjob.py'
--- lib/lp/soyuz/model/initialisedistroseriesjob.py 2010-10-13 05:04:41 +0000
+++ lib/lp/soyuz/model/initialisedistroseriesjob.py 2010-10-16 18:22:51 +0000
@@ -1,3 +1,4 @@
1<<<<<<< TREE
1# Copyright 2010 Canonical Ltd. This software is licensed under the2# Copyright 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).3# GNU Affero General Public License version 3 (see the file LICENSE).
34
@@ -66,3 +67,53 @@
66 self.rebuild)67 self.rebuild)
67 ids.check()68 ids.check()
68 ids.initialise()69 ids.initialise()
70=======
71# Copyright 2010 Canonical Ltd. This software is licensed under the
72# GNU Affero General Public License version 3 (see the file LICENSE).
73
74__metaclass__ = type
75
76__all__ = [
77 "InitialiseDistroSeriesJob",
78]
79
80from zope.interface import classProvides, implements
81
82from canonical.launchpad.interfaces.lpstorm import IMasterStore
83
84from lp.soyuz.interfaces.distributionjob import (
85 DistributionJobType,
86 IInitialiseDistroSeriesJob,
87 IInitialiseDistroSeriesJobSource,
88 )
89from lp.soyuz.model.distributionjob import (
90 DistributionJob,
91 DistributionJobDerived,
92 )
93from lp.soyuz.scripts.initialise_distroseries import (
94 InitialiseDistroSeries,
95 )
96
97
98class InitialiseDistroSeriesJob(DistributionJobDerived):
99
100 implements(IInitialiseDistroSeriesJob)
101
102 class_job_type = DistributionJobType.INITIALISE_SERIES
103 classProvides(IInitialiseDistroSeriesJobSource)
104
105 @classmethod
106 def create(cls, distroseries):
107 """See `IInitialiseDistroSeriesJob`."""
108 job = DistributionJob(
109 distroseries.distribution, distroseries, cls.class_job_type,
110 ())
111 IMasterStore(DistributionJob).add(job)
112 return cls(job)
113
114 def run(self):
115 """See `IRunnableJob`."""
116 ids = InitialiseDistroSeries(self.distroseries)
117 ids.check()
118 ids.initialise()
119>>>>>>> MERGE-SOURCE
69120
=== modified file 'lib/lp/soyuz/tests/test_initialisedistroseriesjob.py'
--- lib/lp/soyuz/tests/test_initialisedistroseriesjob.py 2010-10-12 07:46:56 +0000
+++ lib/lp/soyuz/tests/test_initialisedistroseriesjob.py 2010-10-16 18:22:51 +0000
@@ -1,3 +1,4 @@
1<<<<<<< TREE
1# Copyright 2010 Canonical Ltd. This software is licensed under the2# Copyright 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).3# GNU Affero General Public License version 3 (see the file LICENSE).
34
@@ -88,3 +89,79 @@
88 self.assertEqual(naked_job.arches, arches)89 self.assertEqual(naked_job.arches, arches)
89 self.assertEqual(naked_job.packagesets, packagesets)90 self.assertEqual(naked_job.packagesets, packagesets)
90 self.assertEqual(naked_job.rebuild, False)91 self.assertEqual(naked_job.rebuild, False)
92=======
93# Copyright 2010 Canonical Ltd. This software is licensed under the
94# GNU Affero General Public License version 3 (see the file LICENSE).
95
96import transaction
97from canonical.testing import LaunchpadZopelessLayer
98from storm.exceptions import IntegrityError
99from zope.component import getUtility
100from zope.security.proxy import removeSecurityProxy
101from lp.soyuz.interfaces.distributionjob import (
102 IInitialiseDistroSeriesJobSource,
103 )
104from lp.soyuz.model.initialisedistroseriesjob import (
105 InitialiseDistroSeriesJob,
106 )
107from lp.soyuz.scripts.initialise_distroseries import InitialisationError
108from lp.testing import TestCaseWithFactory
109
110
111class InitialiseDistroSeriesJobTests(TestCaseWithFactory):
112 """Test case for InitialiseDistroSeriesJob."""
113
114 layer = LaunchpadZopelessLayer
115
116 def test_getOopsVars(self):
117 distroseries = self.factory.makeDistroSeries()
118 job = getUtility(IInitialiseDistroSeriesJobSource).create(
119 distroseries)
120 vars = job.getOopsVars()
121 naked_job = removeSecurityProxy(job)
122 self.assertIn(
123 ('distribution_id', distroseries.distribution.id), vars)
124 self.assertIn(('distroseries_id', distroseries.id), vars)
125 self.assertIn(('distribution_job_id', naked_job.context.id), vars)
126
127 def _getJobs(self):
128 """Return the pending InitialiseDistroSeriesJobs as a list."""
129 return list(InitialiseDistroSeriesJob.iterReady())
130
131 def _getJobCount(self):
132 """Return the number of InitialiseDistroSeriesJobs in the
133 queue."""
134 return len(self._getJobs())
135
136 def test_create_only_creates_one(self):
137 distroseries = self.factory.makeDistroSeries()
138 # If there's already a InitialiseDistroSeriesJob for a
139 # DistroSeries, InitialiseDistroSeriesJob.create() won't create
140 # a new one.
141 job = getUtility(IInitialiseDistroSeriesJobSource).create(
142 distroseries)
143 transaction.commit()
144
145 # There will now be one job in the queue.
146 self.assertEqual(1, self._getJobCount())
147
148 new_job = getUtility(IInitialiseDistroSeriesJobSource).create(
149 distroseries)
150
151 # This is less than ideal
152 self.assertRaises(IntegrityError, self._getJobCount)
153
154 def test_run(self):
155 """Test that InitialiseDistroSeriesJob.run() actually
156 initialises builds and copies from the parent."""
157 distroseries = self.factory.makeDistroSeries()
158
159 job = getUtility(IInitialiseDistroSeriesJobSource).create(
160 distroseries)
161
162 # Since our new distroseries doesn't have a parent set, and the first
163 # thing that run() will execute is checking the distroseries, if it
164 # returns an InitialisationError, then it's good.
165 self.assertRaisesWithContent(
166 InitialisationError, "Parent series required.", job.run)
167>>>>>>> MERGE-SOURCE
91168
=== modified file 'versions.cfg'
--- versions.cfg 2010-10-06 11:46:51 +0000
+++ versions.cfg 2010-10-16 18:22:51 +0000
@@ -19,7 +19,7 @@
19epydoc = 3.0.119epydoc = 3.0.1
20FeedParser = 4.120FeedParser = 4.1
21feedvalidator = 0.0.0DEV-r104921feedvalidator = 0.0.0DEV-r1049
22fixtures = 0.322fixtures = 0.3.1
23functest = 0.8.723functest = 0.8.7
24funkload = 1.10.024funkload = 1.10.0
25grokcore.component = 1.625grokcore.component = 1.6