Launchpad itself

Merge lp:~mwhudson/launchpad/bzr-svn-monitor into lp:launchpad

  1. bzr-svn-monitor
  2. Merge into devel
Proposed by Michael Hudson-Doyle
Status: Merged
Approved by: Tim Penhey
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~mwhudson/launchpad/bzr-svn-monitor
Merge into: lp:launchpad
Diff against target: 428 lines (+153/-22)
8 files modified
lib/lp/code/doc/codeimport.txt (+65/-12)
lib/lp/code/mail/codeimport.py (+2/-1)
lib/lp/code/model/codeimport.py (+4/-1)
lib/lp/code/model/codeimportevent.py (+2/-1)
lib/lp/codehosting/codeimport/tests/servers.py (+40/-2)
lib/lp/codehosting/codeimport/tests/test_workermonitor.py (+26/-1)
lib/lp/codehosting/codeimport/worker.py (+5/-1)
lib/lp/testing/factory.py (+9/-3)
To merge this branch: bzr merge lp:~mwhudson/launchpad/bzr-svn-monitor
Reviewer Review Type Date Requested Status
Tim Penhey (community) Approve
Review via email: mp+15264@code.launchpad.net

Commit message

A test for bzr-svn at the worker-monitor level and enough to make it pass.

To post a comment you must log in.
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Hi,

This branch adds a test for bzr-svn at the worker-monitor level and makes it pass. It also tests some very basic stuff in codeimport.txt.

Cheers,
mwh

Revision history for this message
Tim Penhey (thumper) wrote :

As much as it hurts me to say it, but can you please ReSTify the codeimport doc test titles? The current feeling is that as you touch the old files they get converted.

Other than that, looks good.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/code/doc/codeimport.txt'
--- lib/lp/code/doc/codeimport.txt 2009-10-03 17:10:28 +0000
+++ lib/lp/code/doc/codeimport.txt 2009-12-07 03:01:12 +0000
@@ -1,4 +1,5 @@
1= Code Imports =1Code Imports
2============
23
3CodeImport objects model the process surrounding the code import4CodeImport objects model the process surrounding the code import
4service of Launchpad. A CodeImport object is created by a user5service of Launchpad. A CodeImport object is created by a user
@@ -14,7 +15,8 @@
14 >>> login('david.allouche@canonical.com')15 >>> login('david.allouche@canonical.com')
1516
1617
17== Code import set utility ==18Code import set utility
19-----------------------
1820
19CodeImports are created and found using the ICodeImportSet interface,21CodeImports are created and found using the ICodeImportSet interface,
20which is registered as a utility.22which is registered as a utility.
@@ -34,7 +36,8 @@
34 >>> nopriv = getUtility(IPersonSet).getByName('no-priv')36 >>> nopriv = getUtility(IPersonSet).getByName('no-priv')
3537
3638
37== CodeImport events ==39CodeImport events
40-----------------
3841
39Most mutating operations affecting code imports should create42Most mutating operations affecting code imports should create
40CodeImportEvent objects in the database to provide an audit trail.43CodeImportEvent objects in the database to provide an audit trail.
@@ -43,7 +46,8 @@
43 >>> event_set = getUtility(ICodeImportEventSet)46 >>> event_set = getUtility(ICodeImportEventSet)
4447
4548
46== Supported source systems ==49Supported source systems
50------------------------
4751
48The rcs_type field, which indicates whether the import is from CVS or52The rcs_type field, which indicates whether the import is from CVS or
49Subversion, takes values from the 'RevisionControlSystems' vocabulary.53Subversion, takes values from the 'RevisionControlSystems' vocabulary.
@@ -57,7 +61,8 @@
57 Git61 Git
5862
5963
60=== Import from CVS ===64Import from CVS
65+++++++++++++++
6166
62Code imports from CVS specify the CVSROOT value, and the path to import67Code imports from CVS specify the CVSROOT value, and the path to import
63in the repository, known as the "module".68in the repository, known as the "module".
@@ -113,7 +118,8 @@
113 True118 True
114119
115120
116=== Import from Subversion ===121Import from Subversion
122++++++++++++++++++++++
117123
118Code imports from Subversion specify the URL used with "svn checkout" to124Code imports from Subversion specify the URL used with "svn checkout" to
119retrieve the tree to import.125retrieve the tree to import.
@@ -140,7 +146,33 @@
140 >>> svn_import == existing_import146 >>> svn_import == existing_import
141 True147 True
142148
143=== Import from Git ===149
150Import from Subversion via bzr-svn
151++++++++++++++++++++++++++++++++++
152
153Code imports from Subversion can also specify that they should be
154imported with 'bzr-svn' rather than cscvs. In most respects these
155imports are similar to the Subversion via cscvs imports.
156
157 >>> bzr_svn = RevisionControlSystems.BZR_SVN
158 >>> bzr_svn_url = 'svn://svn.example.com/for-bzr-svn/trunk'
159 >>> bzr_svn_import = code_import_set.new(
160 ... registrant=nopriv, product=product, branch_name='trunk-bzr-svn',
161 ... rcs_type=bzr_svn, svn_branch_url=bzr_svn_url)
162 >>> verifyObject(ICodeImport, removeSecurityProxy(svn_import))
163 True
164
165The CodeImportSet.getBySVNDetails is also able to find bzr-svn
166imports.
167
168 >>> existing_bzr_svn_import = code_import_set.getBySVNDetails(
169 ... svn_branch_url=bzr_svn_url)
170 >>> bzr_svn_import == existing_bzr_svn_import
171 True
172
173
174Import from Git
175+++++++++++++++
144176
145Code imports from Git specify the URL used with "git clone" to177Code imports from Git specify the URL used with "git clone" to
146retrieve the branch to import.178retrieve the branch to import.
@@ -167,7 +199,9 @@
167 >>> git_import == existing_import199 >>> git_import == existing_import
168 True200 True
169201
170== Updating code import details ==202
203Updating code import details
204----------------------------
171205
172Members of the VCS Imports team (import operators), or Launchpad206Members of the VCS Imports team (import operators), or Launchpad
173administrators can update the details of the code import, including207administrators can update the details of the code import, including
@@ -267,7 +301,8 @@
267 ...301 ...
268302
269303
270== Update intervals ==304Update intervals
305----------------
271306
272After an import is initially completed, it must be updated regularly. Each307After an import is initially completed, it must be updated regularly. Each
273code import can specify a custom update interval, or use a default value.308code import can specify a custom update interval, or use a default value.
@@ -304,6 +339,8 @@
304 datetime.timedelta(0, 21600)339 datetime.timedelta(0, 21600)
305 >>> svn_import.effective_update_interval340 >>> svn_import.effective_update_interval
306 datetime.timedelta(0, 21600)341 datetime.timedelta(0, 21600)
342 >>> bzr_svn_import.effective_update_interval
343 datetime.timedelta(0, 21600)
307344
308 >>> default_interval_git345 >>> default_interval_git
309 datetime.timedelta(0, 21600)346 datetime.timedelta(0, 21600)
@@ -325,7 +362,9 @@
325 >>> svn_import.effective_update_interval362 >>> svn_import.effective_update_interval
326 datetime.timedelta(0, 3600)363 datetime.timedelta(0, 3600)
327364
328== Retreiving CodeImports ==365
366Retrieving CodeImports
367----------------------
329368
330You can retrieve all imports with the `getAll` method of ICodeImport.369You can retrieve all imports with the `getAll` method of ICodeImport.
331370
@@ -361,7 +400,8 @@
361 NotFoundError: -10400 NotFoundError: -10
362401
363402
364== Canonical URLs ==403Canonical URLs
404--------------
365405
366We've registered the ICodeImportSet utility on the 'code' part of the406We've registered the ICodeImportSet utility on the 'code' part of the
367site:407site:
@@ -380,7 +420,8 @@
380 NoCanonicalUrl: ...420 NoCanonicalUrl: ...
381421
382422
383== Modifying CodeImports ==423Modifying CodeImports
424---------------------
384425
385Modifications to CodeImport objects must be done using setter methods426Modifications to CodeImport objects must be done using setter methods
386that create CodeImportEvent objects when appropriate. This is enforced427that create CodeImportEvent objects when appropriate. This is enforced
@@ -449,6 +490,18 @@
449 instead of:490 instead of:
450 git://git.example.com/hello.git491 git://git.example.com/hello.git
451492
493Imports via bzr-svn are also similar.
494
495 >>> data = {'svn_branch_url': 'http://svn.example.com/for-bzr-svn/trunk'}
496 >>> modify_event = bzr_svn_import.updateFromData(data, nopriv)
497 >>> print make_email_body_for_code_import_update(
498 ... bzr_svn_import, modify_event, None)
499 ~no-priv/firefox/trunk-bzr-svn is now being imported from:
500 http://svn.example.com/for-bzr-svn/trunk
501 instead of:
502 svn://svn.example.com/for-bzr-svn/trunk
503
504
452In addition, updateFromData can be used to set the branch whiteboard,505In addition, updateFromData can be used to set the branch whiteboard,
453which is also described in the email that is sent.506which is also described in the email that is sent.
454507
455508
=== modified file 'lib/lp/code/mail/codeimport.py'
--- lib/lp/code/mail/codeimport.py 2009-10-13 14:00:24 +0000
+++ lib/lp/code/mail/codeimport.py 2009-12-07 03:01:12 +0000
@@ -95,7 +95,8 @@
95 body.append(95 body.append(
96 details_change_prefix + '\n' + new_details +96 details_change_prefix + '\n' + new_details +
97 "\ninstead of:\n" + old_details)97 "\ninstead of:\n" + old_details)
98 elif code_import.rcs_type == RevisionControlSystems.SVN:98 elif code_import.rcs_type in (RevisionControlSystems.SVN,
99 RevisionControlSystems.BZR_SVN):
99 if CodeImportEventDataType.OLD_SVN_BRANCH_URL in event_data:100 if CodeImportEventDataType.OLD_SVN_BRANCH_URL in event_data:
100 old_url = event_data[CodeImportEventDataType.OLD_SVN_BRANCH_URL]101 old_url = event_data[CodeImportEventDataType.OLD_SVN_BRANCH_URL]
101 body.append(102 body.append(
102103
=== modified file 'lib/lp/code/model/codeimport.py'
--- lib/lp/code/model/codeimport.py 2009-10-03 17:10:28 +0000
+++ lib/lp/code/model/codeimport.py 2009-12-07 03:01:12 +0000
@@ -105,6 +105,8 @@
105 config.codeimport.default_interval_cvs,105 config.codeimport.default_interval_cvs,
106 RevisionControlSystems.SVN:106 RevisionControlSystems.SVN:
107 config.codeimport.default_interval_subversion,107 config.codeimport.default_interval_subversion,
108 RevisionControlSystems.BZR_SVN:
109 config.codeimport.default_interval_subversion,
108 RevisionControlSystems.GIT:110 RevisionControlSystems.GIT:
109 config.codeimport.default_interval_git,111 config.codeimport.default_interval_git,
110 }112 }
@@ -218,7 +220,8 @@
218 assert cvs_root is not None and cvs_module is not None220 assert cvs_root is not None and cvs_module is not None
219 assert svn_branch_url is None221 assert svn_branch_url is None
220 assert git_repo_url is None222 assert git_repo_url is None
221 elif rcs_type == RevisionControlSystems.SVN:223 elif rcs_type in (RevisionControlSystems.SVN,
224 RevisionControlSystems.BZR_SVN):
222 assert cvs_root is None and cvs_module is None225 assert cvs_root is None and cvs_module is None
223 assert svn_branch_url is not None226 assert svn_branch_url is not None
224 assert git_repo_url is None227 assert git_repo_url is None
225228
=== modified file 'lib/lp/code/model/codeimportevent.py'
--- lib/lp/code/model/codeimportevent.py 2009-07-17 00:26:05 +0000
+++ lib/lp/code/model/codeimportevent.py 2009-12-07 03:01:12 +0000
@@ -256,7 +256,8 @@
256256
257 def _iterSourceDetails(self, code_import):257 def _iterSourceDetails(self, code_import):
258 """Yield key-value tuples describing the source of the import."""258 """Yield key-value tuples describing the source of the import."""
259 if code_import.rcs_type == RevisionControlSystems.SVN:259 if code_import.rcs_type in (RevisionControlSystems.SVN,
260 RevisionControlSystems.BZR_SVN):
260 yield 'SVN_BRANCH_URL', code_import.svn_branch_url261 yield 'SVN_BRANCH_URL', code_import.svn_branch_url
261 elif code_import.rcs_type == RevisionControlSystems.CVS:262 elif code_import.rcs_type == RevisionControlSystems.CVS:
262 yield 'CVS_ROOT', code_import.cvs_root263 yield 'CVS_ROOT', code_import.cvs_root
263264
=== modified file 'lib/lp/codehosting/codeimport/tests/servers.py'
--- lib/lp/codehosting/codeimport/tests/servers.py 2009-11-19 04:33:06 +0000
+++ lib/lp/codehosting/codeimport/tests/servers.py 2009-12-07 03:01:12 +0000
@@ -3,6 +3,8 @@
33
4"""Server classes that know how to create various kinds of foreign archive."""4"""Server classes that know how to create various kinds of foreign archive."""
55
6from __future__ import with_statement
7
6__all__ = [8__all__ = [
7 'CVSServer',9 'CVSServer',
8 'GitServer',10 'GitServer',
@@ -13,7 +15,10 @@
1315
14import os16import os
15import shutil17import shutil
18import signal
19import subprocess
16import tempfile20import tempfile
21import time
1722
18import CVS23import CVS
19import pysvn24import pysvn
@@ -60,9 +65,10 @@
60class SubversionServer(Server):65class SubversionServer(Server):
61 """A controller for an Subversion repository, used for testing."""66 """A controller for an Subversion repository, used for testing."""
6267
63 def __init__(self, repository_path):68 def __init__(self, repository_path, use_svn_serve=False):
64 super(SubversionServer, self).__init__()69 super(SubversionServer, self).__init__()
65 self.repository_path = os.path.abspath(repository_path)70 self.repository_path = os.path.abspath(repository_path)
71 self._use_svn_serve = use_svn_serve
6672
67 def createRepository(self, path):73 def createRepository(self, path):
68 """Create a Subversion repository at `path`."""74 """Create a Subversion repository at `path`."""
@@ -70,11 +76,43 @@
7076
71 def get_url(self):77 def get_url(self):
72 """Return a URL to the Subversion repository."""78 """Return a URL to the Subversion repository."""
73 return local_path_to_url(self.repository_path)79 if self._use_svn_serve:
80 return 'svn://localhost/'
81 else:
82 return local_path_to_url(self.repository_path)
7483
75 def setUp(self):84 def setUp(self):
76 super(SubversionServer, self).setUp()85 super(SubversionServer, self).setUp()
77 self.createRepository(self.repository_path)86 self.createRepository(self.repository_path)
87 if self._use_svn_serve:
88 conf_path = os.path.join(
89 self.repository_path, 'conf/svnserve.conf')
90 with open(conf_path , 'w') as conf_file:
91 conf_file.write('[general]\nanon-access = write\n')
92 self._svnserve = subprocess.Popen(
93 ['svnserve', '--daemon', '--foreground', '--root',
94 self.repository_path])
95 delay = 0.1
96 for i in range(10):
97 try:
98 client = pysvn.Client()
99 client.ls(self.get_url())
100 except pysvn.ClientError, e:
101 if 'Connection refused' in str(e):
102 time.sleep(delay)
103 delay *= 1.5
104 continue
105 else:
106 break
107 else:
108 raise AssertionError(
109 "svnserve didn't start accepting connections")
110
111 def tearDown(self):
112 super(SubversionServer, self).tearDown()
113 if self._use_svn_serve:
114 os.kill(self._svnserve.pid, signal.SIGINT)
115 self._svnserve.communicate()
78116
79 @run_in_temporary_directory117 @run_in_temporary_directory
80 def makeBranch(self, branch_name, tree_contents):118 def makeBranch(self, branch_name, tree_contents):
81119
=== modified file 'lib/lp/codehosting/codeimport/tests/test_workermonitor.py'
--- lib/lp/codehosting/codeimport/tests/test_workermonitor.py 2009-11-25 07:00:06 +0000
+++ lib/lp/codehosting/codeimport/tests/test_workermonitor.py 2009-12-07 03:01:12 +0000
@@ -29,7 +29,8 @@
29 TwistedLayer, TwistedLaunchpadZopelessLayer)29 TwistedLayer, TwistedLaunchpadZopelessLayer)
30from canonical.twistedsupport.tests.test_processmonitor import (30from canonical.twistedsupport.tests.test_processmonitor import (
31 makeFailure, ProcessTestsMixin)31 makeFailure, ProcessTestsMixin)
32from lp.code.enums import CodeImportResultStatus, CodeImportReviewStatus32from lp.code.enums import (
33 CodeImportResultStatus, CodeImportReviewStatus, RevisionControlSystems)
33from lp.code.interfaces.codeimport import ICodeImportSet34from lp.code.interfaces.codeimport import ICodeImportSet
34from lp.code.interfaces.codeimportjob import (35from lp.code.interfaces.codeimportjob import (
35 ICodeImportJobSet, ICodeImportJobWorkflow)36 ICodeImportJobSet, ICodeImportJobWorkflow)
@@ -476,6 +477,20 @@
476 return self.factory.makeCodeImport(477 return self.factory.makeCodeImport(
477 svn_branch_url=svn_branch_url)478 svn_branch_url=svn_branch_url)
478479
480 def makeBzrSvnCodeImport(self):
481 """Make a `CodeImport` that points to a real Subversion repository."""
482 self.subversion_server = SubversionServer(
483 self.repo_path, use_svn_serve=True)
484 self.subversion_server.setUp()
485 self.addCleanup(self.subversion_server.tearDown)
486 svn_branch_url = self.subversion_server.makeBranch(
487 'trunk', [('README', 'contents')])
488 self.foreign_commit_count = 2
489
490 return self.factory.makeCodeImport(
491 svn_branch_url=svn_branch_url,
492 rcs_type=RevisionControlSystems.BZR_SVN)
493
479 def makeGitCodeImport(self):494 def makeGitCodeImport(self):
480 """Make a `CodeImport` that points to a real Git repository."""495 """Make a `CodeImport` that points to a real Git repository."""
481 load_optional_plugin('git')496 load_optional_plugin('git')
@@ -579,6 +594,15 @@
579 result = self.performImport(job_id)594 result = self.performImport(job_id)
580 return result.addCallback(self.assertImported, code_import_id)595 return result.addCallback(self.assertImported, code_import_id)
581596
597 def test_import_bzrsvn(self):
598 # Create a Subversion-via-bzr-svn CodeImport and import it.
599 job = self.getStartedJobForImport(self.makeBzrSvnCodeImport())
600 code_import_id = job.code_import.id
601 job_id = job.id
602 self.layer.txn.commit()
603 result = self.performImport(job_id)
604 return result.addCallback(self.assertImported, code_import_id)
605
582606
583class DeferredOnExit(protocol.ProcessProtocol):607class DeferredOnExit(protocol.ProcessProtocol):
584608
@@ -591,6 +615,7 @@
591 else:615 else:
592 self._deferred.errback(reason)616 self._deferred.errback(reason)
593617
618
594class TestWorkerMonitorIntegrationScript(TestWorkerMonitorIntegration):619class TestWorkerMonitorIntegrationScript(TestWorkerMonitorIntegration):
595 """Tests for CodeImportWorkerMonitor that execute a child process."""620 """Tests for CodeImportWorkerMonitor that execute a child process."""
596621
597622
=== modified file 'lib/lp/codehosting/codeimport/worker.py'
--- lib/lp/codehosting/codeimport/worker.py 2009-12-06 23:01:26 +0000
+++ lib/lp/codehosting/codeimport/worker.py 2009-12-07 03:01:12 +0000
@@ -152,6 +152,10 @@
152 rcstype = 'svn'152 rcstype = 'svn'
153 svn_branch_url = str(code_import.svn_branch_url)153 svn_branch_url = str(code_import.svn_branch_url)
154 cvs_root = cvs_module = git_repo_url = None154 cvs_root = cvs_module = git_repo_url = None
155 elif code_import.rcs_type == RevisionControlSystems.BZR_SVN:
156 rcstype = 'bzr-svn'
157 svn_branch_url = str(code_import.svn_branch_url)
158 cvs_root = cvs_module = git_repo_url = None
155 elif code_import.rcs_type == RevisionControlSystems.CVS:159 elif code_import.rcs_type == RevisionControlSystems.CVS:
156 rcstype = 'cvs'160 rcstype = 'cvs'
157 svn_branch_url = git_repo_url = None161 svn_branch_url = git_repo_url = None
@@ -162,7 +166,7 @@
162 svn_branch_url = cvs_root = cvs_module = None166 svn_branch_url = cvs_root = cvs_module = None
163 git_repo_url = str(code_import.git_repo_url)167 git_repo_url = str(code_import.git_repo_url)
164 else:168 else:
165 raise AssertionError("Unknown rcstype %r." % rcstype)169 raise AssertionError("Unknown rcstype %r." % code_import.rcs_type)
166 return cls(170 return cls(
167 code_import.branch.id, rcstype, svn_branch_url,171 code_import.branch.id, rcstype, svn_branch_url,
168 cvs_root, cvs_module, git_repo_url)172 cvs_root, cvs_module, git_repo_url)
169173
=== modified file 'lib/lp/testing/factory.py'
--- lib/lp/testing/factory.py 2009-11-27 12:50:16 +0000
+++ lib/lp/testing/factory.py 2009-12-07 03:01:12 +0000
@@ -1210,7 +1210,7 @@
12101210
1211 def makeCodeImport(self, svn_branch_url=None, cvs_root=None,1211 def makeCodeImport(self, svn_branch_url=None, cvs_root=None,
1212 cvs_module=None, product=None, branch_name=None,1212 cvs_module=None, product=None, branch_name=None,
1213 git_repo_url=None, registrant=None):1213 git_repo_url=None, registrant=None, rcs_type=None):
1214 """Create and return a new, arbitrary code import.1214 """Create and return a new, arbitrary code import.
12151215
1216 The type of code import will be inferred from the source details1216 The type of code import will be inferred from the source details
@@ -1229,16 +1229,22 @@
12291229
1230 code_import_set = getUtility(ICodeImportSet)1230 code_import_set = getUtility(ICodeImportSet)
1231 if svn_branch_url is not None:1231 if svn_branch_url is not None:
1232 if rcs_type is None:
1233 rcs_type = RevisionControlSystems.SVN
1234 else:
1235 assert rcs_type in (RevisionControlSystems.SVN,
1236 RevisionControlSystems.BZR_SVN)
1232 return code_import_set.new(1237 return code_import_set.new(
1233 registrant, product, branch_name,1238 registrant, product, branch_name, rcs_type=rcs_type,
1234 rcs_type=RevisionControlSystems.SVN,
1235 svn_branch_url=svn_branch_url)1239 svn_branch_url=svn_branch_url)
1236 elif git_repo_url is not None:1240 elif git_repo_url is not None:
1241 assert rcs_type in (None, RevisionControlSystems.GIT)
1237 return code_import_set.new(1242 return code_import_set.new(
1238 registrant, product, branch_name,1243 registrant, product, branch_name,
1239 rcs_type=RevisionControlSystems.GIT,1244 rcs_type=RevisionControlSystems.GIT,
1240 git_repo_url=git_repo_url)1245 git_repo_url=git_repo_url)
1241 else:1246 else:
1247 assert rcs_type in (None, RevisionControlSystems.CVS)
1242 return code_import_set.new(1248 return code_import_set.new(
1243 registrant, product, branch_name,1249 registrant, product, branch_name,
1244 rcs_type=RevisionControlSystems.CVS,1250 rcs_type=RevisionControlSystems.CVS,