Launchpad itself

Merge lp:~mwhudson/launchpad/bzr-svn-monitor into lp:launchpad

  1. bzr-svn-monitor
  2. Merge into devel
Proposed by Michael Hudson-Doyle
Status: Merged
Approved by: Tim Penhey
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~mwhudson/launchpad/bzr-svn-monitor
Merge into: lp:launchpad
Diff against target: 428 lines (+153/-22)
8 files modified
lib/lp/code/doc/codeimport.txt (+65/-12)
lib/lp/code/mail/codeimport.py (+2/-1)
lib/lp/code/model/codeimport.py (+4/-1)
lib/lp/code/model/codeimportevent.py (+2/-1)
lib/lp/codehosting/codeimport/tests/servers.py (+40/-2)
lib/lp/codehosting/codeimport/tests/test_workermonitor.py (+26/-1)
lib/lp/codehosting/codeimport/worker.py (+5/-1)
lib/lp/testing/factory.py (+9/-3)
To merge this branch: bzr merge lp:~mwhudson/launchpad/bzr-svn-monitor
Reviewer Review Type Date Requested Status
Tim Penhey (community) Approve
Review via email: mp+15264@code.launchpad.net

Commit message

A test for bzr-svn at the worker-monitor level and enough to make it pass.

To post a comment you must log in.
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Hi,

This branch adds a test for bzr-svn at the worker-monitor level and makes it pass. It also tests some very basic stuff in codeimport.txt.

Cheers,
mwh

Revision history for this message
Tim Penhey (thumper) wrote :

As much as it hurts me to say it, but can you please ReSTify the codeimport doc test titles? The current feeling is that as you touch the old files they get converted.

Other than that, looks good.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/code/doc/codeimport.txt'
2--- lib/lp/code/doc/codeimport.txt 2009-10-03 17:10:28 +0000
3+++ lib/lp/code/doc/codeimport.txt 2009-12-07 03:01:12 +0000
4@@ -1,4 +1,5 @@
5-= Code Imports =
6+Code Imports
7+============
8
9 CodeImport objects model the process surrounding the code import
10 service of Launchpad. A CodeImport object is created by a user
11@@ -14,7 +15,8 @@
12 >>> login('david.allouche@canonical.com')
13
14
15-== Code import set utility ==
16+Code import set utility
17+-----------------------
18
19 CodeImports are created and found using the ICodeImportSet interface,
20 which is registered as a utility.
21@@ -34,7 +36,8 @@
22 >>> nopriv = getUtility(IPersonSet).getByName('no-priv')
23
24
25-== CodeImport events ==
26+CodeImport events
27+-----------------
28
29 Most mutating operations affecting code imports should create
30 CodeImportEvent objects in the database to provide an audit trail.
31@@ -43,7 +46,8 @@
32 >>> event_set = getUtility(ICodeImportEventSet)
33
34
35-== Supported source systems ==
36+Supported source systems
37+------------------------
38
39 The rcs_type field, which indicates whether the import is from CVS or
40 Subversion, takes values from the 'RevisionControlSystems' vocabulary.
41@@ -57,7 +61,8 @@
42 Git
43
44
45-=== Import from CVS ===
46+Import from CVS
47++++++++++++++++
48
49 Code imports from CVS specify the CVSROOT value, and the path to import
50 in the repository, known as the "module".
51@@ -113,7 +118,8 @@
52 True
53
54
55-=== Import from Subversion ===
56+Import from Subversion
57+++++++++++++++++++++++
58
59 Code imports from Subversion specify the URL used with "svn checkout" to
60 retrieve the tree to import.
61@@ -140,7 +146,33 @@
62 >>> svn_import == existing_import
63 True
64
65-=== Import from Git ===
66+
67+Import from Subversion via bzr-svn
68+++++++++++++++++++++++++++++++++++
69+
70+Code imports from Subversion can also specify that they should be
71+imported with 'bzr-svn' rather than cscvs. In most respects these
72+imports are similar to the Subversion via cscvs imports.
73+
74+ >>> bzr_svn = RevisionControlSystems.BZR_SVN
75+ >>> bzr_svn_url = 'svn://svn.example.com/for-bzr-svn/trunk'
76+ >>> bzr_svn_import = code_import_set.new(
77+ ... registrant=nopriv, product=product, branch_name='trunk-bzr-svn',
78+ ... rcs_type=bzr_svn, svn_branch_url=bzr_svn_url)
79+ >>> verifyObject(ICodeImport, removeSecurityProxy(svn_import))
80+ True
81+
82+The CodeImportSet.getBySVNDetails is also able to find bzr-svn
83+imports.
84+
85+ >>> existing_bzr_svn_import = code_import_set.getBySVNDetails(
86+ ... svn_branch_url=bzr_svn_url)
87+ >>> bzr_svn_import == existing_bzr_svn_import
88+ True
89+
90+
91+Import from Git
92++++++++++++++++
93
94 Code imports from Git specify the URL used with "git clone" to
95 retrieve the branch to import.
96@@ -167,7 +199,9 @@
97 >>> git_import == existing_import
98 True
99
100-== Updating code import details ==
101+
102+Updating code import details
103+----------------------------
104
105 Members of the VCS Imports team (import operators), or Launchpad
106 administrators can update the details of the code import, including
107@@ -267,7 +301,8 @@
108 ...
109
110
111-== Update intervals ==
112+Update intervals
113+----------------
114
115 After an import is initially completed, it must be updated regularly. Each
116 code import can specify a custom update interval, or use a default value.
117@@ -304,6 +339,8 @@
118 datetime.timedelta(0, 21600)
119 >>> svn_import.effective_update_interval
120 datetime.timedelta(0, 21600)
121+ >>> bzr_svn_import.effective_update_interval
122+ datetime.timedelta(0, 21600)
123
124 >>> default_interval_git
125 datetime.timedelta(0, 21600)
126@@ -325,7 +362,9 @@
127 >>> svn_import.effective_update_interval
128 datetime.timedelta(0, 3600)
129
130-== Retreiving CodeImports ==
131+
132+Retrieving CodeImports
133+----------------------
134
135 You can retrieve all imports with the `getAll` method of ICodeImport.
136
137@@ -361,7 +400,8 @@
138 NotFoundError: -10
139
140
141-== Canonical URLs ==
142+Canonical URLs
143+--------------
144
145 We've registered the ICodeImportSet utility on the 'code' part of the
146 site:
147@@ -380,7 +420,8 @@
148 NoCanonicalUrl: ...
149
150
151-== Modifying CodeImports ==
152+Modifying CodeImports
153+---------------------
154
155 Modifications to CodeImport objects must be done using setter methods
156 that create CodeImportEvent objects when appropriate. This is enforced
157@@ -449,6 +490,18 @@
158 instead of:
159 git://git.example.com/hello.git
160
161+Imports via bzr-svn are also similar.
162+
163+ >>> data = {'svn_branch_url': 'http://svn.example.com/for-bzr-svn/trunk'}
164+ >>> modify_event = bzr_svn_import.updateFromData(data, nopriv)
165+ >>> print make_email_body_for_code_import_update(
166+ ... bzr_svn_import, modify_event, None)
167+ ~no-priv/firefox/trunk-bzr-svn is now being imported from:
168+ http://svn.example.com/for-bzr-svn/trunk
169+ instead of:
170+ svn://svn.example.com/for-bzr-svn/trunk
171+
172+
173 In addition, updateFromData can be used to set the branch whiteboard,
174 which is also described in the email that is sent.
175
176
177=== modified file 'lib/lp/code/mail/codeimport.py'
178--- lib/lp/code/mail/codeimport.py 2009-10-13 14:00:24 +0000
179+++ lib/lp/code/mail/codeimport.py 2009-12-07 03:01:12 +0000
180@@ -95,7 +95,8 @@
181 body.append(
182 details_change_prefix + '\n' + new_details +
183 "\ninstead of:\n" + old_details)
184- elif code_import.rcs_type == RevisionControlSystems.SVN:
185+ elif code_import.rcs_type in (RevisionControlSystems.SVN,
186+ RevisionControlSystems.BZR_SVN):
187 if CodeImportEventDataType.OLD_SVN_BRANCH_URL in event_data:
188 old_url = event_data[CodeImportEventDataType.OLD_SVN_BRANCH_URL]
189 body.append(
190
191=== modified file 'lib/lp/code/model/codeimport.py'
192--- lib/lp/code/model/codeimport.py 2009-10-03 17:10:28 +0000
193+++ lib/lp/code/model/codeimport.py 2009-12-07 03:01:12 +0000
194@@ -105,6 +105,8 @@
195 config.codeimport.default_interval_cvs,
196 RevisionControlSystems.SVN:
197 config.codeimport.default_interval_subversion,
198+ RevisionControlSystems.BZR_SVN:
199+ config.codeimport.default_interval_subversion,
200 RevisionControlSystems.GIT:
201 config.codeimport.default_interval_git,
202 }
203@@ -218,7 +220,8 @@
204 assert cvs_root is not None and cvs_module is not None
205 assert svn_branch_url is None
206 assert git_repo_url is None
207- elif rcs_type == RevisionControlSystems.SVN:
208+ elif rcs_type in (RevisionControlSystems.SVN,
209+ RevisionControlSystems.BZR_SVN):
210 assert cvs_root is None and cvs_module is None
211 assert svn_branch_url is not None
212 assert git_repo_url is None
213
214=== modified file 'lib/lp/code/model/codeimportevent.py'
215--- lib/lp/code/model/codeimportevent.py 2009-07-17 00:26:05 +0000
216+++ lib/lp/code/model/codeimportevent.py 2009-12-07 03:01:12 +0000
217@@ -256,7 +256,8 @@
218
219 def _iterSourceDetails(self, code_import):
220 """Yield key-value tuples describing the source of the import."""
221- if code_import.rcs_type == RevisionControlSystems.SVN:
222+ if code_import.rcs_type in (RevisionControlSystems.SVN,
223+ RevisionControlSystems.BZR_SVN):
224 yield 'SVN_BRANCH_URL', code_import.svn_branch_url
225 elif code_import.rcs_type == RevisionControlSystems.CVS:
226 yield 'CVS_ROOT', code_import.cvs_root
227
228=== modified file 'lib/lp/codehosting/codeimport/tests/servers.py'
229--- lib/lp/codehosting/codeimport/tests/servers.py 2009-11-19 04:33:06 +0000
230+++ lib/lp/codehosting/codeimport/tests/servers.py 2009-12-07 03:01:12 +0000
231@@ -3,6 +3,8 @@
232
233 """Server classes that know how to create various kinds of foreign archive."""
234
235+from __future__ import with_statement
236+
237 __all__ = [
238 'CVSServer',
239 'GitServer',
240@@ -13,7 +15,10 @@
241
242 import os
243 import shutil
244+import signal
245+import subprocess
246 import tempfile
247+import time
248
249 import CVS
250 import pysvn
251@@ -60,9 +65,10 @@
252 class SubversionServer(Server):
253 """A controller for an Subversion repository, used for testing."""
254
255- def __init__(self, repository_path):
256+ def __init__(self, repository_path, use_svn_serve=False):
257 super(SubversionServer, self).__init__()
258 self.repository_path = os.path.abspath(repository_path)
259+ self._use_svn_serve = use_svn_serve
260
261 def createRepository(self, path):
262 """Create a Subversion repository at `path`."""
263@@ -70,11 +76,43 @@
264
265 def get_url(self):
266 """Return a URL to the Subversion repository."""
267- return local_path_to_url(self.repository_path)
268+ if self._use_svn_serve:
269+ return 'svn://localhost/'
270+ else:
271+ return local_path_to_url(self.repository_path)
272
273 def setUp(self):
274 super(SubversionServer, self).setUp()
275 self.createRepository(self.repository_path)
276+ if self._use_svn_serve:
277+ conf_path = os.path.join(
278+ self.repository_path, 'conf/svnserve.conf')
279+ with open(conf_path , 'w') as conf_file:
280+ conf_file.write('[general]\nanon-access = write\n')
281+ self._svnserve = subprocess.Popen(
282+ ['svnserve', '--daemon', '--foreground', '--root',
283+ self.repository_path])
284+ delay = 0.1
285+ for i in range(10):
286+ try:
287+ client = pysvn.Client()
288+ client.ls(self.get_url())
289+ except pysvn.ClientError, e:
290+ if 'Connection refused' in str(e):
291+ time.sleep(delay)
292+ delay *= 1.5
293+ continue
294+ else:
295+ break
296+ else:
297+ raise AssertionError(
298+ "svnserve didn't start accepting connections")
299+
300+ def tearDown(self):
301+ super(SubversionServer, self).tearDown()
302+ if self._use_svn_serve:
303+ os.kill(self._svnserve.pid, signal.SIGINT)
304+ self._svnserve.communicate()
305
306 @run_in_temporary_directory
307 def makeBranch(self, branch_name, tree_contents):
308
309=== modified file 'lib/lp/codehosting/codeimport/tests/test_workermonitor.py'
310--- lib/lp/codehosting/codeimport/tests/test_workermonitor.py 2009-11-25 07:00:06 +0000
311+++ lib/lp/codehosting/codeimport/tests/test_workermonitor.py 2009-12-07 03:01:12 +0000
312@@ -29,7 +29,8 @@
313 TwistedLayer, TwistedLaunchpadZopelessLayer)
314 from canonical.twistedsupport.tests.test_processmonitor import (
315 makeFailure, ProcessTestsMixin)
316-from lp.code.enums import CodeImportResultStatus, CodeImportReviewStatus
317+from lp.code.enums import (
318+ CodeImportResultStatus, CodeImportReviewStatus, RevisionControlSystems)
319 from lp.code.interfaces.codeimport import ICodeImportSet
320 from lp.code.interfaces.codeimportjob import (
321 ICodeImportJobSet, ICodeImportJobWorkflow)
322@@ -476,6 +477,20 @@
323 return self.factory.makeCodeImport(
324 svn_branch_url=svn_branch_url)
325
326+ def makeBzrSvnCodeImport(self):
327+ """Make a `CodeImport` that points to a real Subversion repository."""
328+ self.subversion_server = SubversionServer(
329+ self.repo_path, use_svn_serve=True)
330+ self.subversion_server.setUp()
331+ self.addCleanup(self.subversion_server.tearDown)
332+ svn_branch_url = self.subversion_server.makeBranch(
333+ 'trunk', [('README', 'contents')])
334+ self.foreign_commit_count = 2
335+
336+ return self.factory.makeCodeImport(
337+ svn_branch_url=svn_branch_url,
338+ rcs_type=RevisionControlSystems.BZR_SVN)
339+
340 def makeGitCodeImport(self):
341 """Make a `CodeImport` that points to a real Git repository."""
342 load_optional_plugin('git')
343@@ -579,6 +594,15 @@
344 result = self.performImport(job_id)
345 return result.addCallback(self.assertImported, code_import_id)
346
347+ def test_import_bzrsvn(self):
348+ # Create a Subversion-via-bzr-svn CodeImport and import it.
349+ job = self.getStartedJobForImport(self.makeBzrSvnCodeImport())
350+ code_import_id = job.code_import.id
351+ job_id = job.id
352+ self.layer.txn.commit()
353+ result = self.performImport(job_id)
354+ return result.addCallback(self.assertImported, code_import_id)
355+
356
357 class DeferredOnExit(protocol.ProcessProtocol):
358
359@@ -591,6 +615,7 @@
360 else:
361 self._deferred.errback(reason)
362
363+
364 class TestWorkerMonitorIntegrationScript(TestWorkerMonitorIntegration):
365 """Tests for CodeImportWorkerMonitor that execute a child process."""
366
367
368=== modified file 'lib/lp/codehosting/codeimport/worker.py'
369--- lib/lp/codehosting/codeimport/worker.py 2009-12-06 23:01:26 +0000
370+++ lib/lp/codehosting/codeimport/worker.py 2009-12-07 03:01:12 +0000
371@@ -152,6 +152,10 @@
372 rcstype = 'svn'
373 svn_branch_url = str(code_import.svn_branch_url)
374 cvs_root = cvs_module = git_repo_url = None
375+ elif code_import.rcs_type == RevisionControlSystems.BZR_SVN:
376+ rcstype = 'bzr-svn'
377+ svn_branch_url = str(code_import.svn_branch_url)
378+ cvs_root = cvs_module = git_repo_url = None
379 elif code_import.rcs_type == RevisionControlSystems.CVS:
380 rcstype = 'cvs'
381 svn_branch_url = git_repo_url = None
382@@ -162,7 +166,7 @@
383 svn_branch_url = cvs_root = cvs_module = None
384 git_repo_url = str(code_import.git_repo_url)
385 else:
386- raise AssertionError("Unknown rcstype %r." % rcstype)
387+ raise AssertionError("Unknown rcstype %r." % code_import.rcs_type)
388 return cls(
389 code_import.branch.id, rcstype, svn_branch_url,
390 cvs_root, cvs_module, git_repo_url)
391
392=== modified file 'lib/lp/testing/factory.py'
393--- lib/lp/testing/factory.py 2009-11-27 12:50:16 +0000
394+++ lib/lp/testing/factory.py 2009-12-07 03:01:12 +0000
395@@ -1210,7 +1210,7 @@
396
397 def makeCodeImport(self, svn_branch_url=None, cvs_root=None,
398 cvs_module=None, product=None, branch_name=None,
399- git_repo_url=None, registrant=None):
400+ git_repo_url=None, registrant=None, rcs_type=None):
401 """Create and return a new, arbitrary code import.
402
403 The type of code import will be inferred from the source details
404@@ -1229,16 +1229,22 @@
405
406 code_import_set = getUtility(ICodeImportSet)
407 if svn_branch_url is not None:
408+ if rcs_type is None:
409+ rcs_type = RevisionControlSystems.SVN
410+ else:
411+ assert rcs_type in (RevisionControlSystems.SVN,
412+ RevisionControlSystems.BZR_SVN)
413 return code_import_set.new(
414- registrant, product, branch_name,
415- rcs_type=RevisionControlSystems.SVN,
416+ registrant, product, branch_name, rcs_type=rcs_type,
417 svn_branch_url=svn_branch_url)
418 elif git_repo_url is not None:
419+ assert rcs_type in (None, RevisionControlSystems.GIT)
420 return code_import_set.new(
421 registrant, product, branch_name,
422 rcs_type=RevisionControlSystems.GIT,
423 git_repo_url=git_repo_url)
424 else:
425+ assert rcs_type in (None, RevisionControlSystems.CVS)
426 return code_import_set.new(
427 registrant, product, branch_name,
428 rcs_type=RevisionControlSystems.CVS,