Merge lp:~leonardr/launchpad/no-mutator-named-operations into lp:launchpad/db-devel

Proposed by Leonard Richardson
Status: Merged
Merge reported by: Leonard Richardson
Merged at revision: not available
Proposed branch: lp:~leonardr/launchpad/no-mutator-named-operations
Merge into: lp:launchpad/db-devel
Diff against target: 908 lines (+239/-199)
19 files modified
.bzrignore (+1/-1)
cronscripts/expire-archive-files.py (+3/-3)
lib/canonical/launchpad/doc/timeout.txt (+1/-1)
lib/canonical/launchpad/pagetests/basics/demo-and-lpnet.txt (+7/-6)
lib/canonical/launchpad/pagetests/webservice/multiversion.txt (+32/-0)
lib/canonical/launchpad/rest/configuration.py (+1/-0)
lib/canonical/launchpad/templates/launchpad-requestexpired.pt (+9/-20)
lib/canonical/launchpad/templates/oops.pt (+24/-0)
lib/canonical/launchpad/webapp/error.py (+3/-3)
lib/canonical/launchpad/webapp/tests/test_servers.py (+1/-1)
lib/lp/app/stories/basics/xx-beta-testers-redirection.txt (+7/-7)
lib/lp/app/templates/base-layout-macros.pt (+8/-10)
lib/lp/bugs/stories/webservice/xx-bug.txt (+9/-40)
lib/lp/registry/browser/productseries.py (+1/-1)
lib/lp/registry/browser/tests/productseries-views.txt (+5/-0)
lib/lp/registry/templates/productseries-codesummary.pt (+1/-1)
lib/lp/soyuz/scripts/expire_archive_files.py (+21/-15)
lib/lp/soyuz/scripts/tests/test_expire_archive_files.py (+103/-88)
versions.cfg (+2/-2)
To merge this branch: bzr merge lp:~leonardr/launchpad/no-mutator-named-operations
Reviewer Review Type Date Requested Status
Guilherme Salgado (community) code Approve
Review via email: mp+20777@code.launchpad.net

Description of the change

This branch integrates the latest versions of lazr.restful and launchpadlib into Launchpad. launchpadlib didn't need any changes; it's just a more useful version of the software. To integrate lazr.restful I had to make some changes.

Specifically, this branch introduces the first real difference between the 'beta' and '1.0' versions of the web service. As of '1.0', named operations that duplicate the functionality of fields (such as IBugTask.transitionToStatus) are not published on the web service. In 'beta', those operations are still available. This is controlled by the configuration variable last_version_with_mutator_named_operations.

I changed the xx-bug.txt test to stop testing for the presence of the named operation, even though that test is still using 'beta' and so the named operation is still available. I plan to do a test soon that changes the default version used by the tests to 'devel', and I'm going to have to search-and-replace a whole lot of URLs in tests to make the tests work again. I don't want to also have to make a more complex change to the web service tests.

I made a minor fix to the .bzrignore file so that it would ignore all versions of the generated WADL. (There used to be only one; I recently changed the Makefile to generate many such files, but I didn't think to change the .bzrignore then.)

Finally, I changed test_servers.py to import WebServiceTestCase and other helpful classes from lazr.restful.testing.webservice. In the old version of lazr.restful those classes were buried in lazr.restful's web service unit test module; in the new version, I moved them to a place more appropriate to import from.

To post a comment you must log in.
Revision history for this message
Guilherme Salgado (salgado) wrote :

This looks good, although the diff currently shows the changes from one of my branches that recently landed on devel. As discussed on IRC, this is because you had branched off of db-devel and then merged devel after I landed my changes.

Also, one thing that concerns me is that we're losing test coverage for things that don't exist in the latest version of the api (e.g. transitionToStatus). Is the plan really to test just whatever is supported by the latest version or am I missing something?

review: Approve (code)
Revision history for this message
Leonard Richardson (leonardr) wrote :

This is a little different from other version-specific changes because it's one change that affects _every_ mutator named operation, not a bunch of decisions made on a case-by-case basis. So I think it's sufficient to show that, in general, named operations are available in beta but not in 1.0, which is what the transitionToImportance test does.

In general, the plan is to pick a few applications written against beta, and make sure they continue to work against beta as we change 1.0, rather than write a test for every change. But even by the standards of "write a test for every change", I think in this case a single representative test is good enough.

9076. By Leonard Richardson

Merge with trunk.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2010-03-09 02:42:40 +0000
3+++ .bzrignore 2010-03-11 21:49:26 +0000
4@@ -32,7 +32,7 @@
5 database/sampledata/lintdata-dev.sql
6 lib/canonical/launchpad/apidoc/index.html
7 xxx-report.*
8-lib/canonical/launchpad/apidoc/wadl-development.xml
9+lib/canonical/launchpad/apidoc/wadl-development-*.xml
10 lib/canonical/launchpad/apidoc/wadl-test-playground.xml
11 lib/canonical/launchpad/icing/build/*
12 lib/canonical/launchpad/icing/combo.css
13
14=== renamed file 'cronscripts/expire-ppa-files.py' => 'cronscripts/expire-archive-files.py'
15--- cronscripts/expire-ppa-files.py 2010-01-20 10:32:32 +0000
16+++ cronscripts/expire-archive-files.py 2010-03-11 21:49:26 +0000
17@@ -12,11 +12,11 @@
18 import _pythonpath
19
20 from canonical.config import config
21-from lp.soyuz.scripts.expire_ppa_binaries import PPABinaryExpirer
22+from lp.soyuz.scripts.expire_archive_files import ArchiveExpirer
23
24
25 if __name__ == '__main__':
26- script = PPABinaryExpirer(
27- 'expire-ppa-files', dbuser=config.binaryfile_expire.dbuser)
28+ script = ArchiveExpirer(
29+ 'expire-archive-files', dbuser=config.binaryfile_expire.dbuser)
30 script.lock_and_run()
31
32
33=== modified file 'lib/canonical/launchpad/doc/timeout.txt'
34--- lib/canonical/launchpad/doc/timeout.txt 2008-05-17 02:36:00 +0000
35+++ lib/canonical/launchpad/doc/timeout.txt 2010-03-11 21:49:26 +0000
36@@ -12,7 +12,7 @@
37 >>> from canonical.lazr.timeout import get_default_timeout_function
38 >>> from canonical.launchpad.webapp.servers import (
39 ... set_launchpad_default_timeout)
40- >>> old_func = get_default_timeout_function()
41+ >>> old_func = get_default_timeout_function()
42
43 >>> from zope.app.appsetup import ProcessStarting
44
45
46=== modified file 'lib/canonical/launchpad/pagetests/basics/demo-and-lpnet.txt'
47--- lib/canonical/launchpad/pagetests/basics/demo-and-lpnet.txt 2010-02-09 17:25:13 +0000
48+++ lib/canonical/launchpad/pagetests/basics/demo-and-lpnet.txt 2010-03-11 21:49:26 +0000
49@@ -103,16 +103,18 @@
50 ... 'a', onclick="setBetaRedirect(false)"))
51 Disable edge redirect.
52
53-The disable-redirect link will not appear in the site_message when
54-browsed by non-beta users.
55+The disable-redirect link will also appear in the site_message when browsed by
56+non-beta/anonymous users. This is to reduce the annoyance when users are
57+logged into launchpad.net but haven't noticed yet that they need to log into
58+edge as well (https://launchpad.net/bugs/160191).
59
60 >>> browser.open('http://launchpad.dev/ubuntu')
61 >>> print extract_text(find_tags_by_class(
62 ... browser.contents, 'sitemessage')[0])
63- This is a beta site.
64+ This is a beta site. Disable edge redirect.
65
66-Similarly, once the redirection has been inhibited, the link changes to
67-enable redirects..
68+Once the redirection has been inhibited, the link changes to enable
69+redirects.
70
71 # Workaround bug in mechanize where you cannot use the Cookie
72 # header with the CookieJar
73@@ -135,7 +137,6 @@
74 ... 'a', onclick="setBetaRedirect(true)"))
75 Enable edge redirect.
76
77-
78 # Remove the specific site-message config data before continuing.
79 >>> dummy = config.pop('edge_config_data')
80
81
82=== added file 'lib/canonical/launchpad/pagetests/webservice/multiversion.txt'
83--- lib/canonical/launchpad/pagetests/webservice/multiversion.txt 1970-01-01 00:00:00 +0000
84+++ lib/canonical/launchpad/pagetests/webservice/multiversion.txt 2010-03-11 21:49:26 +0000
85@@ -0,0 +1,32 @@
86+Differences between versions
87+----------------------------
88+
89+In the 'beta' version of the web service, mutator methods like
90+IBugTask.transitionToStatus are published as named operations. In
91+subsequent versions, those named operations are not published.
92+
93+ >>> def get_bugtask_path(version):
94+ ... bug_one = webservice.get("/bugs/1", api_version=version).jsonBody()
95+ ... bug_one_bugtasks_url = bug_one['bug_tasks_collection_link']
96+ ... bug_one_bugtasks = sorted(webservice.get(
97+ ... bug_one_bugtasks_url).jsonBody()['entries'])
98+ ... bugtask_path = bug_one_bugtasks[0]['self_link']
99+ ... return bugtask_path
100+
101+Here's the 'beta' version, where the named operation works.
102+
103+ >>> url = get_bugtask_path('beta')
104+ >>> print webservice.named_post(
105+ ... url, 'transitionToImportance', importance='Low',
106+ ... api_version='beta')
107+ HTTP/1.1 200 Ok
108+ ...
109+
110+Now let's try the same thing in the '1.0' version, where it fails.
111+
112+ >>> url = get_bugtask_path('1.0')
113+ >>> print webservice.named_post(
114+ ... url, 'transitionToImportance', importance='Low',
115+ ... api_version='devel')
116+ HTTP/1.1 405 Method Not Allowed
117+ ...
118
119=== modified file 'lib/canonical/launchpad/rest/configuration.py'
120--- lib/canonical/launchpad/rest/configuration.py 2010-02-15 02:09:32 +0000
121+++ lib/canonical/launchpad/rest/configuration.py 2010-03-11 21:49:26 +0000
122@@ -24,6 +24,7 @@
123
124 path_override = "api"
125 active_versions = ["beta", "1.0", "devel"]
126+ last_version_with_mutator_named_operations = "beta"
127 view_permission = "launchpad.View"
128 set_hop_by_hop_headers = True
129
130
131=== modified file 'lib/canonical/launchpad/templates/launchpad-requestexpired.pt'
132--- lib/canonical/launchpad/templates/launchpad-requestexpired.pt 2009-07-28 17:19:42 +0000
133+++ lib/canonical/launchpad/templates/launchpad-requestexpired.pt 2010-03-11 21:49:26 +0000
134@@ -9,27 +9,16 @@
135 <body>
136 <div class="top-portlet" metal:fill-slot="main">
137 <h1 class="exception">Timeout error</h1>
138- <div
139- id="redirect_notice" style="display:none" class="informational message">
140- <p>Our edge server has a lower timeout threshold than launchpad.net,
141- so we can catch those before they hit a wider audience.
142- As a member of the Launchpad Beta Testers team, you're more
143- likely to experience them. If this is blocking your work, you
144- can disable redirection.</p>
145- <p><button onclick="setBetaRedirect(false)">Disable redirection
146- for 2 hours</button></p>
147+ <div tal:condition="is_edge"
148+ id="redirect_notice" class="informational message">
149+ <p>Our edge server has a lower timeout threshold than launchpad.net,
150+ so we can catch those before they hit a wider audience.
151+ If this is blocking your work and you are a member of the Launchpad
152+ Beta Testers team, you can disable automatic redirection
153+ to edge in order to use launchpad.net.</p>
154+ <p><button onclick="setBetaRedirect(false)">Disable redirection
155+ for 2 hours</button></p>
156 </div>
157- <script type="text/javascript">
158- var edge_cookie = document.cookie.match('edge');
159- var edge_host = document.location.hostname.match('edge');
160-
161- // Logic has been inverted in the next line to avoid breaking
162- // XHTML compliance of the template due to ampersand usage.
163- if (!(edge_cookie || edge_host)) {
164- redirect_notice_div = document.getElementById('redirect_notice');
165- redirect_notice_div.style.display = "block";
166- };
167- </script>
168 <p>
169 Sorry, something just went wrong in Launchpad.
170 </p>
171
172=== modified file 'lib/canonical/launchpad/templates/oops.pt'
173--- lib/canonical/launchpad/templates/oops.pt 2010-01-13 17:58:30 +0000
174+++ lib/canonical/launchpad/templates/oops.pt 2010-03-11 21:49:26 +0000
175@@ -9,6 +9,9 @@
176 <style type="text/css" media="screen, print">
177 @import url(/+icing/rev5/combo.css);
178 </style>
179+ <script type="text/javascript"
180+ tal:content="string:var cookie_scope = '${request/lp:cookie_scope}';"></script>
181+ <script type="text/javascript" src="/+icing/rev5/build/lp/lp.js" />
182 </head>
183 <body>
184 <div class="yui-d0">
185@@ -27,6 +30,27 @@
186 (Error <abbr>ID</abbr>:
187 <tal:oops replace="structure view/oops_id_text" />)
188 </p>
189+
190+ <div id="redirect_notice" style="display:none"
191+ class="informational message">
192+ <p>This server runs pre-release code, so it's possible this problem
193+ doesn't affect launchpad.net. If you're a member of the Launchpad
194+ Beta Testers team, you can disable redirection in order to use
195+ launchpad.net.</p>
196+ <p><button onclick="setBetaRedirect(false)">Disable redirection
197+ for 2 hours</button></p>
198+ </div>
199+ <tal:comment condition="nothing">
200+ Can't use the 'is_edge' global because we don't use our page
201+ macros here.
202+ </tal:comment>
203+ <script type="text/javascript">
204+ if (document.location.hostname.match('edge.')) {
205+ redirect_notice_div = document.getElementById('redirect_notice');
206+ redirect_notice_div.style.display = "block";
207+ };
208+ </script>
209+
210 <tal:traceback replace="structure view/maybeShowTraceback" />
211
212 <div class="related">
213
214=== modified file 'lib/canonical/launchpad/webapp/error.py'
215--- lib/canonical/launchpad/webapp/error.py 2009-12-23 16:17:12 +0000
216+++ lib/canonical/launchpad/webapp/error.py 2010-03-11 21:49:26 +0000
217@@ -26,12 +26,13 @@
218 from canonical.cachedproperty import cachedproperty
219 from canonical.config import config
220 import canonical.launchpad.layers
221+from canonical.launchpad.webapp.publisher import LaunchpadView
222 from canonical.launchpad.webapp.adapter import (
223 clear_request_started, set_request_started)
224 from canonical.launchpad.webapp.interfaces import ILaunchBag
225
226
227-class SystemErrorView:
228+class SystemErrorView(LaunchpadView):
229 """Helper class for views on exceptions.
230
231 Also, sets a 500 response code.
232@@ -64,8 +65,7 @@
233 safe_to_show_in_restricted_mode = False
234
235 def __init__(self, context, request):
236- self.context = context
237- self.request = request
238+ super(SystemErrorView, self).__init__(context, request)
239 self.request.response.removeAllNotifications()
240 if self.response_code is not None:
241 self.request.response.setStatus(self.response_code)
242
243=== modified file 'lib/canonical/launchpad/webapp/tests/test_servers.py'
244--- lib/canonical/launchpad/webapp/tests/test_servers.py 2010-02-24 14:45:33 +0000
245+++ lib/canonical/launchpad/webapp/tests/test_servers.py 2010-03-11 21:49:26 +0000
246@@ -14,7 +14,7 @@
247 from lazr.restful.interfaces import (
248 IServiceRootResource, IWebServiceConfiguration)
249 from lazr.restful.simple import RootResource
250-from lazr.restful.tests.test_webservice import (
251+from lazr.restful.testing.webservice import (
252 IGenericCollection, IGenericEntry, WebServiceTestCase)
253
254 from lp.testing import TestCase
255
256=== modified file 'lib/lp/app/stories/basics/xx-beta-testers-redirection.txt'
257--- lib/lp/app/stories/basics/xx-beta-testers-redirection.txt 2010-02-09 17:25:13 +0000
258+++ lib/lp/app/stories/basics/xx-beta-testers-redirection.txt 2010-03-11 21:49:26 +0000
259@@ -216,11 +216,12 @@
260
261 >>> logout()
262
263-Decrease the timeout values for launchpad.dev
264+Decrease the timeout values for launchpad.dev and pretend we're on the edge
265+server.
266
267 >>> beta_data = """
268 ... [launchpad]
269- ... beta_testers_redirection_host = edge.launchpad.dev
270+ ... is_edge: True
271 ... [database]
272 ... db_statement_timeout: 1
273 ... soft_request_timeout: 2
274@@ -240,15 +241,14 @@
275 <title>Error: Timeout</title>
276 ...
277 <h1 class="exception">Timeout error</h1>
278- <div id="redirect_notice" style="display:none" class="informational message">
279+ ...
280 <p>Our edge server has a lower timeout threshold than launchpad.net,
281 so we can catch those before they hit a wider audience.
282- As a member of the Launchpad Beta Testers team, you're more
283- likely to experience them. If this is blocking your work, you
284- can disable redirection.</p>
285+ If this is blocking your work and you are a member of the Launchpad
286+ Beta Testers team, you can disable automatic redirection
287+ to edge in order to use launchpad.net.</p>
288 <p><button onclick="setBetaRedirect(false)">Disable redirection
289 for 2 hours</button></p>
290- </div>
291 ...
292
293 Restore the config to state it was in at the start of the test.
294
295=== modified file 'lib/lp/app/templates/base-layout-macros.pt'
296--- lib/lp/app/templates/base-layout-macros.pt 2010-03-05 02:57:25 +0000
297+++ lib/lp/app/templates/base-layout-macros.pt 2010-03-11 21:49:26 +0000
298@@ -379,16 +379,14 @@
299 This site is running pre-release code.
300 </tal:site_message>
301 <tal:edge_only condition="is_edge">
302- <tal:beta_user condition="view/macro:isbetauser">
303- <a href="#" class="js-action" onclick="setBetaRedirect(false)"
304- tal:condition="not:view/isRedirectInhibited">
305- Disable edge redirect.
306- </a>
307- <a href="#" class="js-action" onclick="setBetaRedirect(true)"
308- tal:condition="view/isRedirectInhibited">
309- Enable edge redirect.
310- </a>
311- </tal:beta_user>
312+ <a href="#" class="js-action" onclick="setBetaRedirect(false)"
313+ tal:condition="not:view/isRedirectInhibited">
314+ Disable edge redirect.
315+ </a>
316+ <a href="#" class="js-action" onclick="setBetaRedirect(true)"
317+ tal:condition="view/isRedirectInhibited">
318+ Enable edge redirect.
319+ </a>
320 </tal:edge_only>
321 </div>
322 </metal:site-message>
323
324=== modified file 'lib/lp/bugs/stories/webservice/xx-bug.txt'
325--- lib/lp/bugs/stories/webservice/xx-bug.txt 2010-03-10 14:14:38 +0000
326+++ lib/lp/bugs/stories/webservice/xx-bug.txt 2010-03-11 21:49:26 +0000
327@@ -357,7 +357,7 @@
328 http://.../~cprov
329
330
331-The task's importance can be modified directly...
332+The task's importance can be modified directly.
333
334 >>> body = webservice.get(bugtask_path).jsonBody()
335 >>> body['importance']
336@@ -371,20 +371,6 @@
337 >>> body['importance']
338 u'High'
339
340-...or through a named operation.
341-
342- >>> print webservice.named_post(
343- ... bugtask_path, 'transitionToImportance', importance='Low')
344- HTTP/1.1 200 Ok
345- ...
346- Content-Type: application/json
347- ...
348- <BLANKLINE>
349- null
350- >>> body = webservice.get(bugtask_path).jsonBody()
351- >>> body['importance']
352- u'Low'
353-
354 Only bug supervisors or people who can otherwise edit the bugtask's
355 pillar are authorised to edit the importance.
356
357@@ -394,10 +380,9 @@
358
359 >>> body = webservice.get(bugtask_path).jsonBody()
360 >>> body['importance']
361- u'Low'
362+ u'High'
363
364-The task's status is another aspect of the task that can be modified
365-either directly or through a named operation.
366+The task's status can also be modified directly.
367
368 >>> print webservice.get(bugtask_path).jsonBody()['status']
369 Confirmed
370@@ -409,21 +394,14 @@
371 >>> print webservice.get(bugtask_path).jsonBody()['status']
372 Fix Committed
373
374- >>> print webservice.named_post(
375- ... bugtask_path, 'transitionToStatus', status='New')
376- HTTP/1.1 200 Ok...
377-
378- >>> print webservice.get(bugtask_path).jsonBody()['status']
379- New
380-
381 If an error occurs during a request that sets both 'status' and
382 'importance', neither one will be set.
383
384 >>> task = webservice.get(bugtask_path).jsonBody()
385 >>> print task['status']
386- New
387+ Fix Committed
388 >>> print task['importance']
389- Low
390+ High
391
392 >>> patch = {u'importance': 'High', u'status': u'No Such Status'}
393 >>> print webservice.patch(bugtask_path, 'application/json', dumps(patch))
394@@ -431,12 +409,11 @@
395
396 >>> task = webservice.get(bugtask_path).jsonBody()
397 >>> print task['status']
398- New
399+ Fix Committed
400 >>> print task['importance']
401- Low
402+ High
403
404-Like the importance, the milestone can be set either directly or via a
405-named operation, and only by appropriately privileged users.
406+The milestone can only be set by appropriately privileged users.
407
408 >>> print webservice.get(bugtask_path).jsonBody()['milestone_link']
409 None
410@@ -449,20 +426,12 @@
411 >>> print webservice.get(bugtask_path).jsonBody()['milestone_link']
412 http://.../debian/+milestone/3.1
413
414- >>> print webservice.named_post(
415- ... bugtask_path, 'transitionToMilestone',
416- ... milestone=webservice.getAbsoluteUrl('/debian/+milestone/3.1-rc1'))
417- HTTP/1.1 200 Ok...
418-
419- >>> print webservice.get(bugtask_path).jsonBody()['milestone_link']
420- http://.../debian/+milestone/3.1-rc1
421-
422 >>> print user_webservice.patch(
423 ... bugtask_path, 'application/json', dumps(patch))
424 HTTP/1.1 401 Unauthorized...
425
426 >>> print webservice.get(bugtask_path).jsonBody()['milestone_link']
427- http://.../debian/+milestone/3.1-rc1
428+ http://.../debian/+milestone/3.1
429
430 We can change the task's target. Here we change the task's target from
431 the mozilla-firefox package to alsa-utils.
432
433=== modified file 'lib/lp/registry/browser/productseries.py'
434--- lib/lp/registry/browser/productseries.py 2010-02-08 20:11:17 +0000
435+++ lib/lp/registry/browser/productseries.py 2010-03-11 21:49:26 +0000
436@@ -309,7 +309,7 @@
437 @property
438 def request_import_link(self):
439 """A link to the page for requesting a new code import."""
440- return canonical_url(getUtility(ICodeImportSet), view_name='+new')
441+ return canonical_url(self.context.product, view_name='+new-import')
442
443 @property
444 def user_branch_visible(self):
445
446=== modified file 'lib/lp/registry/browser/tests/productseries-views.txt'
447--- lib/lp/registry/browser/tests/productseries-views.txt 2010-02-09 05:13:05 +0000
448+++ lib/lp/registry/browser/tests/productseries-views.txt 2010-03-11 21:49:26 +0000
449@@ -107,6 +107,11 @@
450 >>> print view.latest_release_with_download_files.version
451 0.9.2
452
453+The view also provides a link to register a new code import.
454+
455+ >>> print view.request_import_link
456+ http://launchpad.dev/firefox/+new-import
457+
458
459 Edit ProductSeries
460 ------------------
461
462=== modified file 'lib/lp/registry/templates/productseries-codesummary.pt'
463--- lib/lp/registry/templates/productseries-codesummary.pt 2009-11-13 20:14:41 +0000
464+++ lib/lp/registry/templates/productseries-codesummary.pt 2010-03-11 21:49:26 +0000
465@@ -54,7 +54,7 @@
466 </li>
467
468 <li>
469- If the code is in git, CVS or Subversion you can
470+ If the code is in Git, Mercurial, CVS or Subversion you can
471 <a tal:attributes="href view/request_import_link">request that the branch be imported to Bazaar</a>.
472 </li>
473 </ul>
474
475=== renamed file 'lib/lp/soyuz/scripts/expire_ppa_binaries.py' => 'lib/lp/soyuz/scripts/expire_archive_files.py'
476--- lib/lp/soyuz/scripts/expire_ppa_binaries.py 2010-01-31 19:36:27 +0000
477+++ lib/lp/soyuz/scripts/expire_archive_files.py 2010-03-11 21:49:26 +0000
478@@ -34,7 +34,7 @@
479 """.split()
480
481
482-class PPABinaryExpirer(LaunchpadCronScript):
483+class ArchiveExpirer(LaunchpadCronScript):
484 """Helper class for expiring old PPA binaries.
485
486 Any PPA binary older than 30 days that is superseded or deleted
487@@ -60,6 +60,7 @@
488 from lp.soyuz.interfaces.archive import ArchivePurpose
489
490 stay_of_execution = '%d days' % num_days
491+ archive_types = (ArchivePurpose.PPA, ArchivePurpose.PARTNER)
492
493 # The subquery here has to repeat the checks for privacy and
494 # blacklisting on *other* publications that are also done in
495@@ -79,7 +80,7 @@
496 AND spph.dateremoved < (
497 CURRENT_TIMESTAMP AT TIME ZONE 'UTC' - interval %s)
498 AND spph.archive = archive.id
499- AND archive.purpose = %s
500+ AND archive.purpose IN %s
501 AND lfa.expires IS NULL
502 EXCEPT
503 SELECT sprf.libraryfile
504@@ -95,15 +96,15 @@
505 AND spph.archive = a.id
506 AND p.id = a.owner
507 AND (
508- p.name IN %s
509+ (p.name IN %s AND a.purpose = %s)
510 OR a.private IS TRUE
511- OR a.purpose != %s
512+ OR a.purpose NOT IN %s
513 OR dateremoved >
514 CURRENT_TIMESTAMP AT TIME ZONE 'UTC' - interval %s
515 OR dateremoved IS NULL);
516 """ % sqlvalues(
517- stay_of_execution, ArchivePurpose.PPA, self.blacklist,
518- ArchivePurpose.PPA, stay_of_execution))
519+ stay_of_execution, archive_types, self.blacklist,
520+ ArchivePurpose.PPA, archive_types, stay_of_execution))
521
522 lfa_ids = results.get_all()
523 return lfa_ids
524@@ -114,6 +115,7 @@
525 from lp.soyuz.interfaces.archive import ArchivePurpose
526
527 stay_of_execution = '%d days' % num_days
528+ archive_types = (ArchivePurpose.PPA, ArchivePurpose.PARTNER)
529
530 # The subquery here has to repeat the checks for privacy and
531 # blacklisting on *other* publications that are also done in
532@@ -131,9 +133,10 @@
533 AND bpr.id = bpf.binarypackagerelease
534 AND bpph.binarypackagerelease = bpr.id
535 AND bpph.dateremoved < (
536- CURRENT_TIMESTAMP AT TIME ZONE 'UTC' - interval %s)
537+ CURRENT_TIMESTAMP AT TIME ZONE 'UTC' -
538+ interval %(stay_of_execution)s)
539 AND bpph.archive = archive.id
540- AND archive.purpose = %s
541+ AND archive.purpose IN %(archive_types)s
542 AND lfa.expires IS NULL
543 EXCEPT
544 SELECT bpf.libraryfile
545@@ -149,15 +152,18 @@
546 AND bpph.archive = a.id
547 AND p.id = a.owner
548 AND (
549- p.name IN %s
550+ (p.name IN %(blacklist)s AND a.purpose = %(ppa)s)
551 OR a.private IS TRUE
552- OR a.purpose != %s
553- OR dateremoved >
554- CURRENT_TIMESTAMP AT TIME ZONE 'UTC' - interval %s
555- OR dateremoved IS NULL);
556+ OR a.purpose NOT IN %(archive_types)s
557+ OR dateremoved > (
558+ CURRENT_TIMESTAMP AT TIME ZONE 'UTC' -
559+ interval %(stay_of_execution)s)
560+ OR dateremoved IS NULL)
561 """ % sqlvalues(
562- stay_of_execution, ArchivePurpose.PPA, self.blacklist,
563- ArchivePurpose.PPA, stay_of_execution))
564+ stay_of_execution=stay_of_execution,
565+ archive_types=archive_types,
566+ blacklist=self.blacklist,
567+ ppa=ArchivePurpose.PPA))
568
569 lfa_ids = results.get_all()
570 return lfa_ids
571
572=== renamed file 'lib/lp/soyuz/scripts/tests/test_expire_ppa_bins.py' => 'lib/lp/soyuz/scripts/tests/test_expire_archive_files.py'
573--- lib/lp/soyuz/scripts/tests/test_expire_ppa_bins.py 2010-02-27 20:20:03 +0000
574+++ lib/lp/soyuz/scripts/tests/test_expire_archive_files.py 2010-03-11 21:49:26 +0000
575@@ -1,12 +1,10 @@
576 # Copyright 2009 Canonical Ltd. This software is licensed under the
577 # GNU Affero General Public License version 3 (see the file LICENSE).
578
579-"""Test the expire-ppa-binaries.py script. """
580+"""Test the expire-archive-files.py script. """
581
582+from datetime import datetime, timedelta
583 import pytz
584-
585-from datetime import datetime, timedelta
586-
587 import unittest
588
589 from zope.component import getUtility
590@@ -15,35 +13,20 @@
591 from canonical.launchpad.scripts import QuietFakeLogger
592 from canonical.testing.layers import LaunchpadZopelessLayer
593 from lp.registry.interfaces.distribution import IDistributionSet
594-from lp.registry.interfaces.person import IPersonSet
595-from lp.soyuz.scripts.expire_ppa_binaries import PPABinaryExpirer
596+from lp.soyuz.interfaces.archive import ArchivePurpose
597+from lp.soyuz.scripts.expire_archive_files import ArchiveExpirer
598 from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
599 from lp.testing import TestCaseWithFactory
600
601
602-class TestPPABinaryExpiry(TestCaseWithFactory):
603- """Test the expire-ppa-binaries.py script."""
604-
605+class ArchiveExpiryTestBase(TestCaseWithFactory):
606+ """base class for the expire-archive-files.py script tests."""
607 layer = LaunchpadZopelessLayer
608 dbuser = config.binaryfile_expire.dbuser
609
610- # We need to test several cases are handled properly:
611- # - publications with no "dateremoved" are not expired
612- # - publications with dateremoved <= 30 days ago are not expired
613- # - publications with dateremoved > 30 days ago are expired
614- # - publications with dateremoved > 30 days ago but refer to a
615- # binary published elsewhere with no dateremoved are not
616- # expired
617- # - publications with dateremoved > 30 days ago but refer to a
618- # binary published elsewhere with dateremoved <= 30 days ago
619- # are not expired
620- # - publications with dateremoved > 30 days ago but refer to a
621- # binary published elsewhere with dateremoved > 30 days ago
622- # are expired.
623-
624 def setUp(self):
625 """Set up some test publications."""
626- super(TestPPABinaryExpiry, self).setUp()
627+ super(ArchiveExpiryTestBase, self).setUp()
628 # Configure the test publisher.
629 self.layer.switchDbUser("launchpad")
630 self.stp = SoyuzTestPublisher()
631@@ -54,16 +37,12 @@
632 self.under_threshold_date = self.now - timedelta(days=29)
633 self.over_threshold_date = self.now - timedelta(days=31)
634
635- # Prepare two PPAs for the tests to use.
636- self.ppa = self.factory.makeArchive()
637- self.ppa2 = self.factory.makeArchive()
638-
639 def getScript(self, test_args=None):
640- """Return a PPABinaryExpirer instance."""
641+ """Return a ArchiveExpirer instance."""
642 if test_args is None:
643 test_args = []
644 test_args.extend(['--expire-after', '30'])
645- script = PPABinaryExpirer("test expirer", test_args=test_args)
646+ script = ArchiveExpirer("test expirer", test_args=test_args)
647 script.logger = QuietFakeLogger()
648 script.txn = self.layer.txn
649 return script
650@@ -75,6 +54,24 @@
651 self.layer.switchDbUser(self.dbuser)
652 script.main()
653
654+ def _setUpExpirablePublications(self, archive=None):
655+ """Helper to set up two publications that are both expirable."""
656+ if archive is None:
657+ archive = self.archive
658+ pkg5 = self.stp.getPubSource(
659+ sourcename="pkg5", architecturehintlist="i386", archive=archive,
660+ dateremoved=self.over_threshold_date)
661+ other_source = pkg5.copyTo(
662+ pkg5.distroseries, pkg5.pocket, self.archive2)
663+ other_source.dateremoved = self.over_threshold_date
664+ [pub] = self.stp.getPubBinaries(
665+ pub_source=pkg5, dateremoved=self.over_threshold_date,
666+ archive=archive)
667+ [other_binary] = pub.copyTo(
668+ pub.distroarchseries.distroseries, pub.pocket, self.archive2)
669+ other_binary.dateremoved = self.over_threshold_date
670+ return pkg5, pub
671+
672 def assertBinaryExpired(self, publication):
673 self.assertNotEqual(
674 publication.binarypackagerelease.files[0].libraryfile.expires,
675@@ -99,13 +96,19 @@
676 None,
677 "lfa.expires should be None, but it's not.")
678
679+
680+class ArchiveExpiryCommonTests(object):
681+ """Common source/binary expiration test cases.
682+
683+ These will be shared irrespective of archive type (ppa/partner).
684+ """
685 def testNoExpirationWithNoDateremoved(self):
686 """Test that no expiring happens if no dateremoved set."""
687 pkg1 = self.stp.getPubSource(
688- sourcename="pkg1", architecturehintlist="i386", archive=self.ppa,
689- dateremoved=None)
690+ sourcename="pkg1", architecturehintlist="i386",
691+ archive=self.archive, dateremoved=None)
692 [pub] = self.stp.getPubBinaries(
693- pub_source=pkg1, dateremoved=None, archive=self.ppa)
694+ pub_source=pkg1, dateremoved=None, archive=self.archive)
695
696 self.runScript()
697 self.assertSourceNotExpired(pkg1)
698@@ -114,11 +117,11 @@
699 def testNoExpirationWithDateUnderThreshold(self):
700 """Test no expiring if dateremoved too recent."""
701 pkg2 = self.stp.getPubSource(
702- sourcename="pkg2", architecturehintlist="i386", archive=self.ppa,
703- dateremoved=self.under_threshold_date)
704+ sourcename="pkg2", architecturehintlist="i386",
705+ archive=self.archive, dateremoved=self.under_threshold_date)
706 [pub] = self.stp.getPubBinaries(
707 pub_source=pkg2, dateremoved=self.under_threshold_date,
708- archive=self.ppa)
709+ archive=self.archive)
710
711 self.runScript()
712 self.assertSourceNotExpired(pkg2)
713@@ -127,11 +130,11 @@
714 def testExpirationWithDateOverThreshold(self):
715 """Test expiring works if dateremoved old enough."""
716 pkg3 = self.stp.getPubSource(
717- sourcename="pkg3", architecturehintlist="i386", archive=self.ppa,
718- dateremoved=self.over_threshold_date)
719+ sourcename="pkg3", architecturehintlist="i386",
720+ archive=self.archive, dateremoved=self.over_threshold_date)
721 [pub] = self.stp.getPubBinaries(
722 pub_source=pkg3, dateremoved=self.over_threshold_date,
723- archive=self.ppa)
724+ archive=self.archive)
725
726 self.runScript()
727 self.assertSourceExpired(pkg3)
728@@ -140,16 +143,16 @@
729 def testNoExpirationWithDateOverThresholdAndOtherValidPublication(self):
730 """Test no expiry if dateremoved old enough but other publication."""
731 pkg4 = self.stp.getPubSource(
732- sourcename="pkg4", architecturehintlist="i386", archive=self.ppa,
733- dateremoved=self.over_threshold_date)
734+ sourcename="pkg4", architecturehintlist="i386",
735+ archive=self.archive, dateremoved=self.over_threshold_date)
736 other_source = pkg4.copyTo(
737- pkg4.distroseries, pkg4.pocket, self.ppa2)
738+ pkg4.distroseries, pkg4.pocket, self.archive2)
739 other_source.dateremoved = None
740 [pub] = self.stp.getPubBinaries(
741 pub_source=pkg4, dateremoved=self.over_threshold_date,
742- archive=self.ppa)
743+ archive=self.archive)
744 [other_binary] = pub.copyTo(
745- pub.distroarchseries.distroseries, pub.pocket, self.ppa2)
746+ pub.distroarchseries.distroseries, pub.pocket, self.archive2)
747 other_binary.dateremoved = None
748
749 self.runScript()
750@@ -163,40 +166,22 @@
751 not over date threshold.
752 """
753 pkg5 = self.stp.getPubSource(
754- sourcename="pkg5", architecturehintlist="i386", archive=self.ppa,
755- dateremoved=self.over_threshold_date)
756+ sourcename="pkg5", architecturehintlist="i386",
757+ archive=self.archive, dateremoved=self.over_threshold_date)
758 other_source = pkg5.copyTo(
759- pkg5.distroseries, pkg5.pocket, self.ppa2)
760+ pkg5.distroseries, pkg5.pocket, self.archive2)
761 other_source.dateremoved = self.under_threshold_date
762 [pub] = self.stp.getPubBinaries(
763 pub_source=pkg5, dateremoved=self.over_threshold_date,
764- archive=self.ppa)
765+ archive=self.archive)
766 [other_binary] = pub.copyTo(
767- pub.distroarchseries.distroseries, pub.pocket, self.ppa2)
768+ pub.distroarchseries.distroseries, pub.pocket, self.archive2)
769 other_binary.dateremoved = self.under_threshold_date
770
771 self.runScript()
772 self.assertSourceNotExpired(pkg5)
773 self.assertBinaryNotExpired(pub)
774
775- def _setUpExpirablePublications(self, archive=None):
776- """Helper to set up two publications that are both expirable."""
777- if archive is None:
778- archive = self.ppa
779- pkg5 = self.stp.getPubSource(
780- sourcename="pkg5", architecturehintlist="i386", archive=archive,
781- dateremoved=self.over_threshold_date)
782- other_source = pkg5.copyTo(
783- pkg5.distroseries, pkg5.pocket, self.ppa2)
784- other_source.dateremoved = self.over_threshold_date
785- [pub] = self.stp.getPubBinaries(
786- pub_source=pkg5, dateremoved=self.over_threshold_date,
787- archive=archive)
788- [other_binary] = pub.copyTo(
789- pub.distroarchseries.distroseries, pub.pocket, self.ppa2)
790- other_binary.dateremoved = self.over_threshold_date
791- return pkg5, pub
792-
793 def testNoExpirationWithDateOverThresholdAndOtherPubOverThreshold(self):
794 """Test expiring works.
795
796@@ -208,26 +193,6 @@
797 self.assertSourceExpired(source)
798 self.assertBinaryExpired(binary)
799
800- def testBlacklistingWorks(self):
801- """Test that blacklisted PPAs are not expired."""
802- source, binary = self._setUpExpirablePublications(archive=self.ppa)
803- script = self.getScript()
804- script.blacklist = [self.ppa.owner.name, ]
805- self.layer.txn.commit()
806- self.layer.switchDbUser(self.dbuser)
807- script.main()
808- self.assertSourceNotExpired(source)
809- self.assertBinaryNotExpired(binary)
810-
811- def testPrivatePPAsNotExpired(self):
812- """Test that private PPAs are not expired."""
813- self.ppa.private = True
814- self.ppa.buildd_secret = "foo"
815- source, binary = self._setUpExpirablePublications()
816- self.runScript()
817- self.assertSourceNotExpired(source)
818- self.assertBinaryNotExpired(binary)
819-
820 def testDryRun(self):
821 """Test that when dryrun is specified, nothing is expired."""
822 source, binary = self._setUpExpirablePublications()
823@@ -240,7 +205,7 @@
824 self.assertSourceNotExpired(source)
825 self.assertBinaryNotExpired(binary)
826
827- def testDoesNotAffectNonPPA(self):
828+ def testDoesNotAffectPrimary(self):
829 """Test that expiry does not happen for non-PPA publications."""
830 ubuntu_archive = getUtility(IDistributionSet)['ubuntu'].main_archive
831 source, binary = self._setUpExpirablePublications(ubuntu_archive)
832@@ -249,5 +214,55 @@
833 self.assertBinaryNotExpired(binary)
834
835
836+class TestPPAExpiry(ArchiveExpiryTestBase, ArchiveExpiryCommonTests):
837+ """Test the expire-archive-files.py script.
838+
839+ Here we make use of the common test cases defined in the base class but
840+ also add tests specific to PPAs (excluding particular PPAs from expiry
841+ based on a "black list" or on the fact that PPA is private).
842+ """
843+
844+ def setUp(self):
845+ """Set up some test publications."""
846+ super(TestPPAExpiry, self).setUp()
847+ # Prepare two PPAs for the tests to use.
848+ self.archive = self.factory.makeArchive()
849+ self.archive2 = self.factory.makeArchive()
850+
851+ def testBlacklistingWorks(self):
852+ """Test that blacklisted PPAs are not expired."""
853+ source, binary = self._setUpExpirablePublications(
854+ archive=self.archive)
855+ script = self.getScript()
856+ script.blacklist = [self.archive.owner.name, ]
857+ self.layer.txn.commit()
858+ self.layer.switchDbUser(self.dbuser)
859+ script.main()
860+ self.assertSourceNotExpired(source)
861+ self.assertBinaryNotExpired(binary)
862+
863+ def testPrivatePPAsNotExpired(self):
864+ """Test that private PPAs are not expired."""
865+ self.archive.private = True
866+ self.archive.buildd_secret = "foo"
867+ source, binary = self._setUpExpirablePublications()
868+ self.runScript()
869+ self.assertSourceNotExpired(source)
870+ self.assertBinaryNotExpired(binary)
871+
872+
873+class TestPartnerExpiry(ArchiveExpiryTestBase, ArchiveExpiryCommonTests):
874+ """Test the expire-archive-files.py script on partner archives."""
875+
876+ def setUp(self):
877+ """Set up the partner archives under test."""
878+ super(TestPartnerExpiry, self).setUp()
879+ # Prepare two partner archives for the tests to use.
880+ self.archive = self.factory.makeArchive(
881+ purpose=ArchivePurpose.PARTNER)
882+ self.archive2 = self.factory.makeArchive(
883+ purpose=ArchivePurpose.PARTNER)
884+
885+
886 def test_suite():
887 return unittest.TestLoader().loadTestsFromName(__name__)
888
889=== modified file 'versions.cfg'
890--- versions.cfg 2010-03-08 22:26:29 +0000
891+++ versions.cfg 2010-03-11 21:49:26 +0000
892@@ -21,14 +21,14 @@
893 grokcore.component = 1.6
894 httplib2 = 0.6.0
895 ipython = 0.9.1
896-launchpadlib = 1.5.5
897+launchpadlib = 1.5.6
898 lazr.authentication = 0.1.1
899 lazr.batchnavigator = 1.1
900 lazr.config = 1.1.3
901 lazr.delegates = 1.1.0
902 lazr.enum = 1.1.2
903 lazr.lifecycle = 1.1
904-lazr.restful = 0.9.21
905+lazr.restful = 0.9.22
906 lazr.restfulclient = 0.9.11
907 lazr.smtptest = 1.1
908 lazr.testing = 0.1.1

Subscribers

People subscribed via source and target branches

to status/vote changes: