Merge lp:~leonardr/launchpad/multiversion-apidoc into lp:launchpad/db-devel

Proposed by Leonard Richardson
Status: Merged
Approved by: Aaron Bentley
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~leonardr/launchpad/multiversion-apidoc
Merge into: lp:launchpad/db-devel
Prerequisite: lp:~leonardr/launchpad/no-mutator-named-operations
Diff against target: 1156 lines (+535/-157)
21 files modified
.bzrignore (+2/-2)
Makefile (+5/-9)
lib/canonical/buildd/debian/launchpad-buildd.init (+1/-1)
lib/canonical/launchpad/pagetests/webservice/apidoc.txt (+17/-3)
lib/canonical/launchpad/pagetests/webservice/multiversion.txt (+32/-0)
lib/canonical/launchpad/rest/configuration.py (+21/-0)
lib/canonical/launchpad/webapp/tests/test_servers.py (+1/-1)
lib/lp/bugs/stories/webservice/xx-bug.txt (+9/-40)
lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py (+1/-1)
lib/lp/buildmaster/model/buildfarmjobbehavior.py (+94/-10)
lib/lp/buildmaster/tests/test_buildfarmjobbehavior.py (+80/-8)
lib/lp/code/model/recipebuilder.py (+11/-2)
lib/lp/scripts/utilities/apiindex.py (+0/-21)
lib/lp/soyuz/doc/buildd-slavescanner.txt (+4/-3)
lib/lp/testing/factory.py (+87/-40)
lib/lp/translations/doc/translationtemplatesbuildbehavior.txt (+41/-12)
lib/lp/translations/model/translationtemplatesbuildbehavior.py (+17/-1)
lib/lp/translations/tests/test_translationtemplatesbuildbehavior.py (+37/-0)
utilities/apidoc-index.pt (+37/-0)
utilities/create-lp-wadl-and-apidoc.py (+35/-0)
versions.cfg (+3/-3)
To merge this branch: bzr merge lp:~leonardr/launchpad/multiversion-apidoc
Reviewer Review Type Date Requested Status
Aaron Bentley (community) Approve
Review via email: mp+21181@code.launchpad.net

Description of the change

This branch makes Launchpad generate an HTML apidoc for every version of the web service, not just the devel version. The apidoc for version "foo" is written to lib/canonical/launchpad/apidoc/foo.html, and index.html is now a short list of links to the version-specific documentation files.

This branch renames create-lp-wadl.py to create-lp-wadl-and-apidoc.py, giving it repsonsibility for converting the WADL into the API doc and generating the index.html. The apiindex.py file is removed. (I couldn't this file used anywhere but the Makefile.)

I would really like feedback on the following issues:

1. Is there a way to put apidoc-index.pt somewhere else? Maybe in lib/canonical/launchpad/templates? I couldn't get this to work with pkg_resources because l.c.l.t isn't a module.
2. Is there a way to look up config/active_versions/$version in the template file? If so, I can get rid of versions_and_descriptions.

This branch still needs a little bit of work--I need to fix up the apidoc test so it tests the new system.

To post a comment you must log in.
Revision history for this message
Leonard Richardson (leonardr) wrote :

The diff is screwed up due to the precursor branch being out-of-date, but here's a paste: http://pastebin.ubuntu.com/393568/

Revision history for this message
Leonard Richardson (leonardr) wrote :

One test on EC2 failed: the title of the generated apidoc is now "About this service", not "Launchpad Web Service API". Our XSLT stylesheet (which is based on Mark Nottingham's general WADL stylesheet) has this code:

        <xsl:variable name="title">
            <xsl:choose>
                <xsl:when test="wadl:doc[@title]">
                    <xsl:value-of select="wadl:doc[@title][1]/@title"/>
                </xsl:when>
                <xsl:otherwise>Launchpad Web Service API</xsl:otherwise>
            </xsl:choose>
        </xsl:variable>

Previously the WADL's <application> tag had no <doc> children, so we used the default. Now there are two <doc> tags, "About this service" and "About version {version}".

This is actually pretty nice; I was afraid I would have to hack the XSLT to add the <doc> tags to the HTML, but it already processes them. The only problem is that the XSLT stylesheet uses the title of the first <doc> tag as the title of the whole document, and ignores the title of all subsequent <doc> tags. I was hoping for this (#1):

---
<h1>Launchpad Web Service API</h1>

<h2>About this service</h2>

The Launchpad web service allows automated clients to access most of the functionality available on the Launchpad web site. For help getting started, see the help wiki.

<h2>About version beta</h2>

This is the first version of the web service ever published. Its end-of-life date is April 2011, the same as the Ubuntu release "Karmic Koala".
---

And upon reflection I think this would be even better (#2):

---
<h1>Launchpad web service, version beta</h1>

The Launchpad web service allows automated clients to access most of the functionality available on the Launchpad web site. For help getting started, see the help wiki.

This is the first version of the web service ever published. Its end-of-life date is April 2011, the same as the Ubuntu release "Karmic Koala".
---

Instead, we have this (#3):

---
<h1>About this service</h1>

The Launchpad web service allows automated clients to access most of the functionality available on the Launchpad web site. For help getting started, see the help wiki.

This is the first version of the web service ever published. Its end-of-life date is April 2011, the same as the Ubuntu release "Karmic Koala".
---

We can change the stylesheet to get #1 and change lazr.restful to get #2. But I think what we have now is good enough that I can file a bug and come back to fix the generated documentation later. What do you think?

Revision history for this message
Leonard Richardson (leonardr) wrote :

And here's another diff against the precursor branch: http://paste.ubuntu.com/395661

Revision history for this message
Aaron Bentley (abentley) wrote :

Please use extract_text if possible. Otherwise, this looks fine.

review: Approve
Revision history for this message
Gary Poster (gary) wrote :

Leonard asked me to take a look at this because of his two specific questions.

1. Is there a way to put apidoc-index.pt somewhere else? Maybe in lib/canonical/launchpad/templates? I couldn't get this to work with pkg_resources because l.c.l.t isn't a module.

The FilePageTemplate just expects a path. I'm relatively confident that you can use os.path fun, combined with module __file__ fun, to do what you describe. However, as we agreed on IRC, I'm fine with what you are doing now.

2. Is there a way to look up config/active_versions/$version in the template file? If so, I can get rid of versions_and_descriptions.

The infrequently used syntax for this is "?," I believe. In other words, if you have a variable named "config" and a variable named "version" that contains the string "beta", "config/active_versions/?version" is supposed to be equivalent to "config/active_versions/beta".

On a separate note, also as we discussed on IRC, this segment from create-lp-wadl-and-apidoc.py looks to me like it ought to be simplified.

1115 + # Finally, create an index.html with links to all the HTML
1116 + # documentation files we just generated.
1117 + template_file = 'apidoc-index.pt'
1118 + template = PageTemplateFile(template_file)
1119 + namespace = template.pt_getContext()
1120 + namespace['config'] = config
1121 + versions_and_descriptions = []
1122 + for version in config.active_versions:
1123 + versions_and_descriptions.append(
1124 + dict(version=version,
1125 + description=config.version_descriptions[version]))
1126 + namespace['versions_and_descriptions'] = versions_and_descriptions
1127 +
1128 + f = open(os.path.join(directory, "index.html"), 'w')
1129 + f.write(template.pt_render(namespace))

This could be simplified to the following:

    template_file = 'apidoc-index.pt'
    template = PageTemplateFile(template_file)
    versions_and_descriptions = []
    for version in config.active_versions:
        versions_and_descriptions.append(
            dict(version=version,
                 description=config.version_descriptions[version]))

    f = open(os.path.join(directory, "index.html"), 'w')
    f.write(template(config=config,
                     versions_and_descriptions=versions_and_descriptions))

Then in your template you would access config and versions_and_descriptions off of the "options" object in the page template.

If you use the ? syntax I mentioned above for options/config/version_descriptions/?version, of course, then this could be further simplified to just this:

    template_file = 'apidoc-index.pt'
    template = PageTemplateFile(template_file)
    f = open(os.path.join(directory, "index.html"), 'w')
    f.write(template(config=config))

That's way nicer, I think.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2010-03-09 02:42:40 +0000
3+++ .bzrignore 2010-03-15 21:11:07 +0000
4@@ -30,9 +30,9 @@
5 database/sampledata/newsampledata-dev.sql
6 database/sampledata/lintdata.sql
7 database/sampledata/lintdata-dev.sql
8-lib/canonical/launchpad/apidoc/index.html
9+lib/canonical/launchpad/apidoc/*.html
10 xxx-report.*
11-lib/canonical/launchpad/apidoc/wadl-development.xml
12+lib/canonical/launchpad/apidoc/wadl-development-*.xml
13 lib/canonical/launchpad/apidoc/wadl-test-playground.xml
14 lib/canonical/launchpad/icing/build/*
15 lib/canonical/launchpad/icing/combo.css
16
17=== modified file 'Makefile'
18--- Makefile 2010-03-05 21:47:03 +0000
19+++ Makefile 2010-03-15 21:11:07 +0000
20@@ -30,7 +30,6 @@
21
22 APIDOC_DIR = lib/canonical/launchpad/apidoc
23 WADL_TEMPLATE = $(APIDOC_DIR).tmp/wadl-$(LPCONFIG)-%(version)s.xml
24-DEVEL_WADL_FILE = $(APIDOC_DIR)/wadl-$(LPCONFIG)-devel.xml
25 API_INDEX = $(APIDOC_DIR)/index.html
26
27 # Do not add bin/buildout to this list.
28@@ -60,16 +59,12 @@
29 hosted_branches: $(PY)
30 $(PY) ./utilities/make-dummy-hosted-branches
31
32-$(DEVEL_WADL_FILE): $(BZR_VERSION_INFO)
33+$(API_INDEX): $(BZR_VERSION_INFO)
34 mkdir $(APIDOC_DIR).tmp
35- LPCONFIG=$(LPCONFIG) $(PY) ./utilities/create-lp-wadl.py "$(WADL_TEMPLATE)"
36+ LPCONFIG=$(LPCONFIG) $(PY) ./utilities/create-lp-wadl-and-apidoc.py "$(WADL_TEMPLATE)"
37 mv $(APIDOC_DIR).tmp/* $(APIDOC_DIR)
38 rmdir $(APIDOC_DIR).tmp
39
40-$(API_INDEX): $(DEVEL_WADL_FILE)
41- bin/apiindex $(DEVEL_WADL_FILE) > $@.tmp
42- mv $@.tmp $@
43-
44 apidoc: compile $(API_INDEX)
45
46 check_merge: $(PY)
47@@ -259,7 +254,7 @@
48 $(BZR_VERSION_INFO):
49 scripts/update-bzr-version-info.sh
50
51-support_files: $(DEVEL_WADL_FILE) $(BZR_VERSION_INFO)
52+support_files: $(API_INDEX) $(BZR_VERSION_INFO)
53
54 # Intended for use on developer machines
55 start: inplace stop support_files initscript-start
56@@ -330,9 +325,10 @@
57 $(RM) -r $(CODEHOSTING_ROOT)
58 mv $(APIDOC_DIR)/wadl-testrunner-devel.xml \
59 $(APIDOC_DIR)/wadl-testrunner-devel.xml.bak
60- $(RM) $(APIDOC_DIR)/wadl*.xml $(API_INDEX)
61+ $(RM) $(APIDOC_DIR)/wadl*.xml $(APIDOC_DIR)/*.html
62 mv $(APIDOC_DIR)/wadl-testrunner-devel.xml.bak \
63 $(APIDOC_DIR)/wadl-testrunner-devel.xml
64+ $(RM) -rf $(APIDOC_DIR).tmp
65 $(RM) $(BZR_VERSION_INFO)
66 $(RM) _pythonpath.py
67 $(RM) -rf \
68
69=== modified file 'lib/canonical/buildd/debian/launchpad-buildd.init'
70--- lib/canonical/buildd/debian/launchpad-buildd.init 2009-12-16 00:16:24 +0000
71+++ lib/canonical/buildd/debian/launchpad-buildd.init 2010-03-15 21:11:07 +0000
72@@ -29,7 +29,7 @@
73 CONF=$1
74 PIDFILE="$PIDROOT"/"$CONF".pid
75 LOGFILE="$LOGROOT"/"$CONF".log
76- su - buildd -c "BUILDD_SLAVE_CONFIG=$CONFROOT/$CONF PYTHONPATH=/usr/share/launchpad-buildd twistd --no_save --pidfile $PIDFILE --python $TACFILE --logfile $LOGFILE"
77+ su - buildd -c "BUILDD_SLAVE_CONFIG=$CONFROOT/$CONF PYTHONPATH=/usr/share/launchpad-buildd twistd --no_save --pidfile $PIDFILE --python $TACFILE --logfile $LOGFILE --umask 022"
78 }
79
80 #
81
82=== modified file 'lib/canonical/launchpad/pagetests/webservice/apidoc.txt'
83--- lib/canonical/launchpad/pagetests/webservice/apidoc.txt 2008-07-31 21:23:02 +0000
84+++ lib/canonical/launchpad/pagetests/webservice/apidoc.txt 2010-03-15 21:11:07 +0000
85@@ -1,9 +1,23 @@
86 = API Documentation =
87
88-The up-to-date Launchpad API documentation is available from
89-http://launchpad.dev/+apidoc.
90+The Launchpad API documentation portal is available at
91+http://launchpad.dev/+apidoc. It contains summaries of the different
92+web service versions, and links to version-specific documents.
93
94 >>> browser.open('http://launchpad.dev/+apidoc')
95+ >>> print extract_text(browser.contents)
96+ Launchpad Web Service API
97+ Launchpad web service API documentation
98+ ...
99+ Active versions
100+ ...
101+ devel: This version of the web service reflects the most recent...
102+ ...
103+
104+The documentation for a specific version is located at
105+http://launchpad.dev/+apidoc/{version}.html.
106+
107+ >>> browser.open('http://launchpad.dev/+apidoc/devel.html')
108 >>> print browser.title
109- Launchpad Web Service API
110+ About this service
111
112
113=== added file 'lib/canonical/launchpad/pagetests/webservice/multiversion.txt'
114--- lib/canonical/launchpad/pagetests/webservice/multiversion.txt 1970-01-01 00:00:00 +0000
115+++ lib/canonical/launchpad/pagetests/webservice/multiversion.txt 2010-03-15 21:11:07 +0000
116@@ -0,0 +1,32 @@
117+Differences between versions
118+----------------------------
119+
120+In the 'beta' version of the web service, mutator methods like
121+IBugTask.transitionToStatus are published as named operations. In
122+subsequent versions, those named operations are not published.
123+
124+ >>> def get_bugtask_path(version):
125+ ... bug_one = webservice.get("/bugs/1", api_version=version).jsonBody()
126+ ... bug_one_bugtasks_url = bug_one['bug_tasks_collection_link']
127+ ... bug_one_bugtasks = sorted(webservice.get(
128+ ... bug_one_bugtasks_url).jsonBody()['entries'])
129+ ... bugtask_path = bug_one_bugtasks[0]['self_link']
130+ ... return bugtask_path
131+
132+Here's the 'beta' version, where the named operation works.
133+
134+ >>> url = get_bugtask_path('beta')
135+ >>> print webservice.named_post(
136+ ... url, 'transitionToImportance', importance='Low',
137+ ... api_version='beta')
138+ HTTP/1.1 200 Ok
139+ ...
140+
141+Now let's try the same thing in the '1.0' version, where it fails.
142+
143+ >>> url = get_bugtask_path('1.0')
144+ >>> print webservice.named_post(
145+ ... url, 'transitionToImportance', importance='Low',
146+ ... api_version='devel')
147+ HTTP/1.1 405 Method Not Allowed
148+ ...
149
150=== modified file 'lib/canonical/launchpad/rest/configuration.py'
151--- lib/canonical/launchpad/rest/configuration.py 2010-02-15 02:09:32 +0000
152+++ lib/canonical/launchpad/rest/configuration.py 2010-03-15 21:11:07 +0000
153@@ -24,9 +24,30 @@
154
155 path_override = "api"
156 active_versions = ["beta", "1.0", "devel"]
157+ last_version_with_mutator_named_operations = "beta"
158 view_permission = "launchpad.View"
159 set_hop_by_hop_headers = True
160
161+ service_description = """The Launchpad web service allows automated
162+ clients to access most of the functionality available on the
163+ Launchpad web site. For help getting started, see
164+ <a href="https://help.launchpad.net/API/">the help wiki.</a>"""
165+
166+ version_descriptions = {
167+ "beta": """This is the first version of the web service ever
168+ published. Its end-of-life date is April 2011, the same as the
169+ Ubuntu release "Karmic Koala".""",
170+
171+ "1.0": """This version of the web service removes unnecessary
172+ named operations. It was introduced in March 2010, and its
173+ end-of-life date is April 2015, the same as the server version
174+ of the Ubuntu release "Lucid Lynx".""",
175+
176+ "devel": """This version of the web service reflects the most
177+ recent changes made. It may abruptly change without
178+ warning. Periodically, these changes are bundled up and given a
179+ permanent version number.""" }
180+
181 @property
182 def use_https(self):
183 return config.vhosts.use_https
184
185=== modified file 'lib/canonical/launchpad/webapp/tests/test_servers.py'
186--- lib/canonical/launchpad/webapp/tests/test_servers.py 2010-02-24 14:45:33 +0000
187+++ lib/canonical/launchpad/webapp/tests/test_servers.py 2010-03-15 21:11:07 +0000
188@@ -14,7 +14,7 @@
189 from lazr.restful.interfaces import (
190 IServiceRootResource, IWebServiceConfiguration)
191 from lazr.restful.simple import RootResource
192-from lazr.restful.tests.test_webservice import (
193+from lazr.restful.testing.webservice import (
194 IGenericCollection, IGenericEntry, WebServiceTestCase)
195
196 from lp.testing import TestCase
197
198=== modified file 'lib/lp/bugs/stories/webservice/xx-bug.txt'
199--- lib/lp/bugs/stories/webservice/xx-bug.txt 2010-03-10 14:14:38 +0000
200+++ lib/lp/bugs/stories/webservice/xx-bug.txt 2010-03-15 21:11:07 +0000
201@@ -357,7 +357,7 @@
202 http://.../~cprov
203
204
205-The task's importance can be modified directly...
206+The task's importance can be modified directly.
207
208 >>> body = webservice.get(bugtask_path).jsonBody()
209 >>> body['importance']
210@@ -371,20 +371,6 @@
211 >>> body['importance']
212 u'High'
213
214-...or through a named operation.
215-
216- >>> print webservice.named_post(
217- ... bugtask_path, 'transitionToImportance', importance='Low')
218- HTTP/1.1 200 Ok
219- ...
220- Content-Type: application/json
221- ...
222- <BLANKLINE>
223- null
224- >>> body = webservice.get(bugtask_path).jsonBody()
225- >>> body['importance']
226- u'Low'
227-
228 Only bug supervisors or people who can otherwise edit the bugtask's
229 pillar are authorised to edit the importance.
230
231@@ -394,10 +380,9 @@
232
233 >>> body = webservice.get(bugtask_path).jsonBody()
234 >>> body['importance']
235- u'Low'
236+ u'High'
237
238-The task's status is another aspect of the task that can be modified
239-either directly or through a named operation.
240+The task's status can also be modified directly.
241
242 >>> print webservice.get(bugtask_path).jsonBody()['status']
243 Confirmed
244@@ -409,21 +394,14 @@
245 >>> print webservice.get(bugtask_path).jsonBody()['status']
246 Fix Committed
247
248- >>> print webservice.named_post(
249- ... bugtask_path, 'transitionToStatus', status='New')
250- HTTP/1.1 200 Ok...
251-
252- >>> print webservice.get(bugtask_path).jsonBody()['status']
253- New
254-
255 If an error occurs during a request that sets both 'status' and
256 'importance', neither one will be set.
257
258 >>> task = webservice.get(bugtask_path).jsonBody()
259 >>> print task['status']
260- New
261+ Fix Committed
262 >>> print task['importance']
263- Low
264+ High
265
266 >>> patch = {u'importance': 'High', u'status': u'No Such Status'}
267 >>> print webservice.patch(bugtask_path, 'application/json', dumps(patch))
268@@ -431,12 +409,11 @@
269
270 >>> task = webservice.get(bugtask_path).jsonBody()
271 >>> print task['status']
272- New
273+ Fix Committed
274 >>> print task['importance']
275- Low
276+ High
277
278-Like the importance, the milestone can be set either directly or via a
279-named operation, and only by appropriately privileged users.
280+The milestone can only be set by appropriately privileged users.
281
282 >>> print webservice.get(bugtask_path).jsonBody()['milestone_link']
283 None
284@@ -449,20 +426,12 @@
285 >>> print webservice.get(bugtask_path).jsonBody()['milestone_link']
286 http://.../debian/+milestone/3.1
287
288- >>> print webservice.named_post(
289- ... bugtask_path, 'transitionToMilestone',
290- ... milestone=webservice.getAbsoluteUrl('/debian/+milestone/3.1-rc1'))
291- HTTP/1.1 200 Ok...
292-
293- >>> print webservice.get(bugtask_path).jsonBody()['milestone_link']
294- http://.../debian/+milestone/3.1-rc1
295-
296 >>> print user_webservice.patch(
297 ... bugtask_path, 'application/json', dumps(patch))
298 HTTP/1.1 401 Unauthorized...
299
300 >>> print webservice.get(bugtask_path).jsonBody()['milestone_link']
301- http://.../debian/+milestone/3.1-rc1
302+ http://.../debian/+milestone/3.1
303
304 We can change the task's target. Here we change the task's target from
305 the mozilla-firefox package to alsa-utils.
306
307=== modified file 'lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py'
308--- lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2010-01-21 05:03:16 +0000
309+++ lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2010-03-15 21:11:07 +0000
310@@ -1,4 +1,4 @@
311-# Copyright 2009 Canonical Ltd. This software is licensed under the
312+# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
313 # GNU Affero General Public License version 3 (see the file LICENSE).
314
315 # pylint: disable-msg=E0211,E0213
316
317=== modified file 'lib/lp/buildmaster/model/buildfarmjobbehavior.py'
318--- lib/lp/buildmaster/model/buildfarmjobbehavior.py 2010-03-05 13:52:32 +0000
319+++ lib/lp/buildmaster/model/buildfarmjobbehavior.py 2010-03-15 21:11:07 +0000
320@@ -1,4 +1,4 @@
321-# Copyright 2009 Canonical Ltd. This software is licensed under the
322+# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
323 # GNU Affero General Public License version 3 (see the file LICENSE).
324
325 # pylint: disable-msg=E0211,E0213
326@@ -17,17 +17,22 @@
327 import xmlrpclib
328
329 from sqlobject import SQLObjectNotFound
330+
331 from zope.component import getUtility
332 from zope.interface import implements
333-from zope.security.proxy import removeSecurityProxy
334+from zope.security.proxy import removeSecurityProxy, isinstance as zisinstance
335
336 from canonical import encoding
337 from canonical.librarian.interfaces import ILibrarianClient
338+
339+from canonical.launchpad.webapp.interfaces import NotFoundError
340 from lp.buildmaster.interfaces.builder import CorruptBuildID
341 from lp.buildmaster.interfaces.buildfarmjobbehavior import (
342 BuildBehaviorMismatch, IBuildFarmJobBehavior)
343 from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
344+from lp.buildmaster.model.buildqueue import BuildQueue
345 from lp.services.job.interfaces.job import JobStatus
346+from lp.soyuz.interfaces.build import IBuildSet
347
348
349 class BuildFarmJobBehaviorBase:
350@@ -59,22 +64,101 @@
351 The default behavior is that we don't add any extra values."""
352 return {}
353
354+ def _helpVerifyBuildIDComponent(self, raw_id, item_type, finder):
355+ """Helper for verifying parts of a `BuildFarmJob` name.
356+
357+ Different `IBuildFarmJob` implementations can have different
358+ ways of constructing their identifying names. The names are
359+ produced by `IBuildFarmJob.getName` and verified by
360+ `IBuildFarmJobBehavior.verifySlaveBuildID`.
361+
362+ This little helper makes it easier to verify an object id
363+ embedded in that name, check that it's a valid number, and
364+ retrieve the associated database object.
365+
366+ :param raw_id: An unverified id string as extracted from the
367+ build name. The method will verify that it is a number, and
368+ try to retrieve the associated object.
369+ :param item_type: The type of object this id represents. Should
370+ be a class.
371+ :param finder: A function that, given an integral id, finds the
372+ associated object of type `item_type`.
373+ :raise CorruptBuildID: If `raw_id` is malformed in some way or
374+ the associated `item_type` object is not found.
375+ :return: An object that is an instance of `item_type`.
376+ """
377+ type_name = item_type.__name__
378+ try:
379+ numeric_id = int(raw_id)
380+ except ValueError:
381+ raise CorruptBuildID(
382+ "%s ID is not a number: '%s'" % (type_name, raw_id))
383+
384+ try:
385+ item = finder(numeric_id)
386+ except (NotFoundError, SQLObjectNotFound), reason:
387+ raise CorruptBuildID(
388+ "%s %d is not available: %s" % (
389+ type_name, numeric_id, reason))
390+ except Exception, reason:
391+ raise CorruptBuildID(
392+ "Error while looking up %s %d: %s" % (
393+ type_name, numeric_id, reason))
394+
395+ if item is None:
396+ raise CorruptBuildID("There is no %s with id %d." % (
397+ type_name, numeric_id))
398+
399+ assert zisinstance(item, item_type), (
400+ "Looked for %s, but found %s." % (type_name, repr(item)))
401+
402+ return item
403+
404+ def getVerifiedBuild(self, raw_id):
405+ """Verify and retrieve the `Build` component of a slave build id.
406+
407+ This does part of the verification for `verifySlaveBuildID`.
408+
409+ By default, a `BuildFarmJob` has an identifying name of the form
410+ "b-q", where b is the id of its `Build` and q is the id of its
411+ `BuildQueue` record.
412+
413+ Use `getVerifiedBuild` to verify the "b" part, and retrieve the
414+ associated `Build`.
415+ """
416+ # Avoid circular import.
417+ from lp.soyuz.model.build import Build
418+
419+ return self._helpVerifyBuildIDComponent(
420+ raw_id, Build, getUtility(IBuildSet).getByBuildID)
421+
422+ def getVerifiedBuildQueue(self, raw_id):
423+ """Verify and retrieve the `BuildQueue` component of a slave build id.
424+
425+ This does part of the verification for `verifySlaveBuildID`.
426+
427+ By default, a `BuildFarmJob` has an identifying name of the form
428+ "b-q", where b is the id of its `Build` and q is the id of its
429+ `BuildQueue` record.
430+
431+ Use `getVerifiedBuildQueue` to verify the "q" part, and retrieve
432+ the associated `BuildQueue` object.
433+ """
434+ return self._helpVerifyBuildIDComponent(
435+ raw_id, BuildQueue, getUtility(IBuildQueueSet).get)
436+
437 def verifySlaveBuildID(self, slave_build_id):
438 """See `IBuildFarmJobBehavior`."""
439 # Extract information from the identifier.
440 try:
441 build_id, queue_item_id = slave_build_id.split('-')
442- build_id = int(build_id)
443- queue_item_id = int(queue_item_id)
444 except ValueError:
445 raise CorruptBuildID('Malformed build ID')
446+
447+ build = self.getVerifiedBuild(build_id)
448+ queue_item = self.getVerifiedBuildQueue(queue_item_id)
449
450- try:
451- queue_item = getUtility(IBuildQueueSet).get(queue_item_id)
452- # Check whether build and buildqueue are properly related.
453- except SQLObjectNotFound, reason:
454- raise CorruptBuildID(str(reason))
455- if queue_item.specific_job.build.id != build_id:
456+ if build != queue_item.specific_job.build:
457 raise CorruptBuildID('Job build entry mismatch')
458
459 def updateBuild(self, queueItem):
460
461=== modified file 'lib/lp/buildmaster/tests/test_buildfarmjobbehavior.py'
462--- lib/lp/buildmaster/tests/test_buildfarmjobbehavior.py 2010-03-06 04:57:40 +0000
463+++ lib/lp/buildmaster/tests/test_buildfarmjobbehavior.py 2010-03-15 21:11:07 +0000
464@@ -3,10 +3,18 @@
465
466 """Unit tests for BuildFarmJobBehaviorBase."""
467
468-from unittest import TestCase, TestLoader
469+from unittest import TestLoader
470+
471+from zope.component import getUtility
472+
473+from canonical.testing.layers import ZopelessDatabaseLayer
474+from lp.testing import TestCaseWithFactory
475
476 from lp.buildmaster.interfaces.buildbase import BuildStatus
477+from lp.buildmaster.interfaces.builder import CorruptBuildID
478 from lp.buildmaster.model.buildfarmjobbehavior import BuildFarmJobBehaviorBase
479+from lp.registry.interfaces.pocket import PackagePublishingPocket
480+from lp.soyuz.interfaces.processor import IProcessorFamilySet
481
482
483 class FakeBuildFarmJob:
484@@ -14,27 +22,91 @@
485 pass
486
487
488-class TestBuildFarmJobBehaviorBase(TestCase):
489+class TestBuildFarmJobBehaviorBase(TestCaseWithFactory):
490 """Test very small, basic bits of BuildFarmJobBehaviorBase."""
491
492- def setUp(self):
493- self.behavior = BuildFarmJobBehaviorBase(FakeBuildFarmJob())
494+ layer = ZopelessDatabaseLayer
495+
496+ def _makeBehavior(self, buildfarmjob=None):
497+ """Create a `BuildFarmJobBehaviorBase`."""
498+ if buildfarmjob is None:
499+ buildfarmjob = FakeBuildFarmJob()
500+ return BuildFarmJobBehaviorBase(buildfarmjob)
501+
502+ def _makeBuild(self):
503+ """Create a `Build` object."""
504+ x86 = getUtility(IProcessorFamilySet).getByName('x86')
505+ distroarchseries = self.factory.makeDistroArchSeries(
506+ architecturetag='x86', processorfamily=x86)
507+ distroseries = distroarchseries.distroseries
508+ archive = self.factory.makeArchive(
509+ distribution=distroseries.distribution)
510+ pocket = PackagePublishingPocket.RELEASE
511+ spr = self.factory.makeSourcePackageRelease(
512+ distroseries=distroseries, archive=archive)
513+
514+ return spr.createBuild(
515+ distroarchseries=distroarchseries, pocket=pocket, archive=archive)
516+
517+ def _makeBuildQueue(self):
518+ """Create a `BuildQueue` object."""
519+ return self.factory.makeSourcePackageRecipeBuildJob()
520
521 def test_extractBuildStatus_baseline(self):
522 # extractBuildStatus picks the name of the build status out of a
523 # dict describing the slave's status.
524 slave_status = {'build_status': 'BuildStatus.BUILDING'}
525+ behavior = self._makeBehavior()
526 self.assertEqual(
527 BuildStatus.BUILDING.name,
528- self.behavior.extractBuildStatus(slave_status))
529+ behavior.extractBuildStatus(slave_status))
530
531 def test_extractBuildStatus_malformed(self):
532 # extractBuildStatus errors out when the status string is not
533 # of the form it expects.
534 slave_status = {'build_status': 'BUILDING'}
535- self.assertRaises(
536- AssertionError,
537- self.behavior.extractBuildStatus, slave_status)
538+ behavior = self._makeBehavior()
539+ self.assertRaises(
540+ AssertionError, behavior.extractBuildStatus, slave_status)
541+
542+ def test_getVerifiedBuild_success(self):
543+ build = self._makeBuild()
544+ behavior = self._makeBehavior()
545+ raw_id = str(build.id)
546+
547+ self.assertEqual(build, behavior.getVerifiedBuild(raw_id))
548+
549+ def test_getVerifiedBuild_malformed(self):
550+ behavior = self._makeBehavior()
551+ self.assertRaises(CorruptBuildID, behavior.getVerifiedBuild, 'hi!')
552+
553+ def test_getVerifiedBuild_notfound(self):
554+ build = self._makeBuild()
555+ behavior = self._makeBehavior()
556+ nonexistent_id = str(build.id + 99)
557+
558+ self.assertRaises(
559+ CorruptBuildID, behavior.getVerifiedBuild, nonexistent_id)
560+
561+ def test_getVerifiedBuildQueue_success(self):
562+ buildqueue = self._makeBuildQueue()
563+ behavior = self._makeBehavior()
564+ raw_id = str(buildqueue.id)
565+
566+ self.assertEqual(buildqueue, behavior.getVerifiedBuildQueue(raw_id))
567+
568+ def test_getVerifiedBuildQueue_malformed(self):
569+ behavior = self._makeBehavior()
570+ self.assertRaises(
571+ CorruptBuildID, behavior.getVerifiedBuildQueue, 'bye!')
572+
573+ def test_getVerifiedBuildQueue_notfound(self):
574+ buildqueue = self._makeBuildQueue()
575+ behavior = self._makeBehavior()
576+ nonexistent_id = str(buildqueue.id + 99)
577+
578+ self.assertRaises(
579+ CorruptBuildID, behavior.getVerifiedBuildQueue, nonexistent_id)
580
581
582 def test_suite():
583
584=== modified file 'lib/lp/code/model/recipebuilder.py'
585--- lib/lp/code/model/recipebuilder.py 2010-02-05 15:06:28 +0000
586+++ lib/lp/code/model/recipebuilder.py 2010-03-15 21:11:07 +0000
587@@ -8,7 +8,7 @@
588 'RecipeBuildBehavior',
589 ]
590
591-from zope.component import adapts
592+from zope.component import adapts, getUtility
593 from zope.interface import implements
594
595 from lp.archiveuploader.permission import check_upload_to_pocket
596@@ -18,7 +18,8 @@
597 from lp.buildmaster.model.buildfarmjobbehavior import (
598 BuildFarmJobBehaviorBase)
599 from lp.code.interfaces.sourcepackagerecipebuild import (
600- ISourcePackageRecipeBuildJob)
601+ ISourcePackageRecipeBuildJob, ISourcePackageRecipeBuildSource)
602+from lp.code.model.sourcepackagerecipebuild import SourcePackageRecipeBuild
603 from lp.registry.interfaces.pocket import PackagePublishingPocket
604 from lp.soyuz.adapters.archivedependencies import (
605 get_primary_current_component, get_sources_list_for_building)
606@@ -168,3 +169,11 @@
607 extra_info['logtail'] = raw_slave_status[2]
608
609 return extra_info
610+
611+ def getVerifiedBuild(self, raw_id):
612+ """See `IBuildFarmJobBehavior`."""
613+ # This type of job has a build that is of type BuildBase but not
614+ # actually a Build.
615+ return self._helpVerifyBuildIDComponent(
616+ raw_id, SourcePackageRecipeBuild,
617+ getUtility(ISourcePackageRecipeBuildSource).getById)
618
619=== removed file 'lib/lp/scripts/utilities/apiindex.py'
620--- lib/lp/scripts/utilities/apiindex.py 2009-06-30 16:56:07 +0000
621+++ lib/lp/scripts/utilities/apiindex.py 1970-01-01 00:00:00 +0000
622@@ -1,21 +0,0 @@
623-# Copyright 2009 Canonical Ltd. This software is licensed under the
624-# GNU Affero General Public License version 3 (see the file LICENSE).
625-
626-"""A script to build reference html from wadl.
627-
628-Uses an xsl from the launchpadlib package."""
629-
630-__metaclass__ = type
631-__all__ = ['main']
632-
633-
634-import pkg_resources
635-import subprocess
636-import sys
637-
638-def main():
639- source = sys.argv[1]
640- stylesheet = pkg_resources.resource_filename(
641- 'launchpadlib', 'wadl-to-refhtml.xsl')
642- subprocess.call(['xsltproc', stylesheet, source])
643- pkg_resources.cleanup_resources()
644
645=== modified file 'lib/lp/soyuz/doc/buildd-slavescanner.txt'
646--- lib/lp/soyuz/doc/buildd-slavescanner.txt 2010-03-10 12:50:18 +0000
647+++ lib/lp/soyuz/doc/buildd-slavescanner.txt 2010-03-15 21:11:07 +0000
648@@ -95,15 +95,16 @@
649 >>> lostbuilding_builder = MockBuilder(
650 ... 'Lost Building Slave', LostBuildingSlave())
651 >>> buildergroup.rescueBuilderIfLost(lostbuilding_builder)
652- WARNING:root:Builder 'Lost Building Slave' rescued from
653- '1000-10000': 'Object not found'
654+ WARNING:root:Builder 'Lost Building Slave' rescued from '1000-10000':
655+ 'Build 1000 is not available: 'Object not found''
656
657 Then a lost slave in status 'WAITING':
658
659 >>> lostwaiting_builder = MockBuilder(
660 ... 'Lost Waiting Slave', LostWaitingSlave())
661 >>> buildergroup.rescueBuilderIfLost(lostwaiting_builder)
662- WARNING:root:Builder 'Lost Waiting Slave' rescued from '1000-10000': 'Object not found'
663+ WARNING:root:Builder 'Lost Waiting Slave' rescued from '1000-10000':
664+ 'Build 1000 is not available: 'Object not found''
665
666 Both got rescued, as expected.
667
668
669=== modified file 'lib/lp/testing/factory.py'
670--- lib/lp/testing/factory.py 2010-03-11 20:54:36 +0000
671+++ lib/lp/testing/factory.py 2010-03-15 21:11:07 +0000
672@@ -2008,6 +2008,81 @@
673 def getAnySourcePackageUrgency(self):
674 return SourcePackageUrgency.MEDIUM
675
676+ def makeSourcePackageRelease(self, archive=None, sourcepackagename=None,
677+ distroseries=None, maintainer=None,
678+ creator=None, component=None, section=None,
679+ urgency=None, version=None,
680+ builddepends=None, builddependsindep=None,
681+ build_conflicts=None,
682+ build_conflicts_indep=None,
683+ architecturehintlist='all',
684+ dsc_maintainer_rfc822=None,
685+ dsc_standards_version='3.6.2',
686+ dsc_format='1.0', dsc_binaries='foo-bin',
687+ date_uploaded=UTC_NOW):
688+ """Make a `SourcePackageRelease`."""
689+ if distroseries is None:
690+ if archive is None:
691+ distribution = None
692+ else:
693+ distribution = archive.distribution
694+ distroseries = self.makeDistroRelease(distribution=distribution)
695+
696+ if archive is None:
697+ archive = self.makeArchive(
698+ distribution=distroseries.distribution,
699+ purpose=ArchivePurpose.PRIMARY)
700+
701+ if sourcepackagename is None:
702+ sourcepackagename = self.makeSourcePackageName()
703+
704+ if component is None:
705+ component = self.makeComponent()
706+
707+ if urgency is None:
708+ urgency = self.getAnySourcePackageUrgency()
709+
710+ if section is None:
711+ section = self.getUniqueString('section')
712+ section = getUtility(ISectionSet).ensure(section)
713+
714+ if maintainer is None:
715+ maintainer = self.makePerson()
716+
717+ maintainer_email = '%s <%s>' % (
718+ maintainer.displayname,
719+ maintainer.preferredemail.email)
720+
721+ if creator is None:
722+ creator = self.makePerson()
723+
724+ if version is None:
725+ version = self.getUniqueString('version')
726+
727+ return distroseries.createUploadedSourcePackageRelease(
728+ sourcepackagename=sourcepackagename,
729+ maintainer=maintainer,
730+ creator=creator,
731+ component=component,
732+ section=section,
733+ urgency=urgency,
734+ version=version,
735+ builddepends=builddepends,
736+ builddependsindep=builddependsindep,
737+ build_conflicts=build_conflicts,
738+ build_conflicts_indep=build_conflicts_indep,
739+ architecturehintlist=architecturehintlist,
740+ changelog_entry=None,
741+ dsc=None,
742+ copyright=self.getUniqueString(),
743+ dscsigningkey=None,
744+ dsc_maintainer_rfc822=maintainer_email,
745+ dsc_standards_version=dsc_standards_version,
746+ dsc_format=dsc_format,
747+ dsc_binaries=dsc_binaries,
748+ archive=archive,
749+ dateuploaded=date_uploaded)
750+
751 def makeSourcePackagePublishingHistory(self, sourcepackagename=None,
752 distroseries=None, maintainer=None,
753 creator=None, component=None,
754@@ -2027,54 +2102,31 @@
755 dsc_format='1.0',
756 dsc_binaries='foo-bin',
757 ):
758- if sourcepackagename is None:
759- sourcepackagename = self.makeSourcePackageName()
760- spn = sourcepackagename
761-
762+ """Make a `SourcePackagePublishingHistory`."""
763 if distroseries is None:
764- distroseries = self.makeDistroRelease()
765+ if archive is None:
766+ distribution = None
767+ else:
768+ distribution = archive.distribution
769+ distroseries = self.makeDistroRelease(distribution=distribution)
770
771 if archive is None:
772 archive = self.makeArchive(
773 distribution=distroseries.distribution,
774 purpose=ArchivePurpose.PRIMARY)
775
776- if component is None:
777- component = self.makeComponent()
778-
779 if pocket is None:
780 pocket = self.getAnyPocket()
781
782 if status is None:
783 status = PackagePublishingStatus.PENDING
784
785- if urgency is None:
786- urgency = self.getAnySourcePackageUrgency()
787-
788- if section is None:
789- section = self.getUniqueString('section')
790- section = getUtility(ISectionSet).ensure(section)
791-
792- if maintainer is None:
793- maintainer = self.makePerson()
794-
795- maintainer_email = '%s <%s>' % (
796- maintainer.displayname,
797- maintainer.preferredemail.email)
798-
799- if creator is None:
800- creator = self.makePerson()
801-
802- if version is None:
803- version = self.getUniqueString('version')
804-
805- gpg_key = self.makeGPGKey(creator)
806-
807- spr = distroseries.createUploadedSourcePackageRelease(
808- sourcepackagename=spn,
809+ spr = self.makeSourcePackageRelease(
810+ archive=archive,
811+ sourcepackagename=sourcepackagename,
812+ distroseries=distroseries,
813 maintainer=maintainer,
814- creator=creator,
815- component=component,
816+ creator=creator, component=component,
817 section=section,
818 urgency=urgency,
819 version=version,
820@@ -2083,15 +2135,10 @@
821 build_conflicts=build_conflicts,
822 build_conflicts_indep=build_conflicts_indep,
823 architecturehintlist=architecturehintlist,
824- changelog_entry=None,
825- dsc=None,
826- copyright=self.getUniqueString(),
827- dscsigningkey=gpg_key,
828- dsc_maintainer_rfc822=maintainer_email,
829 dsc_standards_version=dsc_standards_version,
830 dsc_format=dsc_format,
831 dsc_binaries=dsc_binaries,
832- archive=archive, dateuploaded=date_uploaded)
833+ date_uploaded=date_uploaded)
834
835 sspph = SourcePackagePublishingHistory(
836 distroseries=distroseries,
837
838=== modified file 'lib/lp/translations/doc/translationtemplatesbuildbehavior.txt'
839--- lib/lp/translations/doc/translationtemplatesbuildbehavior.txt 2010-03-05 13:52:32 +0000
840+++ lib/lp/translations/doc/translationtemplatesbuildbehavior.txt 2010-03-15 21:11:07 +0000
841@@ -2,13 +2,21 @@
842
843 == Setup ==
844
845-Set up build environment.
846+Set up build environment. Clear out the build queue.
847
848 >>> import transaction
849 >>> import logging
850 >>> logger = logging.getLogger()
851 >>> logger.setLevel(logging.CRITICAL)
852
853+ >>> from canonical.database.sqlbase import quote
854+ >>> from canonical.launchpad.interfaces.lpstorm import IMasterStore
855+ >>> from lp.services.job.interfaces.job import JobStatus
856+ >>> from lp.services.job.model.job import Job
857+ >>> store = IMasterStore(Job)
858+ >>> query = store.execute("UPDATE Job SET status = %s" % quote(
859+ ... JobStatus.FAILED))
860+
861 >>> from lp.buildmaster.master import BuilddMaster
862 >>> from canonical.buildd.tests import BuilddSlaveTestSetup
863 >>> bm = BuilddMaster(logger, transaction)
864@@ -39,7 +47,21 @@
865
866 Make a builder to process our build request.
867
868- >>> builder = factory.makeBuilder(virtualized=False, processor=processor)
869+ >>> builder = factory.makeBuilder(
870+ ... virtualized=True, processor=processor, vm_host='hostname')
871+
872+The builder doesn't talk to a real slave. We don't have those in our
873+test suite. But we give it a fake one.
874+
875+ >>> from lp.testing.fakemethod import FakeMethod
876+ >>> class FakeSlave:
877+ ... build = FakeMethod()
878+ ... cacheFile = FakeMethod()
879+ ... resume = FakeMethod(result=('Output here', 'Errors here', 0))
880+
881+ >>> from zope.security.proxy import removeSecurityProxy
882+ >>> slave = FakeSlave()
883+ >>> removeSecurityProxy(builder).slave = slave
884
885
886 == Get a job! ==
887@@ -60,20 +82,27 @@
888 >>> print buildqueue.date_started
889 None
890
891-Dispatch the job to the build slave. The proper method to call here
892-would have been Builder.findAndStartJob, but it has not been generalised
893-to handle new job types yet.
894-
895-# XXX JeroenVermeulen bug=506617: call findAndStartJob instead.
896-
897- >>> from zope.security.proxy import removeSecurityProxy
898- >>> removeSecurityProxy(builder)._dispatchBuildCandidate(buildqueue)
899-
900-The build is now marked as started.
901+Our job is now first in line to be executed.
902+
903+ >>> removeSecurityProxy(builder)._findBuildCandidate() == buildqueue
904+ True
905+
906+Dispatch the first job in line (ours!) to the build slave.
907+
908+ >>> activated_job = builder.findAndStartJob()
909+ >>> activated_job == buildqueue
910+ True
911+
912+Our build is now marked as started.
913
914 >>> buildqueue.date_started is None
915 False
916
917+The slave's build method has been called.
918+
919+ >>> slave.build.call_count
920+ 1
921+
922
923 == Teardown ==
924
925
926=== modified file 'lib/lp/translations/model/translationtemplatesbuildbehavior.py'
927--- lib/lp/translations/model/translationtemplatesbuildbehavior.py 2010-02-11 19:11:11 +0000
928+++ lib/lp/translations/model/translationtemplatesbuildbehavior.py 2010-03-15 21:11:07 +0000
929@@ -17,6 +17,7 @@
930
931 from canonical.launchpad.interfaces import ILaunchpadCelebrities
932
933+from lp.buildmaster.interfaces.builder import CorruptBuildID
934 from lp.buildmaster.interfaces.buildfarmjobbehavior import (
935 IBuildFarmJobBehavior)
936 from lp.buildmaster.model.buildfarmjobbehavior import (
937@@ -40,12 +41,27 @@
938 self._builder.slave.cacheFile(logger, chroot)
939 buildid = self.buildfarmjob.getName()
940
941- args = { 'branch_url': self.buildfarmjob.branch.url }
942+ args = self.buildfarmjob.metadata
943 filemap = {}
944
945 self._builder.slave.build(
946 buildid, self.build_type, chroot_sha1, filemap, args)
947
948+ def verifySlaveBuildID(self, slave_build_id):
949+ """See `IBuildFarmJobBehavior`."""
950+ try:
951+ branch_name, queue_item_id = slave_build_id.rsplit('-', 1)
952+ except ValueError:
953+ raise CorruptBuildID(
954+ "Malformed translation templates build id: '%s'" % (
955+ slave_build_id))
956+
957+ buildqueue = self.getVerifiedBuildQueue(queue_item_id)
958+ if buildqueue.job != self.buildfarmjob.job:
959+ raise CorruptBuildID(
960+ "ID mismatch for translation templates build '%s'" % (
961+ slave_build_id))
962+
963 def _getChroot(self):
964 ubuntu = getUtility(ILaunchpadCelebrities).ubuntu
965 return ubuntu.currentseries.nominatedarchindep.getChroot()
966
967=== modified file 'lib/lp/translations/tests/test_translationtemplatesbuildbehavior.py'
968--- lib/lp/translations/tests/test_translationtemplatesbuildbehavior.py 2010-03-12 08:31:43 +0000
969+++ lib/lp/translations/tests/test_translationtemplatesbuildbehavior.py 2010-03-15 21:11:07 +0000
970@@ -19,6 +19,7 @@
971 from lp.buildmaster.interfaces.buildbase import BuildStatus
972 from lp.buildmaster.interfaces.buildfarmjobbehavior import (
973 IBuildFarmJobBehavior)
974+from lp.buildmaster.interfaces.builder import CorruptBuildID
975 from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
976 from lp.testing import TestCaseWithFactory
977 from lp.testing.fakemethod import FakeMethod
978@@ -123,6 +124,10 @@
979 self.assertEqual(
980 'translation-templates', slave_status['test_build_type'])
981 self.assertIn('branch_url', slave_status['test_build_args'])
982+ # The slave receives the public http URL for the branch.
983+ self.assertEqual(
984+ behavior.buildfarmjob.branch.composePublicURL(),
985+ slave_status['test_build_args']['branch_url'])
986
987 def test_getChroot(self):
988 # _getChroot produces the current chroot for the current Ubuntu
989@@ -170,6 +175,38 @@
990 self.assertEqual(1, builder.cleanSlave.call_count)
991 self.assertEqual(0, behavior._uploadTarball.call_count)
992
993+ def test_verifySlaveBuildID_success(self):
994+ # TranslationTemplatesBuildJob.getName generates slave build ids
995+ # that TranslationTemplatesBuildBehavior.verifySlaveBuildID
996+ # accepts.
997+ behavior = self._makeBehavior()
998+ buildfarmjob = behavior.buildfarmjob
999+ job = buildfarmjob.job
1000+
1001+ # The test is that this not raise CorruptBuildID (or anything
1002+ # else, for that matter).
1003+ behavior.verifySlaveBuildID(behavior.buildfarmjob.getName())
1004+
1005+ def test_verifySlaveBuildID_handles_dashes(self):
1006+ # TranslationTemplatesBuildBehavior.verifySlaveBuildID can deal
1007+ # with dashes in branch names.
1008+ behavior = self._makeBehavior()
1009+ buildfarmjob = behavior.buildfarmjob
1010+ job = buildfarmjob.job
1011+ buildfarmjob.branch.name = 'x-y-z--'
1012+
1013+ # The test is that this not raise CorruptBuildID (or anything
1014+ # else, for that matter).
1015+ behavior.verifySlaveBuildID(behavior.buildfarmjob.getName())
1016+
1017+ def test_verifySlaveBuildID_malformed(self):
1018+ behavior = self._makeBehavior()
1019+ self.assertRaises(CorruptBuildID, behavior.verifySlaveBuildID, 'huh?')
1020+
1021+ def test_verifySlaveBuildID_notfound(self):
1022+ behavior = self._makeBehavior()
1023+ self.assertRaises(CorruptBuildID, behavior.verifySlaveBuildID, '1-1')
1024+
1025
1026 def test_suite():
1027 return TestLoader().loadTestsFromName(__name__)
1028
1029=== added file 'utilities/apidoc-index.pt'
1030--- utilities/apidoc-index.pt 1970-01-01 00:00:00 +0000
1031+++ utilities/apidoc-index.pt 2010-03-15 21:11:07 +0000
1032@@ -0,0 +1,37 @@
1033+<html
1034+ xmlns="http://www.w3.org/1999/xhtml"
1035+ xmlns:tal="http://xml.zope.org/namespaces/tal"
1036+ xmlns:metal="http://xml.zope.org/namespaces/metal"
1037+ xmlns:i18n="http://xml.zope.org/namespaces/i18n"
1038+ i18n:domain="launchpad"
1039+>
1040+ <head>
1041+ <title>Launchpad Web Service API</title>
1042+ </head>
1043+
1044+ <body>
1045+
1046+ <h1>Launchpad web service API documentation</h1>
1047+
1048+ <p tal:content="structure config/service_description">
1049+ Description of the Launchpad web service.
1050+ </p>
1051+
1052+ <h2>Active versions</h2>
1053+
1054+ <ul>
1055+ <li tal:repeat="version_and_description versions_and_descriptions">
1056+
1057+ <tal:version tal:define="version version_and_description/version;
1058+ description version_and_description/description">
1059+
1060+ <a tal:attributes="href string:$version.html"
1061+ tal:content="version">Version name</a>:
1062+ <tal:description replace="description" />
1063+ </tal:version>
1064+ </li>
1065+ </ul>
1066+
1067+ </body>
1068+
1069+</html>
1070
1071=== renamed file 'utilities/create-lp-wadl.py' => 'utilities/create-lp-wadl-and-apidoc.py'
1072--- utilities/create-lp-wadl.py 2010-03-01 21:16:06 +0000
1073+++ utilities/create-lp-wadl-and-apidoc.py 2010-03-15 21:11:07 +0000
1074@@ -12,10 +12,15 @@
1075
1076 import _pythonpath
1077
1078+from cStringIO import StringIO
1079+import os
1080+import pkg_resources
1081+import subprocess
1082 import sys
1083 import urlparse
1084
1085 from zope.component import getUtility
1086+from zope.pagetemplate.pagetemplatefile import PageTemplateFile
1087
1088 from canonical.launchpad.ftests import login, ANONYMOUS
1089 from canonical.launchpad.scripts import execute_zcml_for_scripts
1090@@ -30,6 +35,9 @@
1091 execute_zcml_for_scripts()
1092 config = getUtility(IWebServiceConfiguration)
1093
1094+ stylesheet = pkg_resources.resource_filename(
1095+ 'launchpadlib', 'wadl-to-refhtml.xsl')
1096+
1097 # Request the WADL from the root resource.
1098 # We do this by creating a request object asking for a WADL
1099 # representation.
1100@@ -52,6 +60,33 @@
1101 content = request.publication.getApplication(request)(request)
1102 f.write(content)
1103 f.close()
1104+
1105+ # Now, convert the WADL into an human-readable description and
1106+ # put the HTML in the same directory as the WADL.
1107+ directory, ignore = os.path.split(path_template)
1108+ html_filename = os.path.join(directory, version + ".html")
1109+ print "Writing apidoc for version %s to %s" % (
1110+ version, html_filename)
1111+ stdout = open(html_filename, "w")
1112+ subprocess.Popen(['xsltproc', stylesheet, filename], stdout=stdout)
1113+ stdout.close()
1114+
1115+ # Finally, create an index.html with links to all the HTML
1116+ # documentation files we just generated.
1117+ template_file = 'apidoc-index.pt'
1118+ template = PageTemplateFile(template_file)
1119+ namespace = template.pt_getContext()
1120+ namespace['config'] = config
1121+ versions_and_descriptions = []
1122+ for version in config.active_versions:
1123+ versions_and_descriptions.append(
1124+ dict(version=version,
1125+ description=config.version_descriptions[version]))
1126+ namespace['versions_and_descriptions'] = versions_and_descriptions
1127+
1128+ f = open(os.path.join(directory, "index.html"), 'w')
1129+ f.write(template.pt_render(namespace))
1130+
1131 return 0
1132
1133 if __name__ == '__main__':
1134
1135=== modified file 'versions.cfg'
1136--- versions.cfg 2010-03-08 22:26:29 +0000
1137+++ versions.cfg 2010-03-15 21:11:07 +0000
1138@@ -21,15 +21,15 @@
1139 grokcore.component = 1.6
1140 httplib2 = 0.6.0
1141 ipython = 0.9.1
1142-launchpadlib = 1.5.5
1143+launchpadlib = 1.5.6
1144 lazr.authentication = 0.1.1
1145 lazr.batchnavigator = 1.1
1146 lazr.config = 1.1.3
1147 lazr.delegates = 1.1.0
1148 lazr.enum = 1.1.2
1149 lazr.lifecycle = 1.1
1150-lazr.restful = 0.9.21
1151-lazr.restfulclient = 0.9.11
1152+lazr.restful = 0.9.23
1153+lazr.restfulclient = 0.9.12
1154 lazr.smtptest = 1.1
1155 lazr.testing = 0.1.1
1156 lazr.uri = 1.0.2

Subscribers

People subscribed via source and target branches

to status/vote changes: