lazr.restful

Merge lp:~leonardr/lazr.restful/integrate-optimized-len into lp:lazr.restful

  1. integrate-optimized-len
  2. Merge into trunk
Proposed by Leonard Richardson
Status: Merged
Approved by: Edwin Grubbs
Approved revision: 142
Merged at revision: 140
Proposed branch: lp:~leonardr/lazr.restful/integrate-optimized-len
Merge into: lp:lazr.restful
Diff against target: 283 lines (+90/-20)
9 files modified
src/lazr/restful/NEWS.txt (+7/-0)
src/lazr/restful/_resource.py (+13/-4)
src/lazr/restful/docs/multiversion.txt (+10/-5)
src/lazr/restful/example/base/tests/collection.txt (+28/-3)
src/lazr/restful/example/multiversion/root.py (+2/-0)
src/lazr/restful/example/multiversion/tests/introduction.txt (+12/-0)
src/lazr/restful/example/multiversion/tests/operation.txt (+16/-6)
src/lazr/restful/version.txt (+1/-1)
versions.cfg (+1/-1)
To merge this branch: bzr merge lp:~leonardr/lazr.restful/integrate-optimized-len
Reviewer Review Type Date Requested Status
Edwin Grubbs (community) code Approve
Review via email: mp+33023@code.launchpad.net

Description of the change

Thanks to an optimization in lazr.batchnavigator, it's now very cheap to determine the total size of a list if the current batch is the last one in the list. It's also cheap to determine whether the current batch is the last one or not. (The most interesting case for our purposes is when the entire list fits in one batch, in which case the first batch is also the last.)

This branch backs lazr.restful away a little bit from its absolute insistence on sending 'total_size_link' instead of 'total_size' in responses to named operations. If the total size of the list turns out to be cheap to calculate, lazr.restful calculates it and sends 'total_size' instead of 'total_size_link'.

Most of this branch consists of test changes: creating scenarios in which the optimization doesn't apply, and showing that total_size_link is sent; creating scenarios in which the optimization does apply, and showing that total_size is sent instead. In one case this required creating extra sample data to pad out a collection returned from a named operation.

To post a comment you must log in.
lp:~leonardr/lazr.restful/integrate-optimized-len updated
141. By Leonard Richardson

updated NEWS.

142. By Leonard Richardson

Use total(), which is part of the public interface, instead of .listlength.

Revision history for this message
Edwin Grubbs (edwin-grubbs) wrote :

Hi Leonard,

This looks good.

-Edwin

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/lazr/restful/NEWS.txt'
2--- src/lazr/restful/NEWS.txt 2010-08-13 14:31:39 +0000
3+++ src/lazr/restful/NEWS.txt 2010-08-18 16:44:44 +0000
4@@ -2,6 +2,13 @@
5 NEWS for lazr.restful
6 =====================
7
8+0.11.2 (Unreleased)
9+===================
10+
11+Optimized lazr.restful to send 'total_size' instead of
12+'total_size_link' when 'total_size' is easy to calculate, possibly
13+saving the client from sending another HTTP request.
14+
15 0.11.1 (2010-08-13)
16 ===================
17
18
19=== modified file 'src/lazr/restful/_resource.py'
20--- src/lazr/restful/_resource.py 2010-08-09 20:36:57 +0000
21+++ src/lazr/restful/_resource.py 2010-08-18 16:44:44 +0000
22@@ -640,13 +640,22 @@
23
24 view_permission = getUtility(IWebServiceConfiguration).view_permission
25 batch = { 'start' : navigator.batch.start }
26- total_size_link = self.total_size_link(navigator)
27+ # If this is the last batch, then total() is easy to
28+ # calculate. Let's use it and save the client from having to
29+ # ask for the total size of the collection.
30+ if navigator.batch.nextBatch() is None:
31+ total_size_link = None
32+ else:
33+ total_size_link = self.total_size_link(navigator)
34 if total_size_link is None:
35- batch['total_size'] = navigator.batch.listlength
36+ total_size = navigator.batch.total()
37+ if total_size < 0:
38+ # lazr.batchnavigator uses -1 for an empty list.
39+ # We want to use 0.
40+ total_size = 0
41+ batch['total_size'] = total_size
42 else:
43 batch['total_size_link'] = total_size_link
44- if navigator.batch.start < 0:
45- batch['start'] = None
46 next_url = navigator.nextBatchURL()
47 if next_url != "":
48 batch['next_collection_link'] = next_url
49
50=== modified file 'src/lazr/restful/docs/multiversion.txt'
51--- src/lazr/restful/docs/multiversion.txt 2010-08-13 14:29:52 +0000
52+++ src/lazr/restful/docs/multiversion.txt 2010-08-18 16:44:44 +0000
53@@ -857,11 +857,11 @@
54
55 >>> request_10 = create_web_service_request(
56 ... '/1.0/contacts',
57- ... environ={'QUERY_STRING' : 'ws.op=find&string=Cleo'})
58+ ... environ={'QUERY_STRING' : 'ws.op=find&string=e&ws.size=2'})
59 >>> operation = request_10.traverse(None)
60 >>> result = simplejson.loads(operation())
61 >>> [contact['name'] for contact in result['entries']]
62- ['Cleo Python']
63+ ['Cleo Python', 'Oliver Bluth']
64
65 >>> result['total_size']
66 Traceback (most recent call last):
67@@ -869,15 +869,18 @@
68 KeyError: 'total_size'
69
70 >>> print result['total_size_link']
71- http://.../1.0/contacts?string=Cleo&ws.op=find&ws.show=total_size
72+ http://.../1.0/contacts?string=e&ws.op=find&ws.show=total_size
73 >>> size_request = create_web_service_request(
74 ... '/1.0/contacts',
75 ... environ={'QUERY_STRING' :
76- ... 'string=Cleo&ws.op=find&ws.show=total_size'})
77+ ... 'string=e&ws.op=find&ws.show=total_size'})
78 >>> operation = size_request.traverse(None)
79 >>> result = simplejson.loads(operation())
80 >>> print result
81- 1
82+ 3
83+
84+If the resultset fits on a single page, total_size will be provided
85+instead of total_size_link, as a convenience.
86
87 >>> request_10 = create_web_service_request(
88 ... '/1.0/contacts',
89@@ -886,6 +889,8 @@
90 >>> result = simplejson.loads(operation())
91 >>> [contact['fax_number'] for contact in result['entries']]
92 ['111-2121']
93+ >>> result['total_size']
94+ 1
95
96 Attempting to invoke the operation using its 'beta' name won't work.
97
98
99=== modified file 'src/lazr/restful/example/base/tests/collection.txt'
100--- src/lazr/restful/example/base/tests/collection.txt 2010-08-10 11:38:06 +0000
101+++ src/lazr/restful/example/base/tests/collection.txt 2010-08-18 16:44:44 +0000
102@@ -209,6 +209,32 @@
103 >>> print webservice.get(s_recipes['total_size_link']).jsonBody()
104 3
105
106+If the entire collection fits in a single 'page' of results, the
107+'total_size_link' is not present; instead, lazr.restful provides the
108+total size as a convenience to the client.
109+
110+ >>> full_list = search_recipes("chicken", size=100)
111+ >>> len(full_list['entries'])
112+ 3
113+ >>> full_list['total_size']
114+ 3
115+ >>> full_list['total_size_link']
116+ Traceback (most recent call last):
117+ ...
118+ KeyError: 'total_size_link'
119+
120+The same is true if the client requests the last page of a list.
121+
122+ >>> last_page = search_recipes("chicken", start=2, size=2)
123+ >>> len(last_page['entries'])
124+ 1
125+ >>> full_list['total_size']
126+ 3
127+ >>> full_list['total_size_link']
128+ Traceback (most recent call last):
129+ ...
130+ KeyError: 'total_size_link'
131+
132 Custom operations may have error handling. In this case, the error
133 handling is in the validate() method of the 'search' field.
134
135@@ -224,9 +250,8 @@
136
137 >>> general_cookbooks = webservice.get(
138 ... "/cookbooks?ws.op=find_for_cuisine&cuisine=General")
139- >>> print general_cookbooks.jsonBody()['total_size_link']
140- http://cookbooks.dev/devel/cookbooks?cuisine=General&ws.op=find_for_cuisine&ws.show=total_size
141-
142+ >>> print general_cookbooks.jsonBody()['total_size']
143+ 3
144
145 POST operations
146 ===============
147
148=== modified file 'src/lazr/restful/example/multiversion/root.py'
149--- src/lazr/restful/example/multiversion/root.py 2010-08-13 14:29:52 +0000
150+++ src/lazr/restful/example/multiversion/root.py 2010-08-18 16:44:44 +0000
151@@ -47,6 +47,8 @@
152 pairset = PairSet()
153 pairset.pairs = [
154 KeyValuePair(pairset, "foo", "bar"),
155+ KeyValuePair(pairset, "foo2", "bar"),
156+ KeyValuePair(pairset, "foo3", "bar"),
157 KeyValuePair(pairset, "1", "2"),
158 KeyValuePair(pairset, "Some", None),
159 KeyValuePair(pairset, "Delete", "me"),
160
161=== modified file 'src/lazr/restful/example/multiversion/tests/introduction.txt'
162--- src/lazr/restful/example/multiversion/tests/introduction.txt 2010-08-13 14:29:52 +0000
163+++ src/lazr/restful/example/multiversion/tests/introduction.txt 2010-08-18 16:44:44 +0000
164@@ -86,12 +86,16 @@
165 Also delete: me
166 Delete: me
167 foo: bar
168+ foo2: bar
169+ foo3: bar
170
171 >>> show_pairs('1.0')
172 1: 2
173 Also delete: me
174 Delete: me
175 foo: bar
176+ foo2: bar
177+ foo3: bar
178
179 >>> show_pairs('2.0')
180 1: 2
181@@ -99,6 +103,8 @@
182 Delete: me
183 Some: None
184 foo: bar
185+ foo2: bar
186+ foo3: bar
187
188 >>> show_pairs('3.0')
189 1: 2
190@@ -106,6 +112,8 @@
191 Delete: me
192 Some: None
193 foo: bar
194+ foo2: bar
195+ foo3: bar
196
197 Entries
198 =======
199@@ -199,6 +207,8 @@
200 1: 2
201 Also delete: me
202 foo: bar
203+ foo2: bar
204+ foo3: bar
205
206 In '3.0', deleting a key-value pair simply sets its 'deleted' field
207 to True. (This is an abuse of the HTTP DELETE method, but it makes a
208@@ -217,6 +227,8 @@
209 1: 2
210 Also delete: me
211 foo: bar
212+ foo2: bar
213+ foo3: bar
214
215 And in a version which publishes the 'delete' field, we can check the
216 key-value pair's value for that field.
217
218=== modified file 'src/lazr/restful/example/multiversion/tests/operation.txt'
219--- src/lazr/restful/example/multiversion/tests/operation.txt 2010-08-13 14:29:52 +0000
220+++ src/lazr/restful/example/multiversion/tests/operation.txt 2010-08-18 16:44:44 +0000
221@@ -24,26 +24,36 @@
222 >>> print config.first_version_with_total_size_link
223 2.0
224
225-When the 'byValue' operation is invoked in version 1.0, it returns a
226+When the 'byValue' operation is invoked in version 1.0, it always returns a
227 total_size.
228
229- >>> def get_collection(version, op='byValue'):
230- ... url = '/pairs?ws.op=%s&value=bar' % op
231+ >>> def get_collection(version, op='byValue', value="bar", size=2):
232+ ... url = '/pairs?ws.op=%s&value=%s&ws.size=%s' % (op, value, size)
233 ... return webservice.get(url, api_version=version).jsonBody()
234
235 >>> print sorted(get_collection('1.0').keys())
236- [u'entries', u'start', u'total_size']
237+ [u'entries', u'next_collection_link', u'start', u'total_size']
238
239 The operation itself doesn't change between 1.0 and 2.0, but in
240 version 2.0, the operation starts returning total_size_link.
241
242 >>> print sorted(get_collection('2.0').keys())
243- [u'entries', u'start', u'total_size_link']
244+ [u'entries', u'next_collection_link', u'start', u'total_size_link']
245
246 The same happens in 3.0.
247
248 >>> print sorted(get_collection('3.0', 'by_value').keys())
249- [u'entries', u'start', u'total_size_link']
250+ [u'entries', u'next_collection_link', u'start', u'total_size_link']
251+
252+However, if the total size is easy to calculate (for instance, because
253+all the results fit on one page), a total_size is returned instead of
254+total_size_link.
255+
256+ >>> print sorted(get_collection('3.0', 'by_value', size=100).keys())
257+ [u'entries', u'start', u'total_size']
258+
259+ >>> print get_collection('3.0', 'by_value', 'no-such-value')
260+ {u'total_size': 0, u'start': 0, u'entries': []}
261
262 Mutators as named operations
263 ----------------------------
264
265=== modified file 'src/lazr/restful/version.txt'
266--- src/lazr/restful/version.txt 2010-08-13 14:31:39 +0000
267+++ src/lazr/restful/version.txt 2010-08-18 16:44:44 +0000
268@@ -1,1 +1,1 @@
269-0.11.1
270+0.11.2
271
272=== modified file 'versions.cfg'
273--- versions.cfg 2010-08-10 18:36:09 +0000
274+++ versions.cfg 2010-08-18 16:44:44 +0000
275@@ -16,7 +16,7 @@
276 docutils = 0.5
277 epydoc = 3.0.1
278 grokcore.component = 1.6
279-lazr.batchnavigator = 1.2.0
280+lazr.batchnavigator = 1.2.2
281 lazr.delegates = 1.2.0
282 lazr.enum = 1.1.2
283 lazr.lifecycle = 1.0

Subscribers

People subscribed via source and target branches