Merge lp:~leonardr/lazr.restful/multiversion-rename-params into lp:lazr.restful

Proposed by Leonard Richardson
Status: Merged
Approved by: Gary Poster
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~leonardr/lazr.restful/multiversion-rename-params
Merge into: lp:lazr.restful
Diff against target: 355 lines (+226/-84)
2 files modified
src/lazr/restful/declarations.py (+19/-4)
src/lazr/restful/docs/webservice-declarations.txt (+207/-80)
To merge this branch: bzr merge lp:~leonardr/lazr.restful/multiversion-rename-params
Reviewer Review Type Date Requested Status
Gary Poster Approve
Review via email: mp+17741@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Leonard Richardson (leonardr) wrote :

This branch continues my progress towards making all the named operation annotations work in a multi-versioned environment. This branch gets @rename_parameters_as to work by radically simplifying the BleedThroughDict data structure.

Since BleedThroughDict has only been around for a couple days, I'll explain how it works. It's a stack of dicts that acts like a single dict. It's used to gather annotations in a multi-versioned environment: the 1.0 annotations all go into a dict at the bottom of the stack, the 2.0 annotations go into a dict on top of that, the 3.0 annotations go into a dict on top of _that_, and so on.

Annotations for version n+1 are inherited from n unless redefined for version n. For instance, if the name of the method being published as a named operation is 'foo', but in the definition of version 2.0 there's an @export_operation_as("bar"), then the name of the operation is 'foo' (the default) in 1.0, but 'bar' in 2.0 and 3.0.

My original plan was that each dict would only contain the annotations specific to that version, and inheritance would be implemented by moving down the stack until I found the version that defined the appropriate annotation. In the example above, the stack would look like this:

[(3.0, {}),
 (2.0, {'as' : 'bar'}),
 (1.0, {}]

This is elegant, but it only works for annotations that add objects to the annotation dict. The @rename_parameters_as annotation takes an object already in the dict and modifies its __name__ attribute in place. This means that if you have @rename_parameters_as annotations anywhere in your definition, the last one takes precedence, and modifies the name of that parameter for every single version.

So I changedBleedThroughDict to get the same basic behavior in a much cruder way. When you push a new dict onto the stack, it's initialized with a deep copy of the dict that was originally the top of the stack. In the example above, the stack would look like this:

[(3.0, {'as' : 'bar'}),
 (2.0, {'as' : 'bar'}),
 (1.0, {}]

Now a 2.0 @rename_parameters_as will modify only the copy of the parameter objects found on the 2.0 level of the stack. It won't affect the objects on the 1.0 level. The 3.0 level will inherit the change made on the 2.0 level (thanks to the deep copy), but it it contains its own @rename_parameters_as, that declaration won't affect the 2.0 or the 1.0 levels.

The new BleedThroughDict is not nearly as flexible as the old, but we don't need that flexibility. We always define versions in ascending order and we never go back to modify an old version once it's defined.

I'm entertaining ideas about new names for BleedThroughDict, but I think the name is still appropriate (it's just that the bleeding through happens all at once, not whenever you do a lookup).

101. By Leonard Richardson

Renamed BleedThroughDict to VersionedDict in response to feedback.

Revision history for this message
Gary Poster (gary) wrote :

This looks good. I like the simplification.

On IRC I asked how you were going to use this data structure. You said that you would collect the data, then build the top-most version, pop the dict, build the next lower version, pop the dict, and so on. Therefore, this data structure will be very transient, and you don't need an API to access lower versions while the upper versions are still present. That's fine.

At this point, the whole thing could really be accomplished without a new class. Just the ``stack`` list of (name, dict) tuples, and direct calls to deepcopy for a new level in the stack would seem to add very little additional weight in the call sites. I'm tempted to suggest that you go that way, since it would remove code that might have little win, but on balance I'm fine with you seeing how this data structure will work in practice; maybe this class will actually be significantly nicer to work with for your use cases.

For the name, ``VersionedDict`` makes sense to me, particularly with new approach. OTOH, ``BleedThroughDict`` has the vampire-ish connotations that are so in vogue.

Gary

review: Approve
102. By Leonard Richardson

Made sure the cache_for directive works in multiversion.

103. By Leonard Richardson

Implemented operation_removed_in_version.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/lazr/restful/declarations.py'
2--- src/lazr/restful/declarations.py 2010-01-20 15:21:38 +0000
3+++ src/lazr/restful/declarations.py 2010-01-20 21:27:13 +0000
4@@ -299,7 +299,7 @@
5 # in the interface method specification.
6 annotations = method.__dict__.get(LAZR_WEBSERVICE_EXPORTED, None)
7 if annotations is None:
8- # Create a new bleed-through dict which associates
9+ # Create a new versuibed dict which associates
10 # annotation data with the earliest active version of the
11 # web service. Future @webservice_version annotations will
12 # push later versions onto the VersionedDict, allowing
13@@ -475,12 +475,27 @@
14 def annotate_method(self, method, annotations):
15 """See `_method_annotator`."""
16 # The annotations dict is a VersionedDict. Push a new dict
17- # onto its stack, labeled with the version number, so that
18- # future annotations can override old annotations without
19- # destroying them.
20+ # onto its stack, labeled with the version number, and copy in
21+ # the old version's annotations so that this version can
22+ # modify those annotations without destroying them.
23 annotations.push(self.version)
24
25
26+class operation_removed_in_version(operation_for_version):
27+ """Decoration removing this operation from the web service.
28+
29+ This operation will not be present in the given version of the web
30+ service, or any subsequent version, unless it's re-published with
31+ an export_*_operation method.
32+ """
33+ def annotate_method(self, method, annotations):
34+ """See `_method_annotator`."""
35+ # The annotations dict is a VersionedDict. Push a new dict
36+ # onto its stack, labeled with the version number. Make sure the
37+ # new dict is empty rather than copying the old annotations
38+ annotations.push(self.version, True)
39+
40+
41 class export_operation_as(_method_annotator):
42 """Decorator specifying the name to export the method as."""
43
44
45=== modified file 'src/lazr/restful/docs/webservice-declarations.txt'
46--- src/lazr/restful/docs/webservice-declarations.txt 2010-01-20 13:43:44 +0000
47+++ src/lazr/restful/docs/webservice-declarations.txt 2010-01-20 21:27:13 +0000
48@@ -503,85 +503,6 @@
49 >>> param_defs['optional2'].default
50 u'Default2'
51
52-Versioning
53-----------
54-
55-Different versions of the webservice can publish the same interface
56-method in totally different ways. Here's a simple example. This method
57-appears differently in three versions of the web service: 2.0, 1.0,
58-and in an unnamed pre-1.0 version.
59-
60- >>> from lazr.restful.declarations import operation_for_version
61- >>> class MultiVersionMethod(Interface):
62- ... export_as_webservice_entry()
63- ...
64- ... @call_with(fixed='2.0 value')
65- ... @operation_for_version('2.0')
66- ...
67- ... @call_with(fixed='1.0 value')
68- ... @export_operation_as('new_name')
69- ... @rename_parameters_as(required="required_argument")
70- ... @operation_for_version('1.0')
71- ...
72- ... @call_with(fixed='pre-1.0 value')
73- ... @operation_parameters(
74- ... required=TextLine(),
75- ... fixed=TextLine()
76- ... )
77- ... @export_read_operation()
78- ... def a_method(required, fixed='Fixed value'):
79- ... """Method demonstrating multiversion publication."""
80-
81-The tagged value containing the annotations looks like a dictionary,
82-but it's actually a stack of dictionaries named after the versions.
83-
84- >>> dictionary = MultiVersionMethod['a_method'].getTaggedValue(
85- ... 'lazr.restful.exported')
86- >>> dictionary.dict_names
87- [None, '1.0', '2.0']
88-
89-The dictionary on top of the stack is for the 2.0 version of the web
90-service. In 2.0, the method is published as 'new_name' and its 'fixed'
91-argument is fixed to the string '2.0 value'.
92-
93- >>> print dictionary['as']
94- new_name
95- >>> dictionary['call_with']
96- {'fixed': '2.0 value'}
97-
98-The published name of the 'required' argument is 'required_argument',
99-not 'required'.
100-
101- >>> print dictionary['params']['required'].__name__
102- required_argument
103-
104-Let's pop the 2.0 version off the stack. Now we can see how the method
105-looks in 1.0. It's still called 'new_name', and its 'required'
106-argument is still called 'required_argument', but its 'fixed' argument
107-is fixed to the string '1.0 value'.
108-
109- >>> ignored = dictionary.pop()
110- >>> print dictionary['as']
111- new_name
112- >>> dictionary['call_with']
113- {'fixed': '1.0 value'}
114- >>> print dictionary['params']['required'].__name__
115- required_argument
116-
117-Let's pop one more time to see how the method looks in the pre-1.0
118-version. It hasn't yet been renamed to 'new_name', its 'required'
119-argument hasn't yet been renamed to 'required_argument', and its
120-'fixed' argument is fixed to the string 'pre-1.0 value'.
121-
122- >>> ignored = dictionary.pop()
123- >>> print dictionary.get('as')
124- None
125- >>> print dictionary['params']['required'].__name__
126- required
127- >>> dictionary['call_with']
128- {'fixed': 'pre-1.0 value'}
129-
130-
131 Error handling
132 --------------
133
134@@ -1526,8 +1447,8 @@
135 It is possible to cache a server response in the browser cache using
136 the @cache_for decorator:
137
138+ >>> from lazr.restful.testing.webservice import FakeRequest
139 >>> from lazr.restful.declarations import cache_for
140- >>> from lazr.restful.testing.webservice import FakeRequest
141 >>>
142 >>> class ICachedBookSet(IBookSet):
143 ... """IBookSet supporting caching."""
144@@ -1577,6 +1498,212 @@
145 ...
146 ValueError: Caching duration should be a positive number: -15
147
148+Versioned services
149+==================
150+
151+Different versions of the webservice can publish the same data model
152+object in totally different ways.
153+
154+Named operations
155+----------------
156+
157+It's easy to reflect the most common changes between versions:
158+operations and arguments being renamed, changes in fixed values, etc.
159+This method appears differently in three versions of the web service:
160+2.0, 1.0, and in an unnamed pre-1.0 version.
161+
162+ >>> from lazr.restful.declarations import operation_for_version
163+ >>> class MultiVersionMethod(Interface):
164+ ... export_as_webservice_entry()
165+ ...
166+ ... @call_with(fixed='2.0 value')
167+ ... @cache_for(300)
168+ ... @operation_for_version('2.0')
169+ ...
170+ ... @call_with(fixed='1.0 value')
171+ ... @export_operation_as('new_name')
172+ ... @rename_parameters_as(required="required_argument")
173+ ... @operation_for_version('1.0')
174+ ...
175+ ... @call_with(fixed='pre-1.0 value')
176+ ... @cache_for(100)
177+ ... @operation_parameters(
178+ ... required=TextLine(),
179+ ... fixed=TextLine()
180+ ... )
181+ ... @export_read_operation()
182+ ... def a_method(required, fixed='Fixed value'):
183+ ... """Method demonstrating multiversion publication."""
184+
185+The tagged value containing the annotations looks like a dictionary,
186+but it's actually a stack of dictionaries named after the versions.
187+
188+ >>> dictionary = MultiVersionMethod['a_method'].getTaggedValue(
189+ ... 'lazr.restful.exported')
190+ >>> dictionary.dict_names
191+ [None, '1.0', '2.0']
192+
193+The dictionary on top of the stack is for the 2.0 version of the web
194+service. In 2.0, the method is published as 'new_name' and its 'fixed'
195+argument is fixed to the string '2.0 value'.
196+
197+ >>> print dictionary['as']
198+ new_name
199+ >>> dictionary['call_with']
200+ {'fixed': '2.0 value'}
201+ >>> dictionary['cache_for']
202+ 300
203+
204+The published name of the 'required' argument is 'required_argument',
205+not 'required'.
206+
207+ >>> print dictionary['params']['required'].__name__
208+ required_argument
209+
210+Let's pop the 2.0 version off the stack. Now we can see how the method
211+looks in 1.0. It's still called 'new_name', and its 'required'
212+argument is still called 'required_argument', but its 'fixed' argument
213+is fixed to the string '1.0 value'.
214+
215+ >>> ignored = dictionary.pop()
216+ >>> print dictionary['as']
217+ new_name
218+ >>> dictionary['call_with']
219+ {'fixed': '1.0 value'}
220+ >>> print dictionary['params']['required'].__name__
221+ required_argument
222+ >>> dictionary['cache_for']
223+ 100
224+
225+Let's pop one more time to see how the method looks in the pre-1.0
226+version. It hasn't yet been renamed to 'new_name', its 'required'
227+argument hasn't yet been renamed to 'required_argument', and its
228+'fixed' argument is fixed to the string 'pre-1.0 value'.
229+
230+ >>> ignored = dictionary.pop()
231+ >>> print dictionary.get('as')
232+ None
233+ >>> print dictionary['params']['required'].__name__
234+ required
235+ >>> dictionary['call_with']
236+ {'fixed': 'pre-1.0 value'}
237+ >>> dictionary['cache_for']
238+ 100
239+
240+@operation_removed_in_version
241+*****************************
242+
243+Sometimes you want version n+1 to remove a named operation that was
244+present in version n. The @operation_removed_in_version declaration
245+does just this.
246+
247+Let's define an operation that's introduced in 1.0 and removed in 2.0.
248+
249+ >>> from lazr.restful.declarations import operation_removed_in_version
250+ >>> class DisappearingMultiversionMethod(Interface):
251+ ... export_as_webservice_entry()
252+ ... @operation_removed_in_version(2.0)
253+ ... @operation_parameters(arg=Float())
254+ ... @export_read_operation()
255+ ... @operation_for_version(1.0)
256+ ... def method(arg):
257+ ... """A doomed method."""
258+
259+ >>> dictionary = DisappearingMultiversionMethod[
260+ ... 'method'].getTaggedValue('lazr.restful.exported')
261+
262+The method is not present in 2.0:
263+
264+ >>> version, attrs = dictionary.pop()
265+ >>> print version
266+ 2.0
267+ >>> sorted(attrs.items())
268+ []
269+
270+It is present in 1.0:
271+
272+ >>> version, attrs = dictionary.pop()
273+ >>> print version
274+ 1.0
275+ >>> print attrs['type']
276+ read_operation
277+ >>> print attrs['params']['arg']
278+ <zope.schema._field.Float object...>
279+
280+But it's not present in the unnamed pre-1.0 version, since it hadn't
281+been defined yet:
282+
283+ >>> print dictionary.pop()
284+ (None, {})
285+
286+The @operation_removed_in_version declaration can also be used to
287+reset a named operation's definition if you need to completely re-do
288+it.
289+
290+For instance, ordinarily you can't change the type of an operation, or
291+totally redefine its parameters--and you shouldn't really need
292+to. It's usually easier to publish two different operations that have
293+the same name in different versions. But you can do it with a single
294+operation, by removing the operation with
295+@operation_removed_in_version and defining it again--either in the
296+same version or in some later version.
297+
298+In this example, the type of the operation, the type and number of the
299+arguments, and the return value change in version 1.0.
300+
301+ >>> class ReadOrWriteMethod(Interface):
302+ ... export_as_webservice_entry()
303+ ...
304+ ... @operation_parameters(arg=TextLine(), arg2=TextLine())
305+ ... @export_write_operation()
306+ ... @operation_removed_in_version(1.0)
307+ ...
308+ ... @operation_parameters(arg=Float())
309+ ... @operation_returns_collection_of(Interface)
310+ ... @export_read_operation()
311+ ... def method(arg, arg2='default'):
312+ ... """A read *or* a write operation, depending on version."""
313+
314+ >>> dictionary = ReadOrWriteMethod[
315+ ... 'method'].getTaggedValue('lazr.restful.exported')
316+
317+In version 1.0, the 'method' named operation is a write operation that
318+takes two TextLine arguments and has no special return value.
319+
320+ >>> version, attrs = dictionary.pop()
321+ >>> print version
322+ 1.0
323+ >>> print attrs['type']
324+ write_operation
325+ >>> attrs['params']['arg']
326+ <zope.schema._bootstrapfields.TextLine object...>
327+ >>> attrs['params']['arg2']
328+ <zope.schema._bootstrapfields.TextLine object...>
329+ >>> print attrs.get('return_type')
330+ None
331+
332+In the unnamed pre-1.0 version, the 'method' operation is a read
333+operation that takes a single Float argument and returns a collection.
334+
335+ >>> version, attrs = dictionary.pop()
336+ >>> print attrs['type']
337+ read_operation
338+
339+ >>> attrs['params']['arg']
340+ <zope.schema._field.Float object...>
341+ >>> attrs['params'].keys()
342+ ['arg']
343+
344+ >>> attrs['return_type']
345+ <lazr.restful.fields.CollectionField object...>
346+
347+[XXX leonardr mutator_for modifies the field, not the method, so it
348+won't work until I add multiversion support for fields. Also, it's
349+possible to remove a named operation from a certain version and then
350+uselessly annotate it some more. The best place to catch this error
351+turns out to be when generating the adapter classes, not when
352+collecting annotations.]
353+
354 Security
355 ========
356

Subscribers

People subscribed via source and target branches