Merge lp:~leonardr/lazr.restful/multiversion-rename-params into lp:lazr.restful

Proposed by Leonard Richardson
Status: Merged
Approved by: Gary Poster
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~leonardr/lazr.restful/multiversion-rename-params
Merge into: lp:lazr.restful
Diff against target: 355 lines (+226/-84)
2 files modified
src/lazr/restful/declarations.py (+19/-4)
src/lazr/restful/docs/webservice-declarations.txt (+207/-80)
To merge this branch: bzr merge lp:~leonardr/lazr.restful/multiversion-rename-params
Reviewer Review Type Date Requested Status
Gary Poster Approve
Review via email: mp+17741@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Leonard Richardson (leonardr) wrote :

This branch continues my progress towards making all the named operation annotations work in a multi-versioned environment. This branch gets @rename_parameters_as to work by radically simplifying the BleedThroughDict data structure.

Since BleedThroughDict has only been around for a couple days, I'll explain how it works. It's a stack of dicts that acts like a single dict. It's used to gather annotations in a multi-versioned environment: the 1.0 annotations all go into a dict at the bottom of the stack, the 2.0 annotations go into a dict on top of that, the 3.0 annotations go into a dict on top of _that_, and so on.

Annotations for version n+1 are inherited from n unless redefined for version n. For instance, if the name of the method being published as a named operation is 'foo', but in the definition of version 2.0 there's an @export_operation_as("bar"), then the name of the operation is 'foo' (the default) in 1.0, but 'bar' in 2.0 and 3.0.

My original plan was that each dict would only contain the annotations specific to that version, and inheritance would be implemented by moving down the stack until I found the version that defined the appropriate annotation. In the example above, the stack would look like this:

[(3.0, {}),
 (2.0, {'as' : 'bar'}),
 (1.0, {}]

This is elegant, but it only works for annotations that add objects to the annotation dict. The @rename_parameters_as annotation takes an object already in the dict and modifies its __name__ attribute in place. This means that if you have @rename_parameters_as annotations anywhere in your definition, the last one takes precedence, and modifies the name of that parameter for every single version.

So I changedBleedThroughDict to get the same basic behavior in a much cruder way. When you push a new dict onto the stack, it's initialized with a deep copy of the dict that was originally the top of the stack. In the example above, the stack would look like this:

[(3.0, {'as' : 'bar'}),
 (2.0, {'as' : 'bar'}),
 (1.0, {}]

Now a 2.0 @rename_parameters_as will modify only the copy of the parameter objects found on the 2.0 level of the stack. It won't affect the objects on the 1.0 level. The 3.0 level will inherit the change made on the 2.0 level (thanks to the deep copy), but it it contains its own @rename_parameters_as, that declaration won't affect the 2.0 or the 1.0 levels.

The new BleedThroughDict is not nearly as flexible as the old, but we don't need that flexibility. We always define versions in ascending order and we never go back to modify an old version once it's defined.

I'm entertaining ideas about new names for BleedThroughDict, but I think the name is still appropriate (it's just that the bleeding through happens all at once, not whenever you do a lookup).

101. By Leonard Richardson

Renamed BleedThroughDict to VersionedDict in response to feedback.

Revision history for this message
Gary Poster (gary) wrote :

This looks good. I like the simplification.

On IRC I asked how you were going to use this data structure. You said that you would collect the data, then build the top-most version, pop the dict, build the next lower version, pop the dict, and so on. Therefore, this data structure will be very transient, and you don't need an API to access lower versions while the upper versions are still present. That's fine.

At this point, the whole thing could really be accomplished without a new class. Just the ``stack`` list of (name, dict) tuples, and direct calls to deepcopy for a new level in the stack would seem to add very little additional weight in the call sites. I'm tempted to suggest that you go that way, since it would remove code that might have little win, but on balance I'm fine with you seeing how this data structure will work in practice; maybe this class will actually be significantly nicer to work with for your use cases.

For the name, ``VersionedDict`` makes sense to me, particularly with new approach. OTOH, ``BleedThroughDict`` has the vampire-ish connotations that are so in vogue.

Gary

review: Approve
102. By Leonard Richardson

Made sure the cache_for directive works in multiversion.

103. By Leonard Richardson

Implemented operation_removed_in_version.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'src/lazr/restful/declarations.py'
--- src/lazr/restful/declarations.py 2010-01-20 15:21:38 +0000
+++ src/lazr/restful/declarations.py 2010-01-20 21:27:13 +0000
@@ -299,7 +299,7 @@
299 # in the interface method specification.299 # in the interface method specification.
300 annotations = method.__dict__.get(LAZR_WEBSERVICE_EXPORTED, None)300 annotations = method.__dict__.get(LAZR_WEBSERVICE_EXPORTED, None)
301 if annotations is None:301 if annotations is None:
302 # Create a new bleed-through dict which associates302 # Create a new versuibed dict which associates
303 # annotation data with the earliest active version of the303 # annotation data with the earliest active version of the
304 # web service. Future @webservice_version annotations will304 # web service. Future @webservice_version annotations will
305 # push later versions onto the VersionedDict, allowing305 # push later versions onto the VersionedDict, allowing
@@ -475,12 +475,27 @@
475 def annotate_method(self, method, annotations):475 def annotate_method(self, method, annotations):
476 """See `_method_annotator`."""476 """See `_method_annotator`."""
477 # The annotations dict is a VersionedDict. Push a new dict477 # The annotations dict is a VersionedDict. Push a new dict
478 # onto its stack, labeled with the version number, so that478 # onto its stack, labeled with the version number, and copy in
479 # future annotations can override old annotations without479 # the old version's annotations so that this version can
480 # destroying them.480 # modify those annotations without destroying them.
481 annotations.push(self.version)481 annotations.push(self.version)
482482
483483
484class operation_removed_in_version(operation_for_version):
485 """Decoration removing this operation from the web service.
486
487 This operation will not be present in the given version of the web
488 service, or any subsequent version, unless it's re-published with
489 an export_*_operation method.
490 """
491 def annotate_method(self, method, annotations):
492 """See `_method_annotator`."""
493 # The annotations dict is a VersionedDict. Push a new dict
494 # onto its stack, labeled with the version number. Make sure the
495 # new dict is empty rather than copying the old annotations
496 annotations.push(self.version, True)
497
498
484class export_operation_as(_method_annotator):499class export_operation_as(_method_annotator):
485 """Decorator specifying the name to export the method as."""500 """Decorator specifying the name to export the method as."""
486501
487502
=== modified file 'src/lazr/restful/docs/webservice-declarations.txt'
--- src/lazr/restful/docs/webservice-declarations.txt 2010-01-20 13:43:44 +0000
+++ src/lazr/restful/docs/webservice-declarations.txt 2010-01-20 21:27:13 +0000
@@ -503,85 +503,6 @@
503 >>> param_defs['optional2'].default503 >>> param_defs['optional2'].default
504 u'Default2'504 u'Default2'
505505
506Versioning
507----------
508
509Different versions of the webservice can publish the same interface
510method in totally different ways. Here's a simple example. This method
511appears differently in three versions of the web service: 2.0, 1.0,
512and in an unnamed pre-1.0 version.
513
514 >>> from lazr.restful.declarations import operation_for_version
515 >>> class MultiVersionMethod(Interface):
516 ... export_as_webservice_entry()
517 ...
518 ... @call_with(fixed='2.0 value')
519 ... @operation_for_version('2.0')
520 ...
521 ... @call_with(fixed='1.0 value')
522 ... @export_operation_as('new_name')
523 ... @rename_parameters_as(required="required_argument")
524 ... @operation_for_version('1.0')
525 ...
526 ... @call_with(fixed='pre-1.0 value')
527 ... @operation_parameters(
528 ... required=TextLine(),
529 ... fixed=TextLine()
530 ... )
531 ... @export_read_operation()
532 ... def a_method(required, fixed='Fixed value'):
533 ... """Method demonstrating multiversion publication."""
534
535The tagged value containing the annotations looks like a dictionary,
536but it's actually a stack of dictionaries named after the versions.
537
538 >>> dictionary = MultiVersionMethod['a_method'].getTaggedValue(
539 ... 'lazr.restful.exported')
540 >>> dictionary.dict_names
541 [None, '1.0', '2.0']
542
543The dictionary on top of the stack is for the 2.0 version of the web
544service. In 2.0, the method is published as 'new_name' and its 'fixed'
545argument is fixed to the string '2.0 value'.
546
547 >>> print dictionary['as']
548 new_name
549 >>> dictionary['call_with']
550 {'fixed': '2.0 value'}
551
552The published name of the 'required' argument is 'required_argument',
553not 'required'.
554
555 >>> print dictionary['params']['required'].__name__
556 required_argument
557
558Let's pop the 2.0 version off the stack. Now we can see how the method
559looks in 1.0. It's still called 'new_name', and its 'required'
560argument is still called 'required_argument', but its 'fixed' argument
561is fixed to the string '1.0 value'.
562
563 >>> ignored = dictionary.pop()
564 >>> print dictionary['as']
565 new_name
566 >>> dictionary['call_with']
567 {'fixed': '1.0 value'}
568 >>> print dictionary['params']['required'].__name__
569 required_argument
570
571Let's pop one more time to see how the method looks in the pre-1.0
572version. It hasn't yet been renamed to 'new_name', its 'required'
573argument hasn't yet been renamed to 'required_argument', and its
574'fixed' argument is fixed to the string 'pre-1.0 value'.
575
576 >>> ignored = dictionary.pop()
577 >>> print dictionary.get('as')
578 None
579 >>> print dictionary['params']['required'].__name__
580 required
581 >>> dictionary['call_with']
582 {'fixed': 'pre-1.0 value'}
583
584
585Error handling506Error handling
586--------------507--------------
587508
@@ -1526,8 +1447,8 @@
1526It is possible to cache a server response in the browser cache using1447It is possible to cache a server response in the browser cache using
1527the @cache_for decorator:1448the @cache_for decorator:
15281449
1450 >>> from lazr.restful.testing.webservice import FakeRequest
1529 >>> from lazr.restful.declarations import cache_for1451 >>> from lazr.restful.declarations import cache_for
1530 >>> from lazr.restful.testing.webservice import FakeRequest
1531 >>>1452 >>>
1532 >>> class ICachedBookSet(IBookSet):1453 >>> class ICachedBookSet(IBookSet):
1533 ... """IBookSet supporting caching."""1454 ... """IBookSet supporting caching."""
@@ -1577,6 +1498,212 @@
1577 ...1498 ...
1578 ValueError: Caching duration should be a positive number: -151499 ValueError: Caching duration should be a positive number: -15
15791500
1501Versioned services
1502==================
1503
1504Different versions of the webservice can publish the same data model
1505object in totally different ways.
1506
1507Named operations
1508----------------
1509
1510It's easy to reflect the most common changes between versions:
1511operations and arguments being renamed, changes in fixed values, etc.
1512This method appears differently in three versions of the web service:
15132.0, 1.0, and in an unnamed pre-1.0 version.
1514
1515 >>> from lazr.restful.declarations import operation_for_version
1516 >>> class MultiVersionMethod(Interface):
1517 ... export_as_webservice_entry()
1518 ...
1519 ... @call_with(fixed='2.0 value')
1520 ... @cache_for(300)
1521 ... @operation_for_version('2.0')
1522 ...
1523 ... @call_with(fixed='1.0 value')
1524 ... @export_operation_as('new_name')
1525 ... @rename_parameters_as(required="required_argument")
1526 ... @operation_for_version('1.0')
1527 ...
1528 ... @call_with(fixed='pre-1.0 value')
1529 ... @cache_for(100)
1530 ... @operation_parameters(
1531 ... required=TextLine(),
1532 ... fixed=TextLine()
1533 ... )
1534 ... @export_read_operation()
1535 ... def a_method(required, fixed='Fixed value'):
1536 ... """Method demonstrating multiversion publication."""
1537
1538The tagged value containing the annotations looks like a dictionary,
1539but it's actually a stack of dictionaries named after the versions.
1540
1541 >>> dictionary = MultiVersionMethod['a_method'].getTaggedValue(
1542 ... 'lazr.restful.exported')
1543 >>> dictionary.dict_names
1544 [None, '1.0', '2.0']
1545
1546The dictionary on top of the stack is for the 2.0 version of the web
1547service. In 2.0, the method is published as 'new_name' and its 'fixed'
1548argument is fixed to the string '2.0 value'.
1549
1550 >>> print dictionary['as']
1551 new_name
1552 >>> dictionary['call_with']
1553 {'fixed': '2.0 value'}
1554 >>> dictionary['cache_for']
1555 300
1556
1557The published name of the 'required' argument is 'required_argument',
1558not 'required'.
1559
1560 >>> print dictionary['params']['required'].__name__
1561 required_argument
1562
1563Let's pop the 2.0 version off the stack. Now we can see how the method
1564looks in 1.0. It's still called 'new_name', and its 'required'
1565argument is still called 'required_argument', but its 'fixed' argument
1566is fixed to the string '1.0 value'.
1567
1568 >>> ignored = dictionary.pop()
1569 >>> print dictionary['as']
1570 new_name
1571 >>> dictionary['call_with']
1572 {'fixed': '1.0 value'}
1573 >>> print dictionary['params']['required'].__name__
1574 required_argument
1575 >>> dictionary['cache_for']
1576 100
1577
1578Let's pop one more time to see how the method looks in the pre-1.0
1579version. It hasn't yet been renamed to 'new_name', its 'required'
1580argument hasn't yet been renamed to 'required_argument', and its
1581'fixed' argument is fixed to the string 'pre-1.0 value'.
1582
1583 >>> ignored = dictionary.pop()
1584 >>> print dictionary.get('as')
1585 None
1586 >>> print dictionary['params']['required'].__name__
1587 required
1588 >>> dictionary['call_with']
1589 {'fixed': 'pre-1.0 value'}
1590 >>> dictionary['cache_for']
1591 100
1592
1593@operation_removed_in_version
1594*****************************
1595
1596Sometimes you want version n+1 to remove a named operation that was
1597present in version n. The @operation_removed_in_version declaration
1598does just this.
1599
1600Let's define an operation that's introduced in 1.0 and removed in 2.0.
1601
1602 >>> from lazr.restful.declarations import operation_removed_in_version
1603 >>> class DisappearingMultiversionMethod(Interface):
1604 ... export_as_webservice_entry()
1605 ... @operation_removed_in_version(2.0)
1606 ... @operation_parameters(arg=Float())
1607 ... @export_read_operation()
1608 ... @operation_for_version(1.0)
1609 ... def method(arg):
1610 ... """A doomed method."""
1611
1612 >>> dictionary = DisappearingMultiversionMethod[
1613 ... 'method'].getTaggedValue('lazr.restful.exported')
1614
1615The method is not present in 2.0:
1616
1617 >>> version, attrs = dictionary.pop()
1618 >>> print version
1619 2.0
1620 >>> sorted(attrs.items())
1621 []
1622
1623It is present in 1.0:
1624
1625 >>> version, attrs = dictionary.pop()
1626 >>> print version
1627 1.0
1628 >>> print attrs['type']
1629 read_operation
1630 >>> print attrs['params']['arg']
1631 <zope.schema._field.Float object...>
1632
1633But it's not present in the unnamed pre-1.0 version, since it hadn't
1634been defined yet:
1635
1636 >>> print dictionary.pop()
1637 (None, {})
1638
1639The @operation_removed_in_version declaration can also be used to
1640reset a named operation's definition if you need to completely re-do
1641it.
1642
1643For instance, ordinarily you can't change the type of an operation, or
1644totally redefine its parameters--and you shouldn't really need
1645to. It's usually easier to publish two different operations that have
1646the same name in different versions. But you can do it with a single
1647operation, by removing the operation with
1648@operation_removed_in_version and defining it again--either in the
1649same version or in some later version.
1650
1651In this example, the type of the operation, the type and number of the
1652arguments, and the return value change in version 1.0.
1653
1654 >>> class ReadOrWriteMethod(Interface):
1655 ... export_as_webservice_entry()
1656 ...
1657 ... @operation_parameters(arg=TextLine(), arg2=TextLine())
1658 ... @export_write_operation()
1659 ... @operation_removed_in_version(1.0)
1660 ...
1661 ... @operation_parameters(arg=Float())
1662 ... @operation_returns_collection_of(Interface)
1663 ... @export_read_operation()
1664 ... def method(arg, arg2='default'):
1665 ... """A read *or* a write operation, depending on version."""
1666
1667 >>> dictionary = ReadOrWriteMethod[
1668 ... 'method'].getTaggedValue('lazr.restful.exported')
1669
1670In version 1.0, the 'method' named operation is a write operation that
1671takes two TextLine arguments and has no special return value.
1672
1673 >>> version, attrs = dictionary.pop()
1674 >>> print version
1675 1.0
1676 >>> print attrs['type']
1677 write_operation
1678 >>> attrs['params']['arg']
1679 <zope.schema._bootstrapfields.TextLine object...>
1680 >>> attrs['params']['arg2']
1681 <zope.schema._bootstrapfields.TextLine object...>
1682 >>> print attrs.get('return_type')
1683 None
1684
1685In the unnamed pre-1.0 version, the 'method' operation is a read
1686operation that takes a single Float argument and returns a collection.
1687
1688 >>> version, attrs = dictionary.pop()
1689 >>> print attrs['type']
1690 read_operation
1691
1692 >>> attrs['params']['arg']
1693 <zope.schema._field.Float object...>
1694 >>> attrs['params'].keys()
1695 ['arg']
1696
1697 >>> attrs['return_type']
1698 <lazr.restful.fields.CollectionField object...>
1699
1700[XXX leonardr mutator_for modifies the field, not the method, so it
1701won't work until I add multiversion support for fields. Also, it's
1702possible to remove a named operation from a certain version and then
1703uselessly annotate it some more. The best place to catch this error
1704turns out to be when generating the adapter classes, not when
1705collecting annotations.]
1706
1580Security1707Security
1581========1708========
15821709

Subscribers

People subscribed via source and target branches