Merge lp:~didrocks/launchpad/expose-sshkeys-bug-357235 into lp:launchpad/db-devel

Proposed by Didier Roche-Tolomelli
Status: Merged
Approved by: Michael Hudson-Doyle
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~didrocks/launchpad/expose-sshkeys-bug-357235
Merge into: lp:launchpad/db-devel
Diff against target: 200 lines (+98/-9)
5 files modified
lib/lp/registry/browser/configure.zcml (+6/-0)
lib/lp/registry/browser/tests/test_sshkey.py (+31/-0)
lib/lp/registry/interfaces/person.py (+8/-3)
lib/lp/registry/interfaces/ssh.py (+10/-5)
lib/lp/registry/stories/webservice/xx-person.txt (+43/-1)
To merge this branch: bzr merge lp:~didrocks/launchpad/expose-sshkeys-bug-357235
Reviewer Review Type Date Requested Status
Francis J. Lacoste (community) Approve
Graham Binns code Pending
Review via email: mp+22067@code.launchpad.net

This proposal supersedes a proposal from 2010-03-09.

Commit message

Expose ssh keys into launchpad API

To post a comment you must log in.
Revision history for this message
Graham Binns (gmb) wrote : Posted in a previous version of this proposal

Hi Didier,

I'm not comfortable reviewing this branch. There's no description of the change in the merge proposal description and I don't know whether you've had a pre-implementation discussion about the branch.

You need to include the following items when you submit a Launchpad branch for review:

 * A description of the change, including a brief list of changes by file
 * The output of `make lint`, run in the root of your branch.
 * Details of the person with whom you had a pre implementation discussion, including (if necessary) details of why you chose the solution you did.

I'm going to reject this branch; please resubmit it with the above items included. If you want to have a pre-implementation discussion this afternoon I'll be happy to make myself available.

review: Needs Resubmitting
Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote : Posted in a previous version of this proposal
Download full text (6.3 KiB)

The branch has been made in a pair programming session with jml, that's why I didn't added any more detail about it.
You can see inspiration and pair session programming on jml's gpg branch: http://bazaar.launchpad.net/~jml/launchpad/expose-gpgkeys-bug-389872/

The change is for exposing ssh key to the API needed for Quickly. We don't allow uploading or setting it directly yet (this will be an UDS discussion).

https://bugs.edge.launchpad.net/launchpad-registry/+bug/357235 is for the corresponding bug report. Again, we don't upload the ssh key/setting is right now.

output of make lint, I added also the testsuite call too:
$ make lint
utilities/shhh.py PYTHONPATH= python2.5 bootstrap.py\
                --ez_setup-source=ez_setup.py \
  --download-base=download-cache/dist --eggs=eggs
Enter passphrase for key '/home/ubuntu/.ssh/id_rsa':
= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/registry/browser/configure.zcml
  lib/lp/registry/browser/tests/test_sshkey.py
  lib/lp/registry/interfaces/person.py
  lib/lp/registry/interfaces/ssh.py
  lib/lp/registry/stories/webservice/xx-person.txt

== Pyflakes notices ==

lib/lp/registry/interfaces/ssh.py
    19: 'export_read_operation' imported but unused
    19: 'export_as_webservice_collection' imported but unused
    19: 'operation_parameters' imported but unused
    19: 'collection_default_content' imported but unused
    19: 'operation_returns_collection_of' imported but unused

== Pylint notices ==

lib/lp/registry/interfaces/person.py
    520: [C0301] Line too long (80/78)
    53: [F0401] Unable to import 'lazr.enum' (No module named enum)
    54: [F0401] Unable to import 'lazr.lifecycle.snapshot' (No module named lifecycle)
    55: [F0401] Unable to import 'lazr.restful.interface' (No module named restful)
    56: [F0401] Unable to import 'lazr.restful.declarations' (No module named restful)
    63: [F0401] Unable to import 'lazr.restful.fields' (No module named restful)
    410: [E1002, PersonNameField._validate] Use super on an old style class
    1404: [C0322, IPersonEditRestricted.addMember] Operator not preceded by a space
    status=copy_field(ITeamMembership['status']),
    ^
    comment=Text(required=False))
    @export_write_operation()
    def addMember(person, reviewer, status=TeamMembershipStatus.APPROVED,
    comment=None, force_team_add=False,
    may_subscribe_to_list=True):
    1445: [C0322, IPersonEditRestricted.acceptInvitationToBeMemberOf] Operator not preceded by a space
    comment=Text())
    ^
    @export_write_operation()
    def acceptInvitationToBeMemberOf(team, comment):
    1457: [C0322, IPersonEditRestricted.declineInvitationToBeMemberOf] Operator not preceded by a space
    comment=Text())
    ^
    @export_write_operation()
    def declineInvitationToBeMemberOf(team, comment):
    1755: [C0322, IPersonSet.newTeam] Operator not preceded by a space
    defaultmembershipperiod='default_membership_period',
    ^
    defaultrenewalperiod='default_renewal_period')
    @operation_parameters(
    subscriptionpolicy=Choice(
    title=_('Subs...

Read more...

Revision history for this message
Graham Binns (gmb) wrote : Posted in a previous version of this proposal

Hi Didier, thanks for adding the details.

== Pyflakes notices ==

> lib/lp/registry/interfaces/ssh.py
> 19: 'export_read_operation' imported but unused
> 19: 'export_as_webservice_collection' imported but unused
> 19: 'operation_parameters' imported but unused
> 19: 'collection_default_content' imported but unused
> 19: 'operation_returns_collection_of' imported but unused
>
> == Pylint notices ==
>
> lib/lp/registry/interfaces/person.py
> 520: [C0301] Line too long (80/78)

You can fix these easily enough. You can pretty much ignore the rest;
Pylint produces a lot of noise and should be taken out and shot.

Other than that I'm happy with this branch. Let me know if you want me
to land it for you (I don't know whether it needs to be landed with
jml's work or not).

review: Approve (code)
Revision history for this message
Francis J. Lacoste (flacoste) wrote : Posted in a previous version of this proposal

On March 9, 2010, Didier Roche wrote:
> class ISSHKey(Interface):
> """SSH public key"""
> - id = Int(title=_("Database ID"), required=True, readonly=True)
> +
> + export_as_webservice_entry('ssh_key')
> +
> + id = exported(Int(title=_("Database ID"), required=True,
> readonly=True)) person = Int(title=_("Owner"), required=True,
> readonly=True)
> personID = Int(title=_('Owner ID'), required=True, readonly=True)
> - keytype = Choice(title=_("Key type"), required=True,
> - vocabulary=SSHKeyType)
> - keytext = TextLine(title=_("Key text"), required=True)
> - comment = TextLine(title=_("Comment describing this key"),
> - required=True)
> + keytype = exported(Choice(title=_("Key type"), required=True,
> + vocabulary=SSHKeyType))
> + keytext = exported(TextLine(title=_("Key text"), required=True))
> + comment = exported(TextLine(title=_("Comment describing this key"),
> + required=True))

These fields should all be exported as readonly=True.

--
Francis J. Lacoste
<email address hidden>

Revision history for this message
Francis J. Lacoste (flacoste) : Posted in a previous version of this proposal
review: Needs Fixing
Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote : Posted in a previous version of this proposal
Download full text (3.5 KiB)

ok, those are fixed now. (readonly, uneeded import and too long lines). testsuite is still ok and here is the new make lint output:

$ make lint
utilities/shhh.py PYTHONPATH= python2.5 bootstrap.py\
                --ez_setup-source=ez_setup.py \
  --download-base=download-cache/dist --eggs=eggs
Enter passphrase for key '/home/ubuntu/.ssh/id_rsa':
Enter passphrase for key '/home/ubuntu/.ssh/id_rsa':
= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/registry/browser/configure.zcml
  lib/lp/registry/browser/tests/test_sshkey.py
  lib/lp/registry/interfaces/person.py
  lib/lp/registry/interfaces/ssh.py
  lib/lp/registry/stories/webservice/xx-person.txt

== Pylint notices ==

lib/lp/registry/interfaces/person.py
    53: [F0401] Unable to import 'lazr.enum' (No module named enum)
    54: [F0401] Unable to import 'lazr.lifecycle.snapshot' (No module named lifecycle)
    55: [F0401] Unable to import 'lazr.restful.interface' (No module named restful)
    56: [F0401] Unable to import 'lazr.restful.declarations' (No module named restful)
    63: [F0401] Unable to import 'lazr.restful.fields' (No module named restful)
    410: [E1002, PersonNameField._validate] Use super on an old style class
    1404: [C0322, IPersonEditRestricted.addMember] Operator not preceded by a space
    status=copy_field(ITeamMembership['status']),
    ^
    comment=Text(required=False))
    @export_write_operation()
    def addMember(person, reviewer, status=TeamMembershipStatus.APPROVED,
    comment=None, force_team_add=False,
    may_subscribe_to_list=True):
    1445: [C0322, IPersonEditRestricted.acceptInvitationToBeMemberOf] Operator not preceded by a space
    comment=Text())
    ^
    @export_write_operation()
    def acceptInvitationToBeMemberOf(team, comment):
    1457: [C0322, IPersonEditRestricted.declineInvitationToBeMemberOf] Operator not preceded by a space
    comment=Text())
    ^
    @export_write_operation()
    def declineInvitationToBeMemberOf(team, comment):
    1755: [C0322, IPersonSet.newTeam] Operator not preceded by a space
    defaultmembershipperiod='default_membership_period',
    ^
    defaultrenewalperiod='default_renewal_period')
    @operation_parameters(
    subscriptionpolicy=Choice(
    title=_('Subscription policy'), vocabulary=TeamSubscriptionPolicy,
    required=False, default=TeamSubscriptionPolicy.MODERATED))
    @export_factory_operation(
    ITeam, ['name', 'displayname', 'teamdescription',
    'defaultmembershipperiod', 'defaultrenewalperiod'])
    def newTeam(teamowner, name, displayname, teamdescription=None,
    subscriptionpolicy=TeamSubscriptionPolicy.MODERATED,
    defaultmembershipperiod=None, defaultrenewalperiod=None):
    1824: [C0322, IPersonSet.findPerson] Operator not preceded by a space
    created_after=Datetime(
    ^
    title=_("Created after"), required=False),
    created_before=Datetime(
    title=_("Created before"), required=False),
    )
    @operation_returns_collection_of(IPerson)
    @export_read_operation()
    def findPerson(text="", exclude_inactive_accounts=True,
    must...

Read more...

Revision history for this message
Karl Fogel (kfogel) wrote : Posted in a previous version of this proposal

I'm not a reviewer, but this looks pretty easy to review (and at least on the surface the change seems correct to me). The formatting fix at @@ -519,8 +520,8 @@ is unrelated -- personally, I find those distracting in a branch containing functional changes, but YMMV.

Revision history for this message
Francis J. Lacoste (flacoste) wrote : Posted in a previous version of this proposal
Download full text (5.9 KiB)

Your test is not working yet, but we are getting there!

> === modified file 'lib/lp/registry/browser/configure.zcml'
> --- lib/lp/registry/browser/configure.zcml 2010-03-08 01:51:58 +0000
> +++ lib/lp/registry/browser/configure.zcml 2010-03-09 19:30:49 +0000
> @@ -2172,4 +2172,9 @@
> classes="
> PersonProductFacets"
> module="lp.registry.browser.personproduct"/>
> + <browser:url
> + for="lp.registry.interfaces.ssh.ISSHKey"
> + path_expression="string:+ssh-keys/${id}"
> + rootsite="api"
> + attribute_to_parent="person" />
> </configure>

We usually refrain from exposing DB id publically. In URLs, especially. We
have a couple of exception and since there is no alternative here, I think
it's fine.

> === modified file 'lib/lp/registry/interfaces/person.py'
> --- lib/lp/registry/interfaces/person.py 2010-03-05 14:50:47 +0000

>
> oauth_request_tokens = Attribute(_("Non-expired request tokens"))
>
> - sshkeys = Attribute(_('List of SSH keys'))
> + sshkeys = exported(
> + CollectionField(
> + title= _('List of SSH keys'),
> + readonly=False, required=False,
> + value_type=Reference(schema=ISSHKey)))

Can we rename that to ssh_keys? Unfortunately, you can't use exported_as
because of bug 546324. I think there isn't that many call sites, but feel free
to push back if this is too daunting a task.

> === modified file 'lib/lp/registry/interfaces/ssh.py'
> --- lib/lp/registry/interfaces/ssh.py 2009-06-25 04:06:00 +0000
> +++ lib/lp/registry/interfaces/ssh.py 2010-03-09 19:30:49 +0000
> @@ -16,6 +16,7 @@
> from zope.schema import Choice, Int, TextLine
> from zope.interface import Interface
> from lazr.enum import DBEnumeratedType, DBItem
> +from lazr.restful.declarations import (export_as_webservice_entry, exported)
>
> from canonical.launchpad import _
>
> @@ -42,14 +43,18 @@
>
> class ISSHKey(Interface):
> """SSH public key"""
> - id = Int(title=_("Database ID"), required=True, readonly=True)
> +
> + export_as_webservice_entry('ssh_key')
> +
> + id = exported(Int(title=_("Database ID"), required=True, readonly=True))
> person = Int(title=_("Owner"), required=True, readonly=True)

We don't want to export the DB id. It's not useful at all. I know it's leaked
into the URL, but that's an artefact.

> personID = Int(title=_('Owner ID'), required=True, readonly=True)
> - keytype = Choice(title=_("Key type"), required=True,
> - vocabulary=SSHKeyType)
> - keytext = TextLine(title=_("Key text"), required=True)
> - comment = TextLine(title=_("Comment describing this key"),
> - required=True)
> + keytype = exported(Choice(title=_("Key type"), required=True,
> + vocabulary=SSHKeyType, readonly=True))
> + keytext = exported(TextLine(title=_("Key text"), required=True,
> + readonly=True))
> + comment = exported(TextLine(title=_("Comment describing this key"),
> + required=True, readonly=True))

> === modified file 'lib/lp/registry/stories/webservice/xx-person.txt'

> +== SSH keys ===
> +
> +People have SSH keys which we can manipulate over the API.
> +
> +The sample person "name12" doesn't have any keys to begin with:
> +
> + ...

Read more...

review: Needs Fixing
Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

Thanks for your review!

Sorry for the test errors. Let's blame being tired about creating a GPG key and listing the existing ssh one :)
That's fixed in creating a new user, as "no-priv" had also an ssh key :)

In fact, keytext and keycomment aren't random (there is just a number appended at the end of generic-stringX. I ellided the number and so, don't thing we need to add a specific comment in makeSSHKey
I removed as well the exposal of id in ssh.py.

"Can we rename that to ssh_keys? Unfortunately, you can't use exported_as
because of bug 546324. I think there isn't that many call sites, but feel free
to push back if this is too daunting a task."
I'm more concerned about this one to be out of my poor knowledge of launchpad. I knew exported_as but as it seems to be broken with CollectionField… To be honest I'm a little bit scared on breaking everything in renaming it as Launchpad is so big :)

Don't hesitate to report any further issue. Thanks again for reviewing this.

Revision history for this message
Francis J. Lacoste (flacoste) wrote :

OK, that's fine.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/registry/browser/configure.zcml'
2--- lib/lp/registry/browser/configure.zcml 2010-03-11 16:48:37 +0000
3+++ lib/lp/registry/browser/configure.zcml 2010-03-25 08:36:44 +0000
4@@ -2174,6 +2174,12 @@
5 module="lp.registry.browser.personproduct"/>
6
7 <browser:url
8+ for="lp.registry.interfaces.ssh.ISSHKey"
9+ path_expression="string:+ssh-keys/${id}"
10+ rootsite="api"
11+ attribute_to_parent="person" />
12+
13+ <browser:url
14 for="lp.registry.interfaces.gpg.IGPGKey"
15 path_expression="string:+gpg-keys/${keyid}"
16 rootsite="api"
17
18=== added file 'lib/lp/registry/browser/tests/test_sshkey.py'
19--- lib/lp/registry/browser/tests/test_sshkey.py 1970-01-01 00:00:00 +0000
20+++ lib/lp/registry/browser/tests/test_sshkey.py 2010-03-25 08:36:44 +0000
21@@ -0,0 +1,31 @@
22+# Copyright 2010 Canonical Ltd. This software is licensed under the
23+# GNU Affero General Public License version 3 (see the file LICENSE).
24+
25+"""Tests for GPG key on the web."""
26+
27+__metaclass__ = type
28+
29+import unittest
30+
31+from canonical.launchpad.webapp import canonical_url
32+from canonical.testing.layers import DatabaseFunctionalLayer
33+from lp.testing import TestCaseWithFactory
34+
35+
36+class TestCanonicalUrl(TestCaseWithFactory):
37+
38+ layer = DatabaseFunctionalLayer
39+
40+ def test_canonical_url(self):
41+ # The canonical URL of a GPG key is ssh-keys
42+ person = self.factory.makePerson()
43+ sshkey = self.factory.makeSSHKey(person)
44+ self.assertEqual(
45+ '%s/+ssh-keys/%s' % (
46+ canonical_url(person, rootsite='api'), sshkey.id),
47+ canonical_url(sshkey))
48+
49+
50+def test_suite():
51+ return unittest.TestLoader().loadTestsFromName(__name__)
52+
53
54=== modified file 'lib/lp/registry/interfaces/person.py'
55--- lib/lp/registry/interfaces/person.py 2010-03-11 20:59:17 +0000
56+++ lib/lp/registry/interfaces/person.py 2010-03-25 08:36:44 +0000
57@@ -94,6 +94,7 @@
58 from lp.registry.interfaces.mailinglistsubscription import (
59 MailingListAutoSubscribePolicy)
60 from lp.registry.interfaces.mentoringoffer import IHasMentoringOffers
61+from lp.registry.interfaces.ssh import ISSHKey
62 from lp.registry.interfaces.teammembership import (
63 ITeamMembership, ITeamParticipation, TeamMembershipStatus)
64 from lp.registry.interfaces.wikiname import IWikiName
65@@ -520,8 +521,8 @@
66 description=_(
67 "An image of exactly 64x64 pixels that will be displayed in "
68 "the heading of all pages related to you. Traditionally this "
69- "is a logo, a small picture or a personal mascot. It should be "
70- "no bigger than 50kb in size.")))
71+ "is a logo, a small picture or a personal mascot. It should "
72+ "be no bigger than 50kb in size.")))
73 logoID = Int(title=_('Logo ID'), required=True, readonly=True)
74
75 mugshot = exported(MugshotImageUpload(
76@@ -606,7 +607,11 @@
77
78 oauth_request_tokens = Attribute(_("Non-expired request tokens"))
79
80- sshkeys = Attribute(_('List of SSH keys'))
81+ sshkeys = exported(
82+ CollectionField(
83+ title= _('List of SSH keys'),
84+ readonly=False, required=False,
85+ value_type=Reference(schema=ISSHKey)))
86
87 account_status = Choice(
88 title=_("The status of this person's account"), required=False,
89
90=== modified file 'lib/lp/registry/interfaces/ssh.py'
91--- lib/lp/registry/interfaces/ssh.py 2009-06-25 04:06:00 +0000
92+++ lib/lp/registry/interfaces/ssh.py 2010-03-25 08:36:44 +0000
93@@ -16,6 +16,7 @@
94 from zope.schema import Choice, Int, TextLine
95 from zope.interface import Interface
96 from lazr.enum import DBEnumeratedType, DBItem
97+from lazr.restful.declarations import (export_as_webservice_entry, exported)
98
99 from canonical.launchpad import _
100
101@@ -42,14 +43,18 @@
102
103 class ISSHKey(Interface):
104 """SSH public key"""
105+
106+ export_as_webservice_entry('ssh_key')
107+
108 id = Int(title=_("Database ID"), required=True, readonly=True)
109 person = Int(title=_("Owner"), required=True, readonly=True)
110 personID = Int(title=_('Owner ID'), required=True, readonly=True)
111- keytype = Choice(title=_("Key type"), required=True,
112- vocabulary=SSHKeyType)
113- keytext = TextLine(title=_("Key text"), required=True)
114- comment = TextLine(title=_("Comment describing this key"),
115- required=True)
116+ keytype = exported(Choice(title=_("Key type"), required=True,
117+ vocabulary=SSHKeyType, readonly=True))
118+ keytext = exported(TextLine(title=_("Key text"), required=True,
119+ readonly=True))
120+ comment = exported(TextLine(title=_("Comment describing this key"),
121+ required=True, readonly=True))
122
123 def destroySelf():
124 """Remove this SSHKey from the database."""
125
126=== modified file 'lib/lp/registry/stories/webservice/xx-person.txt'
127--- lib/lp/registry/stories/webservice/xx-person.txt 2010-03-13 00:32:40 +0000
128+++ lib/lp/registry/stories/webservice/xx-person.txt 2010-03-25 08:36:44 +0000
129@@ -46,6 +46,7 @@
130 proposed_members_collection_link: u'http://.../~salgado/proposed_members'
131 resource_type_link: u'http://.../#person'
132 self_link: u'http://.../~salgado'
133+ sshkeys_collection_link: u'http://.../~salgado/sshkeys'
134 sub_teams_collection_link: u'http://.../~salgado/sub_teams'
135 super_teams_collection_link: u'http://.../~salgado/super_teams'
136 team_owner_link: None
137@@ -96,6 +97,7 @@
138 renewal_policy: u'invite them to apply for renewal'
139 resource_type_link: u'http://.../#team'
140 self_link: u'http://.../~ubuntu-team'
141+ sshkeys_collection_link: u'http://.../~ubuntu-team/sshkeys'
142 sub_teams_collection_link: u'http://.../~ubuntu-team/sub_teams'
143 subscription_policy: u'Moderated Team'
144 super_teams_collection_link: u'http://.../~ubuntu-team/super_teams'
145@@ -151,6 +153,46 @@
146 HTTP/1.1 404 Not Found
147 ...
148
149+== SSH keys ===
150+
151+People have SSH keys which we can manipulate over the API.
152+
153+The sample person "ssh-user" doesn't have any keys to begin with:
154+
155+ >>> login('test@canonical.com')
156+ >>> person = factory.makePerson(name="ssh-user")
157+ >>> logout()
158+ >>> sample_person = webservice.get("/~ssh-user").jsonBody()
159+ >>> sshkeys = sample_person['sshkeys_collection_link']
160+ >>> print sshkeys
161+ http://.../~ssh-user/sshkeys
162+ >>> print_self_link_of_entries(webservice.get(sshkeys).jsonBody())
163+
164+Let's give "ssh-user" a key via the back door of our internal Python APIs:
165+
166+ >>> from zope.component import getUtility
167+ >>> from lp.registry.interfaces.person import IPersonSet
168+ >>> login(ANONYMOUS)
169+ >>> ssh_user = getUtility(IPersonSet).getByName('ssh-user')
170+ >>> ssh_key = factory.makeSSHKey(ssh_user)
171+ >>> logout()
172+
173+Now when we get the sshkey collection for 'sssh-user' again, the key should show
174+up:
175+
176+ >>> keys = webservice.get(sshkeys).jsonBody()
177+ >>> print_self_link_of_entries(keys)
178+ http://.../~ssh-user/+ssh-keys/...
179+
180+
181+And then we can actually retrieve the key:
182+
183+ >>> pprint_entry(keys['entries'][0])
184+ comment: u'generic-string...'
185+ keytext: u'generic-string...'
186+ keytype: u'RSA'
187+ resource_type_link: u'http://.../#ssh_key'
188+ self_link: u'http://.../~ssh-user/+ssh-keys/...'
189
190 === GPG keys ===
191
192@@ -178,7 +220,7 @@
193
194 >>> keys = webservice.get(gpgkeys).jsonBody()
195 >>> print_self_link_of_entries(keys)
196- http://api.launchpad.dev/beta/~name12/+gpg-keys/00000001
197+ http://.../~name12/+gpg-keys/...
198
199
200 And then we can actually retrieve the key:

Subscribers

People subscribed via source and target branches

to status/vote changes: