Merge lp:~benji/launchpad/bug-612754-2 into lp:launchpad

Proposed by Benji York
Status: Merged
Approved by: Benji York
Approved revision: no longer in the source branch.
Merged at revision: 11744
Proposed branch: lp:~benji/launchpad/bug-612754-2
Merge into: lp:launchpad
Diff against target: 76 lines (+27/-7)
2 files modified
lib/lp/services/mail/incoming.py (+5/-1)
lib/lp/services/mail/tests/test_signedmessage.py (+22/-6)
To merge this branch: bzr merge lp:~benji/launchpad/bug-612754-2
Reviewer Review Type Date Requested Status
Edwin Grubbs (community) code Approve
Review via email: mp+37797@code.launchpad.net

Description of the change

Bug 612754 is about people having perfectly valid GPG signed messages
rejected by LP.

It turns out that they were rejected because LP doesn't remove trailing
whitespace before checking the signature (as required by RFC 4880 and
implemented in GPG). The fix was to remove the trailing whitespace as
part of the newline canonicalization.

The test added by this branch can be run by

    bin/test -c test_signedmessage

while more email tests can be run by

    bin/test -c mail

Lint fixed:
./lib/canonical/launchpad/mail/incoming.py
      64: E302 expected 2 blank lines, found 1
     156: W291 trailing whitespace
     345: E202 whitespace before ')'
     358: E202 whitespace before ')'
     407: E202 whitespace before ')'
     156: Line has trailing whitespace.
./lib/lp/services/mail/tests/test_signedmessage.py
      39: E501 line too long (86 characters)
     169: W391 blank line at end of file
      39: Line exceeds 78 characters.

Recommended listening for this MP: http://www.youtube.com/watch?v=o7gw_2W_oCU

To post a comment you must log in.
Revision history for this message
Edwin Grubbs (edwin-grubbs) wrote :
Download full text (3.4 KiB)

Hi Benji,

I think the regex and the test need some work to handle trailing whitespace on any line.

-Edwin

>=== modified file 'lib/canonical/launchpad/mail/incoming.py'
>--- lib/canonical/launchpad/mail/incoming.py 2010-10-03 15:30:06 +0000
>+++ lib/canonical/launchpad/mail/incoming.py 2010-10-06 22:26:05 +0000
>@@ -55,9 +55,11 @@
> from lp.services.mail.signedmessage import signed_message_from_string
>
> # Match '\n' and '\r' line endings. That is, all '\r' that are not
>-# followed by a # '\n', and all '\n' that are not preceded by a '\r'.
>+# followed by a '\n', and all '\n' that are not preceded by a '\r'.
> non_canonicalised_line_endings = re.compile('((?<!\r)\n)|(\r(?!\n))')
>
>+# Match trailing whitespace.
>+trailing_whitespace = re.compile(' *$')

Since you are not using rstrip(), I assume you want the regex to
strip the whitespace at the end of each line. In that case, you need
to tell re.compile() that you want to use a multiline expression.
Also, " *$" will not match " \r\n".

> def canonicalise_line_endings(text):
> r"""Canonicalise the line endings to '\r\n'.
>@@ -73,6 +75,8 @@
> """
> if non_canonicalised_line_endings.search(text):
> text = non_canonicalised_line_endings.sub('\r\n', text)
>+ if trailing_whitespace.search(text):
>+ text = trailing_whitespace.sub('', text)
> return text
>
>
>
>=== modified file 'lib/lp/services/mail/tests/test_signedmessage.py'
>--- lib/lp/services/mail/tests/test_signedmessage.py 2010-08-20 20:31:18 +0000
>+++ lib/lp/services/mail/tests/test_signedmessage.py 2010-10-06 22:26:05 +0000
>@@ -59,17 +59,18 @@
> IWeaklyAuthenticatedPrincipal.providedBy(principle))
> self.assertIs(None, msg.signature)
>
>- def _get_clearsigned_for_person(self, sender):
>+ def _get_clearsigned_for_person(self, sender, body=None):
> # Create a signed message for the sender specified with the test
> # secret key.
> key = import_secret_test_key()
> signing_context = GPGSigningContext(key.fingerprint, password='test')
>- body = dedent("""\
>- This is a multi-line body.
>+ if body is None:
>+ body = dedent("""\
>+ This is a multi-line body.
>
>- Sincerely,
>- Your friendly tester.
>- """)
>+ Sincerely,
>+ Your friendly tester.
>+ """)
> msg = self.factory.makeSignedMessage(
> email_address=sender.preferredemail.email,
> body=body, signing_context=signing_context)
>@@ -97,6 +98,18 @@
> self.assertFalse(
> IWeaklyAuthenticatedPrincipal.providedBy(principle))
>
>+ def test_trailing_whitespace(self):
>+ # Trailing whitespace should be ignored when verifying a message's
>+ # signature.
>+ sender = getUtility(IPersonSet).getByEmail('<email address hidden>')
>+ body = 'A message with trailing whitespace. '

A multiline body with trailing whitespace on each line will fail.

>+ msg = self._get_clearsigned_for_person(sender, body)
>+ principle = authenticateEmail(msg)
>+ self.assertIsNot(None, msg.sig...

Read more...

review: Needs Fixing (code)
Revision history for this message
Benji York (benji) wrote :

On Wed, Oct 6, 2010 at 6:54 PM, Edwin Grubbs <email address hidden> wrote:
> Review: Needs Fixing code
> Hi Benji,
>
> I think the regex and the test need some work to handle trailing whitespace
> on any line.

That's odd. I had made a change along those lines before even
submitting the branch for review but the diff on the MP doesn't reflect
the change.

Darn! I forgot to push after commit. The more appropriate regex and
test should be visible after the diff updates.
--
Benji York

Revision history for this message
Edwin Grubbs (edwin-grubbs) wrote :

Hi Benji,

It looks so much better since I have all your revisions.

-Edwin

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/services/mail/incoming.py'
--- lib/lp/services/mail/incoming.py 2010-10-12 14:03:44 +0000
+++ lib/lp/services/mail/incoming.py 2010-10-18 14:49:45 +0000
@@ -55,9 +55,11 @@
55from lp.services.mail.signedmessage import signed_message_from_string55from lp.services.mail.signedmessage import signed_message_from_string
5656
57# Match '\n' and '\r' line endings. That is, all '\r' that are not57# Match '\n' and '\r' line endings. That is, all '\r' that are not
58# followed by a # '\n', and all '\n' that are not preceded by a '\r'.58# followed by a '\n', and all '\n' that are not preceded by a '\r'.
59non_canonicalised_line_endings = re.compile('((?<!\r)\n)|(\r(?!\n))')59non_canonicalised_line_endings = re.compile('((?<!\r)\n)|(\r(?!\n))')
6060
61# Match trailing whitespace.
62trailing_whitespace = re.compile(r'[ \t]*((?=\r\n)|$)')
6163
62def canonicalise_line_endings(text):64def canonicalise_line_endings(text):
63 r"""Canonicalise the line endings to '\r\n'.65 r"""Canonicalise the line endings to '\r\n'.
@@ -73,6 +75,8 @@
73 """75 """
74 if non_canonicalised_line_endings.search(text):76 if non_canonicalised_line_endings.search(text):
75 text = non_canonicalised_line_endings.sub('\r\n', text)77 text = non_canonicalised_line_endings.sub('\r\n', text)
78 if trailing_whitespace.search(text):
79 text = trailing_whitespace.sub('', text)
76 return text80 return text
7781
7882
7983
=== modified file 'lib/lp/services/mail/tests/test_signedmessage.py'
--- lib/lp/services/mail/tests/test_signedmessage.py 2010-10-11 17:56:08 +0000
+++ lib/lp/services/mail/tests/test_signedmessage.py 2010-10-18 14:49:45 +0000
@@ -59,17 +59,18 @@
59 IWeaklyAuthenticatedPrincipal.providedBy(principle))59 IWeaklyAuthenticatedPrincipal.providedBy(principle))
60 self.assertIs(None, msg.signature)60 self.assertIs(None, msg.signature)
6161
62 def _get_clearsigned_for_person(self, sender):62 def _get_clearsigned_for_person(self, sender, body=None):
63 # Create a signed message for the sender specified with the test63 # Create a signed message for the sender specified with the test
64 # secret key.64 # secret key.
65 key = import_secret_test_key()65 key = import_secret_test_key()
66 signing_context = GPGSigningContext(key.fingerprint, password='test')66 signing_context = GPGSigningContext(key.fingerprint, password='test')
67 body = dedent("""\67 if body is None:
68 This is a multi-line body.68 body = dedent("""\
69 This is a multi-line body.
6970
70 Sincerely,71 Sincerely,
71 Your friendly tester.72 Your friendly tester.
72 """)73 """)
73 msg = self.factory.makeSignedMessage(74 msg = self.factory.makeSignedMessage(
74 email_address=sender.preferredemail.email,75 email_address=sender.preferredemail.email,
75 body=body, signing_context=signing_context)76 body=body, signing_context=signing_context)
@@ -97,6 +98,21 @@
97 self.assertFalse(98 self.assertFalse(
98 IWeaklyAuthenticatedPrincipal.providedBy(principle))99 IWeaklyAuthenticatedPrincipal.providedBy(principle))
99100
101 def test_trailing_whitespace(self):
102 # Trailing whitespace should be ignored when verifying a message's
103 # signature.
104 sender = getUtility(IPersonSet).getByEmail('test@canonical.com')
105 body = (
106 'A message with trailing spaces. \n'+
107 'And tabs\t\t\n'+
108 'Also mixed. \t ')
109 msg = self._get_clearsigned_for_person(sender, body)
110 principle = authenticateEmail(msg)
111 self.assertIsNot(None, msg.signature)
112 self.assertEqual(sender, principle.person)
113 self.assertFalse(
114 IWeaklyAuthenticatedPrincipal.providedBy(principle))
115
100 def _get_detached_message_for_person(self, sender):116 def _get_detached_message_for_person(self, sender):
101 # Return a signed message that contains a detached signature.117 # Return a signed message that contains a detached signature.
102 body = dedent("""\118 body = dedent("""\