Launchpad itself

Merge lp:~al-maisan/launchpad/clog-oops-452070 into lp:launchpad

  1. clog-oops-452070
  2. Merge into devel
Proposed by Muharem Hrnjadovic
Status: Merged
Approved by: Eleanor Berger
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~al-maisan/launchpad/clog-oops-452070
Merge into: lp:launchpad
Diff against target: 659 lines
13 files modified
lib/canonical/launchpad/webapp/tales.py (+9/-2)
lib/lp/bugs/doc/bug.txt (+28/-0)
lib/lp/bugs/interfaces/bug.py (+7/-0)
lib/lp/bugs/model/bug.py (+10/-1)
lib/lp/registry/browser/distributionsourcepackage.py (+46/-4)
lib/lp/registry/configure.zcml (+1/-0)
lib/lp/registry/doc/sourcepackage.txt (+10/-8)
lib/lp/registry/model/distributionsourcepackage.py (+22/-1)
lib/lp/soyuz/browser/configure.zcml (+0/-3)
lib/lp/soyuz/browser/sourcepackagerelease.py (+117/-96)
lib/lp/soyuz/stories/soyuz/xx-distributionsourcepackagerelease-pages.txt (+34/-0)
lib/lp/soyuz/templates/distributionsourcepackagerelease-changes.pt (+5/-4)
lib/lp/soyuz/templates/sourcepackagerelease-change-summary.pt (+0/-10)
To merge this branch: bzr merge lp:~al-maisan/launchpad/clog-oops-452070
Reviewer Review Type Date Requested Status
Eleanor Berger (community) Approve
Review via email: mp+13789@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Muharem Hrnjadovic (al-maisan) wrote :

Hello there!

The branch at hand optimises pages with a long changelog/history like:

    https://launchpad.net/ubuntu/+source/linux/+changelog

These pages time out on a regular basis due to the number of database
queries issued in order to fetch the bugs/persons needed for the so
called "linkification".

You can see what such a page looks like by following the link below
(since it's less prone to time out).

    https://launchpad.net/ubuntu/+source/bash/+changelog

The optimisations introduced are twofold:

  - the numbers of the bugs that appear on the page are all extracted
    and a *single* database query that pre-loads them into the storm
    cache is issued before the linkification starts.
  - Analogously, the email addresses are extracted from the page and an
    email address/person cache is provided to
    FormattersAPI.linkify_email() in order to avoid many/repeated
    database lookups.

Please note that

  - the bulk of the code was merely refactored i.e. most of the private
    `SourcePackageReleaseView` methods have been converted into
    functions so they can be reused.
  - Deryck consented to having the getByNumbers() method added to IBugSet.

Pre-implementation calls with bigjools and jtv.

Tests to run:

    bin/test -v -t xx-distributionsourcepackagerelease-pages.txt -t sourcepackage

More importantly, this branch was tested thoroughly on the Soyuz dogfood
system. The resulting improvements are as follows:

  - the number of database queries was reduced by approx. 66%
  - the overall page loading time was reduced by approx. 33%

Revision history for this message
Eleanor Berger (intellectronica) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/canonical/launchpad/webapp/tales.py'
2--- lib/canonical/launchpad/webapp/tales.py 2009-09-18 21:54:45 +0000
3+++ lib/canonical/launchpad/webapp/tales.py 2009-10-22 21:21:16 +0000
4@@ -2876,7 +2876,7 @@
5 "<<email address hidden>>", "<email address hidden>")
6 return text
7
8- def linkify_email(self):
9+ def linkify_email(self, preloaded_person_data=None):
10 """Linkify any email address recognised in Launchpad.
11
12 If an email address is recognised as one registered in Launchpad,
13@@ -2891,7 +2891,14 @@
14 matches = re.finditer(self._re_email, text)
15 for match in matches:
16 address = match.group()
17- person = getUtility(IPersonSet).getByEmail(address)
18+ person = None
19+ # First try to find the person required in the preloaded person
20+ # data dictionary.
21+ if preloaded_person_data is not None:
22+ person = preloaded_person_data.get(address, None)
23+ else:
24+ # No pre-loaded data -> we need to perform a database lookup.
25+ person = getUtility(IPersonSet).getByEmail(address)
26 # Only linkify if person exists and does not want to hide
27 # their email addresses.
28 if person is not None and not person.hide_email_addresses:
29
30=== modified file 'lib/lp/bugs/doc/bug.txt'
31--- lib/lp/bugs/doc/bug.txt 2009-07-08 03:14:59 +0000
32+++ lib/lp/bugs/doc/bug.txt 2009-10-22 21:21:16 +0000
33@@ -37,6 +37,34 @@
34 ...
35 NotFoundError: 'Unable to locate bug with nickname +bugs.'
36
37+It is also possible to retrieve a number of bugs by specifying the bug numbers
38+of interest.
39+
40+ >>> result_set = bugset.getByNumbers([6, 1234])
41+ >>> print result_set.count()
42+ 1
43+
44+ >>> [the_bug_found] = result_set
45+ >>> print the_bug_found.title
46+ Firefox crashes when Save As dialog for a nonexistent window is closed
47+
48+ >>> result_set = bugset.getByNumbers([6, 4321, 1])
49+ >>> print result_set.count()
50+ 2
51+
52+ >>> second_bug_found = result_set[1]
53+ >>> print second_bug_found.title
54+ Firefox does not support SVG
55+
56+If no bug numbers are specified an empty result set is returned.
57+
58+ >>> result_set = bugset.getByNumbers(None)
59+ >>> print result_set.count()
60+ 0
61+ >>> result_set = bugset.getByNumbers([])
62+ >>> print result_set.count()
63+ 0
64+
65 == Bug creation events ==
66
67 IObjectCreatedEvent events are fired off when a bug is created. First
68
69=== modified file 'lib/lp/bugs/interfaces/bug.py'
70--- lib/lp/bugs/interfaces/bug.py 2009-08-26 11:06:05 +0000
71+++ lib/lp/bugs/interfaces/bug.py 2009-10-22 21:21:16 +0000
72@@ -937,6 +937,13 @@
73 :param limit: The number of distinct Bugs to return.
74 """
75
76+ def getByNumbers(bug_numbers):
77+ """Get `IBug` instances identified by the `bug_numbers` iterable.
78+
79+ :param bug_numbers: An iterable of bug numbers for which we should
80+ return Bugs.
81+ """
82+
83
84 class InvalidBugTargetType(Exception):
85 """Bug target's type is not valid."""
86
87=== modified file 'lib/lp/bugs/model/bug.py'
88--- lib/lp/bugs/model/bug.py 2009-08-26 01:54:39 +0000
89+++ lib/lp/bugs/model/bug.py 2009-10-22 21:21:16 +0000
90@@ -33,7 +33,7 @@
91 from sqlobject import SQLMultipleJoin, SQLRelatedJoin
92 from sqlobject import SQLObjectNotFound
93 from storm.expr import And, Count, In, LeftJoin, Select, SQLRaw, Func
94-from storm.store import Store
95+from storm.store import EmptyResultSet, Store
96
97 from lazr.lifecycle.event import (
98 ObjectCreatedEvent, ObjectDeletedEvent, ObjectModifiedEvent)
99@@ -49,6 +49,7 @@
100 from canonical.launchpad.interfaces.hwdb import IHWSubmissionBugSet
101 from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities
102 from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
103+from canonical.launchpad.interfaces.lpstorm import IStore
104 from canonical.launchpad.interfaces.message import (
105 IMessage, IndexedMessage)
106 from canonical.launchpad.interfaces.structuralsubscription import (
107@@ -1627,6 +1628,14 @@
108
109 return bugs
110
111+ def getByNumbers(self, bug_numbers):
112+ """see `IBugSet`."""
113+ if bug_numbers is None or len(bug_numbers) < 1:
114+ return EmptyResultSet()
115+ store = IStore(Bug)
116+ result_set = store.find(Bug, In(Bug.id, bug_numbers))
117+ return result_set
118+
119
120 class BugAffectsPerson(SQLBase):
121 """A bug is marked as affecting a user."""
122
123=== modified file 'lib/lp/registry/browser/distributionsourcepackage.py'
124--- lib/lp/registry/browser/distributionsourcepackage.py 2009-10-20 17:57:51 +0000
125+++ lib/lp/registry/browser/distributionsourcepackage.py 2009-10-22 21:21:16 +0000
126@@ -27,6 +27,7 @@
127
128 from canonical.cachedproperty import cachedproperty
129 from canonical.launchpad import _
130+from canonical.launchpad.interfaces import IBugSet
131 from lp.answers.interfaces.questionenums import QuestionStatus
132 from lp.soyuz.interfaces.archive import IArchiveSet
133 from lp.soyuz.interfaces.distributionsourcepackagerelease import (
134@@ -49,6 +50,8 @@
135 from canonical.launchpad.webapp.breadcrumb import Breadcrumb
136
137 from lazr.delegates import delegates
138+from lp.soyuz.browser.sourcepackagerelease import (
139+ extract_bug_numbers, extract_email_addresses, linkify_changelog)
140 from canonical.lazr.utils import smartquote
141
142
143@@ -134,11 +137,14 @@
144 """
145 delegates(IDistributionSourcePackageRelease, 'context')
146
147- def __init__(self, distributionsourcepackagerelease,
148- publishing_history, package_diffs):
149+ def __init__(
150+ self, distributionsourcepackagerelease, publishing_history,
151+ package_diffs, person_data, user):
152 self.context = distributionsourcepackagerelease
153 self._publishing_history = publishing_history
154 self._package_diffs = package_diffs
155+ self._person_data = person_data
156+ self._user = user
157
158 @property
159 def publishing_history(self):
160@@ -150,6 +156,12 @@
161 """ See `ISourcePackageRelease`."""
162 return self._package_diffs
163
164+ @property
165+ def change_summary(self):
166+ """ See `ISourcePackageRelease`."""
167+ return linkify_changelog(
168+ self._user, self.context.change_summary, self._person_data)
169+
170
171 class IDistributionSourcePackageActionMenu(Interface):
172 """Marker interface for the action menu."""
173@@ -172,14 +184,43 @@
174 """Common features to all `DistributionSourcePackage` views."""
175
176 def releases(self):
177+ def not_empty(text):
178+ return (
179+ text is not None and isinstance(text, basestring)
180+ and len(text.strip()) > 0)
181+
182 dspr_pubs = self.context.getReleasesAndPublishingHistory()
183
184 # Return as early as possible to avoid unnecessary processing.
185 if len(dspr_pubs) == 0:
186 return []
187
188+ sprs = [dspr.sourcepackagerelease for (dspr, spphs) in dspr_pubs]
189+ # Pre-load the bugs and persons referenced by the +changelog page from
190+ # the database.
191+ # This will improve the performance of the ensuing changelog
192+ # linkification.
193+ the_changelog = '\n'.join(
194+ [spr.changelog_entry for spr in sprs
195+ if not_empty(spr.changelog_entry)])
196+ unique_bugs = extract_bug_numbers(the_changelog)
197+ self._bug_data = list(
198+ getUtility(IBugSet).getByNumbers(unique_bugs.keys()))
199+ # Preload email/person data only if user is logged on. In the opposite
200+ # case the emails in the changelog will be obfuscated anyway and thus
201+ # cause no database lookups.
202+ if self.user:
203+ unique_emails = extract_email_addresses(the_changelog)
204+ # The method below returns a [(EmailAddress,Person]] result set.
205+ result_set = self.context.getPersonsByEmail(unique_emails)
206+ # Ignore the persons who want their email addresses hidden.
207+ self._person_data = dict(
208+ [(email.email,person) for (email,person) in result_set
209+ if not person.hide_email_addresses])
210+ else:
211+ self._person_data = None
212+
213 # Collate diffs for relevant SourcePackageReleases
214- sprs = [dspr.sourcepackagerelease for (dspr, spphs) in dspr_pubs]
215 pkg_diffs = getUtility(IPackageDiffSet).getDiffsToReleases(sprs)
216 spr_diffs = {}
217 for spr, diffs in itertools.groupby(pkg_diffs,
218@@ -188,7 +229,8 @@
219
220 return [
221 DecoratedDistributionSourcePackageRelease(
222- dspr, spphs, spr_diffs.get(dspr.sourcepackagerelease, []))
223+ dspr, spphs, spr_diffs.get(dspr.sourcepackagerelease, []),
224+ self._person_data, self.user)
225 for (dspr, spphs) in dspr_pubs]
226
227
228
229=== modified file 'lib/lp/registry/configure.zcml'
230--- lib/lp/registry/configure.zcml 2009-10-22 09:24:53 +0000
231+++ lib/lp/registry/configure.zcml 2009-10-22 21:21:16 +0000
232@@ -392,6 +392,7 @@
233 getSubscription
234 parent_subscription_target
235 getBugNotificationsRecipients
236+ getPersonsByEmail
237 getReleasesAndPublishingHistory
238 upstream_product
239 target_type_display
240
241=== modified file 'lib/lp/registry/doc/sourcepackage.txt'
242--- lib/lp/registry/doc/sourcepackage.txt 2009-10-19 21:20:31 +0000
243+++ lib/lp/registry/doc/sourcepackage.txt 2009-10-22 21:21:16 +0000
244@@ -370,32 +370,34 @@
245
246 Zero or more spaces may appear between the ":" and the "#".
247
248- >>> spr_view._linkify_bug_numbers("LP: #10")
249+ >>> from lp.soyuz.browser.sourcepackagerelease import (
250+ ... linkify_bug_numbers)
251+ >>> linkify_bug_numbers("LP: #10")
252 u'LP: <a href="/bugs/10" title="another test bug">#10</a>'
253
254- >>> spr_view._linkify_bug_numbers("LP: #10")
255+ >>> linkify_bug_numbers("LP: #10")
256 u'LP: <a href="/bugs/10" title="another test bug">#10</a>'
257
258- >>> spr_view._linkify_bug_numbers("LP:#10")
259+ >>> linkify_bug_numbers("LP:#10")
260 u'LP:<a href="/bugs/10" title="another test bug">#10</a>'
261
262- >>> spr_view._linkify_bug_numbers("LP: #999")
263+ >>> linkify_bug_numbers("LP: #999")
264 'LP: <a href="/bugs/999" title="No such bug">#999</a>'
265
266- >>> spr_view._linkify_bug_numbers("garbage")
267+ >>> linkify_bug_numbers("garbage")
268 'garbage'
269
270- >>> spr_view._linkify_bug_numbers("LP: #10, #7")
271+ >>> linkify_bug_numbers("LP: #10, #7")
272 u'LP: <a href="/bugs/10" title="another test bug">#10</a>,
273 <a href="/bugs/7" title="A test bug">#7</a>'
274
275- >>> spr_view._linkify_bug_numbers("LP: #10 LP: #10")
276+ >>> linkify_bug_numbers("LP: #10 LP: #10")
277 u'LP: <a href="/bugs/10" title="another test bug">#10</a>
278 LP: <a href="/bugs/10" title="another test bug">#10</a>'
279
280 The regex is case-insensitive, so "lp: #nnn" also works.
281
282- >>> spr_view._linkify_bug_numbers("lp: #10")
283+ >>> linkify_bug_numbers("lp: #10")
284 u'lp: <a href="/bugs/10" title="another test bug">#10</a>'
285
286
287
288=== modified file 'lib/lp/registry/model/distributionsourcepackage.py'
289--- lib/lp/registry/model/distributionsourcepackage.py 2009-09-16 04:31:39 +0000
290+++ lib/lp/registry/model/distributionsourcepackage.py 2009-10-22 21:21:16 +0000
291@@ -15,7 +15,8 @@
292 import operator
293
294 from sqlobject.sqlbuilder import SQLConstant
295-from storm.expr import And, Desc, In
296+from storm.expr import And, Desc, In, Join, Lower
297+from storm.store import EmptyResultSet
298 from storm.locals import Int, Reference, Store, Storm, Unicode
299 from zope.interface import implements
300
301@@ -36,10 +37,13 @@
302 from lp.soyuz.model.sourcepackagerelease import (
303 SourcePackageRelease)
304 from lp.registry.model.karma import KarmaTotalCache
305+from lp.registry.model.person import Person
306 from lp.registry.model.sourcepackage import (
307 SourcePackage, SourcePackageQuestionTargetMixin)
308+from canonical.launchpad.database.emailaddress import EmailAddress
309 from canonical.launchpad.database.structuralsubscription import (
310 StructuralSubscriptionTargetMixin)
311+from canonical.launchpad.interfaces.lpstorm import IStore
312
313 from canonical.lazr.utils import smartquote
314
315@@ -408,6 +412,23 @@
316 'BugTask.distribution = %s AND BugTask.sourcepackagename = %s' %
317 sqlvalues(self.distribution, self.sourcepackagename))
318
319+ @staticmethod
320+ def getPersonsByEmail(email_addresses):
321+ """[(EmailAddress,Person), ..] iterable for given email addresses."""
322+ if email_addresses is None or len(email_addresses) < 1:
323+ return EmptyResultSet()
324+ # Perform basic sanitization of email addresses.
325+ email_addresses = [
326+ address.lower().strip() for address in email_addresses]
327+ store = IStore(Person)
328+ origin = [
329+ Person, Join(EmailAddress, EmailAddress.personID == Person.id)]
330+ # Get all persons whose email addresses are in the list.
331+ result_set = store.using(*origin).find(
332+ (EmailAddress, Person),
333+ In(Lower(EmailAddress.email), email_addresses))
334+ return result_set
335+
336
337 class DistributionSourcePackageInDatabase(Storm):
338 """Temporary class to allow access to the database."""
339
340=== modified file 'lib/lp/soyuz/browser/configure.zcml'
341--- lib/lp/soyuz/browser/configure.zcml 2009-09-29 07:21:40 +0000
342+++ lib/lp/soyuz/browser/configure.zcml 2009-10-22 21:21:16 +0000
343@@ -112,9 +112,6 @@
344 name="+files"
345 facet="overview"
346 template="../templates/sourcepackagerelease-files.pt"/>
347- <browser:page
348- name="+change_summary"
349- template="../templates/sourcepackagerelease-change-summary.pt"/>
350 </browser:pages>
351 <facet
352 facet="overview">
353
354=== modified file 'lib/lp/soyuz/browser/sourcepackagerelease.py'
355--- lib/lp/soyuz/browser/sourcepackagerelease.py 2009-09-09 11:52:55 +0000
356+++ lib/lp/soyuz/browser/sourcepackagerelease.py 2009-10-22 21:21:16 +0000
357@@ -16,108 +16,129 @@
358 from canonical.launchpad.webapp.tales import FormattersAPI
359
360
361+def extract_bug_numbers(text):
362+ '''Unique bug numbers matching the "LP: #n(, #n)*" pattern in the text.'''
363+ # FormattersAPI._linkify_substitution requires a match object
364+ # that has named groups "bug" and "bugnum". The matching text for
365+ # the "bug" group is used as the link text and "bugnum" forms part
366+ # of the URL for the link to the bug. Example:
367+ # >>> bm.groupdict( )
368+ # {'bugnum': '400686', 'bug': '#400686'}
369+
370+ # We need to match bug numbers of the form:
371+ # LP: #1, #2, #3
372+ # #4, #5
373+ # over multiple lines.
374+ #
375+ # Writing a single catch-all regex for this has proved rather hard
376+ # so I am taking the strategy of matching LP:(group) first, and
377+ # feeding the result into another regex to pull out the bug and
378+ # bugnum groups.
379+ unique_bug_matches = dict()
380+
381+ line_matches = re.finditer(
382+ 'LP:\s*(?P<buglist>(.+?[^,]))($|\n)', text,
383+ re.DOTALL | re.IGNORECASE)
384+
385+ for line_match in line_matches:
386+ bug_matches = re.finditer(
387+ '\s*((?P<bug>#(?P<bugnum>\d+)),?\s*)',
388+ line_match.group('buglist'))
389+
390+ for bug_match in bug_matches:
391+ bugnum = bug_match.group('bugnum')
392+ if bugnum in unique_bug_matches:
393+ # We got this bug already, ignore it.
394+ continue
395+ unique_bug_matches[bugnum] = bug_match
396+
397+ return unique_bug_matches
398+
399+
400+def linkify_bug_numbers(text):
401+ """Linkify to a bug if LP: #number appears in the (changelog) text."""
402+ unique_bug_matches = extract_bug_numbers(text)
403+ for bug_match in unique_bug_matches.values():
404+ replace_text = bug_match.group('bug')
405+ if replace_text is not None:
406+ # XXX julian 2008-01-10
407+ # Note that re.sub would be far more efficient to use
408+ # instead of string.replace() but this requires a regex
409+ # that matches everything in one go. We're also at danger
410+ # of replacing the wrong thing if string.replace() finds
411+ # other matching substrings. So for example in the
412+ # string:
413+ # "LP: #9, #999"
414+ # replacing #9 with some HTML would also interfere with
415+ # #999. The liklihood of this happening is very, very
416+ # small, however.
417+ text = text.replace(
418+ replace_text,
419+ FormattersAPI._linkify_substitution(bug_match))
420+ return text
421+
422+
423+def extract_email_addresses(text):
424+ '''Unique email adresses in the text.'''
425+ matches = re.finditer(FormattersAPI._re_email, text)
426+ return list(set([match.group() for match in matches]))
427+
428+
429+def obfuscate_email(user, text):
430+ """Obfuscate email addresses if the user is not logged in."""
431+ if not text:
432+ # If there is nothing to obfuscate, the FormattersAPI
433+ # will blow up, so just return.
434+ return text
435+ formatter = FormattersAPI(text)
436+ if user:
437+ return text
438+ else:
439+ return formatter.obfuscate_email()
440+
441+
442+def linkify_email(text, preloaded_person_data):
443+ """Email addresses are linkified to point to the person's profile."""
444+ formatter = FormattersAPI(text)
445+ return formatter.linkify_email(preloaded_person_data)
446+
447+
448+def linkify_changelog(user, changelog, preloaded_person_data=None):
449+ """Linkify the changelog.
450+
451+ This obfuscates email addresses to anonymous users, linkifies
452+ them for non-anonymous and links to the bug page for any bug
453+ numbers mentioned.
454+ """
455+ if changelog is None:
456+ return ''
457+
458+ # Remove any email addresses if the user is not logged in.
459+ changelog = obfuscate_email(user, changelog)
460+
461+ # CGI Escape the changelog here before further replacements
462+ # insert HTML. Email obfuscation does not insert HTML but can insert
463+ # characters that must be escaped.
464+ changelog = cgi.escape(changelog)
465+
466+ # Any email addresses remaining in the changelog were not obfuscated,
467+ # so we linkify them here.
468+ changelog = linkify_email(changelog, preloaded_person_data)
469+
470+ # Ensure any bug numbers are linkified to the bug page.
471+ changelog = linkify_bug_numbers(changelog)
472+
473+ return changelog
474+
475+
476 class SourcePackageReleaseView(LaunchpadView):
477
478 @property
479 def changelog_entry(self):
480 """Return a linkified changelog entry."""
481- return self._linkify_changelog(self.context.changelog_entry)
482+ return linkify_changelog(self.user, self.context.changelog_entry)
483
484 @property
485 def change_summary(self):
486 """Return a linkified change summary."""
487- return self._linkify_changelog(self.context.change_summary)
488-
489- def _obfuscate_email(self, text):
490- """Obfuscate email addresses if the user is not logged in."""
491- if not text:
492- # If there is nothing to obfuscate, the FormattersAPI
493- # will blow up, so just return.
494- return text
495- formatter = FormattersAPI(text)
496- if self.user:
497- return text
498- else:
499- return formatter.obfuscate_email()
500-
501- def _linkify_email(self, text):
502- """Email addresses are linkified to point to the person's profile."""
503- formatter = FormattersAPI(text)
504- return formatter.linkify_email()
505-
506- def _linkify_bug_numbers(self, changelog):
507- """Linkify to a bug if LP: #number appears in the changelog text."""
508- # FormattersAPI._linkify_substitution requires a match object
509- # that has named groups "bug" and "bugnum". The matching text for
510- # the "bug" group is used as the link text and "bugnum" forms part
511- # of the URL for the link to the bug.
512-
513- # We need to match bug numbers of the form:
514- # LP: #1, #2, #3
515- # #4, #5
516- # over multiple lines.
517- #
518- # Writing a single catch-all regex for this has proved rather hard
519- # so I am taking the strategy of matching LP:(group) first, and
520- # feeding the result into another regex to pull out the bug and
521- # bugnum groups.
522- line_matches = re.finditer(
523- 'LP:\s*(?P<buglist>(.+?[^,]))($|\n)', changelog,
524- re.DOTALL | re.IGNORECASE)
525- seen_bugnums = set()
526- for line_match in line_matches:
527- bug_matches = re.finditer(
528- '\s*((?P<bug>#(?P<bugnum>\d+)),?\s*)',
529- line_match.group('buglist'))
530- for bug_match in bug_matches:
531- bugnum = bug_match.group('bug')
532- if bugnum in seen_bugnums:
533- # This bug was already replaced across the entire text of
534- # the changelog.
535- continue
536- seen_bugnums.add(bugnum)
537- replace_text = bug_match.group('bug')
538- if replace_text is not None:
539- # XXX julian 2008-01-10
540- # Note that re.sub would be far more efficient to use
541- # instead of string.replace() but this requires a regex
542- # that matches everything in one go. We're also at danger
543- # of replacing the wrong thing if string.replace() finds
544- # other matching substrings. So for example in the
545- # string:
546- # "LP: #9, #999"
547- # replacing #9 with some HTML would also interfere with
548- # #999. The liklihood of this happening is very, very
549- # small, however.
550- changelog = changelog.replace(
551- replace_text,
552- FormattersAPI._linkify_substitution(bug_match))
553- return changelog
554-
555- def _linkify_changelog(self, changelog):
556- """Linkify the changelog.
557-
558- This obfuscates email addresses to anonymous users, linkifies
559- them for non-anonymous and links to the bug page for any bug
560- numbers mentioned.
561- """
562- if changelog is None:
563- return ''
564-
565- # Remove any email addresses if the user is not logged in.
566- changelog = self._obfuscate_email(changelog)
567-
568- # CGI Escape the changelog here before further replacements
569- # insert HTML. Email obfuscation does not insert HTML but can insert
570- # characters that must be escaped.
571- changelog = cgi.escape(changelog)
572-
573- # Any email addresses remaining in the changelog were not obfuscated,
574- # so we linkify them here.
575- changelog = self._linkify_email(changelog)
576-
577- # Ensure any bug numbers are linkified to the bug page.
578- changelog = self._linkify_bug_numbers(changelog)
579-
580- return changelog
581-
582+ return linkify_changelog(self.user, self.context.change_summary)
583
584=== modified file 'lib/lp/soyuz/stories/soyuz/xx-distributionsourcepackagerelease-pages.txt'
585--- lib/lp/soyuz/stories/soyuz/xx-distributionsourcepackagerelease-pages.txt 2009-09-23 14:58:12 +0000
586+++ lib/lp/soyuz/stories/soyuz/xx-distributionsourcepackagerelease-pages.txt 2009-10-22 21:21:16 +0000
587@@ -270,3 +270,37 @@
588 Binary packages built by this source
589 foo-bin: Foo app is great
590 Well ... it does nothing, though
591+
592+The full change log can be viewed as follows:
593+
594+ >>> user_browser.open(
595+ ... 'http://launchpad.dev/ubuntutest/+source/testing-dspr')
596+ >>> user_browser.getLink('View full change log').click()
597+ >>> print user_browser.url
598+ http://launchpad.dev/ubuntutest/+source/testing-dspr/+changelog
599+
600+ >>> print_tag_with_id(user_browser.contents, 'body_testing-dspr_1.0')
601+ Testing!!!
602+ biscuit@canonical.com
603+ LP: #1234
604+ celso.providelo@canonical.com
605+
606+ >>> print extract_text(find_main_content(user_browser.contents))
607+ Change log for “testing-dspr” package in ubuntutest
608+ ubuntutest
609+ “testing-dspr” package
610+ Change log
611+ 0.9
612+ Pending in breezy-autotest-release
613+ since ...
614+ None
615+ 1.0
616+ Published in breezy-autotest-release
617+ 2 seconds ago
618+ Testing!!!
619+ biscuit@canonical.com
620+ LP: #1234
621+ celso.providelo@canonical.com
622+ Available diffs
623+ 0.9 to 1.0
624+ (7 bytes)
625
626=== modified file 'lib/lp/soyuz/templates/distributionsourcepackagerelease-changes.pt'
627--- lib/lp/soyuz/templates/distributionsourcepackagerelease-changes.pt 2009-07-17 17:59:07 +0000
628+++ lib/lp/soyuz/templates/distributionsourcepackagerelease-changes.pt 2009-10-22 21:21:16 +0000
629@@ -20,10 +20,11 @@
630 </div>
631
632 <div class="boardCommentBody">
633- <div
634- tal:attributes="id string:body_${context/name}_${context/version}"
635- tal:content="structure context/sourcepackagerelease/@@+change_summary" />
636+ <div tal:attributes="id string:body_${context/name}_${context/version}">
637+ <pre style="margin: 0 0 0.3em 0"
638+ tal:attributes="id string:${context/name}_${context/version}"
639+ tal:content="structure context/change_summary" />
640+ </div>
641 <tal:diffs replace="structure context/@@+diffs" />
642 </div>
643-
644 </div>
645
646=== removed file 'lib/lp/soyuz/templates/sourcepackagerelease-change-summary.pt'
647--- lib/lp/soyuz/templates/sourcepackagerelease-change-summary.pt 2009-07-17 17:59:07 +0000
648+++ lib/lp/soyuz/templates/sourcepackagerelease-change-summary.pt 1970-01-01 00:00:00 +0000
649@@ -1,10 +0,0 @@
650-<tal:root
651- xmlns:tal="http://xml.zope.org/namespaces/tal"
652- xmlns:metal="http://xml.zope.org/namespaces/metal"
653- xmlns:i18n="http://xml.zope.org/namespaces/i18n"
654- omit-tag="">
655-
656-<pre style="margin: 0 0 0.3em 0"
657- tal:attributes="id string:${context/name}_${context/version}"
658- tal:content="structure view/change_summary" />
659-</tal:root>