Merge lp:~intellectronica/launchpad/bug-heat-days-active into lp:launchpad/db-devel

Proposed by Eleanor Berger
Status: Merged
Approved by: Eleanor Berger
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~intellectronica/launchpad/bug-heat-days-active
Merge into: lp:launchpad/db-devel
Prerequisite: lp:~intellectronica/launchpad/bug-heat-degrade
Diff against target: 1026 lines (+959/-2)
8 files modified
lib/launchpad_loggerhead/__init__.py (+1/-0)
lib/launchpad_loggerhead/app.py (+215/-0)
lib/launchpad_loggerhead/debug.py (+120/-0)
lib/launchpad_loggerhead/session.py (+73/-0)
lib/launchpad_loggerhead/static/robots.txt (+2/-0)
lib/lp/bugs/scripts/bugheat.py (+16/-1)
lib/lp/bugs/scripts/tests/test_bugheat.py (+21/-1)
lib/lp/code/model/tests/test_sourcepackagerecipe.py (+511/-0)
To merge this branch: bzr merge lp:~intellectronica/launchpad/bug-heat-days-active
Reviewer Review Type Date Requested Status
Michael Nelson (community) code Approve
Review via email: mp+23921@code.launchpad.net

Commit message

Add a proportion of the maximum bug heat to a bug's heat for every day since the bug was created.

Description of the change

This branch is the last in a series of branches to make bug heat more sensitive to bug activity. In this branch we change the formula so that a proportion of the maximum heat for a bug is added for every day since it was created (this is offset by the decrease in bug heat for time since last activity, already in place).

To post a comment you must log in.
Revision history for this message
Michael Nelson (michael.nelson) wrote :

There seems to be an inconsistency between the comment about the formula and the formula itself (regarding days_since_last_activity). From our IRC chat (below) and the related bug title, the comment should be:

> === modified file 'lib/lp/bugs/scripts/bugheat.py'
> --- lib/lp/bugs/scripts/bugheat.py 2010-04-22 12:14:18 +0000
> +++ lib/lp/bugs/scripts/bugheat.py 2010-04-22 12:14:19 +0000
> @@ -86,5 +86,19 @@
> self.bug.date_last_updated.replace(tzinfo=None)).days
> total_heat = int(total_heat * (0.99 ** days))
>
> - return total_heat
> + # Bug heat increases by a quarter of the maximum bug heat divided by
> + # the number of days between the bug's creating and its last activity.

         # the number of days since the bugs last activity.

> + days_since_last_activity = (
> + datetime.utcnow() -
> + max(self.bug.date_last_updated.replace(tzinfo=None),
> + self.bug.date_last_message.replace(tzinfo=None))).days
> + days_since_created = (
> + datetime.utcnow() - self.bug.datecreated.replace(tzinfo=None)).days
> + if days_since_created > 0:
> + max_heat = max(
> + task.target.max_bug_heat for task in self.bug.bugtasks)
> + if max_heat is not None:
> + total_heat = total_heat + (max_heat * 0.25 / days_since_created)

s/days_since_created/days_since_last_activity

> +
> + return int(total_heat)

Similarly in the test:

> + expected = int((fresh_heat * (0.99 ** 10)) + (100 * 0.25 / 20))
s/20/10

{{{
14:17 < noodles775> intellectronica: I can't see that days_since_last_activity is being used for anything?
14:18 < noodles775> Did you mean to do (max_heat * 0.25 / (days_since_created - days_since_last_activity)) or something similar, looking at the comment?
14:19 < noodles775> Ah, or looking at the related bug title, I'm guessing it should be s/dasy_since_created/days_since_last_activity on line 21 on the MP diff?
14:21 < intellectronica> noodles775: no, i think that's a cut-n-paste error.
14:27 < noodles775> intellectronica: so should it be divided by the number of days *since* the bugs last activity (as stated in the bug title), or divided by the difference between the days since the bug was created and it's last activity (as the comment seems to suggest?)
14:28 < intellectronica> noodles775: it should be divided by the days since the bug's creation
14:28 < noodles775> intellectronica: so the bug 567439 title is wrong then, ok.
14:28 < mup> Bug #567439: Add MAX_HEAT / 4 / days since last activity to bug heat <story-bug-heat> <Launchpad Bugs:In Progress by intellectronica> <https://launchpad.net/bugs/567439>
14:29 < intellectronica> noodles775: oh, right, it is. there's another bug for a calculation based on time since last activity, i must have confused them.
}}}

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== removed directory 'lib/canonical/launchpad/apidoc'
=== added directory 'lib/launchpad_loggerhead'
=== added file 'lib/launchpad_loggerhead/__init__.py'
--- lib/launchpad_loggerhead/__init__.py 1970-01-01 00:00:00 +0000
+++ lib/launchpad_loggerhead/__init__.py 2010-04-27 01:35:56 +0000
@@ -0,0 +1,1 @@
1
02
=== added file 'lib/launchpad_loggerhead/app.py'
--- lib/launchpad_loggerhead/app.py 1970-01-01 00:00:00 +0000
+++ lib/launchpad_loggerhead/app.py 2010-04-27 01:39:55 +0000
@@ -0,0 +1,215 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4import logging
5import os
6import threading
7import urllib
8import urlparse
9import xmlrpclib
10
11from bzrlib import errors, lru_cache, urlutils
12
13from loggerhead.apps import favicon_app, static_app
14from loggerhead.apps.branch import BranchWSGIApp
15
16from openid.extensions.sreg import SRegRequest, SRegResponse
17from openid.consumer.consumer import CANCEL, Consumer, FAILURE, SUCCESS
18from openid.store.memstore import MemoryStore
19
20from paste.fileapp import DataApp
21from paste.request import construct_url, parse_querystring, path_info_pop
22from paste.httpexceptions import (
23 HTTPMovedPermanently, HTTPNotFound, HTTPUnauthorized)
24
25from canonical.config import config
26from canonical.launchpad.xmlrpc import faults
27from canonical.launchpad.webapp.vhosts import allvhosts
28from lp.code.interfaces.codehosting import (
29 BRANCH_TRANSPORT, LAUNCHPAD_ANONYMOUS)
30from lp.codehosting.vfs import get_lp_server
31from lp.codehosting.bzrutils import safe_open
32
33robots_txt = '''\
34User-agent: *
35Disallow: /
36'''
37
38robots_app = DataApp(robots_txt, content_type='text/plain')
39
40
41thread_transports = threading.local()
42
43
44def check_fault(fault, *fault_classes):
45 """Check if 'fault's faultCode matches any of 'fault_classes'.
46
47 :param fault: An instance of `xmlrpclib.Fault`.
48 :param fault_classes: Any number of `LaunchpadFault` subclasses.
49 """
50 for cls in fault_classes:
51 if fault.faultCode == cls.error_code:
52 return True
53 return False
54
55
56class RootApp:
57
58 def __init__(self, session_var):
59 self.graph_cache = lru_cache.LRUCache(10)
60 self.branchfs = xmlrpclib.ServerProxy(
61 config.codehosting.codehosting_endpoint)
62 self.session_var = session_var
63 self.store = MemoryStore()
64 self.log = logging.getLogger('lp-loggerhead')
65
66 def get_transports(self):
67 t = getattr(thread_transports, 'transports', None)
68 if t is None:
69 thread_transports.transports = []
70 return thread_transports.transports
71
72 def _make_consumer(self, environ):
73 """Build an OpenID `Consumer` object with standard arguments."""
74 return Consumer(environ[self.session_var], self.store)
75
76 def _begin_login(self, environ, start_response):
77 """Start the process of authenticating with OpenID.
78
79 We redirect the user to Launchpad to identify themselves, asking to be
80 sent their nickname. Launchpad will then redirect them to our +login
81 page with enough information that we can then redirect them again to
82 the page they were looking at, with a cookie that gives us the
83 username.
84 """
85 openid_vhost = config.launchpad.openid_provider_vhost
86 openid_request = self._make_consumer(environ).begin(
87 allvhosts.configs[openid_vhost].rooturl)
88 openid_request.addExtension(
89 SRegRequest(required=['nickname']))
90 back_to = construct_url(environ)
91 raise HTTPMovedPermanently(openid_request.redirectURL(
92 config.codehosting.secure_codebrowse_root,
93 config.codehosting.secure_codebrowse_root + '+login/?'
94 + urllib.urlencode({'back_to':back_to})))
95
96 def _complete_login(self, environ, start_response):
97 """Complete the OpenID authentication process.
98
99 Here we handle the result of the OpenID process. If the process
100 succeeded, we record the username in the session and redirect the user
101 to the page they were trying to view that triggered the login attempt.
102 In the various failures cases we return a 401 Unauthorized response
103 with a brief explanation of what went wrong.
104 """
105 query = dict(parse_querystring(environ))
106 # Passing query['openid.return_to'] here is massive cheating, but
107 # given we control the endpoint who cares.
108 response = self._make_consumer(environ).complete(
109 query, query['openid.return_to'])
110 if response.status == SUCCESS:
111 self.log.error('open id response: SUCCESS')
112 sreg_info = SRegResponse.fromSuccessResponse(response)
113 print sreg_info
114 environ[self.session_var]['user'] = sreg_info['nickname']
115 raise HTTPMovedPermanently(query['back_to'])
116 elif response.status == FAILURE:
117 self.log.error('open id response: FAILURE: %s', response.message)
118 exc = HTTPUnauthorized()
119 exc.explanation = response.message
120 raise exc
121 elif response.status == CANCEL:
122 self.log.error('open id response: CANCEL')
123 exc = HTTPUnauthorized()
124 exc.explanation = "Authetication cancelled."
125 raise exc
126 else:
127 self.log.error('open id response: UNKNOWN')
128 exc = HTTPUnauthorized()
129 exc.explanation = "Unknown OpenID response."
130 raise exc
131
132 def __call__(self, environ, start_response):
133 environ['loggerhead.static.url'] = environ['SCRIPT_NAME']
134 if environ['PATH_INFO'].startswith('/static/'):
135 path_info_pop(environ)
136 return static_app(environ, start_response)
137 elif environ['PATH_INFO'] == '/favicon.ico':
138 return favicon_app(environ, start_response)
139 elif environ['PATH_INFO'] == '/robots.txt':
140 return robots_app(environ, start_response)
141 elif environ['PATH_INFO'].startswith('/+login'):
142 return self._complete_login(environ, start_response)
143 path = environ['PATH_INFO']
144 trailingSlashCount = len(path) - len(path.rstrip('/'))
145 user = environ[self.session_var].get('user', LAUNCHPAD_ANONYMOUS)
146 lp_server = get_lp_server(
147 user, branch_url=config.codehosting.internal_branch_by_id_root)
148 lp_server.start_server()
149 try:
150 try:
151 transport_type, info, trail = self.branchfs.translatePath(
152 user, urlutils.escape(path))
153 except xmlrpclib.Fault, f:
154 if check_fault(f, faults.PathTranslationError):
155 raise HTTPNotFound()
156 elif check_fault(f, faults.PermissionDenied):
157 # If we're not allowed to see the branch...
158 if environ['wsgi.url_scheme'] != 'https':
159 # ... the request shouldn't have come in over http, as
160 # requests for private branches over http should be
161 # redirected to https by the dynamic rewrite script we
162 # use (which runs before this code is reached), but
163 # just in case...
164 env_copy = environ.copy()
165 env_copy['wsgi.url_scheme'] = 'https'
166 raise HTTPMovedPermanently(construct_url(env_copy))
167 elif user != LAUNCHPAD_ANONYMOUS:
168 # ... if the user is already logged in and still can't
169 # see the branch, they lose.
170 exc = HTTPUnauthorized()
171 exc.explanation = "You are logged in as %s." % user
172 raise exc
173 else:
174 # ... otherwise, lets give them a chance to log in
175 # with OpenID.
176 return self._begin_login(environ, start_response)
177 else:
178 raise
179 if transport_type != BRANCH_TRANSPORT:
180 raise HTTPNotFound()
181 trail = urlutils.unescape(trail).encode('utf-8')
182 trail += trailingSlashCount * '/'
183 amount_consumed = len(path) - len(trail)
184 consumed = path[:amount_consumed]
185 branch_name = consumed.strip('/')
186 self.log.info('Using branch: %s', branch_name)
187 if trail and not trail.startswith('/'):
188 trail = '/' + trail
189 environ['PATH_INFO'] = trail
190 environ['SCRIPT_NAME'] += consumed.rstrip('/')
191 branch_url = lp_server.get_url() + branch_name
192 branch_link = urlparse.urljoin(
193 config.codebrowse.launchpad_root, branch_name)
194 cachepath = os.path.join(
195 config.codebrowse.cachepath, branch_name[1:])
196 if not os.path.isdir(cachepath):
197 os.makedirs(cachepath)
198 self.log.info('branch_url: %s', branch_url)
199 try:
200 bzr_branch = safe_open(
201 lp_server.get_url().strip(':/'), branch_url,
202 possible_transports=self.get_transports())
203 except errors.NotBranchError, err:
204 self.log.warning('Not a branch: %s', err)
205 raise HTTPNotFound()
206 bzr_branch.lock_read()
207 try:
208 view = BranchWSGIApp(
209 bzr_branch, branch_name, {'cachepath': cachepath},
210 self.graph_cache, branch_link=branch_link, served_url=None)
211 return view.app(environ, start_response)
212 finally:
213 bzr_branch.unlock()
214 finally:
215 lp_server.stop_server()
0216
=== added file 'lib/launchpad_loggerhead/debug.py'
--- lib/launchpad_loggerhead/debug.py 1970-01-01 00:00:00 +0000
+++ lib/launchpad_loggerhead/debug.py 2010-04-27 01:35:56 +0000
@@ -0,0 +1,120 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4import thread
5import time
6
7from paste.request import construct_url
8
9
10def tabulate(cells):
11 """Format a list of lists of strings in a table.
12
13 The 'cells' are centered.
14
15 >>> print ''.join(tabulate(
16 ... [['title 1', 'title 2'],
17 ... ['short', 'rather longer']]))
18 title 1 title 2
19 short rather longer
20 """
21 widths = {}
22 for row in cells:
23 for col_index, cell in enumerate(row):
24 widths[col_index] = max(len(cell), widths.get(col_index, 0))
25 result = []
26 for row in cells:
27 result_row = ''
28 for col_index, cell in enumerate(row):
29 result_row += cell.center(widths[col_index] + 2)
30 result.append(result_row.rstrip() + '\n')
31 return result
32
33
34def threadpool_debug(app):
35 """Wrap `app` to provide debugging information about the threadpool state.
36
37 The returned application will serve debugging information about the state
38 of the threadpool at '/thread-debug' -- but only when accessed directly,
39 not when accessed through Apache.
40 """
41 def wrapped(environ, start_response):
42 if ('HTTP_X_FORWARDED_SERVER' in environ
43 or environ['PATH_INFO'] != '/thread-debug'):
44 environ['lp.timestarted'] = time.time()
45 return app(environ, start_response)
46 threadpool = environ['paste.httpserver.thread_pool']
47 start_response("200 Ok", [])
48 output = [("url", "time running", "time since last activity")]
49 now = time.time()
50 # Because we're accessing mutable structures without locks here,
51 # we're a bit cautious about things looking like we expect -- if a
52 # worker doesn't seem fully set up, we just ignore it.
53 for worker in threadpool.workers:
54 if not hasattr(worker, 'thread_id'):
55 continue
56 time_started, info = threadpool.worker_tracker.get(
57 worker.thread_id, (None, None))
58 if time_started is not None and info is not None:
59 real_time_started = info.get(
60 'lp.timestarted', time_started)
61 output.append(
62 map(str,
63 (construct_url(info),
64 now - real_time_started,
65 now - time_started,)))
66 return tabulate(output)
67 return wrapped
68
69
70def change_kill_thread_criteria(application):
71 """Interfere with threadpool so that threads are killed for inactivity.
72
73 The usual rules with paste's threadpool is that a thread that takes longer
74 than 'hung_thread_limit' seconds to process a request is considered hung
75 and more than 'kill_thread_limit' seconds is killed.
76
77 Because loggerhead streams its output, how long the entire request takes
78 to process depends on things like how fast the users internet connection
79 is. What we'd like to do is kill threads that don't _start_ to produce
80 output for 'kill_thread_limit' seconds.
81
82 What this class actually does is arrange things so that threads that
83 produce no output for 'kill_thread_limit' are killed, because that's the
84 rule Apache uses when interpreting ProxyTimeout.
85 """
86 def wrapped_application(environ, start_response):
87 threadpool = environ['paste.httpserver.thread_pool']
88 def reset_timer():
89 """Make this thread safe for another 'kill_thread_limit' seconds.
90
91 We do this by hacking the threadpool's record of when this thread
92 started to pretend that it started right now. Hacky, but it's
93 enough to fool paste.httpserver.ThreadPool.kill_hung_threads and
94 that's what matters.
95 """
96 threadpool.worker_tracker[thread.get_ident()][0] = time.time()
97 def response_hook(status, response_headers, exc_info=None):
98 # We reset the timer when the HTTP headers are sent...
99 reset_timer()
100 writer = start_response(status, response_headers, exc_info)
101 def wrapped_writer(arg):
102 # ... and whenever more output has been generated.
103 reset_timer()
104 return writer(arg)
105 return wrapped_writer
106 result = application(environ, response_hook)
107 # WSGI allows the application to return an iterable, which could be a
108 # generator that does significant processing between successive items,
109 # so we should reset the timer between each item.
110 #
111 # This isn't really necessary as loggerhead doesn't return any
112 # non-trivial iterables to the WSGI server. But it's probably better
113 # to cope with this case to avoid nasty suprises if loggerhead
114 # changes.
115 def reset_timer_between_items(iterable):
116 for item in iterable:
117 reset_timer()
118 yield item
119 return reset_timer_between_items(result)
120 return wrapped_application
0121
=== added file 'lib/launchpad_loggerhead/session.py'
--- lib/launchpad_loggerhead/session.py 1970-01-01 00:00:00 +0000
+++ lib/launchpad_loggerhead/session.py 2010-04-27 01:35:56 +0000
@@ -0,0 +1,73 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Simple paste-y session manager tuned for the needs of launchpad-loggerhead.
5"""
6
7import pickle
8
9from paste.auth.cookie import AuthCookieHandler, AuthCookieSigner
10
11
12class MyAuthCookieSigner(AuthCookieSigner):
13 """Fix a bug in AuthCookieSigner."""
14
15 def sign(self, content):
16 # XXX 2008-01-13 Michael Hudson: paste.auth.cookie generates bogus
17 # cookies when the value is long:
18 # http://trac.pythonpaste.org/pythonpaste/ticket/257. This is fixed
19 # now, so when a new version is released and packaged we can remove
20 # this class.
21 r = AuthCookieSigner.sign(self, content)
22 return r.replace('\n', '')
23
24
25class SessionHandler(object):
26 """Middleware that provides a cookie-based session.
27
28 The session dict is stored, pickled (and HMACed), in a cookie, so don't
29 store very much in the session!
30 """
31
32 def __init__(self, application, session_var, secret=None):
33 """Initialize a SessionHandler instance.
34
35 :param application: This is the wrapped application which will have
36 access to the ``environ[session_var]`` dictionary managed by this
37 middleware.
38 :param session_var: The key under which to store the session
39 dictionary in the environment.
40 :param secret: A secret value used for signing the cookie. If not
41 supplied, a new secret will be used for each instantiation of the
42 SessionHandler.
43 """
44 self.application = application
45 self.cookie_handler = AuthCookieHandler(
46 self._process, scanlist=[session_var],
47 signer=MyAuthCookieSigner(secret))
48 self.session_var = session_var
49
50 def __call__(self, environ, start_response):
51 # We need to put the request through the cookie handler first, so we
52 # can access the validated string in the environ in `_process` below.
53 return self.cookie_handler(environ, start_response)
54
55 def _process(self, environ, start_response):
56 """Process a request.
57
58 AuthCookieHandler takes care of getting the text value of the session
59 in and out of the cookie (and validating the text using HMAC) so we
60 just need to convert that string to and from a real dictionary using
61 pickle.
62 """
63 if self.session_var in environ:
64 session = pickle.loads(environ[self.session_var])
65 else:
66 session = {}
67 environ[self.session_var] = session
68 def response_hook(status, response_headers, exc_info=None):
69 session = environ.pop(self.session_var)
70 if session:
71 environ[self.session_var] = pickle.dumps(session)
72 return start_response(status, response_headers, exc_info)
73 return self.application(environ, response_hook)
074
=== added directory 'lib/launchpad_loggerhead/static'
=== added file 'lib/launchpad_loggerhead/static/robots.txt'
--- lib/launchpad_loggerhead/static/robots.txt 1970-01-01 00:00:00 +0000
+++ lib/launchpad_loggerhead/static/robots.txt 2010-04-27 01:35:56 +0000
@@ -0,0 +1,2 @@
1User-agent: *
2Disallow: /
03
=== modified file 'lib/lp/bugs/scripts/bugheat.py'
--- lib/lp/bugs/scripts/bugheat.py 2010-04-28 21:40:12 +0000
+++ lib/lp/bugs/scripts/bugheat.py 2010-04-29 14:53:44 +0000
@@ -90,4 +90,19 @@
90 self.bug.date_last_updated.replace(tzinfo=None)).days90 self.bug.date_last_updated.replace(tzinfo=None)).days
91 total_heat = int(total_heat * (0.99 ** days))91 total_heat = int(total_heat * (0.99 ** days))
9292
93 return total_heat93 if days > 0:
94 # Bug heat increases by a quarter of the maximum bug heat divided
95 # by the number of days since the bug's creation date.
96 days_since_last_activity = (
97 datetime.utcnow() -
98 max(self.bug.date_last_updated.replace(tzinfo=None),
99 self.bug.date_last_message.replace(tzinfo=None))).days
100 days_since_created = (
101 datetime.utcnow() - self.bug.datecreated.replace(tzinfo=None)).days
102 max_heat = max(
103 task.target.max_bug_heat for task in self.bug.bugtasks)
104 if max_heat is not None and days_since_created > 0:
105 total_heat = total_heat + (max_heat * 0.25 / days_since_created)
106
107 return int(total_heat)
108
94109
=== modified file 'lib/lp/bugs/scripts/tests/test_bugheat.py'
--- lib/lp/bugs/scripts/tests/test_bugheat.py 2010-04-28 21:40:12 +0000
+++ lib/lp/bugs/scripts/tests/test_bugheat.py 2010-04-29 14:53:44 +0000
@@ -7,7 +7,7 @@
77
8import unittest8import unittest
99
10from datetime import timedelta10from datetime import datetime, timedelta
1111
12from canonical.testing import LaunchpadZopelessLayer12from canonical.testing import LaunchpadZopelessLayer
1313
@@ -15,6 +15,9 @@
15from lp.bugs.scripts.bugheat import BugHeatCalculator, BugHeatConstants15from lp.bugs.scripts.bugheat import BugHeatCalculator, BugHeatConstants
16from lp.testing import TestCaseWithFactory16from lp.testing import TestCaseWithFactory
1717
18from zope.security.proxy import removeSecurityProxy
19
20
18class TestBugHeatCalculator(TestCaseWithFactory):21class TestBugHeatCalculator(TestCaseWithFactory):
19 """Tests for the BugHeatCalculator class."""22 """Tests for the BugHeatCalculator class."""
20 # If you change the way that bug heat is calculated, remember to update23 # If you change the way that bug heat is calculated, remember to update
@@ -231,6 +234,23 @@
231 "Expected bug heat did not match actual bug heat. "234 "Expected bug heat did not match actual bug heat. "
232 "Expected %s, got %s" % (expected, heat))235 "Expected %s, got %s" % (expected, heat))
233236
237 def test_getBugHeat_activity(self):
238 # Bug heat increases by a quarter of the maximum bug heat divided by
239 # the number of days between the bug's creating and its last activity.
240 active_bug = removeSecurityProxy(self.factory.makeBug())
241 fresh_heat = BugHeatCalculator(active_bug).getBugHeat()
242 active_bug.date_last_updated = (
243 active_bug.date_last_updated - timedelta(days=10))
244 active_bug.datecreated = (active_bug.datecreated - timedelta(days=20))
245 active_bug.default_bugtask.target.setMaxBugHeat(100)
246 expected = int((fresh_heat * (0.99 ** 20)) + (100 * 0.25 / 20))
247 heat = BugHeatCalculator(active_bug).getBugHeat()
248 self.assertEqual(
249 expected, heat,
250 "Expected bug heat did not match actual bug heat. "
251 "Expected %s, got %s" % (expected, heat))
252
253
234254
235def test_suite():255def test_suite():
236 return unittest.TestLoader().loadTestsFromName(__name__)256 return unittest.TestLoader().loadTestsFromName(__name__)
237257
=== added file 'lib/lp/code/model/tests/test_sourcepackagerecipe.py'
--- lib/lp/code/model/tests/test_sourcepackagerecipe.py 1970-01-01 00:00:00 +0000
+++ lib/lp/code/model/tests/test_sourcepackagerecipe.py 2010-04-20 03:27:10 +0000
@@ -0,0 +1,511 @@
1# Copyright 2009, 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Tests for the SourcePackageRecipe content type."""
5
6from __future__ import with_statement
7
8__metaclass__ = type
9
10from datetime import datetime
11import textwrap
12import unittest
13
14from bzrlib.plugins.builder.recipe import RecipeParser
15
16from pytz import UTC
17from storm.locals import Store
18
19from zope.component import getUtility
20from zope.security.interfaces import Unauthorized
21from zope.security.proxy import removeSecurityProxy
22
23from canonical.testing.layers import DatabaseFunctionalLayer
24
25from canonical.launchpad.webapp.authorization import check_permission
26from lp.archiveuploader.permission import (
27 ArchiveDisabled, CannotUploadToArchive, InvalidPocketForPPA)
28from lp.buildmaster.interfaces.buildqueue import IBuildQueue
29from lp.buildmaster.model.buildqueue import BuildQueue
30from lp.code.interfaces.sourcepackagerecipe import (
31 ForbiddenInstruction, ISourcePackageRecipe, ISourcePackageRecipeSource,
32 TooNewRecipeFormat)
33from lp.code.interfaces.sourcepackagerecipebuild import (
34 ISourcePackageRecipeBuild, ISourcePackageRecipeBuildJob)
35from lp.code.model.sourcepackagerecipebuild import (
36 SourcePackageRecipeBuildJob)
37from lp.code.model.sourcepackagerecipe import (
38 NonPPABuildRequest)
39from lp.registry.interfaces.pocket import PackagePublishingPocket
40from lp.services.job.interfaces.job import (
41 IJob, JobStatus)
42from lp.soyuz.interfaces.archive import ArchivePurpose
43from lp.testing import (
44 login_person, person_logged_in, TestCaseWithFactory)
45
46class TestSourcePackageRecipe(TestCaseWithFactory):
47 """Tests for `SourcePackageRecipe` objects."""
48
49 layer = DatabaseFunctionalLayer
50
51 def makeSourcePackageRecipeFromBuilderRecipe(self, builder_recipe):
52 """Make a SourcePackageRecipe from a recipe with arbitrary other data.
53 """
54 registrant = self.factory.makePerson()
55 owner = self.factory.makeTeam(owner=registrant)
56 distroseries = self.factory.makeDistroSeries()
57 sourcepackagename = self.factory.makeSourcePackageName()
58 name = self.factory.getUniqueString(u'recipe-name')
59 description = self.factory.getUniqueString(u'recipe-description')
60 return getUtility(ISourcePackageRecipeSource).new(
61 registrant=registrant, owner=owner, distroseries=[distroseries],
62 sourcepackagename=sourcepackagename, name=name,
63 description=description, builder_recipe=builder_recipe)
64
65 def test_creation(self):
66 # The metadata supplied when a SourcePackageRecipe is created is
67 # present on the new object.
68 registrant = self.factory.makePerson()
69 owner = self.factory.makeTeam(owner=registrant)
70 distroseries = self.factory.makeDistroSeries()
71 sourcepackagename = self.factory.makeSourcePackageName()
72 name = self.factory.getUniqueString(u'recipe-name')
73 description = self.factory.getUniqueString(u'recipe-description')
74 builder_recipe = self.factory.makeRecipe()
75 recipe = getUtility(ISourcePackageRecipeSource).new(
76 registrant=registrant, owner=owner, distroseries=[distroseries],
77 sourcepackagename=sourcepackagename, name=name,
78 description=description, builder_recipe=builder_recipe)
79 self.assertEquals(
80 (registrant, owner, set([distroseries]), sourcepackagename, name),
81 (recipe.registrant, recipe.owner, set(recipe.distroseries),
82 recipe.sourcepackagename, recipe.name))
83
84 def test_source_implements_interface(self):
85 # The SourcePackageRecipe class implements ISourcePackageRecipeSource.
86 self.assertProvides(
87 getUtility(ISourcePackageRecipeSource),
88 ISourcePackageRecipeSource)
89
90 def test_recipe_implements_interface(self):
91 # SourcePackageRecipe objects implement ISourcePackageRecipe.
92 recipe = self.makeSourcePackageRecipeFromBuilderRecipe(
93 self.factory.makeRecipe())
94 self.assertProvides(recipe, ISourcePackageRecipe)
95
96 def test_base_branch(self):
97 # When a recipe is created, we can access its base branch.
98 branch = self.factory.makeAnyBranch()
99 builder_recipe = self.factory.makeRecipe(branch)
100 sp_recipe = self.makeSourcePackageRecipeFromBuilderRecipe(
101 builder_recipe)
102 self.assertEquals(branch, sp_recipe.base_branch)
103
104 def test_branch_links_created(self):
105 # When a recipe is created, we can query it for links to the branch
106 # it references.
107 branch = self.factory.makeAnyBranch()
108 builder_recipe = self.factory.makeRecipe(branch)
109 sp_recipe = self.makeSourcePackageRecipeFromBuilderRecipe(
110 builder_recipe)
111 self.assertEquals([branch], list(sp_recipe.getReferencedBranches()))
112
113 def test_multiple_branch_links_created(self):
114 # If a recipe links to more than one branch, getReferencedBranches()
115 # returns all of them.
116 branch1 = self.factory.makeAnyBranch()
117 branch2 = self.factory.makeAnyBranch()
118 builder_recipe = self.factory.makeRecipe(branch1, branch2)
119 sp_recipe = self.makeSourcePackageRecipeFromBuilderRecipe(
120 builder_recipe)
121 self.assertEquals(
122 sorted([branch1, branch2]),
123 sorted(sp_recipe.getReferencedBranches()))
124
125 def test_random_user_cant_edit(self):
126 # An arbitrary user can't set attributes.
127 branch1 = self.factory.makeAnyBranch()
128 builder_recipe1 = self.factory.makeRecipe(branch1)
129 sp_recipe = self.makeSourcePackageRecipeFromBuilderRecipe(
130 builder_recipe1)
131 branch2 = self.factory.makeAnyBranch()
132 builder_recipe2 = self.factory.makeRecipe(branch2)
133 login_person(self.factory.makePerson())
134 self.assertRaises(
135 Unauthorized, setattr, sp_recipe, 'builder_recipe',
136 builder_recipe2)
137
138 def test_set_recipe_text_resets_branch_references(self):
139 # When the recipe_text is replaced, getReferencedBranches returns
140 # (only) the branches referenced by the new recipe.
141 branch1 = self.factory.makeAnyBranch()
142 builder_recipe1 = self.factory.makeRecipe(branch1)
143 sp_recipe = self.makeSourcePackageRecipeFromBuilderRecipe(
144 builder_recipe1)
145 branch2 = self.factory.makeAnyBranch()
146 builder_recipe2 = self.factory.makeRecipe(branch2)
147 login_person(sp_recipe.owner.teamowner)
148 #import pdb; pdb.set_trace()
149 sp_recipe.builder_recipe = builder_recipe2
150 self.assertEquals([branch2], list(sp_recipe.getReferencedBranches()))
151
152 def test_rejects_run_command(self):
153 recipe_text = '''\
154 # bzr-builder format 0.2 deb-version 0.1-{revno}
155 %(base)s
156 run touch test
157 ''' % dict(base=self.factory.makeAnyBranch().bzr_identity)
158 parser = RecipeParser(textwrap.dedent(recipe_text))
159 builder_recipe = parser.parse()
160 self.assertRaises(
161 ForbiddenInstruction,
162 self.makeSourcePackageRecipeFromBuilderRecipe, builder_recipe)
163
164 def test_run_rejected_without_mangling_recipe(self):
165 branch1 = self.factory.makeAnyBranch()
166 builder_recipe1 = self.factory.makeRecipe(branch1)
167 sp_recipe = self.makeSourcePackageRecipeFromBuilderRecipe(
168 builder_recipe1)
169 recipe_text = '''\
170 # bzr-builder format 0.2 deb-version 0.1-{revno}
171 %(base)s
172 run touch test
173 ''' % dict(base=self.factory.makeAnyBranch().bzr_identity)
174 parser = RecipeParser(textwrap.dedent(recipe_text))
175 builder_recipe2 = parser.parse()
176 login_person(sp_recipe.owner.teamowner)
177 self.assertRaises(
178 ForbiddenInstruction, setattr, sp_recipe, 'builder_recipe',
179 builder_recipe2)
180 self.assertEquals([branch1], list(sp_recipe.getReferencedBranches()))
181
182 def test_reject_newer_formats(self):
183 builder_recipe = self.factory.makeRecipe()
184 builder_recipe.format = 0.3
185 self.assertRaises(
186 TooNewRecipeFormat,
187 self.makeSourcePackageRecipeFromBuilderRecipe, builder_recipe)
188
189 def test_requestBuild(self):
190 recipe = self.factory.makeSourcePackageRecipe()
191 (distroseries,) = list(recipe.distroseries)
192 ppa = self.factory.makeArchive()
193 build = recipe.requestBuild(ppa, ppa.owner, distroseries,
194 PackagePublishingPocket.RELEASE)
195 self.assertProvides(build, ISourcePackageRecipeBuild)
196 self.assertEqual(build.archive, ppa)
197 self.assertEqual(build.distroseries, distroseries)
198 self.assertEqual(build.requester, ppa.owner)
199 store = Store.of(build)
200 store.flush()
201 build_job = store.find(SourcePackageRecipeBuildJob,
202 SourcePackageRecipeBuildJob.build_id==build.id).one()
203 self.assertProvides(build_job, ISourcePackageRecipeBuildJob)
204 self.assertTrue(build_job.virtualized)
205 job = build_job.job
206 self.assertProvides(job, IJob)
207 self.assertEquals(job.status, JobStatus.WAITING)
208 build_queue = store.find(BuildQueue, BuildQueue.job==job.id).one()
209 self.assertProvides(build_queue, IBuildQueue)
210 self.assertTrue(build_queue.virtualized)
211
212 def test_requestBuildRejectsNotPPA(self):
213 recipe = self.factory.makeSourcePackageRecipe()
214 not_ppa = self.factory.makeArchive(purpose=ArchivePurpose.PRIMARY)
215 (distroseries,) = list(recipe.distroseries)
216 self.assertRaises(NonPPABuildRequest, recipe.requestBuild, not_ppa,
217 not_ppa.owner, distroseries, PackagePublishingPocket.RELEASE)
218
219 def test_requestBuildRejectsNoPermission(self):
220 recipe = self.factory.makeSourcePackageRecipe()
221 ppa = self.factory.makeArchive()
222 requester = self.factory.makePerson()
223 (distroseries,) = list(recipe.distroseries)
224 self.assertRaises(CannotUploadToArchive, recipe.requestBuild, ppa,
225 requester, distroseries, PackagePublishingPocket.RELEASE)
226
227 def test_requestBuildRejectsInvalidPocket(self):
228 recipe = self.factory.makeSourcePackageRecipe()
229 ppa = self.factory.makeArchive()
230 (distroseries,) = list(recipe.distroseries)
231 self.assertRaises(InvalidPocketForPPA, recipe.requestBuild, ppa,
232 ppa.owner, distroseries, PackagePublishingPocket.BACKPORTS)
233
234 def test_requestBuildRejectsDisabledArchive(self):
235 recipe = self.factory.makeSourcePackageRecipe()
236 ppa = self.factory.makeArchive()
237 removeSecurityProxy(ppa).disable()
238 (distroseries,) = list(recipe.distroseries)
239 self.assertRaises(ArchiveDisabled, recipe.requestBuild, ppa,
240 ppa.owner, distroseries, PackagePublishingPocket.RELEASE)
241
242 def test_sourcepackagerecipe_description(self):
243 """Ensure that the SourcePackageRecipe has a proper description."""
244 description = u'The whoozits and whatzits.'
245 source_package_recipe = self.factory.makeSourcePackageRecipe(
246 description=description)
247 self.assertEqual(description, source_package_recipe.description)
248
249 def test_distroseries(self):
250 """Test that the distroseries behaves as a set."""
251 recipe = self.factory.makeSourcePackageRecipe()
252 distroseries = self.factory.makeDistroSeries()
253 (old_distroseries,) = recipe.distroseries
254 recipe.distroseries.add(distroseries)
255 self.assertEqual(
256 set([distroseries, old_distroseries]), set(recipe.distroseries))
257 recipe.distroseries.remove(distroseries)
258 self.assertEqual([old_distroseries], list(recipe.distroseries))
259 recipe.distroseries.clear()
260 self.assertEqual([], list(recipe.distroseries))
261
262 def test_build_daily(self):
263 """Test that build_daily behaves as a bool."""
264 recipe = self.factory.makeSourcePackageRecipe()
265 self.assertFalse(recipe.build_daily)
266 login_person(recipe.owner)
267 recipe.build_daily = True
268 self.assertTrue(recipe.build_daily)
269
270 def test_view_public(self):
271 """Anyone can view a recipe with public branches."""
272 owner = self.factory.makePerson()
273 branch = self.factory.makeAnyBranch(owner=owner)
274 with person_logged_in(owner):
275 recipe = self.factory.makeSourcePackageRecipe(branches=[branch])
276 self.assertTrue(check_permission('launchpad.View', recipe))
277 with person_logged_in(self.factory.makePerson()):
278 self.assertTrue(check_permission('launchpad.View', recipe))
279 self.assertTrue(check_permission('launchpad.View', recipe))
280
281 def test_view_private(self):
282 """Recipes with private branches are restricted."""
283 owner = self.factory.makePerson()
284 branch = self.factory.makeAnyBranch(owner=owner, private=True)
285 with person_logged_in(owner):
286 recipe = self.factory.makeSourcePackageRecipe(branches=[branch])
287 self.assertTrue(check_permission('launchpad.View', recipe))
288 with person_logged_in(self.factory.makePerson()):
289 self.assertFalse(check_permission('launchpad.View', recipe))
290 self.assertFalse(check_permission('launchpad.View', recipe))
291
292 def test_edit(self):
293 """Only the owner can edit a sourcepackagerecipe."""
294 recipe = self.factory.makeSourcePackageRecipe()
295 self.assertFalse(check_permission('launchpad.Edit', recipe))
296 with person_logged_in(self.factory.makePerson()):
297 self.assertFalse(check_permission('launchpad.Edit', recipe))
298 with person_logged_in(recipe.owner):
299 self.assertTrue(check_permission('launchpad.Edit', recipe))
300
301 def test_destroySelf(self):
302 """Should destroy associated builds, distroseries, etc."""
303 # Recipe should have at least one datainstruction.
304 branches = [self.factory.makeBranch() for count in range(2)]
305 recipe = self.factory.makeSourcePackageRecipe(branches=branches)
306 pending_build = self.factory.makeSourcePackageRecipeBuild(
307 recipe=recipe)
308 self.factory.makeSourcePackageRecipeBuildJob(
309 recipe_build=pending_build)
310 past_build = self.factory.makeSourcePackageRecipeBuild(
311 recipe=recipe)
312 self.factory.makeSourcePackageRecipeBuildJob(
313 recipe_build=past_build)
314 removeSecurityProxy(past_build).datebuilt = datetime.now(UTC)
315 recipe.destroySelf()
316 # Show no database constraints were violated
317 Store.of(recipe).flush()
318
319
320class TestRecipeBranchRoundTripping(TestCaseWithFactory):
321
322 layer = DatabaseFunctionalLayer
323
324 def setUp(self):
325 super(TestRecipeBranchRoundTripping, self).setUp()
326 self.base_branch = self.factory.makeAnyBranch()
327 self.nested_branch = self.factory.makeAnyBranch()
328 self.merged_branch = self.factory.makeAnyBranch()
329 self.branch_identities = {
330 'base': self.base_branch.bzr_identity,
331 'nested': self.nested_branch.bzr_identity,
332 'merged': self.merged_branch.bzr_identity,
333 }
334
335 def get_recipe(self, recipe_text):
336 builder_recipe = RecipeParser(textwrap.dedent(recipe_text)).parse()
337 registrant = self.factory.makePerson()
338 owner = self.factory.makeTeam(owner=registrant)
339 distroseries = self.factory.makeDistroSeries()
340 sourcepackagename = self.factory.makeSourcePackageName()
341 name = self.factory.getUniqueString(u'recipe-name')
342 description = self.factory.getUniqueString(u'recipe-description')
343 recipe = getUtility(ISourcePackageRecipeSource).new(
344 registrant=registrant, owner=owner, distroseries=[distroseries],
345 sourcepackagename=sourcepackagename, name=name,
346 description=description, builder_recipe=builder_recipe)
347 return recipe.builder_recipe
348
349 def check_base_recipe_branch(self, branch, url, revspec=None,
350 num_child_branches=0, revid=None, deb_version=None):
351 self.check_recipe_branch(branch, None, url, revspec=revspec,
352 num_child_branches=num_child_branches, revid=revid)
353 self.assertEqual(deb_version, branch.deb_version)
354
355 def check_recipe_branch(self, branch, name, url, revspec=None,
356 num_child_branches=0, revid=None):
357 self.assertEqual(name, branch.name)
358 self.assertEqual(url, branch.url)
359 self.assertEqual(revspec, branch.revspec)
360 self.assertEqual(revid, branch.revid)
361 self.assertEqual(num_child_branches, len(branch.child_branches))
362
363 def test_builds_simplest_recipe(self):
364 recipe_text = '''\
365 # bzr-builder format 0.2 deb-version 0.1-{revno}
366 %(base)s
367 ''' % self.branch_identities
368 base_branch = self.get_recipe(recipe_text)
369 self.check_base_recipe_branch(
370 base_branch, self.base_branch.bzr_identity,
371 deb_version='0.1-{revno}')
372
373 def test_builds_recipe_with_merge(self):
374 recipe_text = '''\
375 # bzr-builder format 0.2 deb-version 0.1-{revno}
376 %(base)s
377 merge bar %(merged)s
378 ''' % self.branch_identities
379 base_branch = self.get_recipe(recipe_text)
380 self.check_base_recipe_branch(
381 base_branch, self.base_branch.bzr_identity, num_child_branches=1,
382 deb_version='0.1-{revno}')
383 child_branch, location = base_branch.child_branches[0].as_tuple()
384 self.assertEqual(None, location)
385 self.check_recipe_branch(
386 child_branch, "bar", self.merged_branch.bzr_identity)
387
388 def test_builds_recipe_with_nest(self):
389 recipe_text = '''\
390 # bzr-builder format 0.2 deb-version 0.1-{revno}
391 %(base)s
392 nest bar %(nested)s baz
393 ''' % self.branch_identities
394 base_branch = self.get_recipe(recipe_text)
395 self.check_base_recipe_branch(
396 base_branch, self.base_branch.bzr_identity, num_child_branches=1,
397 deb_version='0.1-{revno}')
398 child_branch, location = base_branch.child_branches[0].as_tuple()
399 self.assertEqual("baz", location)
400 self.check_recipe_branch(
401 child_branch, "bar", self.nested_branch.bzr_identity)
402
403 def test_builds_recipe_with_nest_then_merge(self):
404 recipe_text = '''\
405 # bzr-builder format 0.2 deb-version 0.1-{revno}
406 %(base)s
407 nest bar %(nested)s baz
408 merge zam %(merged)s
409 ''' % self.branch_identities
410 base_branch = self.get_recipe(recipe_text)
411 self.check_base_recipe_branch(
412 base_branch, self.base_branch.bzr_identity, num_child_branches=2,
413 deb_version='0.1-{revno}')
414 child_branch, location = base_branch.child_branches[0].as_tuple()
415 self.assertEqual("baz", location)
416 self.check_recipe_branch(
417 child_branch, "bar", self.nested_branch.bzr_identity)
418 child_branch, location = base_branch.child_branches[1].as_tuple()
419 self.assertEqual(None, location)
420 self.check_recipe_branch(
421 child_branch, "zam", self.merged_branch.bzr_identity)
422
423 def test_builds_recipe_with_merge_then_nest(self):
424 recipe_text = '''\
425 # bzr-builder format 0.2 deb-version 0.1-{revno}
426 %(base)s
427 merge zam %(merged)s
428 nest bar %(nested)s baz
429 ''' % self.branch_identities
430 base_branch = self.get_recipe(recipe_text)
431 self.check_base_recipe_branch(
432 base_branch, self.base_branch.bzr_identity, num_child_branches=2,
433 deb_version='0.1-{revno}')
434 child_branch, location = base_branch.child_branches[0].as_tuple()
435 self.assertEqual(None, location)
436 self.check_recipe_branch(
437 child_branch, "zam", self.merged_branch.bzr_identity)
438 child_branch, location = base_branch.child_branches[1].as_tuple()
439 self.assertEqual("baz", location)
440 self.check_recipe_branch(
441 child_branch, "bar", self.nested_branch.bzr_identity)
442
443 def test_builds_a_merge_in_to_a_nest(self):
444 recipe_text = '''\
445 # bzr-builder format 0.2 deb-version 0.1-{revno}
446 %(base)s
447 nest bar %(nested)s baz
448 merge zam %(merged)s
449 ''' % self.branch_identities
450 base_branch = self.get_recipe(recipe_text)
451 self.check_base_recipe_branch(
452 base_branch, self.base_branch.bzr_identity, num_child_branches=1,
453 deb_version='0.1-{revno}')
454 child_branch, location = base_branch.child_branches[0].as_tuple()
455 self.assertEqual("baz", location)
456 self.check_recipe_branch(
457 child_branch, "bar", self.nested_branch.bzr_identity,
458 num_child_branches=1)
459 child_branch, location = child_branch.child_branches[0].as_tuple()
460 self.assertEqual(None, location)
461 self.check_recipe_branch(
462 child_branch, "zam", self.merged_branch.bzr_identity)
463
464 def tests_builds_nest_into_a_nest(self):
465 nested2 = self.factory.makeAnyBranch()
466 self.branch_identities['nested2'] = nested2.bzr_identity
467 recipe_text = '''\
468 # bzr-builder format 0.2 deb-version 0.1-{revno}
469 %(base)s
470 nest bar %(nested)s baz
471 nest zam %(nested2)s zoo
472 ''' % self.branch_identities
473 base_branch = self.get_recipe(recipe_text)
474 self.check_base_recipe_branch(
475 base_branch, self.base_branch.bzr_identity, num_child_branches=1,
476 deb_version='0.1-{revno}')
477 child_branch, location = base_branch.child_branches[0].as_tuple()
478 self.assertEqual("baz", location)
479 self.check_recipe_branch(
480 child_branch, "bar", self.nested_branch.bzr_identity,
481 num_child_branches=1)
482 child_branch, location = child_branch.child_branches[0].as_tuple()
483 self.assertEqual("zoo", location)
484 self.check_recipe_branch(child_branch, "zam", nested2.bzr_identity)
485
486 def tests_builds_recipe_with_revspecs(self):
487 recipe_text = '''\
488 # bzr-builder format 0.2 deb-version 0.1-{revno}
489 %(base)s revid:a
490 nest bar %(nested)s baz tag:b
491 merge zam %(merged)s 2
492 ''' % self.branch_identities
493 base_branch = self.get_recipe(recipe_text)
494 self.check_base_recipe_branch(
495 base_branch, self.base_branch.bzr_identity, num_child_branches=2,
496 revspec="revid:a", deb_version='0.1-{revno}')
497 instruction = base_branch.child_branches[0]
498 child_branch = instruction.recipe_branch
499 location = instruction.nest_path
500 self.assertEqual("baz", location)
501 self.check_recipe_branch(
502 child_branch, "bar", self.nested_branch.bzr_identity,
503 revspec="tag:b")
504 child_branch, location = base_branch.child_branches[1].as_tuple()
505 self.assertEqual(None, location)
506 self.check_recipe_branch(
507 child_branch, "zam", self.merged_branch.bzr_identity, revspec="2")
508
509
510def test_suite():
511 return unittest.TestLoader().loadTestsFromName(__name__)

Subscribers

People subscribed via source and target branches

to status/vote changes: