Launchpad itself

Merge ~andrey-fedoseev/launchpad:jira-bug-watch into launchpad:master

  1. Git
  2. lp:~andrey-fedoseev/launchpad
  3. jira-bug-watch
  4. Merge into master
Proposed by Andrey Fedoseev
Status: Needs review
Proposed branch: ~andrey-fedoseev/launchpad:jira-bug-watch
Merge into: launchpad:master
Diff against target: 873 lines (+742/-5)
9 files modified
lib/lp/bugs/externalbugtracker/__init__.py (+2/-0)
lib/lp/bugs/externalbugtracker/base.py (+8/-3)
lib/lp/bugs/externalbugtracker/jira.py (+268/-0)
lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py (+1/-1)
lib/lp/bugs/externalbugtracker/tests/test_jira.py (+432/-0)
lib/lp/bugs/interfaces/bugtracker.py (+9/-0)
lib/lp/bugs/model/bugwatch.py (+11/-0)
lib/lp/bugs/tests/test_bugwatch.py (+9/-0)
lib/lp/services/config/schema-lazr.conf (+2/-1)
Reviewer Review Type Date Requested Status
Colin Watson (community) Approve
Review via email: mp+433103@code.launchpad.net

Commit message

Add external bug tracker for JIRA

To post a comment you must log in.
Revision history for this message
Colin Watson (cjwatson) wrote :

This looks like a perfectly reasonable first-pass implementation, and I'm fine with landing it as far as it goes.

We'll need to be careful about the credentials handling. The existing `checkwatches.credentials` stuff was intended for cases where the bug tracker itself is essentially public but we need some kind of credentials for Launchpad to connect to it anyway, either out of politeness (credentials allow our sync script to be identified unambiguously), or to gain access to higher rate limits (as in the GitHub/GitLab cases), or because we need to push comments (for Bugzilla). In this case, though, the credentials are partly also being used because the remote bug tracker is private, which is a very different matter: if we were to configure these credentials on production, it would mean that anyone could discover information about the status of a given Jira issue by guessing its URL and adding a bug watch for it. Not a very serious information leak since it only tells you the remote status and importance, but nevertheless probably not something we should leave designed into the system in case somebody wants to extend it in future to gather more information. I think it's fine to leave an XXX comment about this for now, though, as it doesn't become a problem until we configure credentials; perhaps we could change some other part of the system to restrict who can add such bug watches, or restrict them to certain projects, or something like that.

Having gathered requirements for Launchpad/Jira integration, I also think this will probably not address those requirements on its own (though it may be a component of the eventual solution). I've belatedly written down what I know so far here: https://docs.google.com/document/d/1CiEgo-CHX8Go0lTAdryqKCeFxaVnBq49QVkfBshX28M

review: Approve
Reply

Unmerged commits

9f871dc... by Andrey Fedoseev

Add external bug tracker for JIRA

Succeeded
[SUCCEEDED] docs:0 (build)
[SUCCEEDED] lint:0 (build)
[SUCCEEDED] mypy:0 (build)
1 → 3 of 3 results First • Previous • Next • Last

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/lib/lp/bugs/externalbugtracker/__init__.py b/lib/lp/bugs/externalbugtracker/__init__.py
index 65c5333..d4b887d 100644
--- a/lib/lp/bugs/externalbugtracker/__init__.py
+++ b/lib/lp/bugs/externalbugtracker/__init__.py
@@ -51,6 +51,7 @@ from lp.bugs.externalbugtracker.bugzilla import Bugzilla
51from lp.bugs.externalbugtracker.debbugs import DebBugs, DebBugsDatabaseNotFound51from lp.bugs.externalbugtracker.debbugs import DebBugs, DebBugsDatabaseNotFound
52from lp.bugs.externalbugtracker.github import GitHub52from lp.bugs.externalbugtracker.github import GitHub
53from lp.bugs.externalbugtracker.gitlab import GitLab53from lp.bugs.externalbugtracker.gitlab import GitLab
54from lp.bugs.externalbugtracker.jira import Jira
54from lp.bugs.externalbugtracker.mantis import Mantis55from lp.bugs.externalbugtracker.mantis import Mantis
55from lp.bugs.externalbugtracker.roundup import Roundup56from lp.bugs.externalbugtracker.roundup import Roundup
56from lp.bugs.externalbugtracker.rt import RequestTracker57from lp.bugs.externalbugtracker.rt import RequestTracker
@@ -68,6 +69,7 @@ BUG_TRACKER_CLASSES = {
68 BugTrackerType.ROUNDUP: Roundup,69 BugTrackerType.ROUNDUP: Roundup,
69 BugTrackerType.RT: RequestTracker,70 BugTrackerType.RT: RequestTracker,
70 BugTrackerType.SOURCEFORGE: SourceForge,71 BugTrackerType.SOURCEFORGE: SourceForge,
72 BugTrackerType.JIRA: Jira,
71}73}
7274
7375
diff --git a/lib/lp/bugs/externalbugtracker/base.py b/lib/lp/bugs/externalbugtracker/base.py
index ee7be38..0d2efd4 100644
--- a/lib/lp/bugs/externalbugtracker/base.py
+++ b/lib/lp/bugs/externalbugtracker/base.py
@@ -279,16 +279,17 @@ class ExternalBugTracker:
279 except requests.RequestException as e:279 except requests.RequestException as e:
280 raise BugTrackerConnectError(self.baseurl, e)280 raise BugTrackerConnectError(self.baseurl, e)
281281
282 def _postPage(self, page, form, repost_on_redirect=False):282 def _postPage(self, page, data, repost_on_redirect=False, json=False):
283 """POST to the specified page and form.283 """POST to the specified page and form.
284284
285 :param form: is a dict of form variables being POSTed.285 :param data: is a dict of form variables being POSTed.
286 :param repost_on_redirect: override RFC-compliant redirect handling.286 :param repost_on_redirect: override RFC-compliant redirect handling.
287 By default, if the POST receives a redirect response, the287 By default, if the POST receives a redirect response, the
288 request to the redirection's target URL will be a GET. If288 request to the redirection's target URL will be a GET. If
289 `repost_on_redirect` is True, this method will do a second POST289 `repost_on_redirect` is True, this method will do a second POST
290 instead. Do this only if you are sure that repeated POST to290 instead. Do this only if you are sure that repeated POST to
291 this page is safe, as is usually the case with search forms.291 this page is safe, as is usually the case with search forms.
292 :param json: if True, the data will be JSON encoded.
292 :return: A `requests.Response` object.293 :return: A `requests.Response` object.
293 """294 """
294 hooks = (295 hooks = (
@@ -301,8 +302,12 @@ class ExternalBugTracker:
301 if not url.endswith("/"):302 if not url.endswith("/"):
302 url += "/"303 url += "/"
303 url = urljoin(url, page)304 url = urljoin(url, page)
305 if json:
306 kwargs = {"json": data}
307 else:
308 kwargs = {"data": data}
304 response = self.makeRequest(309 response = self.makeRequest(
305 "POST", url, headers=self._getHeaders(), data=form, hooks=hooks310 "POST", url, headers=self._getHeaders(), hooks=hooks, **kwargs
306 )311 )
307 raise_for_status_redacted(response)312 raise_for_status_redacted(response)
308 return response313 return response
diff --git a/lib/lp/bugs/externalbugtracker/jira.py b/lib/lp/bugs/externalbugtracker/jira.py
309new file mode 100644314new file mode 100644
index 0000000..b529c5b
--- /dev/null
+++ b/lib/lp/bugs/externalbugtracker/jira.py
@@ -0,0 +1,268 @@
1# Copyright 2022 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Jira ExternalBugTracker utility."""
5
6__all__ = [
7 "Jira",
8 "JiraCredentials",
9 "JiraBug",
10 "JiraStatus",
11 "JiraPriority",
12]
13
14import base64
15import datetime
16from enum import Enum
17from typing import Dict, Iterable, NamedTuple, Optional, Tuple
18from urllib.parse import urlunsplit
19
20import dateutil.parser
21
22from lp.bugs.externalbugtracker import (
23 BugTrackerConnectError,
24 ExternalBugTracker,
25)
26from lp.bugs.interfaces.bugtask import BugTaskImportance, BugTaskStatus
27from lp.services.config import config
28from lp.services.webapp.url import urlsplit
29
30JiraCredentials = NamedTuple(
31 "JiraCredentials",
32 (
33 ("username", str),
34 ("password", str),
35 ),
36)
37
38
39class JiraStatus(Enum):
40
41 UNDEFINED = "undefined"
42 NEW = "new"
43 INDETERMINATE = "indeterminate"
44 DONE = "done"
45
46 @property
47 def launchpad_status(self):
48 if self == JiraStatus.UNDEFINED:
49 return BugTaskStatus.UNKNOWN
50 elif self == JiraStatus.NEW:
51 return BugTaskStatus.NEW
52 elif self == JiraStatus.INDETERMINATE:
53 return BugTaskStatus.INPROGRESS
54 elif self == JiraStatus.DONE:
55 return BugTaskStatus.FIXRELEASED
56 else:
57 raise AssertionError()
58
59
60class JiraPriority(Enum):
61
62 UNDEFINED = "undefined"
63 LOWEST = "Lowest"
64 LOW = "Low"
65 MEDIUM = "Medium"
66 HIGH = "High"
67 HIGHEST = "Highest"
68
69 @property
70 def launchpad_importance(self):
71 if self == JiraPriority.UNDEFINED:
72 return BugTaskImportance.UNKNOWN
73 elif self == JiraPriority.LOWEST:
74 return BugTaskImportance.WISHLIST
75 elif self == JiraPriority.LOW:
76 return BugTaskImportance.LOW
77 elif self == JiraPriority.MEDIUM:
78 return BugTaskImportance.MEDIUM
79 elif self == JiraPriority.HIGH:
80 return BugTaskImportance.HIGH
81 elif self == JiraPriority.HIGHEST:
82 return BugTaskImportance.CRITICAL
83 else:
84 raise AssertionError()
85
86
87class JiraBug:
88 def __init__(self, key: str, status: JiraStatus, priority: JiraPriority):
89 self.key = key
90 self.status = status
91 self.priority = priority
92
93 @classmethod
94 def from_api_data(cls, bug_data) -> "JiraBug":
95 try:
96 status = JiraStatus(
97 bug_data["fields"]["status"]["statusCategory"]["key"]
98 )
99 except ValueError:
100 status = JiraStatus.UNDEFINED
101
102 try:
103 priority = JiraPriority(bug_data["fields"]["priority"]["name"])
104 except ValueError:
105 priority = JiraPriority.UNDEFINED
106
107 return cls(
108 key=bug_data["key"],
109 status=status,
110 priority=priority,
111 )
112
113 def __eq__(self, other):
114 if not isinstance(other, JiraBug):
115 raise ValueError()
116 return (
117 self.key == other.key
118 and self.status == other.status
119 and self.priority == other.priority
120 )
121
122
123class Jira(ExternalBugTracker):
124 """An `ExternalBugTracker` for dealing with Jira issues."""
125
126 batch_query_threshold = 0 # Always use the batch method.
127
128 def __init__(self, baseurl):
129 _, host, path, query, fragment = urlsplit(baseurl)
130 path = "/rest/api/2/"
131 baseurl = urlunsplit(("https", host, path, "", ""))
132 super().__init__(baseurl)
133 self.cached_bugs = {} # type: Dict[str, Optional[JiraBug]]
134
135 @property
136 def credentials(self) -> Optional[JiraCredentials]:
137 credentials_config = config["checkwatches.credentials"]
138 # lazr.config.Section doesn't support get().
139 try:
140 username = credentials_config["{}.username".format(self.basehost)]
141 password = credentials_config["{}.password".format(self.basehost)]
142 return JiraCredentials(
143 username=username,
144 password=password,
145 )
146 except KeyError:
147 return
148
149 def getCurrentDBTime(self):
150 # See https://docs.atlassian.com/software/jira/docs/api/REST/9.3.1/#api/2/serverInfo-getServerInfo # noqa
151 response_data = self._getPage("serverInfo").json()
152 return dateutil.parser.parse(response_data["serverTime"]).astimezone(
153 datetime.timezone.utc
154 )
155
156 def getModifiedRemoteBugs(self, bug_ids, last_accessed):
157 """See `IExternalBugTracker`."""
158 modified_bugs = self.getRemoteBugBatch(
159 bug_ids, last_accessed=last_accessed
160 )
161 self.cached_bugs.update(modified_bugs)
162 return list(modified_bugs)
163
164 def getRemoteBug(self, bug_id: str) -> Tuple[str, Optional[JiraBug]]:
165 """See `ExternalBugTracker`."""
166 if bug_id not in self.cached_bugs:
167 self.cached_bugs[bug_id] = self._loadJiraBug(bug_id)
168 return bug_id, self.cached_bugs[bug_id]
169
170 def getRemoteBugBatch(
171 self, bug_ids, last_accessed=None
172 ) -> Dict[str, Optional[JiraBug]]:
173 """See `ExternalBugTracker`."""
174 bugs = {
175 bug_id: self.cached_bugs[bug_id]
176 for bug_id in bug_ids
177 if bug_id in self.cached_bugs
178 }
179 if set(bugs) == set(bug_ids):
180 return bugs
181
182 for jira_bug in self._loadJiraBugs(
183 bug_ids, last_accessed=last_accessed
184 ):
185 if jira_bug.key not in bug_ids:
186 continue
187 bugs[jira_bug.key] = self.cached_bugs[jira_bug.key] = jira_bug
188
189 return bugs
190
191 def getRemoteImportance(self, bug_id) -> str:
192 """See `ExternalBugTracker`."""
193 remote_bug = self.bugs[bug_id] # type: JiraBug
194 return remote_bug.priority.value
195
196 def getRemoteStatus(self, bug_id) -> str:
197 """See `ExternalBugTracker`."""
198 remote_bug = self.bugs[bug_id] # type: JiraBug
199 return remote_bug.status.value
200
201 def convertRemoteImportance(
202 self, remote_importance: str
203 ) -> BugTaskImportance:
204 """See `IExternalBugTracker`."""
205 return JiraPriority(remote_importance).launchpad_importance
206
207 def convertRemoteStatus(self, remote_status: str) -> BugTaskStatus:
208 """See `IExternalBugTracker`."""
209 return JiraStatus(remote_status).launchpad_status
210
211 def _getHeaders(self):
212 headers = super()._getHeaders()
213 credentials = self.credentials
214 if credentials:
215 headers["Authorization"] = "Basic {}".format(
216 base64.b64encode(
217 "{}:{}".format(
218 credentials.username, credentials.password
219 ).encode()
220 ).decode()
221 )
222 return headers
223
224 def _loadJiraBug(self, bug_id: str) -> Optional[JiraBug]:
225 # See https://docs.atlassian.com/software/jira/docs/api/REST/9.3.1/#api/2/issue-getIssue # noqa
226 try:
227 response = self._getPage(
228 "issue/{}".format(bug_id),
229 params={
230 "fields": "status,priority",
231 },
232 )
233 except BugTrackerConnectError:
234 return
235
236 return JiraBug.from_api_data(response.json())
237
238 def _loadJiraBugs(
239 self, bug_ids, last_accessed=None, start_at=0
240 ) -> Iterable[JiraBug]:
241 # See https://docs.atlassian.com/software/jira/docs/api/REST/9.3.1/#api/2/search-searchUsingSearchRequest # noqa
242
243 jql_query = "id in ({})".format(",".join(bug_ids))
244 if last_accessed is not None:
245 jql_query = "{} AND updated >= {}".format(
246 jql_query, last_accessed.strftime("%Y-%m-%d %H:%M")
247 )
248
249 params = {
250 "jql": jql_query,
251 "fields": ["status", "priority"],
252 "startAt": start_at,
253 }
254
255 response_data = self._postPage("search", data=params, json=True).json()
256
257 max_results = response_data["maxResults"]
258 total = response_data["total"]
259
260 for bug_data in response_data["issues"]:
261 yield JiraBug.from_api_data(bug_data)
262
263 if total > (start_at + max_results):
264 yield from self._loadJiraBugs(
265 bug_ids,
266 last_accessed=last_accessed,
267 start_at=start_at + max_results,
268 )
diff --git a/lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py b/lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py
index 03b3197..2da0866 100644
--- a/lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py
+++ b/lib/lp/bugs/externalbugtracker/tests/test_externalbugtracker.py
@@ -167,7 +167,7 @@ class TestCheckwatchesConfig(TestCase):
167 )167 )
168 responses.add("POST", base_url + target, body=fake_form)168 responses.add("POST", base_url + target, body=fake_form)
169169
170 bugtracker._postPage(form, form={}, repost_on_redirect=True)170 bugtracker._postPage(form, {}, repost_on_redirect=True)
171171
172 requests = [call.request for call in responses.calls]172 requests = [call.request for call in responses.calls]
173 self.assertThat(173 self.assertThat(
diff --git a/lib/lp/bugs/externalbugtracker/tests/test_jira.py b/lib/lp/bugs/externalbugtracker/tests/test_jira.py
174new file mode 100644174new file mode 100644
index 0000000..1301dce
--- /dev/null
+++ b/lib/lp/bugs/externalbugtracker/tests/test_jira.py
@@ -0,0 +1,432 @@
1# Copyright 2022 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3import datetime
4import json
5
6import responses
7import transaction
8from testtools.matchers import (
9 ContainsDict,
10 Equals,
11 MatchesListwise,
12 MatchesStructure,
13 StartsWith,
14)
15from zope.component import getUtility
16
17from lp.app.interfaces.launchpad import ILaunchpadCelebrities
18from lp.bugs.externalbugtracker import get_external_bugtracker
19from lp.bugs.externalbugtracker.jira import (
20 Jira,
21 JiraBug,
22 JiraCredentials,
23 JiraPriority,
24 JiraStatus,
25)
26from lp.bugs.interfaces.bugtask import BugTaskStatus
27from lp.bugs.interfaces.bugtracker import BugTrackerType
28from lp.bugs.interfaces.externalbugtracker import IExternalBugTracker
29from lp.bugs.scripts.checkwatches import CheckwatchesMaster
30from lp.services.log.logger import BufferLogger
31from lp.testing import TestCase, TestCaseWithFactory, verifyObject
32from lp.testing.layers import ZopelessDatabaseLayer, ZopelessLayer
33
34
35class TestJira(TestCase):
36
37 layer = ZopelessLayer
38
39 def setUp(self):
40 super().setUp()
41 self.jira = Jira("https://warthogs.atlassian.net")
42 self.pushConfig(
43 "checkwatches.credentials",
44 **{
45 "warthogs.atlassian.net.username": "launchpad",
46 "warthogs.atlassian.net.password": "launchpad",
47 },
48 )
49
50 def test_implements_interface(self):
51 self.assertTrue(verifyObject(IExternalBugTracker, self.jira))
52
53 def test_convert_jira_url_to_api_endpoint(self):
54 self.assertEqual(
55 "https://warthogs.atlassian.net/rest/api/2", self.jira.baseurl
56 )
57
58 def test_credentials(self):
59 self.assertEqual(
60 JiraCredentials(
61 username="launchpad",
62 password="launchpad",
63 ),
64 self.jira.credentials,
65 )
66
67 def test_getHeaders(self):
68 headers = self.jira._getHeaders()
69 self.assertThat(
70 headers,
71 ContainsDict(
72 {"Authorization": Equals("Basic bGF1bmNocGFkOmxhdW5jaHBhZA==")}
73 ),
74 )
75
76 @responses.activate
77 def test_getCurrentDBTime(self):
78 responses.add(
79 "GET",
80 self.jira.baseurl + "/serverInfo",
81 json={
82 "baseUrl": "https://warthogs.atlassian.net",
83 "buildDate": "2022-11-15T06:27:18.000+0800",
84 "buildNumber": 100210,
85 "defaultLocale": {"locale": "en_US"},
86 "deploymentType": "Cloud",
87 "scmInfo": "28a36363a81be3fec088cc03de57ea0d3b868a26",
88 "serverTime": "2022-11-15T14:11:11.818+0800",
89 "serverTitle": "Jira",
90 "version": "1001.0.0-SNAPSHOT",
91 "versionNumbers": [1001, 0, 0],
92 },
93 )
94 self.assertEqual(
95 self.jira.getCurrentDBTime(),
96 datetime.datetime(
97 2022, 11, 15, 6, 11, 11, 818000, tzinfo=datetime.timezone.utc
98 ),
99 )
100 requests = [call.request for call in responses.calls]
101 self.assertThat(
102 requests,
103 MatchesListwise(
104 [
105 MatchesStructure(
106 method=Equals("GET"),
107 path_url=Equals("/rest/api/2/serverInfo"),
108 headers=ContainsDict(
109 {"Authorization": StartsWith("Basic ")}
110 ),
111 ),
112 ]
113 ),
114 )
115
116 @responses.activate
117 def test_getRemoteBug(self):
118 responses.add(
119 "GET",
120 self.jira.baseurl + "/issue/LP-984",
121 json={
122 "fields": {
123 "priority": {"name": "Medium"},
124 "status": {"statusCategory": {"key": "indeterminate"}},
125 },
126 "key": "LP-984",
127 },
128 )
129 responses.add("GET", self.jira.baseurl + "/issue/LP-123", status=404)
130 self.assertEqual(
131 (
132 "LP-984",
133 JiraBug(
134 key="LP-984",
135 status=JiraStatus.INDETERMINATE,
136 priority=JiraPriority.MEDIUM,
137 ),
138 ),
139 self.jira.getRemoteBug("LP-984"),
140 )
141 self.assertEqual(("LP-123", None), self.jira.getRemoteBug("LP-123"))
142
143 requests = [call.request for call in responses.calls]
144 self.assertThat(
145 requests,
146 MatchesListwise(
147 [
148 MatchesStructure(
149 method=Equals("GET"),
150 path_url=Equals(
151 "/rest/api/2/issue/LP-984?fields=status%2Cpriority"
152 ),
153 ),
154 MatchesStructure(
155 method=Equals("GET"),
156 path_url=Equals(
157 "/rest/api/2/issue/LP-123?fields=status%2Cpriority"
158 ),
159 ),
160 ]
161 ),
162 )
163
164 # Getting the same bug the second time should fetch it from the cache
165 # without making another request to JIRA API
166 self.assertEqual(
167 (
168 "LP-984",
169 JiraBug(
170 key="LP-984",
171 status=JiraStatus.INDETERMINATE,
172 priority=JiraPriority.MEDIUM,
173 ),
174 ),
175 self.jira.getRemoteBug("LP-984"),
176 )
177 self.assertEqual(("LP-123", None), self.jira.getRemoteBug("LP-123"))
178 self.assertEqual(2, len(responses.calls))
179
180 @responses.activate
181 def test_getRemoteBugBatch(self):
182
183 existing_bugs = [
184 {
185 "fields": {
186 "priority": {"name": "High"},
187 "status": {"statusCategory": {"key": "indeterminate"}},
188 },
189 "key": "1",
190 },
191 {
192 "fields": {
193 "priority": {"name": "Medium"},
194 "status": {"statusCategory": {"key": "done"}},
195 },
196 "key": "2",
197 },
198 ]
199
200 def search_callback(request):
201 payload = json.loads(request.body.decode())
202 start_at = payload["startAt"]
203
204 if start_at >= len(existing_bugs):
205 return 404, {}, ""
206
207 return (
208 200,
209 {},
210 json.dumps(
211 {
212 "issues": existing_bugs[start_at : start_at + 1],
213 "total": len(existing_bugs),
214 "startAt": start_at,
215 "maxResults": 1,
216 }
217 ),
218 )
219
220 responses.add_callback(
221 "POST",
222 self.jira.baseurl + "/search",
223 callback=search_callback,
224 content_type="application/json",
225 )
226
227 self.assertDictEqual(
228 {
229 "1": JiraBug(
230 key="1",
231 status=JiraStatus.INDETERMINATE,
232 priority=JiraPriority.HIGH,
233 ),
234 "2": JiraBug(
235 key="2",
236 status=JiraStatus.DONE,
237 priority=JiraPriority.MEDIUM,
238 ),
239 },
240 self.jira.getRemoteBugBatch(["1", "2"]),
241 )
242
243 requests = [call.request for call in responses.calls]
244 self.assertThat(
245 requests,
246 MatchesListwise(
247 [
248 MatchesStructure(
249 method=Equals("POST"),
250 path_url=Equals("/rest/api/2/search"),
251 ),
252 MatchesStructure(
253 method=Equals("POST"),
254 path_url=Equals("/rest/api/2/search"),
255 ),
256 ]
257 ),
258 )
259
260 for i, call in enumerate(responses.calls):
261 payload = json.loads(call.request.body.decode())
262 self.assertEqual("id in (1,2)", payload["jql"])
263 self.assertEqual(["status", "priority"], payload["fields"])
264 self.assertEqual(i, payload["startAt"])
265
266 # Getting the same bugs the second time should fetch it from the cache
267 # without making another request to JIRA API
268 self.assertDictEqual(
269 {
270 "1": JiraBug(
271 key="1",
272 status=JiraStatus.INDETERMINATE,
273 priority=JiraPriority.HIGH,
274 ),
275 "2": JiraBug(
276 key="2",
277 status=JiraStatus.DONE,
278 priority=JiraPriority.MEDIUM,
279 ),
280 },
281 self.jira.getRemoteBugBatch(["1", "2"]),
282 )
283 self.assertEqual(2, len(responses.calls))
284
285 # Verify JQL query when `last_accessed` is specified
286 self.jira.getRemoteBugBatch(
287 ["3"], last_accessed=datetime.datetime(2000, 1, 1, 1, 2, 3)
288 )
289 payload = json.loads(responses.calls[-1].request.body.decode())
290 self.assertEqual(
291 "id in (3) AND updated >= 2000-01-01 01:02", payload["jql"]
292 )
293
294
295class TestJiraUpdateBugWatches(TestCaseWithFactory):
296
297 layer = ZopelessDatabaseLayer
298
299 @responses.activate
300 def test_process_one(self):
301 responses.add(
302 "GET",
303 "https://warthogs.atlassian.net/rest/api/2/issue/LP-984",
304 json={
305 "fields": {
306 "priority": {"name": "Medium"},
307 "status": {"statusCategory": {"key": "indeterminate"}},
308 },
309 "key": "LP-984",
310 },
311 )
312 responses.add(
313 "GET",
314 "https://warthogs.atlassian.net/rest/api/2/serverInfo",
315 json={
316 "serverTime": datetime.datetime.now(
317 tz=datetime.timezone.utc
318 ).isoformat()
319 },
320 )
321 bug_tracker = self.factory.makeBugTracker(
322 base_url="https://warthogs.atlassian.net",
323 bugtrackertype=BugTrackerType.JIRA,
324 )
325 bug = self.factory.makeBug()
326 bug.addWatch(
327 bug_tracker, "LP-984", getUtility(ILaunchpadCelebrities).janitor
328 )
329 self.assertEqual(
330 [("LP-984", None)],
331 [
332 (watch.remotebug, watch.remotestatus)
333 for watch in bug_tracker.watches
334 ],
335 )
336 transaction.commit()
337 logger = BufferLogger()
338 bug_watch_updater = CheckwatchesMaster(transaction, logger=logger)
339 jira = get_external_bugtracker(bug_tracker)
340 jira.batch_query_threshold = 1
341 bug_watch_updater.updateBugWatches(jira, bug_tracker.watches)
342 self.assertEqual(
343 "INFO Updating 1 watches for 1 bugs on "
344 "https://warthogs.atlassian.net/rest/api/2\n",
345 logger.getLogBuffer(),
346 )
347 self.assertEqual(
348 [("LP-984", BugTaskStatus.INPROGRESS)],
349 [
350 (
351 watch.remotebug,
352 jira.convertRemoteStatus(watch.remotestatus),
353 )
354 for watch in bug_tracker.watches
355 ],
356 )
357
358 @responses.activate
359 def test_process_many(self):
360 remote_bugs = [
361 {
362 "fields": {
363 "priority": {"name": "Medium"},
364 "status": {
365 "statusCategory": {
366 "key": "indeterminate"
367 if (bug_id % 2) == 0
368 else "done"
369 }
370 },
371 },
372 "key": str(bug_id),
373 }
374 for bug_id in range(1000, 1010)
375 ]
376 responses.add(
377 "POST",
378 "https://warthogs.atlassian.net/rest/api/2/search",
379 json={
380 "startAt": 0,
381 "maxResults": 100,
382 "total": len(remote_bugs),
383 "issues": remote_bugs,
384 },
385 )
386 responses.add(
387 "GET",
388 "https://warthogs.atlassian.net/rest/api/2/serverInfo",
389 json={
390 "serverTime": datetime.datetime.now(
391 tz=datetime.timezone.utc
392 ).isoformat()
393 },
394 )
395 bug = self.factory.makeBug()
396 bug_tracker = self.factory.makeBugTracker(
397 base_url="https://warthogs.atlassian.net",
398 bugtrackertype=BugTrackerType.JIRA,
399 )
400 for remote_bug in remote_bugs:
401 bug.addWatch(
402 bug_tracker,
403 remote_bug["key"],
404 getUtility(ILaunchpadCelebrities).janitor,
405 )
406 transaction.commit()
407 logger = BufferLogger()
408 bug_watch_updater = CheckwatchesMaster(transaction, logger=logger)
409 jira = get_external_bugtracker(bug_tracker)
410 bug_watch_updater.updateBugWatches(jira, bug_tracker.watches)
411 self.assertEqual(
412 "INFO Updating 10 watches for 10 bugs on "
413 "https://warthogs.atlassian.net/rest/api/2\n",
414 logger.getLogBuffer(),
415 )
416 self.assertContentEqual(
417 [
418 (str(bug_id), BugTaskStatus.INPROGRESS)
419 for bug_id in (1000, 1002, 1004, 1006, 1008)
420 ]
421 + [
422 (str(bug_id), BugTaskStatus.FIXRELEASED)
423 for bug_id in (1001, 1003, 1005, 1007, 1009)
424 ],
425 [
426 (
427 watch.remotebug,
428 jira.convertRemoteStatus(watch.remotestatus),
429 )
430 for watch in bug_tracker.watches
431 ],
432 )
diff --git a/lib/lp/bugs/interfaces/bugtracker.py b/lib/lp/bugs/interfaces/bugtracker.py
index 3f383e3..a44c99b 100644
--- a/lib/lp/bugs/interfaces/bugtracker.py
+++ b/lib/lp/bugs/interfaces/bugtracker.py
@@ -219,6 +219,15 @@ class BugTrackerType(DBEnumeratedType):
219 """,219 """,
220 )220 )
221221
222 JIRA = DBItem(
223 14,
224 """
225 JIRA Issues
226
227 The issue tracker for JIRA-based projects.
228 """,
229 )
230
222231
223# A list of the BugTrackerTypes that don't need a remote product to be232# A list of the BugTrackerTypes that don't need a remote product to be
224# able to return a bug filing URL. We use a whitelist rather than a233# able to return a bug filing URL. We use a whitelist rather than a
diff --git a/lib/lp/bugs/model/bugwatch.py b/lib/lp/bugs/model/bugwatch.py
index 2bbef55..83fe3cd 100644
--- a/lib/lp/bugs/model/bugwatch.py
+++ b/lib/lp/bugs/model/bugwatch.py
@@ -70,6 +70,7 @@ BUG_TRACKER_URL_FORMATS = {
70 BugTrackerType.TRAC: "ticket/%s",70 BugTrackerType.TRAC: "ticket/%s",
71 BugTrackerType.SAVANE: "bugs/?%s",71 BugTrackerType.SAVANE: "bugs/?%s",
72 BugTrackerType.PHPPROJECT: "bug.php?id=%s",72 BugTrackerType.PHPPROJECT: "bug.php?id=%s",
73 BugTrackerType.JIRA: "%s",
73}74}
7475
7576
@@ -418,6 +419,7 @@ class BugWatchSet:
418 BugTrackerType.SAVANE: self.parseSavaneURL,419 BugTrackerType.SAVANE: self.parseSavaneURL,
419 BugTrackerType.SOURCEFORGE: self.parseSourceForgeLikeURL,420 BugTrackerType.SOURCEFORGE: self.parseSourceForgeLikeURL,
420 BugTrackerType.TRAC: self.parseTracURL,421 BugTrackerType.TRAC: self.parseTracURL,
422 BugTrackerType.JIRA: self.parseJiraURL,
421 }423 }
422424
423 def get(self, watch_id):425 def get(self, watch_id):
@@ -745,6 +747,15 @@ class BugWatchSet:
745 base_url = urlunsplit((scheme, host, base_path, "", ""))747 base_url = urlunsplit((scheme, host, base_path, "", ""))
746 return base_url, remote_bug748 return base_url, remote_bug
747749
750 def parseJiraURL(self, scheme, host, path, query):
751 """Extract a JIRA issue base URL and bug ID."""
752 match = re.match(r"^/browse/([A-Z]{1,10}-\d+)$", path)
753 if not match:
754 return None
755 remote_bug = match.group(1)
756 base_url = urlunsplit((scheme, host, "/", "", ""))
757 return base_url, remote_bug
758
748 def extractBugTrackerAndBug(self, url):759 def extractBugTrackerAndBug(self, url):
749 """See `IBugWatchSet`."""760 """See `IBugWatchSet`."""
750 for trackertype, parse_func in self.bugtracker_parse_functions.items():761 for trackertype, parse_func in self.bugtracker_parse_functions.items():
diff --git a/lib/lp/bugs/tests/test_bugwatch.py b/lib/lp/bugs/tests/test_bugwatch.py
index a71a7d0..275761b 100644
--- a/lib/lp/bugs/tests/test_bugwatch.py
+++ b/lib/lp/bugs/tests/test_bugwatch.py
@@ -224,6 +224,15 @@ class ExtractBugTrackerAndBugTest(WithScenarios, TestCase):
224 "bug_id": "12345",224 "bug_id": "12345",
225 },225 },
226 ),226 ),
227 (
228 "JIRA",
229 {
230 "bugtracker_type": BugTrackerType.JIRA,
231 "bug_url": "https://warthogs.atlassian.net/browse/LP-984",
232 "base_url": "https://warthogs.atlassian.net/",
233 "bug_id": "LP-984",
234 },
235 ),
227 ]236 ]
228237
229 layer = LaunchpadFunctionalLayer238 layer = LaunchpadFunctionalLayer
diff --git a/lib/lp/services/config/schema-lazr.conf b/lib/lp/services/config/schema-lazr.conf
index 640865d..da9477e 100644
--- a/lib/lp/services/config/schema-lazr.conf
+++ b/lib/lp/services/config/schema-lazr.conf
@@ -237,7 +237,8 @@ api.github.com.token: none
237gitlab.com.token: none237gitlab.com.token: none
238gitlab.gnome.org.token: none238gitlab.gnome.org.token: none
239salsa.debian.org.token: none239salsa.debian.org.token: none
240240warthogs.atlassian.net.username: none
241warthogs.atlassian.net.password: none
241242
242[cibuild.soss]243[cibuild.soss]
243# value is a JSON Object244# value is a JSON Object

Subscribers

People subscribed via source and target branches

to status/vote changes: