Launchpad itself

Merge ~ines-almeida/launchpad:fetch-service-buildd-manager-end-session into launchpad:master

  1. Git
  2. lp:~ines-almeida/launchpad
  3. fetch-service-buildd-manager-end-session
  4. Merge into master
Proposed by Ines Almeida
Status: Merged
Approved by: Ines Almeida
Approved revision: dd90403bd5235d9f35a9844db9ce857871eed418
Merge reported by: Otto Co-Pilot
Merged at revision: not available
Proposed branch: ~ines-almeida/launchpad:fetch-service-buildd-manager-end-session
Merge into: launchpad:master
Prerequisite: ~ines-almeida/launchpad:fetch-service-buildd-manager-refactor
Diff against target: 596 lines (+348/-34)
9 files modified
lib/lp/buildmaster/builderproxy.py (+85/-5)
lib/lp/buildmaster/downloader.py (+93/-21)
lib/lp/buildmaster/model/buildfarmjobbehaviour.py (+11/-0)
lib/lp/buildmaster/tests/fetchservice.py (+54/-4)
lib/lp/charms/model/charmrecipebuildbehaviour.py (+1/-1)
lib/lp/code/model/cibuildbehaviour.py (+1/-1)
lib/lp/oci/model/ocirecipebuildbehaviour.py (+1/-1)
lib/lp/snappy/model/snapbuildbehaviour.py (+5/-1)
lib/lp/snappy/tests/test_snapbuildbehaviour.py (+97/-0)
Reviewer Review Type Date Requested Status
Jürgen Gmach Approve
Review via email: mp+464312@code.launchpad.net

Commit message

buildd-manager: Add logic to end fetch service session and retrieve metadata.

This adds the necessary code to the BuilderProxyMixin to be able to request metadata to the fetch service, store it in a file, and end the fetch service session; as well as the logic needed to run end end-of-session logc when the build finishes successfully.

Description of the change

All tests in `lp.snappy.tests.test_snapbuildbehaviour` ran successfully, as well as `lp.charms.tests.test_charmrecipebuildbehaviour` (doesn't use fetch service, but uses proxy).

To post a comment you must log in.
Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Thanks! Looks good so far!

Revision history for this message
Ines Almeida (ines-almeida) :
Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Looks great! (also see your notes you took during the meeting)

review: Approve
~ines-almeida/launchpad:fetch-service-buildd-manager-end-session updated
df62eb0... by Ines Almeida

buildd-manager: rename method to start Proxy session

Revision history for this message
Ines Almeida (ines-almeida) wrote :

I wasn't able to make `FetchService()` start the session because it requires an async call to the fetch service, which is quite complicated to get working for `__init__` methods.

Otherwise, all comments were addressed.

~ines-almeida/launchpad:fetch-service-buildd-manager-end-session updated
a1595b9... by Ines Almeida

buildd-manager: Add logic to end fetch service session and retrieve metadata.

This adds the necessary code to the BuilderProxyMixin to be able to request metadata to the fetch service, store it in a file, and end the fetch service session.

dd90403... by Ines Almeida

buildd-master: add logic to call `endProxySession` at the end of a successfull build.

Currently, only snap builds use the fetch service and require any post-build cleanup; for the remaining build types this wil simply `pass`

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/lib/lp/buildmaster/builderproxy.py b/lib/lp/buildmaster/builderproxy.py
2index 304c258..1d08e03 100644
3--- a/lib/lp/buildmaster/builderproxy.py
4+++ b/lib/lp/buildmaster/builderproxy.py
5@@ -15,14 +15,17 @@ __all__ = [
6 ]
7
8 import base64
9+import os
10 import time
11 from typing import Dict, Generator, Type
12
13 from twisted.internet import defer
14
15 from lp.buildmaster.downloader import (
16+ EndFetchServiceSessionCommand,
17 RequestFetchServiceSessionCommand,
18 RequestProxyTokenCommand,
19+ RetrieveFetchServiceSessionCommand,
20 )
21 from lp.buildmaster.interactor import BuilderWorker
22 from lp.buildmaster.interfaces.builder import CannotBuild
23@@ -30,6 +33,8 @@ from lp.buildmaster.interfaces.buildfarmjobbehaviour import BuildArgs
24 from lp.buildmaster.model.buildfarmjob import BuildFarmJob
25 from lp.services.config import config
26
27+BUILD_METADATA_FILENAME_FORMAT = "{build_id}_metadata.json"
28+
29
30 def _get_proxy_config(name):
31 """Get a config item from builddmaster (current) or snappy (deprecated)."""
32@@ -50,15 +55,20 @@ class BuilderProxyMixin:
33 build: BuildFarmJob
34 _worker: BuilderWorker
35
36+ def __init__(self, *args, **kwargs):
37+ super().__init__(*args, **kwargs)
38+ self._use_fetch_service = False
39+ self._proxy_service = None
40+
41 @defer.inlineCallbacks
42- def addProxyArgs(
43+ def startProxySession(
44 self,
45 args: BuildArgs,
46 allow_internet: bool = True,
47 use_fetch_service: bool = False,
48 ) -> Generator[None, Dict[str, str], None]:
49
50- self._proxy_service = None
51+ self._use_fetch_service = use_fetch_service
52
53 if not allow_internet:
54 return
55@@ -83,9 +93,44 @@ class BuilderProxyMixin:
56 self._proxy_service = proxy_service(
57 build_id=self.build.build_cookie, worker=self._worker
58 )
59- new_session = yield self._proxy_service.startSession()
60- args["proxy_url"] = new_session["proxy_url"]
61- args["revocation_endpoint"] = new_session["revocation_endpoint"]
62+ session_data = yield self._proxy_service.startSession()
63+
64+ args["proxy_url"] = session_data["proxy_url"]
65+ args["revocation_endpoint"] = session_data["revocation_endpoint"]
66+
67+ @defer.inlineCallbacks
68+ def endProxySession(self, upload_path: str):
69+ """Handles all the necessary cleanup to be done at the end of a build.
70+
71+ For the fetch service case, this means:
72+ - Retrieving the metadata, and storing into a file in `upload_path`
73+ - Closing the fetch service session (which deletes the session data
74+ from the fetch service system)
75+
76+ Note that if retrieving or storing the metadata file fails, an
77+ exception will be raised, and we won't close the session. This could be
78+ useful for debbugging.
79+ Sessions will be closed automatically within the Fetch Service after
80+ a certain amount of time configured by its charm (default 6 hours).
81+ """
82+ if not self._use_fetch_service:
83+ # No cleanup needed for the builder proxy
84+ return
85+
86+ if self._proxy_service is None:
87+ # A session was never started. This can happen if the proxy configs
88+ # are not set (see `startProxySession`)
89+ return
90+
91+ metadata_file_name = BUILD_METADATA_FILENAME_FORMAT.format(
92+ build_id=self.build.build_cookie
93+ )
94+ file_path = os.path.join(upload_path, metadata_file_name)
95+ yield self._proxy_service.retrieveMetadataFromSession(
96+ save_content_to=file_path
97+ )
98+
99+ yield self._proxy_service.endSession()
100
101
102 class IProxyService:
103@@ -242,3 +287,38 @@ class FetchService(IProxyService):
104 "proxy_url": proxy_url,
105 "revocation_endpoint": revocation_endpoint,
106 }
107+
108+ @defer.inlineCallbacks
109+ def retrieveMetadataFromSession(self, save_content_to: str):
110+ """Make request to retrieve metadata from the current session.
111+
112+ Data is stored directly into a file whose path is `save_content_to`
113+
114+ :raises: RequestException if request to Fetch Service fails
115+ """
116+ url = self.RETRIEVE_METADATA_ENDPOINT.format(
117+ control_endpoint=self.control_endpoint,
118+ session_id=self.session_id,
119+ )
120+ yield self.worker.process_pool.doWork(
121+ RetrieveFetchServiceSessionCommand,
122+ url=url,
123+ auth_header=self.auth_header,
124+ save_content_to=save_content_to,
125+ )
126+
127+ @defer.inlineCallbacks
128+ def endSession(self):
129+ """End the proxy session and do any cleanup needed.
130+
131+ :raises: RequestException if request to Fetch Service fails
132+ """
133+ url = self.END_SESSION_ENDPOINT.format(
134+ control_endpoint=self.control_endpoint,
135+ session_id=self.session_id,
136+ )
137+ yield self.worker.process_pool.doWork(
138+ EndFetchServiceSessionCommand,
139+ url=url,
140+ auth_header=self.auth_header,
141+ )
142diff --git a/lib/lp/buildmaster/downloader.py b/lib/lp/buildmaster/downloader.py
143index ba3eaf2..ca256d7 100644
144--- a/lib/lp/buildmaster/downloader.py
145+++ b/lib/lp/buildmaster/downloader.py
146@@ -9,9 +9,11 @@ anything from the rest of Launchpad.
147
148 __all__ = [
149 "DownloadCommand",
150+ "EndFetchServiceSessionCommand",
151 "RequestFetchServiceSessionCommand",
152 "RequestProcess",
153 "RequestProxyTokenCommand",
154+ "RetrieveFetchServiceSessionCommand",
155 ]
156
157 import os.path
158@@ -39,7 +41,7 @@ class DownloadCommand(amp.Command):
159
160
161 class RequestFetchServiceSessionCommand(amp.Command):
162- """Fetch service API Command subclass
163+ """Fetch service API Command subclass to start a session.
164
165 It defines arguments, response values, and error conditions.
166 For reference:
167@@ -59,6 +61,42 @@ class RequestFetchServiceSessionCommand(amp.Command):
168 }
169
170
171+class RetrieveFetchServiceSessionCommand(amp.Command):
172+ """Fetch service API Command subclass to retrieve data from a session.
173+
174+ It defines arguments and error conditions. For reference:
175+ https://docs.twisted.org/en/twisted-18.7.0/core/howto/amp.html
176+ """
177+
178+ arguments = [
179+ (b"url", amp.Unicode()),
180+ (b"auth_header", amp.String()),
181+ (b"save_content_to", amp.Unicode()),
182+ ]
183+ response: List[Tuple[bytes, amp.Argument]] = []
184+
185+ errors = {
186+ RequestException: b"REQUEST_ERROR",
187+ }
188+
189+
190+class EndFetchServiceSessionCommand(amp.Command):
191+ """Fetch service API Command subclass to end a session.
192+
193+ It defines arguments and error conditions. For reference:
194+ https://docs.twisted.org/en/twisted-18.7.0/core/howto/amp.html
195+ """
196+
197+ arguments = [
198+ (b"url", amp.Unicode()),
199+ (b"auth_header", amp.String()),
200+ ]
201+ response: List[Tuple[bytes, amp.Argument]] = []
202+ errors = {
203+ RequestException: b"REQUEST_ERROR",
204+ }
205+
206+
207 class RequestProxyTokenCommand(amp.Command):
208 arguments = [
209 (b"url", amp.Unicode()),
210@@ -78,32 +116,41 @@ class RequestProxyTokenCommand(amp.Command):
211 class RequestProcess(AMPChild):
212 """A subprocess that performs requests for buildd-manager."""
213
214+ @staticmethod
215+ def _saveResponseToFile(streamed_response, path_to_write):
216+ """Helper method to save a streamed response to a given path.
217+
218+ :param streamed_response: response from a request with `stream=True`.
219+ :param path_to_write: os path (incl. filename) where to write data to.
220+ """
221+ try:
222+ os.makedirs(os.path.dirname(path_to_write))
223+ except FileExistsError:
224+ pass
225+ f = tempfile.NamedTemporaryFile(
226+ mode="wb",
227+ prefix=os.path.basename(path_to_write) + "_",
228+ dir=os.path.dirname(path_to_write),
229+ delete=False,
230+ )
231+ try:
232+ stream.stream_response_to_file(streamed_response, path=f)
233+ except Exception:
234+ f.close()
235+ os.unlink(f.name)
236+ raise
237+ else:
238+ f.close()
239+ os.rename(f.name, path_to_write)
240+ return {}
241+
242 @DownloadCommand.responder
243 def downloadCommand(self, file_url, path_to_write, timeout):
244 with Session() as session:
245 session.trust_env = False
246 response = session.get(file_url, timeout=timeout, stream=True)
247 response.raise_for_status()
248- try:
249- os.makedirs(os.path.dirname(path_to_write))
250- except FileExistsError:
251- pass
252- f = tempfile.NamedTemporaryFile(
253- mode="wb",
254- prefix=os.path.basename(path_to_write) + "_",
255- dir=os.path.dirname(path_to_write),
256- delete=False,
257- )
258- try:
259- stream.stream_response_to_file(response, path=f)
260- except Exception:
261- f.close()
262- os.unlink(f.name)
263- raise
264- else:
265- f.close()
266- os.rename(f.name, path_to_write)
267- return {}
268+ return self._saveResponseToFile(response, path_to_write)
269
270 @RequestProxyTokenCommand.responder
271 def requestProxyTokenCommand(self, url, auth_header, proxy_username):
272@@ -135,3 +182,28 @@ class RequestProcess(AMPChild):
273 )
274 response.raise_for_status()
275 return response.json()
276+
277+ @RetrieveFetchServiceSessionCommand.responder
278+ def retrieveFetchServiceSessionCommand(
279+ self, url, auth_header, save_content_to
280+ ):
281+ with Session() as session:
282+ session.trust_env = False
283+ response = session.get(
284+ url,
285+ headers={"Authorization": auth_header},
286+ stream=True,
287+ )
288+ response.raise_for_status()
289+ return self._saveResponseToFile(response, save_content_to)
290+
291+ @EndFetchServiceSessionCommand.responder
292+ def endFetchServiceSessionCommand(self, url, auth_header):
293+ with Session() as session:
294+ session.trust_env = False
295+ response = session.delete(
296+ url,
297+ headers={"Authorization": auth_header},
298+ )
299+ response.raise_for_status()
300+ return {}
301diff --git a/lib/lp/buildmaster/model/buildfarmjobbehaviour.py b/lib/lp/buildmaster/model/buildfarmjobbehaviour.py
302index ae81888..e8d82d2 100644
303--- a/lib/lp/buildmaster/model/buildfarmjobbehaviour.py
304+++ b/lib/lp/buildmaster/model/buildfarmjobbehaviour.py
305@@ -405,6 +405,11 @@ class BuildFarmJobBehaviourBase:
306 yield self._worker.getFiles(filenames_to_download, logger=logger)
307
308 @defer.inlineCallbacks
309+ def _saveBuildSpecificFiles(self, upload_path):
310+ # Specific for each build type
311+ yield
312+
313+ @defer.inlineCallbacks
314 def handleSuccess(self, worker_status, logger):
315 """Handle a package that built successfully.
316
317@@ -447,6 +452,12 @@ class BuildFarmJobBehaviourBase:
318
319 yield self._downloadFiles(worker_status, upload_path, logger)
320
321+ # Certain builds might require saving specific files. This is the case,
322+ # for example, for snap builds that use the fetch service as their
323+ # proxy, which require a metadata file to be retrieve in the end of the
324+ # build.
325+ yield self._saveBuildSpecificFiles(upload_path)
326+
327 transaction.commit()
328
329 # Move the directory used to grab the binaries into incoming
330diff --git a/lib/lp/buildmaster/tests/fetchservice.py b/lib/lp/buildmaster/tests/fetchservice.py
331index 34b6122..b23535a 100644
332--- a/lib/lp/buildmaster/tests/fetchservice.py
333+++ b/lib/lp/buildmaster/tests/fetchservice.py
334@@ -25,6 +25,7 @@ class FetchServiceControlEndpoint(resource.Resource):
335 def __init__(self):
336 resource.Resource.__init__(self)
337 self.requests = []
338+ self.responses = []
339 self.session_id = "2ea9c9f759044f9b9aff469fe90429a5"
340
341 def render_POST(self, request):
342@@ -38,12 +39,61 @@ class FetchServiceControlEndpoint(resource.Resource):
343 "json": content,
344 }
345 )
346- return json.dumps(
347+ response = {
348+ "id": self.session_id,
349+ "token": uuid.uuid4().hex,
350+ }
351+ self.responses.append(response)
352+ return json.dumps(response).encode("UTF-8")
353+
354+ def render_GET(self, request):
355+ """We make a GET request to get a fetch service session metadata"""
356+ self.requests.append(
357 {
358- "id": self.session_id,
359- "token": uuid.uuid4().hex,
360+ "method": request.method,
361+ "uri": request.uri,
362+ "headers": dict(request.requestHeaders.getAllRawHeaders()),
363 }
364- ).encode("UTF-8")
365+ )
366+
367+ request.setHeader(b"Content-Type", b"application/json")
368+ response = {
369+ "session-id": self.session_id,
370+ "start-time": "2024-04-17T16:25:02.631557582Z",
371+ "end-time": "2024-04-17T16:26:23.505219343Z",
372+ "inspectors": [
373+ "pip.simple-index",
374+ "pip.wheel",
375+ "deb",
376+ "apt.release",
377+ "apt.packages",
378+ "default",
379+ ],
380+ "spool-path": (
381+ f"/var/snap/fetch-service/common/spool/{self.session_id}"
382+ ),
383+ "policy": "test",
384+ "processed-requests": 0,
385+ "processed-artefacts": 0,
386+ "rejected-requests": 0,
387+ "rejected-artefacts": 0,
388+ "artefacts": [],
389+ }
390+ self.responses.append(response)
391+ return json.dumps(response).encode("UTF-8")
392+
393+ def render_DELETE(self, request):
394+ """We make a DELETE request to end a fetch service session"""
395+ self.requests.append(
396+ {
397+ "method": request.method,
398+ "uri": request.uri,
399+ "headers": dict(request.requestHeaders.getAllRawHeaders()),
400+ }
401+ )
402+ response = {}
403+ self.responses.append(response)
404+ return json.dumps(response).encode("UTF-8")
405
406
407 class InProcessFetchServiceAuthAPIFixture(fixtures.Fixture):
408diff --git a/lib/lp/charms/model/charmrecipebuildbehaviour.py b/lib/lp/charms/model/charmrecipebuildbehaviour.py
409index 5e7bf50..f1ee46c 100644
410--- a/lib/lp/charms/model/charmrecipebuildbehaviour.py
411+++ b/lib/lp/charms/model/charmrecipebuildbehaviour.py
412@@ -79,7 +79,7 @@ class CharmRecipeBuildBehaviour(BuilderProxyMixin, BuildFarmJobBehaviourBase):
413 """
414 build = self.build
415 args: BuildArgs = yield super().extraBuildArgs(logger=logger)
416- yield self.addProxyArgs(args)
417+ yield self.startProxySession(args)
418 args["name"] = build.recipe.store_name or build.recipe.name
419 channels = build.channels or {}
420 # We have to remove the security proxy that Zope applies to this
421diff --git a/lib/lp/code/model/cibuildbehaviour.py b/lib/lp/code/model/cibuildbehaviour.py
422index 648546b..6cceb72 100644
423--- a/lib/lp/code/model/cibuildbehaviour.py
424+++ b/lib/lp/code/model/cibuildbehaviour.py
425@@ -176,7 +176,7 @@ class CIBuildBehaviour(BuilderProxyMixin, BuildFarmJobBehaviourBase):
426 )
427
428 args: BuildArgs = yield super().extraBuildArgs(logger=logger)
429- yield self.addProxyArgs(args)
430+ yield self.startProxySession(args)
431 (
432 args["archives"],
433 args["trusted_keys"],
434diff --git a/lib/lp/oci/model/ocirecipebuildbehaviour.py b/lib/lp/oci/model/ocirecipebuildbehaviour.py
435index dd12112..4482b17 100644
436--- a/lib/lp/oci/model/ocirecipebuildbehaviour.py
437+++ b/lib/lp/oci/model/ocirecipebuildbehaviour.py
438@@ -136,7 +136,7 @@ class OCIRecipeBuildBehaviour(BuilderProxyMixin, BuildFarmJobBehaviourBase):
439 """
440 build = self.build
441 args: BuildArgs = yield super().extraBuildArgs(logger=logger)
442- yield self.addProxyArgs(args, build.recipe.allow_internet)
443+ yield self.startProxySession(args, build.recipe.allow_internet)
444 # XXX twom 2020-02-17 This may need to be more complex, and involve
445 # distribution name.
446 args["name"] = build.recipe.name
447diff --git a/lib/lp/snappy/model/snapbuildbehaviour.py b/lib/lp/snappy/model/snapbuildbehaviour.py
448index 58fdd5f..7f0cf6d 100644
449--- a/lib/lp/snappy/model/snapbuildbehaviour.py
450+++ b/lib/lp/snappy/model/snapbuildbehaviour.py
451@@ -116,7 +116,7 @@ class SnapBuildBehaviour(BuilderProxyMixin, BuildFarmJobBehaviourBase):
452 """
453 build: ISnapBuild = self.build
454 args: BuildArgs = yield super().extraBuildArgs(logger=logger)
455- yield self.addProxyArgs(
456+ yield self.startProxySession(
457 args, build.snap.allow_internet, build.snap.use_fetch_service
458 )
459 args["name"] = build.snap.store_name or build.snap.name
460@@ -209,3 +209,7 @@ class SnapBuildBehaviour(BuilderProxyMixin, BuildFarmJobBehaviourBase):
461 # check does not make sense. We do, however, refuse to build for
462 # obsolete series.
463 assert self.build.distro_series.status != SeriesStatus.OBSOLETE
464+
465+ @defer.inlineCallbacks
466+ def _saveBuildSpecificFiles(self, upload_path):
467+ yield self.endProxySession(upload_path)
468diff --git a/lib/lp/snappy/tests/test_snapbuildbehaviour.py b/lib/lp/snappy/tests/test_snapbuildbehaviour.py
469index 781643c..d48efbc 100644
470--- a/lib/lp/snappy/tests/test_snapbuildbehaviour.py
471+++ b/lib/lp/snappy/tests/test_snapbuildbehaviour.py
472@@ -4,6 +4,7 @@
473 """Test snap package build behaviour."""
474
475 import base64
476+import json
477 import os.path
478 import time
479 import uuid
480@@ -39,6 +40,7 @@ from lp.app.enums import InformationType
481 from lp.archivepublisher.interfaces.archivegpgsigningkey import (
482 IArchiveGPGSigningKey,
483 )
484+from lp.buildmaster.builderproxy import BuilderProxy
485 from lp.buildmaster.enums import BuildBaseImageType, BuildStatus
486 from lp.buildmaster.interactor import shut_down_default_process_pool
487 from lp.buildmaster.interfaces.builder import CannotBuild
488@@ -415,6 +417,91 @@ class TestAsyncSnapBuildBehaviourFetchService(
489 yield job.extraBuildArgs()
490 self.assertEqual([], self.fetch_service_api.sessions.requests)
491
492+ @defer.inlineCallbacks
493+ def test_endProxySession(self):
494+ """By ending a fetch service session, metadata is retrieved from the
495+ fetch service and saved to a file; and call to end the session is made.
496+ """
497+ self.useFixture(
498+ FeatureFixture({SNAP_USE_FETCH_SERVICE_FEATURE_FLAG: "on"})
499+ )
500+ tem_upload_path = self.useFixture(fixtures.TempDir()).path
501+
502+ snap = self.factory.makeSnap(use_fetch_service=True)
503+ request = self.factory.makeSnapBuildRequest(snap=snap)
504+ job = self.makeJob(snap=snap, build_request=request)
505+ yield job.extraBuildArgs()
506+
507+ # End the session
508+ yield job.endProxySession(upload_path=tem_upload_path)
509+
510+ # We expect 3 calls made to the fetch service API, in this order
511+ self.assertEqual(3, len(self.fetch_service_api.sessions.requests))
512+
513+ # Request start a session
514+ start_session_request = self.fetch_service_api.sessions.requests[0]
515+ self.assertEqual(b"POST", start_session_request["method"])
516+ self.assertEqual(b"/session", start_session_request["uri"])
517+ session_id = self.fetch_service_api.sessions.responses[0]["id"]
518+
519+ # Request retrieve metadata
520+ retrieve_metadata_request = self.fetch_service_api.sessions.requests[1]
521+ self.assertEqual(b"GET", retrieve_metadata_request["method"])
522+ self.assertEqual(
523+ f"/session/{session_id}".encode(), retrieve_metadata_request["uri"]
524+ )
525+
526+ # Request end session
527+ end_session_request = self.fetch_service_api.sessions.requests[2]
528+ self.assertEqual(b"DELETE", end_session_request["method"])
529+ self.assertEqual(
530+ f"/session/{session_id}".encode(), end_session_request["uri"]
531+ )
532+
533+ # The expected file is created in the `tem_upload_path`
534+ expected_filename = f"{job.build.build_cookie}_metadata.json"
535+ expected_file_path = os.path.join(tem_upload_path, expected_filename)
536+ with open(expected_file_path) as f:
537+ self.assertEqual(
538+ json.dumps(self.fetch_service_api.sessions.responses[1]),
539+ f.read(),
540+ )
541+
542+ @defer.inlineCallbacks
543+ def test_endProxySession_fetch_Service_false(self):
544+ """When `use_fetch_service` is False, we don't make any calls to the
545+ fetch service API."""
546+ self.useFixture(
547+ FeatureFixture({SNAP_USE_FETCH_SERVICE_FEATURE_FLAG: "on"})
548+ )
549+ snap = self.factory.makeSnap(use_fetch_service=False)
550+ request = self.factory.makeSnapBuildRequest(snap=snap)
551+ job = self.makeJob(snap=snap, build_request=request)
552+ yield job.extraBuildArgs()
553+ yield job.endProxySession(upload_path="test_path")
554+
555+ # No calls go through to the fetch service
556+ self.assertEqual(0, len(self.fetch_service_api.sessions.requests))
557+
558+ @defer.inlineCallbacks
559+ def test_endProxySession_no_proxy_service(self):
560+ """When the `fetch_service_host` is not set, the calls to the fetch
561+ service don't go through."""
562+ self.useFixture(
563+ FeatureFixture({SNAP_USE_FETCH_SERVICE_FEATURE_FLAG: "on"})
564+ )
565+ self.useFixture(FeatureFixture({"fetch_service_host": None}))
566+
567+ snap = self.factory.makeSnap(use_fetch_service=True)
568+ request = self.factory.makeSnapBuildRequest(snap=snap)
569+ job = self.makeJob(snap=snap, build_request=request)
570+ yield job.extraBuildArgs()
571+ yield job.endProxySession(upload_path="test_path")
572+
573+ # No calls go through to the fetch service
574+ self.assertEqual(0, len(self.fetch_service_api.sessions.requests))
575+ self.assertEqual(None, job._proxy_service)
576+
577
578 class TestAsyncSnapBuildBehaviourBuilderProxy(
579 StatsMixin, TestSnapBuildBehaviourBase
580@@ -1363,6 +1450,16 @@ class TestAsyncSnapBuildBehaviourBuilderProxy(
581 args = yield job.extraBuildArgs()
582 self.assertTrue(args["private"])
583
584+ def test_endProxySession(self):
585+ branch = self.factory.makeBranch()
586+ job = self.makeJob(branch=branch)
587+ yield job.extraBuildArgs()
588+
589+ # End the session
590+ yield job.endProxySession(upload_path="test_path")
591+ self.assertFalse(job.use_fetch_service)
592+ self.assertTrue(isinstance(job.proxy_service, BuilderProxy))
593+
594 @defer.inlineCallbacks
595 def test_composeBuildRequest_proxy_url_set(self):
596 job = self.makeJob()

Subscribers

People subscribed via source and target branches

to status/vote changes: