Launchpad itself

Merge lp:~gmb/launchpad/blobjob-cronscript-bug-513190 into lp:launchpad/db-devel

  1. blobjob-cronscript-bug-513190
  2. Merge into db-devel
Proposed by Graham Binns
Status: Merged
Approved by: Graham Binns
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~gmb/launchpad/blobjob-cronscript-bug-513190
Merge into: lp:launchpad/db-devel
Diff against target: 522 lines (+311/-105) (has conflicts)
8 files modified
configs/development/launchpad-lazr.conf (+3/-0)
configs/testrunner/launchpad-lazr.conf (+5/-0)
cronscripts/process-apport-blobs.py (+33/-0)
database/schema/security.cfg (+8/-0)
lib/canonical/config/schema-lazr.conf (+7/-1)
lib/canonical/launchpad/browser/temporaryblobstorage.py (+25/-1)
lib/lp/bugs/tests/test_apportjob.py (+226/-103)
lib/lp/services/mail/sendmail.py (+4/-0)
Text conflict in lib/canonical/launchpad/browser/temporaryblobstorage.py
Text conflict in lib/lp/bugs/tests/test_apportjob.py
Text conflict in lib/lp/services/mail/sendmail.py
To merge this branch: bzr merge lp:~gmb/launchpad/blobjob-cronscript-bug-513190
Reviewer Review Type Date Requested Status
Abel Deuring (community) code Approve
Review via email: mp+19184@code.launchpad.net

Commit message

Add a cronscript for running ProcessApportBlobJobs.

To post a comment you must log in.
Revision history for this message
Graham Binns (gmb) wrote :

This branch adds a cronscript to run ProcessApportBlob jobs. These jobs are created when a new Apport BLOB is uploaded. The idea is that the processing of large BLOBs should be done outside the request, so that +filebug will no longer time out due to the size of uploaded data.

== configs/development/launchpad-lazr.conf ==

 - I've added an error_dir directive for the process_apport_blobs
   script.

== configs/testrunner/launchpad-lazr.conf ==

 - I've added error_dir and oops_prefix directives for the
   process_apport_blobs script.

== cronscripts/process-apport-blobs.py ==

 - I've created a new cronscript based on JobCronScript to run the
   ProcessApportBlob jobs.

== database/schema/security.cfg ==

 - I've added the necessary permissions for the script's db user.

== lib/canonical/config/schema-lazr.conf ==

 - I've added basic config options for the script.

== lib/canonical/launchpad/browser/temporaryblobstorage.py ==

 - I've updated the TemporaryBlobStorageAddView to ensure that errors
   are handled properly if the librarian isn't available for some
   reason.

== lib/lp/bugs/tests/test_apportjob.py ==

 - I've added a test to ensure that the cronscript runs.

No lint.

Revision history for this message
Graham Binns (gmb) wrote :
Download full text (6.2 KiB)

Non-crazy diff:

=== modified file 'configs/development/launchpad-lazr.conf'
--- configs/development/launchpad-lazr.conf 2010-01-22 04:01:17 +0000
+++ configs/development/launchpad-lazr.conf 2010-02-12 15:44:24 +0000
@@ -221,6 +221,9 @@
 error_dir: /var/tmp/poimport
 oops_prefix: POI

+[process_apport_blobs]
+error_dir: /var/tmp/lperr
+
 [supermirror_puller]
 error_dir: /var/tmp/codehosting.test
 oops_prefix: SMP

=== modified file 'configs/testrunner/launchpad-lazr.conf'
--- configs/testrunner/launchpad-lazr.conf 2010-01-29 17:15:31 +0000
+++ configs/testrunner/launchpad-lazr.conf 2010-02-12 15:44:24 +0000
@@ -193,6 +193,11 @@
 error_dir: /var/tmp/poimport.test
 oops_prefix: TPOI

+[process_apport_blobs]
+dbuser: process-apport-blobs
+oops_prefix: TAPPORTBLOB
+error_dir: /var/tmp/lperr.test
+
 [rosettabranches]
 oops_prefix: TRSBR
 error_dir: /var/tmp/rosettabranches.test

=== added file 'cronscripts/process-apport-blobs.py'
--- cronscripts/process-apport-blobs.py 1970-01-01 00:00:00 +0000
+++ cronscripts/process-apport-blobs.py 2010-02-11 18:50:58 +0000
@@ -0,0 +1,33 @@
+#!/usr/bin/python2.5
+#
+# Copyright 2010 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+# pylint: disable-msg=W0403
+
+"""Process uploaded Apport BLOBs."""
+
+__metaclass__ = type
+
+import _pythonpath
+
+from canonical.launchpad.webapp import errorlog
+
+from lp.services.job.runner import JobCronScript
+from lp.bugs.interfaces.apportjob import IProcessApportBlobJobSource
+
+
+class RunProcessApportBlobs(JobCronScript):
+ """Run ProcessApportBlobJobs."""
+
+ config_name = 'process_apport_blobs'
+ source_interface = IProcessApportBlobJobSource
+
+ def main(self):
+ errorlog.globalErrorUtility.configure(self.config_name)
+ return super(RunProcessApportBlobs, self).main()
+
+
+if __name__ == '__main__':
+ script = RunProcessApportBlobs()
+ script.lock_and_run()

=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg 2010-02-02 11:58:46 +0000
+++ database/schema/security.cfg 2010-02-12 15:44:24 +0000
@@ -1897,3 +1897,11 @@
 public.bug = SELECT, UPDATE
 public.job = SELECT, UPDATE, DELETE
 public.bugjob = SELECT, DELETE
+
+[process-apport-blobs]
+type=user
+groups=script,read
+public.job = SELECT, UPDATE, DELETE
+public.apportjob = SELECT, INSERT, UPDATE, DELETE
+public.libraryfilealias = SELECT, INSERT, UPDATE
+public.libraryfilecontent = SELECT, INSERT, UPDATE

=== modified file 'lib/canonical/config/schema-lazr.conf'
--- lib/canonical/config/schema-lazr.conf 2010-01-22 04:01:17 +0000
+++ lib/canonical/config/schema-lazr.conf 2010-02-12 15:44:24 +0000
@@ -28,7 +28,6 @@
 # datatype: string
 base_url: http://ftpmaster.internal/

-
 [calculate_bug_heat]
 # The database user which will be used by this process.
 # datatype: string
@@ -37,6 +36,13 @@
 error_dir: none
 copy_to_zlog: false

+[process_apport_blobs]
+# The database user which will be used by this process.
+# datatype...

Read more...

Revision history for this message
Abel Deuring (adeuring) wrote :

Hi Graham,

anice branch. But please remove the "import pdb; pdb.set_trace()" from temporaryblobstorage.py ;)

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'configs/development/launchpad-lazr.conf'
2--- configs/development/launchpad-lazr.conf 2010-01-22 04:01:17 +0000
3+++ configs/development/launchpad-lazr.conf 2010-02-12 16:13:42 +0000
4@@ -221,6 +221,9 @@
5 error_dir: /var/tmp/poimport
6 oops_prefix: POI
7
8+[process_apport_blobs]
9+error_dir: /var/tmp/lperr
10+
11 [supermirror_puller]
12 error_dir: /var/tmp/codehosting.test
13 oops_prefix: SMP
14
15=== modified file 'configs/testrunner/launchpad-lazr.conf'
16--- configs/testrunner/launchpad-lazr.conf 2010-01-29 17:15:31 +0000
17+++ configs/testrunner/launchpad-lazr.conf 2010-02-12 16:13:42 +0000
18@@ -193,6 +193,11 @@
19 error_dir: /var/tmp/poimport.test
20 oops_prefix: TPOI
21
22+[process_apport_blobs]
23+dbuser: process-apport-blobs
24+oops_prefix: TAPPORTBLOB
25+error_dir: /var/tmp/lperr.test
26+
27 [rosettabranches]
28 oops_prefix: TRSBR
29 error_dir: /var/tmp/rosettabranches.test
30
31=== added file 'cronscripts/process-apport-blobs.py'
32--- cronscripts/process-apport-blobs.py 1970-01-01 00:00:00 +0000
33+++ cronscripts/process-apport-blobs.py 2010-02-12 16:13:41 +0000
34@@ -0,0 +1,33 @@
35+#!/usr/bin/python2.5
36+#
37+# Copyright 2010 Canonical Ltd. This software is licensed under the
38+# GNU Affero General Public License version 3 (see the file LICENSE).
39+
40+# pylint: disable-msg=W0403
41+
42+"""Process uploaded Apport BLOBs."""
43+
44+__metaclass__ = type
45+
46+import _pythonpath
47+
48+from canonical.launchpad.webapp import errorlog
49+
50+from lp.services.job.runner import JobCronScript
51+from lp.bugs.interfaces.apportjob import IProcessApportBlobJobSource
52+
53+
54+class RunProcessApportBlobs(JobCronScript):
55+ """Run ProcessApportBlobJobs."""
56+
57+ config_name = 'process_apport_blobs'
58+ source_interface = IProcessApportBlobJobSource
59+
60+ def main(self):
61+ errorlog.globalErrorUtility.configure(self.config_name)
62+ return super(RunProcessApportBlobs, self).main()
63+
64+
65+if __name__ == '__main__':
66+ script = RunProcessApportBlobs()
67+ script.lock_and_run()
68
69=== modified file 'database/schema/security.cfg'
70--- database/schema/security.cfg 2010-02-02 11:58:46 +0000
71+++ database/schema/security.cfg 2010-02-12 16:13:41 +0000
72@@ -1897,3 +1897,11 @@
73 public.bug = SELECT, UPDATE
74 public.job = SELECT, UPDATE, DELETE
75 public.bugjob = SELECT, DELETE
76+
77+[process-apport-blobs]
78+type=user
79+groups=script,read
80+public.job = SELECT, UPDATE, DELETE
81+public.apportjob = SELECT, INSERT, UPDATE, DELETE
82+public.libraryfilealias = SELECT, INSERT, UPDATE
83+public.libraryfilecontent = SELECT, INSERT, UPDATE
84
85=== modified file 'lib/canonical/config/schema-lazr.conf'
86--- lib/canonical/config/schema-lazr.conf 2010-01-22 04:01:17 +0000
87+++ lib/canonical/config/schema-lazr.conf 2010-02-12 16:13:41 +0000
88@@ -28,7 +28,6 @@
89 # datatype: string
90 base_url: http://ftpmaster.internal/
91
92-
93 [calculate_bug_heat]
94 # The database user which will be used by this process.
95 # datatype: string
96@@ -37,6 +36,13 @@
97 error_dir: none
98 copy_to_zlog: false
99
100+[process_apport_blobs]
101+# The database user which will be used by this process.
102+# datatype: string
103+dbuser: process-apport-blobs
104+oops_prefix: APPORTBLOB
105+error_dir: none
106+copy_to_zlog: false
107
108 [branchscanner]
109 # The database user which will be used by this process.
110
111=== modified file 'lib/canonical/launchpad/browser/temporaryblobstorage.py'
112--- lib/canonical/launchpad/browser/temporaryblobstorage.py 2010-02-10 14:17:10 +0000
113+++ lib/canonical/launchpad/browser/temporaryblobstorage.py 2010-02-12 16:13:41 +0000
114@@ -42,6 +42,7 @@
115 # being named like that.
116 @action('Continue', name='FORM_SUBMIT')
117 def continue_action(self, action, data):
118+<<<<<<< TREE
119 uuid = self.store_blob(data['blob'])
120 self.request.response.setHeader('X-Launchpad-Blob-Token', uuid)
121 self.request.response.addInfoNotification(
122@@ -51,12 +52,35 @@
123 """Store a blob and return its UUID."""
124 try:
125 uuid = getUtility(ITemporaryStorageManager).new(blob)
126+=======
127+ uuid = self.store_blob(data['blob'])
128+ if uuid is not None:
129+ self.request.response.setHeader('X-Launchpad-Blob-Token', uuid)
130+ self.request.response.addInfoNotification(
131+ 'Your ticket is "%s"' % uuid)
132+
133+ def store_blob(self, blob):
134+ """Store a blob and return its UUID."""
135+ try:
136+ uuid = getUtility(ITemporaryStorageManager).new(blob)
137+>>>>>>> MERGE-SOURCE
138 except BlobTooLarge:
139 self.addError('Uploaded file was too large.')
140- except UploadFailed:
141+ return None
142+ except UploadFailed, e:
143+ import pdb; pdb.set_trace()
144 self.addError('File storage unavailable - try again later.')
145+<<<<<<< TREE
146
147 # Create ProcessApportBlobJob for the BLOB.
148 blob = getUtility(ITemporaryStorageManager).fetch(uuid)
149 getUtility(IProcessApportBlobJobSource).create(blob)
150 return uuid
151+=======
152+ return None
153+ else:
154+ # Create ProcessApportBlobJob for the BLOB.
155+ blob = getUtility(ITemporaryStorageManager).fetch(uuid)
156+ getUtility(IProcessApportBlobJobSource).create(blob)
157+ return uuid
158+>>>>>>> MERGE-SOURCE
159
160=== modified file 'lib/lp/bugs/tests/test_apportjob.py'
161--- lib/lp/bugs/tests/test_apportjob.py 2010-02-11 10:59:08 +0000
162+++ lib/lp/bugs/tests/test_apportjob.py 2010-02-12 16:13:42 +0000
163@@ -13,11 +13,20 @@
164
165 from canonical.config import config
166 from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
167-from canonical.launchpad.interfaces.temporaryblobstorage import (
168- ITemporaryStorageManager)
169-from canonical.launchpad.webapp.interfaces import ILaunchpadRoot
170-from canonical.testing import (
171- LaunchpadFunctionalLayer, LaunchpadZopelessLayer)
172+<<<<<<< TREE
173+from canonical.launchpad.interfaces.temporaryblobstorage import (
174+ ITemporaryStorageManager)
175+from canonical.launchpad.webapp.interfaces import ILaunchpadRoot
176+from canonical.testing import (
177+ LaunchpadFunctionalLayer, LaunchpadZopelessLayer)
178+=======
179+from canonical.launchpad.interfaces.temporaryblobstorage import (
180+ ITemporaryStorageManager)
181+from canonical.launchpad.webapp.interfaces import ILaunchpadRoot
182+from canonical.launchpad.scripts.tests import run_script
183+from canonical.testing import (
184+ LaunchpadFunctionalLayer, LaunchpadZopelessLayer)
185+>>>>>>> MERGE-SOURCE
186
187 from lp.bugs.interfaces.apportjob import ApportJobType
188 from lp.bugs.model.apportjob import (
189@@ -162,104 +171,218 @@
190 "LibrarianFileAlias %s" % (
191 attachment['filename'], file_alias.id))
192
193- def test_getByBlobUUID(self):
194- # ProcessApportBlobJob.getByBlobUUID takes a BLOB UUID as a
195- # parameter and returns any jobs for that BLOB.
196- uuid = self.blob.uuid
197-
198- job = ProcessApportBlobJob.create(self.blob)
199- job_from_uuid = ProcessApportBlobJob.getByBlobUUID(uuid)
200- self.assertEqual(
201- job, job_from_uuid,
202- "Job returend by getByBlobUUID() did not match original job.")
203- self.assertEqual(
204- self.blob, job_from_uuid.blob,
205- "BLOB referenced by Job returned by getByBlobUUID() did not "
206- "match original BLOB.")
207-
208- def test_create_job_creates_only_one(self):
209- # ProcessApportBlobJob.create() will create only one
210- # ProcessApportBlobJob for a given BLOB, no matter how many
211- # times it is called.
212- current_jobs = list(ProcessApportBlobJob.iterReady())
213- self.assertEqual(
214- 0, len(current_jobs),
215- "There should be no ProcessApportBlobJobs. Found %s" %
216- len(current_jobs))
217-
218- job = ProcessApportBlobJob.create(self.blob)
219- current_jobs = list(ProcessApportBlobJob.iterReady())
220- self.assertEqual(
221- 1, len(current_jobs),
222- "There should be only one ProcessApportBlobJob. Found %s" %
223- len(current_jobs))
224-
225- another_job = ProcessApportBlobJob.create(self.blob)
226- current_jobs = list(ProcessApportBlobJob.iterReady())
227- self.assertEqual(
228- 1, len(current_jobs),
229- "There should be only one ProcessApportBlobJob. Found %s" %
230- len(current_jobs))
231-
232- # If the job is complete, it will no longer show up in the list
233- # of ready jobs. However, it won't be possible to create a new
234- # job to process the BLOB because each BLOB can only have one
235- # ProcessApportBlobJob.
236- job.job.start()
237- job.job.complete()
238- current_jobs = list(ProcessApportBlobJob.iterReady())
239- self.assertEqual(
240- 0, len(current_jobs),
241- "There should be no ready ProcessApportBlobJobs. Found %s" %
242- len(current_jobs))
243-
244- yet_another_job = ProcessApportBlobJob.create(self.blob)
245- current_jobs = list(ProcessApportBlobJob.iterReady())
246- self.assertEqual(
247- 0, len(current_jobs),
248- "There should be no new ProcessApportBlobJobs. Found %s" %
249- len(current_jobs))
250-
251- # In fact, yet_another_job will be the same job as before, since
252- # it's attached to the same BLOB.
253- self.assertEqual(job.id, yet_another_job.id, "Jobs do not match.")
254-
255-
256-class TestTemporaryBlobStorageAddView(TestCaseWithFactory):
257- """Test case for the TemporaryBlobStorageAddView."""
258-
259- layer = LaunchpadFunctionalLayer
260-
261- def setUp(self):
262- super(TestTemporaryBlobStorageAddView, self).setUp()
263-
264- # Create a BLOB using existing testing data.
265- testfiles = os.path.join(config.root, 'lib/lp/bugs/tests/testfiles')
266- blob_file = open(
267- os.path.join(testfiles, 'extra_filebug_data.msg'))
268- self.blob_data = blob_file.read()
269- blob_file.close()
270-
271- def test_adding_blob_adds_job(self):
272- # Using the TemporaryBlobStorageAddView to upload a new BLOB
273- # will add a new ProcessApportBlobJob for that BLOB.
274- view = create_initialized_view(
275- getUtility(ILaunchpadRoot), '+storeblob')
276-
277- # The view's store_blob method stores the blob in the database
278- # and returns its UUID.
279- blob_uuid = view.store_blob(self.blob_data)
280- transaction.commit()
281-
282- # A new ProcessApportBlobJob will have been created for the
283- # BLOB.
284- blob = getUtility(ITemporaryStorageManager).fetch(blob_uuid)
285- job = ProcessApportBlobJob.getByBlobUUID(blob_uuid)
286-
287- self.assertEqual(
288- blob, job.blob,
289- "BLOB attached to Job returned by getByBlobUUID() did not match "
290- "expected BLOB.")
291+<<<<<<< TREE
292+ def test_getByBlobUUID(self):
293+ # ProcessApportBlobJob.getByBlobUUID takes a BLOB UUID as a
294+ # parameter and returns any jobs for that BLOB.
295+ uuid = self.blob.uuid
296+
297+ job = ProcessApportBlobJob.create(self.blob)
298+ job_from_uuid = ProcessApportBlobJob.getByBlobUUID(uuid)
299+ self.assertEqual(
300+ job, job_from_uuid,
301+ "Job returend by getByBlobUUID() did not match original job.")
302+ self.assertEqual(
303+ self.blob, job_from_uuid.blob,
304+ "BLOB referenced by Job returned by getByBlobUUID() did not "
305+ "match original BLOB.")
306+
307+ def test_create_job_creates_only_one(self):
308+ # ProcessApportBlobJob.create() will create only one
309+ # ProcessApportBlobJob for a given BLOB, no matter how many
310+ # times it is called.
311+ current_jobs = list(ProcessApportBlobJob.iterReady())
312+ self.assertEqual(
313+ 0, len(current_jobs),
314+ "There should be no ProcessApportBlobJobs. Found %s" %
315+ len(current_jobs))
316+
317+ job = ProcessApportBlobJob.create(self.blob)
318+ current_jobs = list(ProcessApportBlobJob.iterReady())
319+ self.assertEqual(
320+ 1, len(current_jobs),
321+ "There should be only one ProcessApportBlobJob. Found %s" %
322+ len(current_jobs))
323+
324+ another_job = ProcessApportBlobJob.create(self.blob)
325+ current_jobs = list(ProcessApportBlobJob.iterReady())
326+ self.assertEqual(
327+ 1, len(current_jobs),
328+ "There should be only one ProcessApportBlobJob. Found %s" %
329+ len(current_jobs))
330+
331+ # If the job is complete, it will no longer show up in the list
332+ # of ready jobs. However, it won't be possible to create a new
333+ # job to process the BLOB because each BLOB can only have one
334+ # ProcessApportBlobJob.
335+ job.job.start()
336+ job.job.complete()
337+ current_jobs = list(ProcessApportBlobJob.iterReady())
338+ self.assertEqual(
339+ 0, len(current_jobs),
340+ "There should be no ready ProcessApportBlobJobs. Found %s" %
341+ len(current_jobs))
342+
343+ yet_another_job = ProcessApportBlobJob.create(self.blob)
344+ current_jobs = list(ProcessApportBlobJob.iterReady())
345+ self.assertEqual(
346+ 0, len(current_jobs),
347+ "There should be no new ProcessApportBlobJobs. Found %s" %
348+ len(current_jobs))
349+
350+ # In fact, yet_another_job will be the same job as before, since
351+ # it's attached to the same BLOB.
352+ self.assertEqual(job.id, yet_another_job.id, "Jobs do not match.")
353+
354+
355+class TestTemporaryBlobStorageAddView(TestCaseWithFactory):
356+ """Test case for the TemporaryBlobStorageAddView."""
357+
358+ layer = LaunchpadFunctionalLayer
359+
360+ def setUp(self):
361+ super(TestTemporaryBlobStorageAddView, self).setUp()
362+
363+ # Create a BLOB using existing testing data.
364+ testfiles = os.path.join(config.root, 'lib/lp/bugs/tests/testfiles')
365+ blob_file = open(
366+ os.path.join(testfiles, 'extra_filebug_data.msg'))
367+ self.blob_data = blob_file.read()
368+ blob_file.close()
369+
370+ def test_adding_blob_adds_job(self):
371+ # Using the TemporaryBlobStorageAddView to upload a new BLOB
372+ # will add a new ProcessApportBlobJob for that BLOB.
373+ view = create_initialized_view(
374+ getUtility(ILaunchpadRoot), '+storeblob')
375+
376+ # The view's store_blob method stores the blob in the database
377+ # and returns its UUID.
378+ blob_uuid = view.store_blob(self.blob_data)
379+ transaction.commit()
380+
381+ # A new ProcessApportBlobJob will have been created for the
382+ # BLOB.
383+ blob = getUtility(ITemporaryStorageManager).fetch(blob_uuid)
384+ job = ProcessApportBlobJob.getByBlobUUID(blob_uuid)
385+
386+ self.assertEqual(
387+ blob, job.blob,
388+ "BLOB attached to Job returned by getByBlobUUID() did not match "
389+ "expected BLOB.")
390+=======
391+ def test_getByBlobUUID(self):
392+ # ProcessApportBlobJob.getByBlobUUID takes a BLOB UUID as a
393+ # parameter and returns any jobs for that BLOB.
394+ uuid = self.blob.uuid
395+
396+ job = ProcessApportBlobJob.create(self.blob)
397+ job_from_uuid = ProcessApportBlobJob.getByBlobUUID(uuid)
398+ self.assertEqual(
399+ job, job_from_uuid,
400+ "Job returend by getByBlobUUID() did not match original job.")
401+ self.assertEqual(
402+ self.blob, job_from_uuid.blob,
403+ "BLOB referenced by Job returned by getByBlobUUID() did not "
404+ "match original BLOB.")
405+
406+ def test_create_job_creates_only_one(self):
407+ # ProcessApportBlobJob.create() will create only one
408+ # ProcessApportBlobJob for a given BLOB, no matter how many
409+ # times it is called.
410+ current_jobs = list(ProcessApportBlobJob.iterReady())
411+ self.assertEqual(
412+ 0, len(current_jobs),
413+ "There should be no ProcessApportBlobJobs. Found %s" %
414+ len(current_jobs))
415+
416+ job = ProcessApportBlobJob.create(self.blob)
417+ current_jobs = list(ProcessApportBlobJob.iterReady())
418+ self.assertEqual(
419+ 1, len(current_jobs),
420+ "There should be only one ProcessApportBlobJob. Found %s" %
421+ len(current_jobs))
422+
423+ another_job = ProcessApportBlobJob.create(self.blob)
424+ current_jobs = list(ProcessApportBlobJob.iterReady())
425+ self.assertEqual(
426+ 1, len(current_jobs),
427+ "There should be only one ProcessApportBlobJob. Found %s" %
428+ len(current_jobs))
429+
430+ # If the job is complete, it will no longer show up in the list
431+ # of ready jobs. However, it won't be possible to create a new
432+ # job to process the BLOB because each BLOB can only have one
433+ # ProcessApportBlobJob.
434+ job.job.start()
435+ job.job.complete()
436+ current_jobs = list(ProcessApportBlobJob.iterReady())
437+ self.assertEqual(
438+ 0, len(current_jobs),
439+ "There should be no ready ProcessApportBlobJobs. Found %s" %
440+ len(current_jobs))
441+
442+ yet_another_job = ProcessApportBlobJob.create(self.blob)
443+ current_jobs = list(ProcessApportBlobJob.iterReady())
444+ self.assertEqual(
445+ 0, len(current_jobs),
446+ "There should be no new ProcessApportBlobJobs. Found %s" %
447+ len(current_jobs))
448+
449+ # In fact, yet_another_job will be the same job as before, since
450+ # it's attached to the same BLOB.
451+ self.assertEqual(job.id, yet_another_job.id, "Jobs do not match.")
452+
453+ def test_cronscript_succeeds(self):
454+ # The process-apport-blobs cronscript will run all pending
455+ # ProcessApportBlobJobs.
456+ ProcessApportBlobJob.create(self.blob)
457+ transaction.commit()
458+
459+ retcode, stdout, stderr = run_script(
460+ 'cronscripts/process-apport-blobs.py', [],
461+ expect_returncode=0)
462+ self.assertEqual('', stdout)
463+ self.assertIn(
464+ 'INFO Ran 1 IProcessApportBlobJobSource jobs.\n', stderr)
465+
466+
467+class TestTemporaryBlobStorageAddView(TestCaseWithFactory):
468+ """Test case for the TemporaryBlobStorageAddView."""
469+
470+ layer = LaunchpadFunctionalLayer
471+
472+ def setUp(self):
473+ super(TestTemporaryBlobStorageAddView, self).setUp()
474+
475+ # Create a BLOB using existing testing data.
476+ testfiles = os.path.join(config.root, 'lib/lp/bugs/tests/testfiles')
477+ blob_file = open(
478+ os.path.join(testfiles, 'extra_filebug_data.msg'))
479+ self.blob_data = blob_file.read()
480+ blob_file.close()
481+
482+ def test_adding_blob_adds_job(self):
483+ # Using the TemporaryBlobStorageAddView to upload a new BLOB
484+ # will add a new ProcessApportBlobJob for that BLOB.
485+ view = create_initialized_view(
486+ getUtility(ILaunchpadRoot), '+storeblob')
487+
488+ # The view's store_blob method stores the blob in the database
489+ # and returns its UUID.
490+ blob_uuid = view.store_blob(self.blob_data)
491+ transaction.commit()
492+
493+ # A new ProcessApportBlobJob will have been created for the
494+ # BLOB.
495+ blob = getUtility(ITemporaryStorageManager).fetch(blob_uuid)
496+ job = ProcessApportBlobJob.getByBlobUUID(blob_uuid)
497+
498+ self.assertEqual(
499+ blob, job.blob,
500+ "BLOB attached to Job returned by getByBlobUUID() did not match "
501+ "expected BLOB.")
502+>>>>>>> MERGE-SOURCE
503
504
505 def test_suite():
506
507=== modified file 'lib/lp/services/mail/sendmail.py'
508--- lib/lp/services/mail/sendmail.py 2010-02-12 09:08:43 +0000
509+++ lib/lp/services/mail/sendmail.py 2010-02-12 16:13:42 +0000
510@@ -30,6 +30,10 @@
511 import sets
512
513 from binascii import b2a_qp
514+<<<<<<< TREE
515+=======
516+import sha
517+>>>>>>> MERGE-SOURCE
518 from email.Encoders import encode_base64
519 from email.Utils import getaddresses, make_msgid, formatdate, formataddr
520 from email.Message import Message
521
522=== modified file 'lib/lp/soyuz/model/publishing.py'

Subscribers

People subscribed via source and target branches

to status/vote changes: