Launchpad itself

Merge lp:~gmb/launchpad/blobjob-cronscript-bug-513190 into lp:launchpad/db-devel

  1. blobjob-cronscript-bug-513190
  2. Merge into db-devel
Proposed by Graham Binns
Status: Merged
Approved by: Graham Binns
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~gmb/launchpad/blobjob-cronscript-bug-513190
Merge into: lp:launchpad/db-devel
Diff against target: 522 lines (+311/-105) (has conflicts)
8 files modified
configs/development/launchpad-lazr.conf (+3/-0)
configs/testrunner/launchpad-lazr.conf (+5/-0)
cronscripts/process-apport-blobs.py (+33/-0)
database/schema/security.cfg (+8/-0)
lib/canonical/config/schema-lazr.conf (+7/-1)
lib/canonical/launchpad/browser/temporaryblobstorage.py (+25/-1)
lib/lp/bugs/tests/test_apportjob.py (+226/-103)
lib/lp/services/mail/sendmail.py (+4/-0)
Text conflict in lib/canonical/launchpad/browser/temporaryblobstorage.py
Text conflict in lib/lp/bugs/tests/test_apportjob.py
Text conflict in lib/lp/services/mail/sendmail.py
To merge this branch: bzr merge lp:~gmb/launchpad/blobjob-cronscript-bug-513190
Reviewer Review Type Date Requested Status
Abel Deuring (community) code Approve
Review via email: mp+19184@code.launchpad.net

Commit message

Add a cronscript for running ProcessApportBlobJobs.

To post a comment you must log in.
Revision history for this message
Graham Binns (gmb) wrote :

This branch adds a cronscript to run ProcessApportBlob jobs. These jobs are created when a new Apport BLOB is uploaded. The idea is that the processing of large BLOBs should be done outside the request, so that +filebug will no longer time out due to the size of uploaded data.

== configs/development/launchpad-lazr.conf ==

 - I've added an error_dir directive for the process_apport_blobs
   script.

== configs/testrunner/launchpad-lazr.conf ==

 - I've added error_dir and oops_prefix directives for the
   process_apport_blobs script.

== cronscripts/process-apport-blobs.py ==

 - I've created a new cronscript based on JobCronScript to run the
   ProcessApportBlob jobs.

== database/schema/security.cfg ==

 - I've added the necessary permissions for the script's db user.

== lib/canonical/config/schema-lazr.conf ==

 - I've added basic config options for the script.

== lib/canonical/launchpad/browser/temporaryblobstorage.py ==

 - I've updated the TemporaryBlobStorageAddView to ensure that errors
   are handled properly if the librarian isn't available for some
   reason.

== lib/lp/bugs/tests/test_apportjob.py ==

 - I've added a test to ensure that the cronscript runs.

No lint.

Revision history for this message
Graham Binns (gmb) wrote :
Download full text (6.2 KiB)

Non-crazy diff:

=== modified file 'configs/development/launchpad-lazr.conf'
--- configs/development/launchpad-lazr.conf 2010-01-22 04:01:17 +0000
+++ configs/development/launchpad-lazr.conf 2010-02-12 15:44:24 +0000
@@ -221,6 +221,9 @@
 error_dir: /var/tmp/poimport
 oops_prefix: POI

+[process_apport_blobs]
+error_dir: /var/tmp/lperr
+
 [supermirror_puller]
 error_dir: /var/tmp/codehosting.test
 oops_prefix: SMP

=== modified file 'configs/testrunner/launchpad-lazr.conf'
--- configs/testrunner/launchpad-lazr.conf 2010-01-29 17:15:31 +0000
+++ configs/testrunner/launchpad-lazr.conf 2010-02-12 15:44:24 +0000
@@ -193,6 +193,11 @@
 error_dir: /var/tmp/poimport.test
 oops_prefix: TPOI

+[process_apport_blobs]
+dbuser: process-apport-blobs
+oops_prefix: TAPPORTBLOB
+error_dir: /var/tmp/lperr.test
+
 [rosettabranches]
 oops_prefix: TRSBR
 error_dir: /var/tmp/rosettabranches.test

=== added file 'cronscripts/process-apport-blobs.py'
--- cronscripts/process-apport-blobs.py 1970-01-01 00:00:00 +0000
+++ cronscripts/process-apport-blobs.py 2010-02-11 18:50:58 +0000
@@ -0,0 +1,33 @@
+#!/usr/bin/python2.5
+#
+# Copyright 2010 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+# pylint: disable-msg=W0403
+
+"""Process uploaded Apport BLOBs."""
+
+__metaclass__ = type
+
+import _pythonpath
+
+from canonical.launchpad.webapp import errorlog
+
+from lp.services.job.runner import JobCronScript
+from lp.bugs.interfaces.apportjob import IProcessApportBlobJobSource
+
+
+class RunProcessApportBlobs(JobCronScript):
+ """Run ProcessApportBlobJobs."""
+
+ config_name = 'process_apport_blobs'
+ source_interface = IProcessApportBlobJobSource
+
+ def main(self):
+ errorlog.globalErrorUtility.configure(self.config_name)
+ return super(RunProcessApportBlobs, self).main()
+
+
+if __name__ == '__main__':
+ script = RunProcessApportBlobs()
+ script.lock_and_run()

=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg 2010-02-02 11:58:46 +0000
+++ database/schema/security.cfg 2010-02-12 15:44:24 +0000
@@ -1897,3 +1897,11 @@
 public.bug = SELECT, UPDATE
 public.job = SELECT, UPDATE, DELETE
 public.bugjob = SELECT, DELETE
+
+[process-apport-blobs]
+type=user
+groups=script,read
+public.job = SELECT, UPDATE, DELETE
+public.apportjob = SELECT, INSERT, UPDATE, DELETE
+public.libraryfilealias = SELECT, INSERT, UPDATE
+public.libraryfilecontent = SELECT, INSERT, UPDATE

=== modified file 'lib/canonical/config/schema-lazr.conf'
--- lib/canonical/config/schema-lazr.conf 2010-01-22 04:01:17 +0000
+++ lib/canonical/config/schema-lazr.conf 2010-02-12 15:44:24 +0000
@@ -28,7 +28,6 @@
 # datatype: string
 base_url: http://ftpmaster.internal/

-
 [calculate_bug_heat]
 # The database user which will be used by this process.
 # datatype: string
@@ -37,6 +36,13 @@
 error_dir: none
 copy_to_zlog: false

+[process_apport_blobs]
+# The database user which will be used by this process.
+# datatype...

Read more...

Revision history for this message
Abel Deuring (adeuring) wrote :

Hi Graham,

anice branch. But please remove the "import pdb; pdb.set_trace()" from temporaryblobstorage.py ;)

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'configs/development/launchpad-lazr.conf'
--- configs/development/launchpad-lazr.conf 2010-01-22 04:01:17 +0000
+++ configs/development/launchpad-lazr.conf 2010-02-12 16:13:42 +0000
@@ -221,6 +221,9 @@
221error_dir: /var/tmp/poimport221error_dir: /var/tmp/poimport
222oops_prefix: POI222oops_prefix: POI
223223
224[process_apport_blobs]
225error_dir: /var/tmp/lperr
226
224[supermirror_puller]227[supermirror_puller]
225error_dir: /var/tmp/codehosting.test228error_dir: /var/tmp/codehosting.test
226oops_prefix: SMP229oops_prefix: SMP
227230
=== modified file 'configs/testrunner/launchpad-lazr.conf'
--- configs/testrunner/launchpad-lazr.conf 2010-01-29 17:15:31 +0000
+++ configs/testrunner/launchpad-lazr.conf 2010-02-12 16:13:42 +0000
@@ -193,6 +193,11 @@
193error_dir: /var/tmp/poimport.test193error_dir: /var/tmp/poimport.test
194oops_prefix: TPOI194oops_prefix: TPOI
195195
196[process_apport_blobs]
197dbuser: process-apport-blobs
198oops_prefix: TAPPORTBLOB
199error_dir: /var/tmp/lperr.test
200
196[rosettabranches]201[rosettabranches]
197oops_prefix: TRSBR202oops_prefix: TRSBR
198error_dir: /var/tmp/rosettabranches.test203error_dir: /var/tmp/rosettabranches.test
199204
=== added file 'cronscripts/process-apport-blobs.py'
--- cronscripts/process-apport-blobs.py 1970-01-01 00:00:00 +0000
+++ cronscripts/process-apport-blobs.py 2010-02-12 16:13:41 +0000
@@ -0,0 +1,33 @@
1#!/usr/bin/python2.5
2#
3# Copyright 2010 Canonical Ltd. This software is licensed under the
4# GNU Affero General Public License version 3 (see the file LICENSE).
5
6# pylint: disable-msg=W0403
7
8"""Process uploaded Apport BLOBs."""
9
10__metaclass__ = type
11
12import _pythonpath
13
14from canonical.launchpad.webapp import errorlog
15
16from lp.services.job.runner import JobCronScript
17from lp.bugs.interfaces.apportjob import IProcessApportBlobJobSource
18
19
20class RunProcessApportBlobs(JobCronScript):
21 """Run ProcessApportBlobJobs."""
22
23 config_name = 'process_apport_blobs'
24 source_interface = IProcessApportBlobJobSource
25
26 def main(self):
27 errorlog.globalErrorUtility.configure(self.config_name)
28 return super(RunProcessApportBlobs, self).main()
29
30
31if __name__ == '__main__':
32 script = RunProcessApportBlobs()
33 script.lock_and_run()
034
=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg 2010-02-02 11:58:46 +0000
+++ database/schema/security.cfg 2010-02-12 16:13:41 +0000
@@ -1897,3 +1897,11 @@
1897public.bug = SELECT, UPDATE1897public.bug = SELECT, UPDATE
1898public.job = SELECT, UPDATE, DELETE1898public.job = SELECT, UPDATE, DELETE
1899public.bugjob = SELECT, DELETE1899public.bugjob = SELECT, DELETE
1900
1901[process-apport-blobs]
1902type=user
1903groups=script,read
1904public.job = SELECT, UPDATE, DELETE
1905public.apportjob = SELECT, INSERT, UPDATE, DELETE
1906public.libraryfilealias = SELECT, INSERT, UPDATE
1907public.libraryfilecontent = SELECT, INSERT, UPDATE
19001908
=== modified file 'lib/canonical/config/schema-lazr.conf'
--- lib/canonical/config/schema-lazr.conf 2010-01-22 04:01:17 +0000
+++ lib/canonical/config/schema-lazr.conf 2010-02-12 16:13:41 +0000
@@ -28,7 +28,6 @@
28# datatype: string28# datatype: string
29base_url: http://ftpmaster.internal/29base_url: http://ftpmaster.internal/
3030
31
32[calculate_bug_heat]31[calculate_bug_heat]
33# The database user which will be used by this process.32# The database user which will be used by this process.
34# datatype: string33# datatype: string
@@ -37,6 +36,13 @@
37error_dir: none36error_dir: none
38copy_to_zlog: false37copy_to_zlog: false
3938
39[process_apport_blobs]
40# The database user which will be used by this process.
41# datatype: string
42dbuser: process-apport-blobs
43oops_prefix: APPORTBLOB
44error_dir: none
45copy_to_zlog: false
4046
41[branchscanner]47[branchscanner]
42# The database user which will be used by this process.48# The database user which will be used by this process.
4349
=== modified file 'lib/canonical/launchpad/browser/temporaryblobstorage.py'
--- lib/canonical/launchpad/browser/temporaryblobstorage.py 2010-02-10 14:17:10 +0000
+++ lib/canonical/launchpad/browser/temporaryblobstorage.py 2010-02-12 16:13:41 +0000
@@ -42,6 +42,7 @@
42 # being named like that.42 # being named like that.
43 @action('Continue', name='FORM_SUBMIT')43 @action('Continue', name='FORM_SUBMIT')
44 def continue_action(self, action, data):44 def continue_action(self, action, data):
45<<<<<<< TREE
45 uuid = self.store_blob(data['blob'])46 uuid = self.store_blob(data['blob'])
46 self.request.response.setHeader('X-Launchpad-Blob-Token', uuid)47 self.request.response.setHeader('X-Launchpad-Blob-Token', uuid)
47 self.request.response.addInfoNotification(48 self.request.response.addInfoNotification(
@@ -51,12 +52,35 @@
51 """Store a blob and return its UUID."""52 """Store a blob and return its UUID."""
52 try:53 try:
53 uuid = getUtility(ITemporaryStorageManager).new(blob)54 uuid = getUtility(ITemporaryStorageManager).new(blob)
55=======
56 uuid = self.store_blob(data['blob'])
57 if uuid is not None:
58 self.request.response.setHeader('X-Launchpad-Blob-Token', uuid)
59 self.request.response.addInfoNotification(
60 'Your ticket is "%s"' % uuid)
61
62 def store_blob(self, blob):
63 """Store a blob and return its UUID."""
64 try:
65 uuid = getUtility(ITemporaryStorageManager).new(blob)
66>>>>>>> MERGE-SOURCE
54 except BlobTooLarge:67 except BlobTooLarge:
55 self.addError('Uploaded file was too large.')68 self.addError('Uploaded file was too large.')
56 except UploadFailed:69 return None
70 except UploadFailed, e:
71 import pdb; pdb.set_trace()
57 self.addError('File storage unavailable - try again later.')72 self.addError('File storage unavailable - try again later.')
73<<<<<<< TREE
5874
59 # Create ProcessApportBlobJob for the BLOB.75 # Create ProcessApportBlobJob for the BLOB.
60 blob = getUtility(ITemporaryStorageManager).fetch(uuid)76 blob = getUtility(ITemporaryStorageManager).fetch(uuid)
61 getUtility(IProcessApportBlobJobSource).create(blob)77 getUtility(IProcessApportBlobJobSource).create(blob)
62 return uuid78 return uuid
79=======
80 return None
81 else:
82 # Create ProcessApportBlobJob for the BLOB.
83 blob = getUtility(ITemporaryStorageManager).fetch(uuid)
84 getUtility(IProcessApportBlobJobSource).create(blob)
85 return uuid
86>>>>>>> MERGE-SOURCE
6387
=== modified file 'lib/lp/bugs/tests/test_apportjob.py'
--- lib/lp/bugs/tests/test_apportjob.py 2010-02-11 10:59:08 +0000
+++ lib/lp/bugs/tests/test_apportjob.py 2010-02-12 16:13:42 +0000
@@ -13,11 +13,20 @@
1313
14from canonical.config import config14from canonical.config import config
15from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet15from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
16from canonical.launchpad.interfaces.temporaryblobstorage import (16<<<<<<< TREE
17 ITemporaryStorageManager)17from canonical.launchpad.interfaces.temporaryblobstorage import (
18from canonical.launchpad.webapp.interfaces import ILaunchpadRoot18 ITemporaryStorageManager)
19from canonical.testing import (19from canonical.launchpad.webapp.interfaces import ILaunchpadRoot
20 LaunchpadFunctionalLayer, LaunchpadZopelessLayer)20from canonical.testing import (
21 LaunchpadFunctionalLayer, LaunchpadZopelessLayer)
22=======
23from canonical.launchpad.interfaces.temporaryblobstorage import (
24 ITemporaryStorageManager)
25from canonical.launchpad.webapp.interfaces import ILaunchpadRoot
26from canonical.launchpad.scripts.tests import run_script
27from canonical.testing import (
28 LaunchpadFunctionalLayer, LaunchpadZopelessLayer)
29>>>>>>> MERGE-SOURCE
2130
22from lp.bugs.interfaces.apportjob import ApportJobType31from lp.bugs.interfaces.apportjob import ApportJobType
23from lp.bugs.model.apportjob import (32from lp.bugs.model.apportjob import (
@@ -162,104 +171,218 @@
162 "LibrarianFileAlias %s" % (171 "LibrarianFileAlias %s" % (
163 attachment['filename'], file_alias.id))172 attachment['filename'], file_alias.id))
164173
165 def test_getByBlobUUID(self):174<<<<<<< TREE
166 # ProcessApportBlobJob.getByBlobUUID takes a BLOB UUID as a175 def test_getByBlobUUID(self):
167 # parameter and returns any jobs for that BLOB.176 # ProcessApportBlobJob.getByBlobUUID takes a BLOB UUID as a
168 uuid = self.blob.uuid177 # parameter and returns any jobs for that BLOB.
169178 uuid = self.blob.uuid
170 job = ProcessApportBlobJob.create(self.blob)179
171 job_from_uuid = ProcessApportBlobJob.getByBlobUUID(uuid)180 job = ProcessApportBlobJob.create(self.blob)
172 self.assertEqual(181 job_from_uuid = ProcessApportBlobJob.getByBlobUUID(uuid)
173 job, job_from_uuid,182 self.assertEqual(
174 "Job returend by getByBlobUUID() did not match original job.")183 job, job_from_uuid,
175 self.assertEqual(184 "Job returend by getByBlobUUID() did not match original job.")
176 self.blob, job_from_uuid.blob,185 self.assertEqual(
177 "BLOB referenced by Job returned by getByBlobUUID() did not "186 self.blob, job_from_uuid.blob,
178 "match original BLOB.")187 "BLOB referenced by Job returned by getByBlobUUID() did not "
179188 "match original BLOB.")
180 def test_create_job_creates_only_one(self):189
181 # ProcessApportBlobJob.create() will create only one190 def test_create_job_creates_only_one(self):
182 # ProcessApportBlobJob for a given BLOB, no matter how many191 # ProcessApportBlobJob.create() will create only one
183 # times it is called.192 # ProcessApportBlobJob for a given BLOB, no matter how many
184 current_jobs = list(ProcessApportBlobJob.iterReady())193 # times it is called.
185 self.assertEqual(194 current_jobs = list(ProcessApportBlobJob.iterReady())
186 0, len(current_jobs),195 self.assertEqual(
187 "There should be no ProcessApportBlobJobs. Found %s" %196 0, len(current_jobs),
188 len(current_jobs))197 "There should be no ProcessApportBlobJobs. Found %s" %
189198 len(current_jobs))
190 job = ProcessApportBlobJob.create(self.blob)199
191 current_jobs = list(ProcessApportBlobJob.iterReady())200 job = ProcessApportBlobJob.create(self.blob)
192 self.assertEqual(201 current_jobs = list(ProcessApportBlobJob.iterReady())
193 1, len(current_jobs),202 self.assertEqual(
194 "There should be only one ProcessApportBlobJob. Found %s" %203 1, len(current_jobs),
195 len(current_jobs))204 "There should be only one ProcessApportBlobJob. Found %s" %
196205 len(current_jobs))
197 another_job = ProcessApportBlobJob.create(self.blob)206
198 current_jobs = list(ProcessApportBlobJob.iterReady())207 another_job = ProcessApportBlobJob.create(self.blob)
199 self.assertEqual(208 current_jobs = list(ProcessApportBlobJob.iterReady())
200 1, len(current_jobs),209 self.assertEqual(
201 "There should be only one ProcessApportBlobJob. Found %s" %210 1, len(current_jobs),
202 len(current_jobs))211 "There should be only one ProcessApportBlobJob. Found %s" %
203212 len(current_jobs))
204 # If the job is complete, it will no longer show up in the list213
205 # of ready jobs. However, it won't be possible to create a new214 # If the job is complete, it will no longer show up in the list
206 # job to process the BLOB because each BLOB can only have one215 # of ready jobs. However, it won't be possible to create a new
207 # ProcessApportBlobJob.216 # job to process the BLOB because each BLOB can only have one
208 job.job.start()217 # ProcessApportBlobJob.
209 job.job.complete()218 job.job.start()
210 current_jobs = list(ProcessApportBlobJob.iterReady())219 job.job.complete()
211 self.assertEqual(220 current_jobs = list(ProcessApportBlobJob.iterReady())
212 0, len(current_jobs),221 self.assertEqual(
213 "There should be no ready ProcessApportBlobJobs. Found %s" %222 0, len(current_jobs),
214 len(current_jobs))223 "There should be no ready ProcessApportBlobJobs. Found %s" %
215224 len(current_jobs))
216 yet_another_job = ProcessApportBlobJob.create(self.blob)225
217 current_jobs = list(ProcessApportBlobJob.iterReady())226 yet_another_job = ProcessApportBlobJob.create(self.blob)
218 self.assertEqual(227 current_jobs = list(ProcessApportBlobJob.iterReady())
219 0, len(current_jobs),228 self.assertEqual(
220 "There should be no new ProcessApportBlobJobs. Found %s" %229 0, len(current_jobs),
221 len(current_jobs))230 "There should be no new ProcessApportBlobJobs. Found %s" %
222231 len(current_jobs))
223 # In fact, yet_another_job will be the same job as before, since232
224 # it's attached to the same BLOB.233 # In fact, yet_another_job will be the same job as before, since
225 self.assertEqual(job.id, yet_another_job.id, "Jobs do not match.")234 # it's attached to the same BLOB.
226235 self.assertEqual(job.id, yet_another_job.id, "Jobs do not match.")
227236
228class TestTemporaryBlobStorageAddView(TestCaseWithFactory):237
229 """Test case for the TemporaryBlobStorageAddView."""238class TestTemporaryBlobStorageAddView(TestCaseWithFactory):
230239 """Test case for the TemporaryBlobStorageAddView."""
231 layer = LaunchpadFunctionalLayer240
232241 layer = LaunchpadFunctionalLayer
233 def setUp(self):242
234 super(TestTemporaryBlobStorageAddView, self).setUp()243 def setUp(self):
235244 super(TestTemporaryBlobStorageAddView, self).setUp()
236 # Create a BLOB using existing testing data.245
237 testfiles = os.path.join(config.root, 'lib/lp/bugs/tests/testfiles')246 # Create a BLOB using existing testing data.
238 blob_file = open(247 testfiles = os.path.join(config.root, 'lib/lp/bugs/tests/testfiles')
239 os.path.join(testfiles, 'extra_filebug_data.msg'))248 blob_file = open(
240 self.blob_data = blob_file.read()249 os.path.join(testfiles, 'extra_filebug_data.msg'))
241 blob_file.close()250 self.blob_data = blob_file.read()
242251 blob_file.close()
243 def test_adding_blob_adds_job(self):252
244 # Using the TemporaryBlobStorageAddView to upload a new BLOB253 def test_adding_blob_adds_job(self):
245 # will add a new ProcessApportBlobJob for that BLOB.254 # Using the TemporaryBlobStorageAddView to upload a new BLOB
246 view = create_initialized_view(255 # will add a new ProcessApportBlobJob for that BLOB.
247 getUtility(ILaunchpadRoot), '+storeblob')256 view = create_initialized_view(
248257 getUtility(ILaunchpadRoot), '+storeblob')
249 # The view's store_blob method stores the blob in the database258
250 # and returns its UUID.259 # The view's store_blob method stores the blob in the database
251 blob_uuid = view.store_blob(self.blob_data)260 # and returns its UUID.
252 transaction.commit()261 blob_uuid = view.store_blob(self.blob_data)
253262 transaction.commit()
254 # A new ProcessApportBlobJob will have been created for the263
255 # BLOB.264 # A new ProcessApportBlobJob will have been created for the
256 blob = getUtility(ITemporaryStorageManager).fetch(blob_uuid)265 # BLOB.
257 job = ProcessApportBlobJob.getByBlobUUID(blob_uuid)266 blob = getUtility(ITemporaryStorageManager).fetch(blob_uuid)
258267 job = ProcessApportBlobJob.getByBlobUUID(blob_uuid)
259 self.assertEqual(268
260 blob, job.blob,269 self.assertEqual(
261 "BLOB attached to Job returned by getByBlobUUID() did not match "270 blob, job.blob,
262 "expected BLOB.")271 "BLOB attached to Job returned by getByBlobUUID() did not match "
272 "expected BLOB.")
273=======
274 def test_getByBlobUUID(self):
275 # ProcessApportBlobJob.getByBlobUUID takes a BLOB UUID as a
276 # parameter and returns any jobs for that BLOB.
277 uuid = self.blob.uuid
278
279 job = ProcessApportBlobJob.create(self.blob)
280 job_from_uuid = ProcessApportBlobJob.getByBlobUUID(uuid)
281 self.assertEqual(
282 job, job_from_uuid,
283 "Job returend by getByBlobUUID() did not match original job.")
284 self.assertEqual(
285 self.blob, job_from_uuid.blob,
286 "BLOB referenced by Job returned by getByBlobUUID() did not "
287 "match original BLOB.")
288
289 def test_create_job_creates_only_one(self):
290 # ProcessApportBlobJob.create() will create only one
291 # ProcessApportBlobJob for a given BLOB, no matter how many
292 # times it is called.
293 current_jobs = list(ProcessApportBlobJob.iterReady())
294 self.assertEqual(
295 0, len(current_jobs),
296 "There should be no ProcessApportBlobJobs. Found %s" %
297 len(current_jobs))
298
299 job = ProcessApportBlobJob.create(self.blob)
300 current_jobs = list(ProcessApportBlobJob.iterReady())
301 self.assertEqual(
302 1, len(current_jobs),
303 "There should be only one ProcessApportBlobJob. Found %s" %
304 len(current_jobs))
305
306 another_job = ProcessApportBlobJob.create(self.blob)
307 current_jobs = list(ProcessApportBlobJob.iterReady())
308 self.assertEqual(
309 1, len(current_jobs),
310 "There should be only one ProcessApportBlobJob. Found %s" %
311 len(current_jobs))
312
313 # If the job is complete, it will no longer show up in the list
314 # of ready jobs. However, it won't be possible to create a new
315 # job to process the BLOB because each BLOB can only have one
316 # ProcessApportBlobJob.
317 job.job.start()
318 job.job.complete()
319 current_jobs = list(ProcessApportBlobJob.iterReady())
320 self.assertEqual(
321 0, len(current_jobs),
322 "There should be no ready ProcessApportBlobJobs. Found %s" %
323 len(current_jobs))
324
325 yet_another_job = ProcessApportBlobJob.create(self.blob)
326 current_jobs = list(ProcessApportBlobJob.iterReady())
327 self.assertEqual(
328 0, len(current_jobs),
329 "There should be no new ProcessApportBlobJobs. Found %s" %
330 len(current_jobs))
331
332 # In fact, yet_another_job will be the same job as before, since
333 # it's attached to the same BLOB.
334 self.assertEqual(job.id, yet_another_job.id, "Jobs do not match.")
335
336 def test_cronscript_succeeds(self):
337 # The process-apport-blobs cronscript will run all pending
338 # ProcessApportBlobJobs.
339 ProcessApportBlobJob.create(self.blob)
340 transaction.commit()
341
342 retcode, stdout, stderr = run_script(
343 'cronscripts/process-apport-blobs.py', [],
344 expect_returncode=0)
345 self.assertEqual('', stdout)
346 self.assertIn(
347 'INFO Ran 1 IProcessApportBlobJobSource jobs.\n', stderr)
348
349
350class TestTemporaryBlobStorageAddView(TestCaseWithFactory):
351 """Test case for the TemporaryBlobStorageAddView."""
352
353 layer = LaunchpadFunctionalLayer
354
355 def setUp(self):
356 super(TestTemporaryBlobStorageAddView, self).setUp()
357
358 # Create a BLOB using existing testing data.
359 testfiles = os.path.join(config.root, 'lib/lp/bugs/tests/testfiles')
360 blob_file = open(
361 os.path.join(testfiles, 'extra_filebug_data.msg'))
362 self.blob_data = blob_file.read()
363 blob_file.close()
364
365 def test_adding_blob_adds_job(self):
366 # Using the TemporaryBlobStorageAddView to upload a new BLOB
367 # will add a new ProcessApportBlobJob for that BLOB.
368 view = create_initialized_view(
369 getUtility(ILaunchpadRoot), '+storeblob')
370
371 # The view's store_blob method stores the blob in the database
372 # and returns its UUID.
373 blob_uuid = view.store_blob(self.blob_data)
374 transaction.commit()
375
376 # A new ProcessApportBlobJob will have been created for the
377 # BLOB.
378 blob = getUtility(ITemporaryStorageManager).fetch(blob_uuid)
379 job = ProcessApportBlobJob.getByBlobUUID(blob_uuid)
380
381 self.assertEqual(
382 blob, job.blob,
383 "BLOB attached to Job returned by getByBlobUUID() did not match "
384 "expected BLOB.")
385>>>>>>> MERGE-SOURCE
263386
264387
265def test_suite():388def test_suite():
266389
=== modified file 'lib/lp/services/mail/sendmail.py'
--- lib/lp/services/mail/sendmail.py 2010-02-12 09:08:43 +0000
+++ lib/lp/services/mail/sendmail.py 2010-02-12 16:13:42 +0000
@@ -30,6 +30,10 @@
30import sets30import sets
3131
32from binascii import b2a_qp32from binascii import b2a_qp
33<<<<<<< TREE
34=======
35import sha
36>>>>>>> MERGE-SOURCE
33from email.Encoders import encode_base6437from email.Encoders import encode_base64
34from email.Utils import getaddresses, make_msgid, formatdate, formataddr38from email.Utils import getaddresses, make_msgid, formatdate, formataddr
35from email.Message import Message39from email.Message import Message
3640
=== modified file 'lib/lp/soyuz/model/publishing.py'

Subscribers

People subscribed via source and target branches

to status/vote changes: