Launchpad itself

Merge lp:~jtv/launchpad/use-fakelibrarian into lp:launchpad

  1. use-fakelibrarian
  2. Merge into devel
Proposed by Jeroen T. Vermeulen
Status: Merged
Approved by: Graham Binns
Approved revision: no longer in the source branch.
Merged at revision: 11427
Proposed branch: lp:~jtv/launchpad/use-fakelibrarian
Merge into: lp:launchpad
Prerequisite: lp:~jtv/launchpad/fakelibrarian-commit
Diff against target: 223 lines (+43/-36)
1 file modified
lib/lp/translations/utilities/tests/test_file_importer.py (+43/-36)
To merge this branch: bzr merge lp:~jtv/launchpad/use-fakelibrarian
Reviewer Review Type Date Requested Status
Graham Binns (community) code Approve
Robert Collins code Pending
Review via email: mp+33324@code.launchpad.net

Commit message

Convert a test to FakeLibrarian.

Description of the change

= Convert a test to FakeLibrarian =

Requires the lp:~jtv/launchpad/fakelibrarian-commit branch.

The Translations import engine annoyingly relies on the Librarian from the very first point where an upload is picked for processing all the way down into the per-format input parsers. I'm hoping to fix that someday but until that day, here's a branch that converts one of the most badly affected tests to use the FakeLibrarian.

I cleaned up a bunch of lint later, so that may be a little distracting. The meat of the matter is in the transaction.commit calls that have been replaced with self.fakelibrarian.pretendCommit calls. This means that a database commit is no longer needed to make each test pass.

The change speeds up the test by about 20% (based on the shortest times used by the second of two pairs of runs each, to eliminate cache variability as much as possible). The percentage is about equal for setup, which no longer requires the LibrarianLayer; test run time; and overall test time. All numbers as reported by "bin/test -vvv."

The test runs were done with LP_PERSISTENT_TEST_SERVICES enabled. The speedup would be much more dramatic without that. On the other hand, the speedup is obviously less dramatic as a part of a larger test run that's likely to set up LibrarianLayer anyway.

Jeroen

To post a comment you must log in.
Revision history for this message
Graham Binns (gmb) :
review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/translations/utilities/tests/test_file_importer.py'
--- lib/lp/translations/utilities/tests/test_file_importer.py 2010-08-20 20:31:18 +0000
+++ lib/lp/translations/utilities/tests/test_file_importer.py 2010-08-22 10:25:01 +0000
@@ -1,19 +1,17 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the1# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).2# GNU Affero General Public License version 3 (see the file LICENSE).
33
4"""Translation File Importer tests."""4"""Translation File Importer tests."""
55
6__metaclass__ = type6__metaclass__ = type
77
8import unittest
9
10import transaction
11from zope.component import getUtility8from zope.component import getUtility
12from zope.security.proxy import removeSecurityProxy9from zope.security.proxy import removeSecurityProxy
1310
14from canonical.testing import LaunchpadZopelessLayer11from canonical.testing import ZopelessDatabaseLayer
15from lp.registry.interfaces.person import IPersonSet12from lp.registry.interfaces.person import IPersonSet
16from lp.testing import TestCaseWithFactory13from lp.testing import TestCaseWithFactory
14from lp.testing.fakelibrarian import FakeLibrarian
17from lp.translations.interfaces.translationgroup import TranslationPermission15from lp.translations.interfaces.translationgroup import TranslationPermission
18from lp.translations.interfaces.translationimporter import (16from lp.translations.interfaces.translationimporter import (
19 OutdatedTranslationError,17 OutdatedTranslationError,
@@ -82,7 +80,8 @@
82#, c-format80#, c-format
83msgid "%s"81msgid "%s"
84msgstr ""82msgstr ""
85''' % (TEST_MSGID_ERROR)83''' % TEST_MSGID_ERROR
84
8685
87TEST_TRANSLATION_FILE_WITH_ERROR = r'''86TEST_TRANSLATION_FILE_WITH_ERROR = r'''
88msgid ""87msgid ""
@@ -95,11 +94,12 @@
95#, c-format94#, c-format
96msgid "%s"95msgid "%s"
97msgstr "format specifier changes %%s"96msgstr "format specifier changes %%s"
98''' % (TEST_MSGID_ERROR)97''' % TEST_MSGID_ERROR
98
9999
100class FileImporterTestCase(TestCaseWithFactory):100class FileImporterTestCase(TestCaseWithFactory):
101 """Class test for translation importer component"""101 """Class test for translation importer component"""
102 layer = LaunchpadZopelessLayer102 layer = ZopelessDatabaseLayer
103103
104 def _createFileImporters(self, pot_content, po_content, is_published):104 def _createFileImporters(self, pot_content, po_content, is_published):
105 """Create queue entries from POT and PO content strings.105 """Create queue entries from POT and PO content strings.
@@ -119,9 +119,8 @@
119 is_published, self.importer_person,119 is_published, self.importer_person,
120 productseries=potemplate.productseries,120 productseries=potemplate.productseries,
121 potemplate=potemplate)121 potemplate=potemplate)
122 transaction.commit()122 self.fake_librarian.pretendCommit()
123 return POTFileImporter(123 return POTFileImporter(template_entry, GettextPOImporter(), None)
124 template_entry, GettextPOImporter(), None )
125124
126 def _createPOFileImporter(self,125 def _createPOFileImporter(self,
127 pot_importer, po_content, is_published, existing_pofile=None,126 pot_importer, po_content, is_published, existing_pofile=None,
@@ -138,9 +137,8 @@
138 translation_entry = self.translation_import_queue.addOrUpdateEntry(137 translation_entry = self.translation_import_queue.addOrUpdateEntry(
139 pofile.path, po_content, is_published, person,138 pofile.path, po_content, is_published, person,
140 productseries=potemplate.productseries, pofile=pofile)139 productseries=potemplate.productseries, pofile=pofile)
141 transaction.commit()140 self.fake_librarian.pretendCommit()
142 return POFileImporter(141 return POFileImporter(translation_entry, GettextPOImporter(), None)
143 translation_entry, GettextPOImporter(), None )
144142
145 def _createImporterForExportedEntries(self):143 def _createImporterForExportedEntries(self):
146 """Set up entries that where exported from LP, i.e. that contain the144 """Set up entries that where exported from LP, i.e. that contain the
@@ -164,15 +162,20 @@
164 False, self.importer_person,162 False, self.importer_person,
165 productseries=potemplate.productseries,163 productseries=potemplate.productseries,
166 potemplate=potemplate)164 potemplate=potemplate)
167 transaction.commit()165 self.fake_librarian.pretendCommit()
168 return FileImporter(166 return FileImporter(template_entry, GettextPOImporter(), None)
169 template_entry, GettextPOImporter(), None )
170167
171 def setUp(self):168 def setUp(self):
172 super(FileImporterTestCase, self).setUp()169 super(FileImporterTestCase, self).setUp()
170 self.fake_librarian = FakeLibrarian()
171 self.fake_librarian.installAsLibrarian()
173 self.translation_import_queue = getUtility(ITranslationImportQueue)172 self.translation_import_queue = getUtility(ITranslationImportQueue)
174 self.importer_person = self.factory.makePerson()173 self.importer_person = self.factory.makePerson()
175174
175 def tearDown(self):
176 self.fake_librarian.uninstall()
177 super(FileImporterTestCase, self).tearDown()
178
176 def test_FileImporter_importMessage_NotImplemented(self):179 def test_FileImporter_importMessage_NotImplemented(self):
177 importer = self._createFileImporter()180 importer = self._createFileImporter()
178 self.failUnlessRaises(NotImplementedError,181 self.failUnlessRaises(NotImplementedError,
@@ -241,7 +244,7 @@
241 product.translationpermission = TranslationPermission.CLOSED244 product.translationpermission = TranslationPermission.CLOSED
242 product.translationgroup = self.factory.makeTranslationGroup(245 product.translationgroup = self.factory.makeTranslationGroup(
243 self.importer_person)246 self.importer_person)
244 transaction.commit()247 self.fake_librarian.pretendCommit()
245248
246 # Get one POTMsgSet to do storeTranslationsInDatabase on.249 # Get one POTMsgSet to do storeTranslationsInDatabase on.
247 message = pot_importer.translation_file.messages[0]250 message = pot_importer.translation_file.messages[0]
@@ -280,7 +283,7 @@
280 "POTFileImporter had no reference to an IPOTemplate.")283 "POTFileImporter had no reference to an IPOTemplate.")
281 self.failUnless(pot_importer.pofile is None or284 self.failUnless(pot_importer.pofile is None or
282 pot_importer.pofile.language == "en",285 pot_importer.pofile.language == "en",
283 "POTFileImporter referenced an IPOFile which was not English." )286 "POTFileImporter referenced an IPOFile which was not English.")
284 # Test if POFileImporter gets initialised correctly.287 # Test if POFileImporter gets initialised correctly.
285 self.failUnless(po_importer.potemplate is not None,288 self.failUnless(po_importer.potemplate is not None,
286 "POTFileImporter had no reference to an IPOTemplate.")289 "POTFileImporter had no reference to an IPOTemplate.")
@@ -340,9 +343,9 @@
340 # Test correct import operation for both343 # Test correct import operation for both
341 # exported and published files.344 # exported and published files.
342 importers = (345 importers = (
343 self._createImporterForExportedEntries(),346 self._createImporterForExportedEntries(),
344 self._createImporterForPublishedEntries()347 self._createImporterForPublishedEntries(),
345 )348 )
346 for (pot_importer, po_importer) in importers:349 for (pot_importer, po_importer) in importers:
347 # Run the import and see if PotMsgSet and TranslationMessage350 # Run the import and see if PotMsgSet and TranslationMessage
348 # entries are correctly created in the DB.351 # entries are correctly created in the DB.
@@ -385,7 +388,7 @@
385 self.failUnlessEqual(len(errors), 0,388 self.failUnlessEqual(len(errors), 0,
386 "POFileImporter.importFile returned errors where there should "389 "POFileImporter.importFile returned errors where there should "
387 "be none.")390 "be none.")
388 transaction.commit()391 self.fake_librarian.pretendCommit()
389392
390 # Create new POFileImporter with an earlier timestamp and393 # Create new POFileImporter with an earlier timestamp and
391 # a different translation (msgstr).394 # a different translation (msgstr).
@@ -397,7 +400,8 @@
397 self.failUnlessEqual(len(errors), 1,400 self.failUnlessEqual(len(errors), 1,
398 "No error detected when importing a pofile with an earlier "401 "No error detected when importing a pofile with an earlier "
399 "export timestamp (update conflict).")402 "export timestamp (update conflict).")
400 self.failUnless( errors[0]['error-message'].find(403 self.failUnless(
404 errors[0]['error-message'].find(
401 u"updated by someone else after you") != -1,405 u"updated by someone else after you") != -1,
402 "importFile() failed to detect a message update conflict.")406 "importFile() failed to detect a message update conflict.")
403407
@@ -441,7 +445,7 @@
441 TEST_TRANSLATION_FILE_WITH_ERROR, False)445 TEST_TRANSLATION_FILE_WITH_ERROR, False)
442 pot_importer.importFile()446 pot_importer.importFile()
443 po_importer.importFile()447 po_importer.importFile()
444 transaction.commit()448 self.fake_librarian.pretendCommit()
445449
446 po_importer2 = self._createPOFileImporter(450 po_importer2 = self._createPOFileImporter(
447 pot_importer, TEST_TRANSLATION_EXPORTED_EARLIER, False,451 pot_importer, TEST_TRANSLATION_EXPORTED_EARLIER, False,
@@ -477,7 +481,7 @@
477 "Last-Translator: Hector Atlas <??@??.??>\\n"481 "Last-Translator: Hector Atlas <??@??.??>\\n"
478 "Content-Type: text/plain; charset=UTF-8\\n"482 "Content-Type: text/plain; charset=UTF-8\\n"
479 "X-Launchpad-Export-Date: 2008-11-05 13:31+0000\\n"483 "X-Launchpad-Export-Date: 2008-11-05 13:31+0000\\n"
480 484
481 msgid "%s"485 msgid "%s"
482 msgstr "Dankuwel"486 msgstr "Dankuwel"
483 """ % TEST_MSGID487 """ % TEST_MSGID
@@ -493,13 +497,19 @@
493497
494class CreateFileImporterTestCase(TestCaseWithFactory):498class CreateFileImporterTestCase(TestCaseWithFactory):
495 """Class test for translation importer creation."""499 """Class test for translation importer creation."""
496 layer = LaunchpadZopelessLayer500 layer = ZopelessDatabaseLayer
497501
498 def setUp(self):502 def setUp(self):
499 super(CreateFileImporterTestCase, self).setUp()503 super(CreateFileImporterTestCase, self).setUp()
504 self.fake_librarian = FakeLibrarian()
505 self.fake_librarian.installAsLibrarian()
500 self.translation_import_queue = getUtility(ITranslationImportQueue)506 self.translation_import_queue = getUtility(ITranslationImportQueue)
501 self.importer_person = self.factory.makePerson()507 self.importer_person = self.factory.makePerson()
502508
509 def tearDown(self):
510 self.fake_librarian.uninstall()
511 super(CreateFileImporterTestCase, self).tearDown()
512
503 def _make_queue_entry(self, is_published):513 def _make_queue_entry(self, is_published):
504 pofile = self.factory.makePOFile('eo')514 pofile = self.factory.makePOFile('eo')
505 # Create a header with a newer date than what is found in515 # Create a header with a newer date than what is found in
@@ -510,18 +520,19 @@
510 queue_entry = self.translation_import_queue.addOrUpdateEntry(520 queue_entry = self.translation_import_queue.addOrUpdateEntry(
511 pofile.path, po_content, is_published, self.importer_person,521 pofile.path, po_content, is_published, self.importer_person,
512 productseries=pofile.potemplate.productseries, pofile=pofile)522 productseries=pofile.potemplate.productseries, pofile=pofile)
513 transaction.commit()523 self.fake_librarian.pretendCommit()
514 return queue_entry524 return queue_entry
515525
516 def test_raises_OutdatedTranslationError_on_user_uploads(self):526 def test_raises_OutdatedTranslationError_on_user_uploads(self):
517 queue_entry = self._make_queue_entry(False)527 queue_entry = self._make_queue_entry(False)
518 self.assertRaises(OutdatedTranslationError, POFileImporter,528 self.assertRaises(
519 queue_entry, GettextPOImporter(), None )529 OutdatedTranslationError,
530 POFileImporter, queue_entry, GettextPOImporter(), None)
520531
521 def test_not_raises_OutdatedTranslationError_on_published_uploads(self):532 def test_not_raises_OutdatedTranslationError_on_published_uploads(self):
522 queue_entry = self._make_queue_entry(True)533 queue_entry = self._make_queue_entry(True)
523 try:534 try:
524 importer = POFileImporter(queue_entry, GettextPOImporter(), None )535 importer = POFileImporter(queue_entry, GettextPOImporter(), None)
525 except OutdatedTranslationError:536 except OutdatedTranslationError:
526 self.fail("OutdatedTranslationError raised.")537 self.fail("OutdatedTranslationError raised.")
527538
@@ -529,9 +540,5 @@
529 queue_entry = self._make_queue_entry(True)540 queue_entry = self._make_queue_entry(True)
530 pofile = queue_entry.pofile541 pofile = queue_entry.pofile
531 old_raw_header = pofile.header542 old_raw_header = pofile.header
532 importer = POFileImporter(queue_entry, GettextPOImporter(), None )543 importer = POFileImporter(queue_entry, GettextPOImporter(), None)
533 self.assertEqual(old_raw_header, pofile.header)544 self.assertEqual(old_raw_header, pofile.header)
534
535
536def test_suite():
537 return unittest.TestLoader().loadTestsFromName(__name__)