Launchpad itself

Merge lp:~mwhudson/launchpad/back-off-failing-imports-bug-413637 into lp:launchpad/db-devel

  1. back-off-failing-imports-bug-413637
  2. Merge into db-devel
Proposed by Michael Hudson-Doyle
Status: Rejected
Rejected by: Michael Hudson-Doyle
Proposed branch: lp:~mwhudson/launchpad/back-off-failing-imports-bug-413637
Merge into: lp:launchpad/db-devel
Diff against target: 873 lines (+431/-102)
13 files modified
lib/canonical/buildd/debian/launchpad-buildd.init (+1/-1)
lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py (+1/-1)
lib/lp/buildmaster/model/buildfarmjobbehavior.py (+94/-10)
lib/lp/buildmaster/tests/test_buildfarmjobbehavior.py (+80/-8)
lib/lp/code/interfaces/codeimportjob.py (+7/-3)
lib/lp/code/model/codeimportjob.py (+26/-21)
lib/lp/code/model/recipebuilder.py (+11/-2)
lib/lp/code/model/tests/test_codeimportjob.py (+25/-0)
lib/lp/soyuz/doc/buildd-slavescanner.txt (+4/-3)
lib/lp/testing/factory.py (+87/-40)
lib/lp/translations/doc/translationtemplatesbuildbehavior.txt (+41/-12)
lib/lp/translations/model/translationtemplatesbuildbehavior.py (+17/-1)
lib/lp/translations/tests/test_translationtemplatesbuildbehavior.py (+37/-0)
To merge this branch: bzr merge lp:~mwhudson/launchpad/back-off-failing-imports-bug-413637
Reviewer Review Type Date Requested Status
Canonical Launchpad Engineering Pending
Review via email: mp+21408@code.launchpad.net

Description of the change

Hi there,

This small branch changes the code import system to exponentially back off on retrying code imports -- see the linked bug for more.

Currently it backs off so that it tries imports at 6, 12, 24 and 48 hours (for subversion) for a total of three and a bit days of trying before marking an import failed -- is that enough? It seems OK, though perhaps a bit on the low side, to me.

Cheers,
mwh

To post a comment you must log in.
Revision history for this message
Jonathan Lange (jml) wrote :
Download full text (6.6 KiB)

On Mon, Mar 15, 2010 at 10:22 PM, Michael Hudson
<email address hidden> wrote:
> Michael Hudson has proposed merging lp:~mwhudson/launchpad/back-off-failing-imports-bug-413637 into lp:launchpad.
>
> Requested reviews:
>  Canonical Launchpad Engineering (launchpad)
> Related bugs:
>  #413637 imports disabled too rapidly
>  https://bugs.launchpad.net/bugs/413637
>
>
> Hi there,
>
> This small branch changes the code import system to exponentially back off on retrying code imports -- see the linked bug for more.
>

I think the branch is bigger than you intended. It's got a bunch of
build farm job stuff in the diff.

> Currently it backs off so that it tries imports at 6, 12, 24 and 48 hours (for subversion) for a total of three and a bit days of trying before marking an import failed -- is that enough?  It seems OK, though perhaps a bit on the low side, to me.
>

Sounds fine to me.

...
> === modified file 'lib/lp/code/model/codeimportjob.py'
> --- lib/lp/code/model/codeimportjob.py  2010-02-24 10:18:16 +0000
> +++ lib/lp/code/model/codeimportjob.py  2010-03-15 22:22:20 +0000
> @@ -10,6 +10,8 @@
>     'CodeImportJobWorkflow',
>     ]
>
> +import datetime
> +
>  from sqlobject import ForeignKey, IntCol, SQLObjectNotFound, StringCol
>
>  from zope.component import getUtility
> @@ -140,7 +142,7 @@
>
>     implements(ICodeImportJobWorkflow)
>
> -    def newJob(self, code_import, date_due=None):
> +    def newJob(self, code_import, interval=None):
>         """See `ICodeImportJobWorkflow`."""
>         assert code_import.review_status == CodeImportReviewStatus.REVIEWED, (
>             "Review status of %s is not REVIEWED: %s" % (
> @@ -149,23 +151,22 @@
>             "Already associated to a CodeImportJob: %s" % (
>             code_import.branch.unique_name))
>
> +        if interval is None:
> +            interval = code_import.effective_update_interval
> +
>         job = CodeImportJob(code_import=code_import, date_due=UTC_NOW)
>
> -        if date_due is None:
> -            # Find the most recent CodeImportResult for this CodeImport. We
> -            # sort by date_created because we do not have an index on
> -            # date_job_started in the database, and that should give the same
> -            # sort order.
> -            most_recent_result_list = list(CodeImportResult.selectBy(
> -                code_import=code_import).orderBy(['-date_created']).limit(1))
> +        # Find the most recent CodeImportResult for this CodeImport. We
> +        # sort by date_created because we do not have an index on
> +        # date_job_started in the database, and that should give the same
> +        # sort order.
> +        most_recent_result_list = list(CodeImportResult.selectBy(
> +            code_import=code_import).orderBy(['-date_created']).limit(1))
>

Why aren't you using Storm, with its 'one()' notation for this?

> @@ -281,15 +282,19 @@
>         # If the import has failed too many times in a row, mark it as
>         # FAILING.
>         failure_limit = config.codeimport.consecutive_failure_limit
> -        if code_import.consecutive_failure_count >= failure_limit:
> +        failure_count = code_import.consecutive_failure_count...

Read more...

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :
Download full text (7.3 KiB)

Jonathan Lange wrote:
> On Mon, Mar 15, 2010 at 10:22 PM, Michael Hudson
> <email address hidden> wrote:
>> Michael Hudson has proposed merging lp:~mwhudson/launchpad/back-off-failing-imports-bug-413637 into lp:launchpad.
>>
>> Requested reviews:
>> Canonical Launchpad Engineering (launchpad)
>> Related bugs:
>> #413637 imports disabled too rapidly
>> https://bugs.launchpad.net/bugs/413637
>>
>>
>> Hi there,
>>
>> This small branch changes the code import system to exponentially back off on retrying code imports -- see the linked bug for more.
>>
>
> I think the branch is bigger than you intended. It's got a bunch of
> build farm job stuff in the diff.

Grar grar, it's proposed into db-devel of course.

I proposed again:
https://code.edge.launchpad.net/~mwhudson/launchpad/back-off-failing-imports-bug-413637/+merge/21411

>> Currently it backs off so that it tries imports at 6, 12, 24 and 48 hours (for subversion) for a total of three and a bit days of trying before marking an import failed -- is that enough? It seems OK, though perhaps a bit on the low side, to me.
>>
>
> Sounds fine to me.
>
> ...
>> === modified file 'lib/lp/code/model/codeimportjob.py'
>> --- lib/lp/code/model/codeimportjob.py 2010-02-24 10:18:16 +0000
>> +++ lib/lp/code/model/codeimportjob.py 2010-03-15 22:22:20 +0000
>> @@ -10,6 +10,8 @@
>> 'CodeImportJobWorkflow',
>> ]
>>
>> +import datetime
>> +
>> from sqlobject import ForeignKey, IntCol, SQLObjectNotFound, StringCol
>>
>> from zope.component import getUtility
>> @@ -140,7 +142,7 @@
>>
>> implements(ICodeImportJobWorkflow)
>>
>> - def newJob(self, code_import, date_due=None):
>> + def newJob(self, code_import, interval=None):
>> """See `ICodeImportJobWorkflow`."""
>> assert code_import.review_status == CodeImportReviewStatus.REVIEWED, (
>> "Review status of %s is not REVIEWED: %s" % (
>> @@ -149,23 +151,22 @@
>> "Already associated to a CodeImportJob: %s" % (
>> code_import.branch.unique_name))
>>
>> + if interval is None:
>> + interval = code_import.effective_update_interval
>> +
>> job = CodeImportJob(code_import=code_import, date_due=UTC_NOW)
>>
>> - if date_due is None:
>> - # Find the most recent CodeImportResult for this CodeImport. We
>> - # sort by date_created because we do not have an index on
>> - # date_job_started in the database, and that should give the same
>> - # sort order.
>> - most_recent_result_list = list(CodeImportResult.selectBy(
>> - code_import=code_import).orderBy(['-date_created']).limit(1))
>> + # Find the most recent CodeImportResult for this CodeImport. We
>> + # sort by date_created because we do not have an index on
>> + # date_job_started in the database, and that should give the same
>> + # sort order.
>> + most_recent_result_list = list(CodeImportResult.selectBy(
>> + code_import=code_import).orderBy(['-date_created']).limit(1))
>>
>
> Why aren't you using Storm, with its 'one()' notation for this?

Because all I did to this code was dedent it. I ...

Read more...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/canonical/buildd/debian/launchpad-buildd.init'
--- lib/canonical/buildd/debian/launchpad-buildd.init 2009-12-16 00:16:24 +0000
+++ lib/canonical/buildd/debian/launchpad-buildd.init 2010-03-15 22:22:20 +0000
@@ -29,7 +29,7 @@
29 CONF=$129 CONF=$1
30 PIDFILE="$PIDROOT"/"$CONF".pid30 PIDFILE="$PIDROOT"/"$CONF".pid
31 LOGFILE="$LOGROOT"/"$CONF".log31 LOGFILE="$LOGROOT"/"$CONF".log
32 su - buildd -c "BUILDD_SLAVE_CONFIG=$CONFROOT/$CONF PYTHONPATH=/usr/share/launchpad-buildd twistd --no_save --pidfile $PIDFILE --python $TACFILE --logfile $LOGFILE"32 su - buildd -c "BUILDD_SLAVE_CONFIG=$CONFROOT/$CONF PYTHONPATH=/usr/share/launchpad-buildd twistd --no_save --pidfile $PIDFILE --python $TACFILE --logfile $LOGFILE --umask 022"
33}33}
3434
35#35#
3636
=== modified file 'lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py'
--- lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2010-01-21 05:03:16 +0000
+++ lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2010-03-15 22:22:20 +0000
@@ -1,4 +1,4 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the1# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).2# GNU Affero General Public License version 3 (see the file LICENSE).
33
4# pylint: disable-msg=E0211,E02134# pylint: disable-msg=E0211,E0213
55
=== modified file 'lib/lp/buildmaster/model/buildfarmjobbehavior.py'
--- lib/lp/buildmaster/model/buildfarmjobbehavior.py 2010-03-05 13:52:32 +0000
+++ lib/lp/buildmaster/model/buildfarmjobbehavior.py 2010-03-15 22:22:20 +0000
@@ -1,4 +1,4 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the1# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).2# GNU Affero General Public License version 3 (see the file LICENSE).
33
4# pylint: disable-msg=E0211,E02134# pylint: disable-msg=E0211,E0213
@@ -17,17 +17,22 @@
17import xmlrpclib17import xmlrpclib
1818
19from sqlobject import SQLObjectNotFound19from sqlobject import SQLObjectNotFound
20
20from zope.component import getUtility21from zope.component import getUtility
21from zope.interface import implements22from zope.interface import implements
22from zope.security.proxy import removeSecurityProxy23from zope.security.proxy import removeSecurityProxy, isinstance as zisinstance
2324
24from canonical import encoding25from canonical import encoding
25from canonical.librarian.interfaces import ILibrarianClient26from canonical.librarian.interfaces import ILibrarianClient
27
28from canonical.launchpad.webapp.interfaces import NotFoundError
26from lp.buildmaster.interfaces.builder import CorruptBuildID29from lp.buildmaster.interfaces.builder import CorruptBuildID
27from lp.buildmaster.interfaces.buildfarmjobbehavior import (30from lp.buildmaster.interfaces.buildfarmjobbehavior import (
28 BuildBehaviorMismatch, IBuildFarmJobBehavior)31 BuildBehaviorMismatch, IBuildFarmJobBehavior)
29from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet32from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
33from lp.buildmaster.model.buildqueue import BuildQueue
30from lp.services.job.interfaces.job import JobStatus34from lp.services.job.interfaces.job import JobStatus
35from lp.soyuz.interfaces.build import IBuildSet
3136
3237
33class BuildFarmJobBehaviorBase:38class BuildFarmJobBehaviorBase:
@@ -59,22 +64,101 @@
59 The default behavior is that we don't add any extra values."""64 The default behavior is that we don't add any extra values."""
60 return {}65 return {}
6166
67 def _helpVerifyBuildIDComponent(self, raw_id, item_type, finder):
68 """Helper for verifying parts of a `BuildFarmJob` name.
69
70 Different `IBuildFarmJob` implementations can have different
71 ways of constructing their identifying names. The names are
72 produced by `IBuildFarmJob.getName` and verified by
73 `IBuildFarmJobBehavior.verifySlaveBuildID`.
74
75 This little helper makes it easier to verify an object id
76 embedded in that name, check that it's a valid number, and
77 retrieve the associated database object.
78
79 :param raw_id: An unverified id string as extracted from the
80 build name. The method will verify that it is a number, and
81 try to retrieve the associated object.
82 :param item_type: The type of object this id represents. Should
83 be a class.
84 :param finder: A function that, given an integral id, finds the
85 associated object of type `item_type`.
86 :raise CorruptBuildID: If `raw_id` is malformed in some way or
87 the associated `item_type` object is not found.
88 :return: An object that is an instance of `item_type`.
89 """
90 type_name = item_type.__name__
91 try:
92 numeric_id = int(raw_id)
93 except ValueError:
94 raise CorruptBuildID(
95 "%s ID is not a number: '%s'" % (type_name, raw_id))
96
97 try:
98 item = finder(numeric_id)
99 except (NotFoundError, SQLObjectNotFound), reason:
100 raise CorruptBuildID(
101 "%s %d is not available: %s" % (
102 type_name, numeric_id, reason))
103 except Exception, reason:
104 raise CorruptBuildID(
105 "Error while looking up %s %d: %s" % (
106 type_name, numeric_id, reason))
107
108 if item is None:
109 raise CorruptBuildID("There is no %s with id %d." % (
110 type_name, numeric_id))
111
112 assert zisinstance(item, item_type), (
113 "Looked for %s, but found %s." % (type_name, repr(item)))
114
115 return item
116
117 def getVerifiedBuild(self, raw_id):
118 """Verify and retrieve the `Build` component of a slave build id.
119
120 This does part of the verification for `verifySlaveBuildID`.
121
122 By default, a `BuildFarmJob` has an identifying name of the form
123 "b-q", where b is the id of its `Build` and q is the id of its
124 `BuildQueue` record.
125
126 Use `getVerifiedBuild` to verify the "b" part, and retrieve the
127 associated `Build`.
128 """
129 # Avoid circular import.
130 from lp.soyuz.model.build import Build
131
132 return self._helpVerifyBuildIDComponent(
133 raw_id, Build, getUtility(IBuildSet).getByBuildID)
134
135 def getVerifiedBuildQueue(self, raw_id):
136 """Verify and retrieve the `BuildQueue` component of a slave build id.
137
138 This does part of the verification for `verifySlaveBuildID`.
139
140 By default, a `BuildFarmJob` has an identifying name of the form
141 "b-q", where b is the id of its `Build` and q is the id of its
142 `BuildQueue` record.
143
144 Use `getVerifiedBuildQueue` to verify the "q" part, and retrieve
145 the associated `BuildQueue` object.
146 """
147 return self._helpVerifyBuildIDComponent(
148 raw_id, BuildQueue, getUtility(IBuildQueueSet).get)
149
62 def verifySlaveBuildID(self, slave_build_id):150 def verifySlaveBuildID(self, slave_build_id):
63 """See `IBuildFarmJobBehavior`."""151 """See `IBuildFarmJobBehavior`."""
64 # Extract information from the identifier.152 # Extract information from the identifier.
65 try:153 try:
66 build_id, queue_item_id = slave_build_id.split('-')154 build_id, queue_item_id = slave_build_id.split('-')
67 build_id = int(build_id)
68 queue_item_id = int(queue_item_id)
69 except ValueError:155 except ValueError:
70 raise CorruptBuildID('Malformed build ID')156 raise CorruptBuildID('Malformed build ID')
157
158 build = self.getVerifiedBuild(build_id)
159 queue_item = self.getVerifiedBuildQueue(queue_item_id)
71160
72 try:161 if build != queue_item.specific_job.build:
73 queue_item = getUtility(IBuildQueueSet).get(queue_item_id)
74 # Check whether build and buildqueue are properly related.
75 except SQLObjectNotFound, reason:
76 raise CorruptBuildID(str(reason))
77 if queue_item.specific_job.build.id != build_id:
78 raise CorruptBuildID('Job build entry mismatch')162 raise CorruptBuildID('Job build entry mismatch')
79163
80 def updateBuild(self, queueItem):164 def updateBuild(self, queueItem):
81165
=== modified file 'lib/lp/buildmaster/tests/test_buildfarmjobbehavior.py'
--- lib/lp/buildmaster/tests/test_buildfarmjobbehavior.py 2010-03-06 04:57:40 +0000
+++ lib/lp/buildmaster/tests/test_buildfarmjobbehavior.py 2010-03-15 22:22:20 +0000
@@ -3,10 +3,18 @@
33
4"""Unit tests for BuildFarmJobBehaviorBase."""4"""Unit tests for BuildFarmJobBehaviorBase."""
55
6from unittest import TestCase, TestLoader6from unittest import TestLoader
7
8from zope.component import getUtility
9
10from canonical.testing.layers import ZopelessDatabaseLayer
11from lp.testing import TestCaseWithFactory
712
8from lp.buildmaster.interfaces.buildbase import BuildStatus13from lp.buildmaster.interfaces.buildbase import BuildStatus
14from lp.buildmaster.interfaces.builder import CorruptBuildID
9from lp.buildmaster.model.buildfarmjobbehavior import BuildFarmJobBehaviorBase15from lp.buildmaster.model.buildfarmjobbehavior import BuildFarmJobBehaviorBase
16from lp.registry.interfaces.pocket import PackagePublishingPocket
17from lp.soyuz.interfaces.processor import IProcessorFamilySet
1018
1119
12class FakeBuildFarmJob:20class FakeBuildFarmJob:
@@ -14,27 +22,91 @@
14 pass22 pass
1523
1624
17class TestBuildFarmJobBehaviorBase(TestCase):25class TestBuildFarmJobBehaviorBase(TestCaseWithFactory):
18 """Test very small, basic bits of BuildFarmJobBehaviorBase."""26 """Test very small, basic bits of BuildFarmJobBehaviorBase."""
1927
20 def setUp(self):28 layer = ZopelessDatabaseLayer
21 self.behavior = BuildFarmJobBehaviorBase(FakeBuildFarmJob())29
30 def _makeBehavior(self, buildfarmjob=None):
31 """Create a `BuildFarmJobBehaviorBase`."""
32 if buildfarmjob is None:
33 buildfarmjob = FakeBuildFarmJob()
34 return BuildFarmJobBehaviorBase(buildfarmjob)
35
36 def _makeBuild(self):
37 """Create a `Build` object."""
38 x86 = getUtility(IProcessorFamilySet).getByName('x86')
39 distroarchseries = self.factory.makeDistroArchSeries(
40 architecturetag='x86', processorfamily=x86)
41 distroseries = distroarchseries.distroseries
42 archive = self.factory.makeArchive(
43 distribution=distroseries.distribution)
44 pocket = PackagePublishingPocket.RELEASE
45 spr = self.factory.makeSourcePackageRelease(
46 distroseries=distroseries, archive=archive)
47
48 return spr.createBuild(
49 distroarchseries=distroarchseries, pocket=pocket, archive=archive)
50
51 def _makeBuildQueue(self):
52 """Create a `BuildQueue` object."""
53 return self.factory.makeSourcePackageRecipeBuildJob()
2254
23 def test_extractBuildStatus_baseline(self):55 def test_extractBuildStatus_baseline(self):
24 # extractBuildStatus picks the name of the build status out of a56 # extractBuildStatus picks the name of the build status out of a
25 # dict describing the slave's status.57 # dict describing the slave's status.
26 slave_status = {'build_status': 'BuildStatus.BUILDING'}58 slave_status = {'build_status': 'BuildStatus.BUILDING'}
59 behavior = self._makeBehavior()
27 self.assertEqual(60 self.assertEqual(
28 BuildStatus.BUILDING.name,61 BuildStatus.BUILDING.name,
29 self.behavior.extractBuildStatus(slave_status))62 behavior.extractBuildStatus(slave_status))
3063
31 def test_extractBuildStatus_malformed(self):64 def test_extractBuildStatus_malformed(self):
32 # extractBuildStatus errors out when the status string is not65 # extractBuildStatus errors out when the status string is not
33 # of the form it expects.66 # of the form it expects.
34 slave_status = {'build_status': 'BUILDING'}67 slave_status = {'build_status': 'BUILDING'}
35 self.assertRaises(68 behavior = self._makeBehavior()
36 AssertionError,69 self.assertRaises(
37 self.behavior.extractBuildStatus, slave_status)70 AssertionError, behavior.extractBuildStatus, slave_status)
71
72 def test_getVerifiedBuild_success(self):
73 build = self._makeBuild()
74 behavior = self._makeBehavior()
75 raw_id = str(build.id)
76
77 self.assertEqual(build, behavior.getVerifiedBuild(raw_id))
78
79 def test_getVerifiedBuild_malformed(self):
80 behavior = self._makeBehavior()
81 self.assertRaises(CorruptBuildID, behavior.getVerifiedBuild, 'hi!')
82
83 def test_getVerifiedBuild_notfound(self):
84 build = self._makeBuild()
85 behavior = self._makeBehavior()
86 nonexistent_id = str(build.id + 99)
87
88 self.assertRaises(
89 CorruptBuildID, behavior.getVerifiedBuild, nonexistent_id)
90
91 def test_getVerifiedBuildQueue_success(self):
92 buildqueue = self._makeBuildQueue()
93 behavior = self._makeBehavior()
94 raw_id = str(buildqueue.id)
95
96 self.assertEqual(buildqueue, behavior.getVerifiedBuildQueue(raw_id))
97
98 def test_getVerifiedBuildQueue_malformed(self):
99 behavior = self._makeBehavior()
100 self.assertRaises(
101 CorruptBuildID, behavior.getVerifiedBuildQueue, 'bye!')
102
103 def test_getVerifiedBuildQueue_notfound(self):
104 buildqueue = self._makeBuildQueue()
105 behavior = self._makeBehavior()
106 nonexistent_id = str(buildqueue.id + 99)
107
108 self.assertRaises(
109 CorruptBuildID, behavior.getVerifiedBuildQueue, nonexistent_id)
38110
39111
40def test_suite():112def test_suite():
41113
=== modified file 'lib/lp/code/interfaces/codeimportjob.py'
--- lib/lp/code/interfaces/codeimportjob.py 2010-02-22 08:10:03 +0000
+++ lib/lp/code/interfaces/codeimportjob.py 2010-03-15 22:22:20 +0000
@@ -16,6 +16,8 @@
16 'ICodeImportJobWorkflow',16 'ICodeImportJobWorkflow',
17 ]17 ]
1818
19import datetime
20
19from zope.interface import Interface21from zope.interface import Interface
20from zope.schema import Choice, Datetime, Int, Object, Text22from zope.schema import Choice, Datetime, Int, Object, Text
2123
@@ -137,7 +139,7 @@
137class ICodeImportJobWorkflow(Interface):139class ICodeImportJobWorkflow(Interface):
138 """Utility to manage `CodeImportJob` objects through their life cycle."""140 """Utility to manage `CodeImportJob` objects through their life cycle."""
139141
140 def newJob(code_import):142 def newJob(code_import, interval=datetime.timedelta(0)):
141 """Create a `CodeImportJob` associated with a reviewed `CodeImport`.143 """Create a `CodeImportJob` associated with a reviewed `CodeImport`.
142144
143 Call this method from `CodeImport.updateFromData` when the145 Call this method from `CodeImport.updateFromData` when the
@@ -218,9 +220,11 @@
218 display for diagnostics. May be None.220 display for diagnostics. May be None.
219 :precondition: `import_job`.state == RUNNING.221 :precondition: `import_job`.state == RUNNING.
220 :postcondition: `import_job` is deleted.222 :postcondition: `import_job` is deleted.
221 :postcondition: `code_import.import_job` is not None.223 :postcondition: `code_import.import_job` is not None unless the job
224 has failed more than consecutive_failure_limit times in a row.
222 :postcondition: `code_import.import_job.date_due` is225 :postcondition: `code_import.import_job.date_due` is
223 import_job.date_due + code_import.effective_update_interval`.226 import_job.date_due + code_import.effective_update_interval`, with
227 scaling to retry failing imports less often.
224 :postcondition: A `CodeImportResult` was created.228 :postcondition: A `CodeImportResult` was created.
225 :postcondition: A FINISH `CodeImportEvent` was created.229 :postcondition: A FINISH `CodeImportEvent` was created.
226 """230 """
227231
=== modified file 'lib/lp/code/model/codeimportjob.py'
--- lib/lp/code/model/codeimportjob.py 2010-02-24 10:18:16 +0000
+++ lib/lp/code/model/codeimportjob.py 2010-03-15 22:22:20 +0000
@@ -10,6 +10,8 @@
10 'CodeImportJobWorkflow',10 'CodeImportJobWorkflow',
11 ]11 ]
1212
13import datetime
14
13from sqlobject import ForeignKey, IntCol, SQLObjectNotFound, StringCol15from sqlobject import ForeignKey, IntCol, SQLObjectNotFound, StringCol
1416
15from zope.component import getUtility17from zope.component import getUtility
@@ -140,7 +142,7 @@
140142
141 implements(ICodeImportJobWorkflow)143 implements(ICodeImportJobWorkflow)
142144
143 def newJob(self, code_import, date_due=None):145 def newJob(self, code_import, interval=None):
144 """See `ICodeImportJobWorkflow`."""146 """See `ICodeImportJobWorkflow`."""
145 assert code_import.review_status == CodeImportReviewStatus.REVIEWED, (147 assert code_import.review_status == CodeImportReviewStatus.REVIEWED, (
146 "Review status of %s is not REVIEWED: %s" % (148 "Review status of %s is not REVIEWED: %s" % (
@@ -149,23 +151,22 @@
149 "Already associated to a CodeImportJob: %s" % (151 "Already associated to a CodeImportJob: %s" % (
150 code_import.branch.unique_name))152 code_import.branch.unique_name))
151153
154 if interval is None:
155 interval = code_import.effective_update_interval
156
152 job = CodeImportJob(code_import=code_import, date_due=UTC_NOW)157 job = CodeImportJob(code_import=code_import, date_due=UTC_NOW)
153158
154 if date_due is None:159 # Find the most recent CodeImportResult for this CodeImport. We
155 # Find the most recent CodeImportResult for this CodeImport. We160 # sort by date_created because we do not have an index on
156 # sort by date_created because we do not have an index on161 # date_job_started in the database, and that should give the same
157 # date_job_started in the database, and that should give the same162 # sort order.
158 # sort order.163 most_recent_result_list = list(CodeImportResult.selectBy(
159 most_recent_result_list = list(CodeImportResult.selectBy(164 code_import=code_import).orderBy(['-date_created']).limit(1))
160 code_import=code_import).orderBy(['-date_created']).limit(1))
161165
162 if len(most_recent_result_list) != 0:166 if len(most_recent_result_list) != 0:
163 [most_recent_result] = most_recent_result_list167 [most_recent_result] = most_recent_result_list
164 interval = code_import.effective_update_interval168 date_due = most_recent_result.date_job_started + interval
165 date_due = most_recent_result.date_job_started + interval169 job.date_due = max(job.date_due, date_due)
166 job.date_due = max(job.date_due, date_due)
167 else:
168 job.date_due = date_due
169170
170 return job171 return job
171172
@@ -281,15 +282,19 @@
281 # If the import has failed too many times in a row, mark it as282 # If the import has failed too many times in a row, mark it as
282 # FAILING.283 # FAILING.
283 failure_limit = config.codeimport.consecutive_failure_limit284 failure_limit = config.codeimport.consecutive_failure_limit
284 if code_import.consecutive_failure_count >= failure_limit:285 failure_count = code_import.consecutive_failure_count
286 if failure_count >= failure_limit:
285 code_import.updateFromData(287 code_import.updateFromData(
286 dict(review_status=CodeImportReviewStatus.FAILING), None)288 dict(review_status=CodeImportReviewStatus.FAILING), None)
289 elif status == CodeImportResultStatus.SUCCESS_PARTIAL:
290 interval = datetime.timedelta(0)
291 elif failure_count > 0:
292 interval = code_import.effective_update_interval * (2**(failure_count - 1))
293 else:
294 interval = code_import.effective_update_interval
287 # Only start a new one if the import is still in the REVIEWED state.295 # Only start a new one if the import is still in the REVIEWED state.
288 if code_import.review_status == CodeImportReviewStatus.REVIEWED:296 if code_import.review_status == CodeImportReviewStatus.REVIEWED:
289 extra = {}297 self.newJob(code_import, interval=interval)
290 if status == CodeImportResultStatus.SUCCESS_PARTIAL:
291 extra['date_due'] = UTC_NOW
292 self.newJob(code_import, **extra)
293 # If the status was successful, update date_last_successful.298 # If the status was successful, update date_last_successful.
294 if status in [CodeImportResultStatus.SUCCESS,299 if status in [CodeImportResultStatus.SUCCESS,
295 CodeImportResultStatus.SUCCESS_NOCHANGE]:300 CodeImportResultStatus.SUCCESS_NOCHANGE]:
@@ -321,7 +326,7 @@
321 import_job, CodeImportResultStatus.RECLAIMED, None)326 import_job, CodeImportResultStatus.RECLAIMED, None)
322 # 3)327 # 3)
323 if code_import.review_status == CodeImportReviewStatus.REVIEWED:328 if code_import.review_status == CodeImportReviewStatus.REVIEWED:
324 self.newJob(code_import, UTC_NOW)329 self.newJob(code_import, datetime.timedelta(0))
325 # 4)330 # 4)
326 getUtility(ICodeImportEventSet).newReclaim(331 getUtility(ICodeImportEventSet).newReclaim(
327 code_import, machine, job_id)332 code_import, machine, job_id)
328333
=== modified file 'lib/lp/code/model/recipebuilder.py'
--- lib/lp/code/model/recipebuilder.py 2010-02-05 15:06:28 +0000
+++ lib/lp/code/model/recipebuilder.py 2010-03-15 22:22:20 +0000
@@ -8,7 +8,7 @@
8 'RecipeBuildBehavior',8 'RecipeBuildBehavior',
9 ]9 ]
1010
11from zope.component import adapts11from zope.component import adapts, getUtility
12from zope.interface import implements12from zope.interface import implements
1313
14from lp.archiveuploader.permission import check_upload_to_pocket14from lp.archiveuploader.permission import check_upload_to_pocket
@@ -18,7 +18,8 @@
18from lp.buildmaster.model.buildfarmjobbehavior import (18from lp.buildmaster.model.buildfarmjobbehavior import (
19 BuildFarmJobBehaviorBase)19 BuildFarmJobBehaviorBase)
20from lp.code.interfaces.sourcepackagerecipebuild import (20from lp.code.interfaces.sourcepackagerecipebuild import (
21 ISourcePackageRecipeBuildJob)21 ISourcePackageRecipeBuildJob, ISourcePackageRecipeBuildSource)
22from lp.code.model.sourcepackagerecipebuild import SourcePackageRecipeBuild
22from lp.registry.interfaces.pocket import PackagePublishingPocket23from lp.registry.interfaces.pocket import PackagePublishingPocket
23from lp.soyuz.adapters.archivedependencies import (24from lp.soyuz.adapters.archivedependencies import (
24 get_primary_current_component, get_sources_list_for_building)25 get_primary_current_component, get_sources_list_for_building)
@@ -168,3 +169,11 @@
168 extra_info['logtail'] = raw_slave_status[2]169 extra_info['logtail'] = raw_slave_status[2]
169170
170 return extra_info171 return extra_info
172
173 def getVerifiedBuild(self, raw_id):
174 """See `IBuildFarmJobBehavior`."""
175 # This type of job has a build that is of type BuildBase but not
176 # actually a Build.
177 return self._helpVerifyBuildIDComponent(
178 raw_id, SourcePackageRecipeBuild,
179 getUtility(ISourcePackageRecipeBuildSource).getById)
171180
=== modified file 'lib/lp/code/model/tests/test_codeimportjob.py'
--- lib/lp/code/model/tests/test_codeimportjob.py 2010-03-14 20:15:34 +0000
+++ lib/lp/code/model/tests/test_codeimportjob.py 2010-03-15 22:22:20 +0000
@@ -778,6 +778,31 @@
778 self.assertEqual(new_job.machine, None)778 self.assertEqual(new_job.machine, None)
779 self.assertSqlAttributeEqualsDate(new_job, 'date_due', UTC_NOW)779 self.assertSqlAttributeEqualsDate(new_job, 'date_due', UTC_NOW)
780780
781 def test_failures_back_off(self):
782 # We wait for longer and longer between retrying failing imports, to
783 # make it less likely that an import is marked failing just because
784 # someone's DNS went down for a day.
785 running_job = self.makeRunningJob()
786 intervals = []
787 interval = running_job.code_import.effective_update_interval
788 expected_intervals = []
789 for i in range(config.codeimport.consecutive_failure_limit - 1):
790 expected_intervals.append(interval)
791 interval *= 2
792 # Fail an import a bunch of times and record how far in the future the
793 # next job was scheduled.
794 for i in range(config.codeimport.consecutive_failure_limit - 1):
795 code_import = running_job.code_import
796 getUtility(ICodeImportJobWorkflow).finishJob(
797 running_job, CodeImportResultStatus.FAILURE, None)
798 intervals.append(
799 code_import.import_job.date_due -
800 code_import.results[-1].date_job_started)
801 running_job = code_import.import_job
802 getUtility(ICodeImportJobWorkflow).startJob(
803 running_job, self.machine)
804 self.assertEqual(expected_intervals, intervals)
805
781 def test_doesntCreateNewJobIfCodeImportNotReviewed(self):806 def test_doesntCreateNewJobIfCodeImportNotReviewed(self):
782 # finishJob() creates a new CodeImportJob for the given CodeImport,807 # finishJob() creates a new CodeImportJob for the given CodeImport,
783 # unless the CodeImport has been suspended or marked invalid.808 # unless the CodeImport has been suspended or marked invalid.
784809
=== modified file 'lib/lp/soyuz/doc/buildd-slavescanner.txt'
--- lib/lp/soyuz/doc/buildd-slavescanner.txt 2010-03-10 12:50:18 +0000
+++ lib/lp/soyuz/doc/buildd-slavescanner.txt 2010-03-15 22:22:20 +0000
@@ -95,15 +95,16 @@
95 >>> lostbuilding_builder = MockBuilder(95 >>> lostbuilding_builder = MockBuilder(
96 ... 'Lost Building Slave', LostBuildingSlave())96 ... 'Lost Building Slave', LostBuildingSlave())
97 >>> buildergroup.rescueBuilderIfLost(lostbuilding_builder)97 >>> buildergroup.rescueBuilderIfLost(lostbuilding_builder)
98 WARNING:root:Builder 'Lost Building Slave' rescued from98 WARNING:root:Builder 'Lost Building Slave' rescued from '1000-10000':
99 '1000-10000': 'Object not found'99 'Build 1000 is not available: 'Object not found''
100100
101Then a lost slave in status 'WAITING':101Then a lost slave in status 'WAITING':
102102
103 >>> lostwaiting_builder = MockBuilder(103 >>> lostwaiting_builder = MockBuilder(
104 ... 'Lost Waiting Slave', LostWaitingSlave())104 ... 'Lost Waiting Slave', LostWaitingSlave())
105 >>> buildergroup.rescueBuilderIfLost(lostwaiting_builder)105 >>> buildergroup.rescueBuilderIfLost(lostwaiting_builder)
106 WARNING:root:Builder 'Lost Waiting Slave' rescued from '1000-10000': 'Object not found'106 WARNING:root:Builder 'Lost Waiting Slave' rescued from '1000-10000':
107 'Build 1000 is not available: 'Object not found''
107108
108Both got rescued, as expected.109Both got rescued, as expected.
109110
110111
=== modified file 'lib/lp/testing/factory.py'
--- lib/lp/testing/factory.py 2010-03-11 20:54:36 +0000
+++ lib/lp/testing/factory.py 2010-03-15 22:22:20 +0000
@@ -2008,6 +2008,81 @@
2008 def getAnySourcePackageUrgency(self):2008 def getAnySourcePackageUrgency(self):
2009 return SourcePackageUrgency.MEDIUM2009 return SourcePackageUrgency.MEDIUM
20102010
2011 def makeSourcePackageRelease(self, archive=None, sourcepackagename=None,
2012 distroseries=None, maintainer=None,
2013 creator=None, component=None, section=None,
2014 urgency=None, version=None,
2015 builddepends=None, builddependsindep=None,
2016 build_conflicts=None,
2017 build_conflicts_indep=None,
2018 architecturehintlist='all',
2019 dsc_maintainer_rfc822=None,
2020 dsc_standards_version='3.6.2',
2021 dsc_format='1.0', dsc_binaries='foo-bin',
2022 date_uploaded=UTC_NOW):
2023 """Make a `SourcePackageRelease`."""
2024 if distroseries is None:
2025 if archive is None:
2026 distribution = None
2027 else:
2028 distribution = archive.distribution
2029 distroseries = self.makeDistroRelease(distribution=distribution)
2030
2031 if archive is None:
2032 archive = self.makeArchive(
2033 distribution=distroseries.distribution,
2034 purpose=ArchivePurpose.PRIMARY)
2035
2036 if sourcepackagename is None:
2037 sourcepackagename = self.makeSourcePackageName()
2038
2039 if component is None:
2040 component = self.makeComponent()
2041
2042 if urgency is None:
2043 urgency = self.getAnySourcePackageUrgency()
2044
2045 if section is None:
2046 section = self.getUniqueString('section')
2047 section = getUtility(ISectionSet).ensure(section)
2048
2049 if maintainer is None:
2050 maintainer = self.makePerson()
2051
2052 maintainer_email = '%s <%s>' % (
2053 maintainer.displayname,
2054 maintainer.preferredemail.email)
2055
2056 if creator is None:
2057 creator = self.makePerson()
2058
2059 if version is None:
2060 version = self.getUniqueString('version')
2061
2062 return distroseries.createUploadedSourcePackageRelease(
2063 sourcepackagename=sourcepackagename,
2064 maintainer=maintainer,
2065 creator=creator,
2066 component=component,
2067 section=section,
2068 urgency=urgency,
2069 version=version,
2070 builddepends=builddepends,
2071 builddependsindep=builddependsindep,
2072 build_conflicts=build_conflicts,
2073 build_conflicts_indep=build_conflicts_indep,
2074 architecturehintlist=architecturehintlist,
2075 changelog_entry=None,
2076 dsc=None,
2077 copyright=self.getUniqueString(),
2078 dscsigningkey=None,
2079 dsc_maintainer_rfc822=maintainer_email,
2080 dsc_standards_version=dsc_standards_version,
2081 dsc_format=dsc_format,
2082 dsc_binaries=dsc_binaries,
2083 archive=archive,
2084 dateuploaded=date_uploaded)
2085
2011 def makeSourcePackagePublishingHistory(self, sourcepackagename=None,2086 def makeSourcePackagePublishingHistory(self, sourcepackagename=None,
2012 distroseries=None, maintainer=None,2087 distroseries=None, maintainer=None,
2013 creator=None, component=None,2088 creator=None, component=None,
@@ -2027,54 +2102,31 @@
2027 dsc_format='1.0',2102 dsc_format='1.0',
2028 dsc_binaries='foo-bin',2103 dsc_binaries='foo-bin',
2029 ):2104 ):
2030 if sourcepackagename is None:2105 """Make a `SourcePackagePublishingHistory`."""
2031 sourcepackagename = self.makeSourcePackageName()
2032 spn = sourcepackagename
2033
2034 if distroseries is None:2106 if distroseries is None:
2035 distroseries = self.makeDistroRelease()2107 if archive is None:
2108 distribution = None
2109 else:
2110 distribution = archive.distribution
2111 distroseries = self.makeDistroRelease(distribution=distribution)
20362112
2037 if archive is None:2113 if archive is None:
2038 archive = self.makeArchive(2114 archive = self.makeArchive(
2039 distribution=distroseries.distribution,2115 distribution=distroseries.distribution,
2040 purpose=ArchivePurpose.PRIMARY)2116 purpose=ArchivePurpose.PRIMARY)
20412117
2042 if component is None:
2043 component = self.makeComponent()
2044
2045 if pocket is None:2118 if pocket is None:
2046 pocket = self.getAnyPocket()2119 pocket = self.getAnyPocket()
20472120
2048 if status is None:2121 if status is None:
2049 status = PackagePublishingStatus.PENDING2122 status = PackagePublishingStatus.PENDING
20502123
2051 if urgency is None:2124 spr = self.makeSourcePackageRelease(
2052 urgency = self.getAnySourcePackageUrgency()2125 archive=archive,
20532126 sourcepackagename=sourcepackagename,
2054 if section is None:2127 distroseries=distroseries,
2055 section = self.getUniqueString('section')
2056 section = getUtility(ISectionSet).ensure(section)
2057
2058 if maintainer is None:
2059 maintainer = self.makePerson()
2060
2061 maintainer_email = '%s <%s>' % (
2062 maintainer.displayname,
2063 maintainer.preferredemail.email)
2064
2065 if creator is None:
2066 creator = self.makePerson()
2067
2068 if version is None:
2069 version = self.getUniqueString('version')
2070
2071 gpg_key = self.makeGPGKey(creator)
2072
2073 spr = distroseries.createUploadedSourcePackageRelease(
2074 sourcepackagename=spn,
2075 maintainer=maintainer,2128 maintainer=maintainer,
2076 creator=creator,2129 creator=creator, component=component,
2077 component=component,
2078 section=section,2130 section=section,
2079 urgency=urgency,2131 urgency=urgency,
2080 version=version,2132 version=version,
@@ -2083,15 +2135,10 @@
2083 build_conflicts=build_conflicts,2135 build_conflicts=build_conflicts,
2084 build_conflicts_indep=build_conflicts_indep,2136 build_conflicts_indep=build_conflicts_indep,
2085 architecturehintlist=architecturehintlist,2137 architecturehintlist=architecturehintlist,
2086 changelog_entry=None,
2087 dsc=None,
2088 copyright=self.getUniqueString(),
2089 dscsigningkey=gpg_key,
2090 dsc_maintainer_rfc822=maintainer_email,
2091 dsc_standards_version=dsc_standards_version,2138 dsc_standards_version=dsc_standards_version,
2092 dsc_format=dsc_format,2139 dsc_format=dsc_format,
2093 dsc_binaries=dsc_binaries,2140 dsc_binaries=dsc_binaries,
2094 archive=archive, dateuploaded=date_uploaded)2141 date_uploaded=date_uploaded)
20952142
2096 sspph = SourcePackagePublishingHistory(2143 sspph = SourcePackagePublishingHistory(
2097 distroseries=distroseries,2144 distroseries=distroseries,
20982145
=== modified file 'lib/lp/translations/doc/translationtemplatesbuildbehavior.txt'
--- lib/lp/translations/doc/translationtemplatesbuildbehavior.txt 2010-03-05 13:52:32 +0000
+++ lib/lp/translations/doc/translationtemplatesbuildbehavior.txt 2010-03-15 22:22:20 +0000
@@ -2,13 +2,21 @@
22
3== Setup ==3== Setup ==
44
5Set up build environment.5Set up build environment. Clear out the build queue.
66
7 >>> import transaction7 >>> import transaction
8 >>> import logging8 >>> import logging
9 >>> logger = logging.getLogger()9 >>> logger = logging.getLogger()
10 >>> logger.setLevel(logging.CRITICAL)10 >>> logger.setLevel(logging.CRITICAL)
1111
12 >>> from canonical.database.sqlbase import quote
13 >>> from canonical.launchpad.interfaces.lpstorm import IMasterStore
14 >>> from lp.services.job.interfaces.job import JobStatus
15 >>> from lp.services.job.model.job import Job
16 >>> store = IMasterStore(Job)
17 >>> query = store.execute("UPDATE Job SET status = %s" % quote(
18 ... JobStatus.FAILED))
19
12 >>> from lp.buildmaster.master import BuilddMaster20 >>> from lp.buildmaster.master import BuilddMaster
13 >>> from canonical.buildd.tests import BuilddSlaveTestSetup21 >>> from canonical.buildd.tests import BuilddSlaveTestSetup
14 >>> bm = BuilddMaster(logger, transaction)22 >>> bm = BuilddMaster(logger, transaction)
@@ -39,7 +47,21 @@
3947
40Make a builder to process our build request.48Make a builder to process our build request.
4149
42 >>> builder = factory.makeBuilder(virtualized=False, processor=processor)50 >>> builder = factory.makeBuilder(
51 ... virtualized=True, processor=processor, vm_host='hostname')
52
53The builder doesn't talk to a real slave. We don't have those in our
54test suite. But we give it a fake one.
55
56 >>> from lp.testing.fakemethod import FakeMethod
57 >>> class FakeSlave:
58 ... build = FakeMethod()
59 ... cacheFile = FakeMethod()
60 ... resume = FakeMethod(result=('Output here', 'Errors here', 0))
61
62 >>> from zope.security.proxy import removeSecurityProxy
63 >>> slave = FakeSlave()
64 >>> removeSecurityProxy(builder).slave = slave
4365
4466
45== Get a job! ==67== Get a job! ==
@@ -60,20 +82,27 @@
60 >>> print buildqueue.date_started82 >>> print buildqueue.date_started
61 None83 None
6284
63Dispatch the job to the build slave. The proper method to call here85Our job is now first in line to be executed.
64would have been Builder.findAndStartJob, but it has not been generalised86
65to handle new job types yet.87 >>> removeSecurityProxy(builder)._findBuildCandidate() == buildqueue
6688 True
67# XXX JeroenVermeulen bug=506617: call findAndStartJob instead.89
6890Dispatch the first job in line (ours!) to the build slave.
69 >>> from zope.security.proxy import removeSecurityProxy91
70 >>> removeSecurityProxy(builder)._dispatchBuildCandidate(buildqueue)92 >>> activated_job = builder.findAndStartJob()
7193 >>> activated_job == buildqueue
72The build is now marked as started.94 True
95
96Our build is now marked as started.
7397
74 >>> buildqueue.date_started is None98 >>> buildqueue.date_started is None
75 False99 False
76100
101The slave's build method has been called.
102
103 >>> slave.build.call_count
104 1
105
77106
78== Teardown ==107== Teardown ==
79108
80109
=== modified file 'lib/lp/translations/model/translationtemplatesbuildbehavior.py'
--- lib/lp/translations/model/translationtemplatesbuildbehavior.py 2010-02-11 19:11:11 +0000
+++ lib/lp/translations/model/translationtemplatesbuildbehavior.py 2010-03-15 22:22:20 +0000
@@ -17,6 +17,7 @@
1717
18from canonical.launchpad.interfaces import ILaunchpadCelebrities18from canonical.launchpad.interfaces import ILaunchpadCelebrities
1919
20from lp.buildmaster.interfaces.builder import CorruptBuildID
20from lp.buildmaster.interfaces.buildfarmjobbehavior import (21from lp.buildmaster.interfaces.buildfarmjobbehavior import (
21 IBuildFarmJobBehavior)22 IBuildFarmJobBehavior)
22from lp.buildmaster.model.buildfarmjobbehavior import (23from lp.buildmaster.model.buildfarmjobbehavior import (
@@ -40,12 +41,27 @@
40 self._builder.slave.cacheFile(logger, chroot)41 self._builder.slave.cacheFile(logger, chroot)
41 buildid = self.buildfarmjob.getName()42 buildid = self.buildfarmjob.getName()
4243
43 args = { 'branch_url': self.buildfarmjob.branch.url }44 args = self.buildfarmjob.metadata
44 filemap = {}45 filemap = {}
4546
46 self._builder.slave.build(47 self._builder.slave.build(
47 buildid, self.build_type, chroot_sha1, filemap, args)48 buildid, self.build_type, chroot_sha1, filemap, args)
4849
50 def verifySlaveBuildID(self, slave_build_id):
51 """See `IBuildFarmJobBehavior`."""
52 try:
53 branch_name, queue_item_id = slave_build_id.rsplit('-', 1)
54 except ValueError:
55 raise CorruptBuildID(
56 "Malformed translation templates build id: '%s'" % (
57 slave_build_id))
58
59 buildqueue = self.getVerifiedBuildQueue(queue_item_id)
60 if buildqueue.job != self.buildfarmjob.job:
61 raise CorruptBuildID(
62 "ID mismatch for translation templates build '%s'" % (
63 slave_build_id))
64
49 def _getChroot(self):65 def _getChroot(self):
50 ubuntu = getUtility(ILaunchpadCelebrities).ubuntu66 ubuntu = getUtility(ILaunchpadCelebrities).ubuntu
51 return ubuntu.currentseries.nominatedarchindep.getChroot()67 return ubuntu.currentseries.nominatedarchindep.getChroot()
5268
=== modified file 'lib/lp/translations/tests/test_translationtemplatesbuildbehavior.py'
--- lib/lp/translations/tests/test_translationtemplatesbuildbehavior.py 2010-03-12 08:31:43 +0000
+++ lib/lp/translations/tests/test_translationtemplatesbuildbehavior.py 2010-03-15 22:22:20 +0000
@@ -19,6 +19,7 @@
19from lp.buildmaster.interfaces.buildbase import BuildStatus19from lp.buildmaster.interfaces.buildbase import BuildStatus
20from lp.buildmaster.interfaces.buildfarmjobbehavior import (20from lp.buildmaster.interfaces.buildfarmjobbehavior import (
21 IBuildFarmJobBehavior)21 IBuildFarmJobBehavior)
22from lp.buildmaster.interfaces.builder import CorruptBuildID
22from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet23from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
23from lp.testing import TestCaseWithFactory24from lp.testing import TestCaseWithFactory
24from lp.testing.fakemethod import FakeMethod25from lp.testing.fakemethod import FakeMethod
@@ -123,6 +124,10 @@
123 self.assertEqual(124 self.assertEqual(
124 'translation-templates', slave_status['test_build_type'])125 'translation-templates', slave_status['test_build_type'])
125 self.assertIn('branch_url', slave_status['test_build_args'])126 self.assertIn('branch_url', slave_status['test_build_args'])
127 # The slave receives the public http URL for the branch.
128 self.assertEqual(
129 behavior.buildfarmjob.branch.composePublicURL(),
130 slave_status['test_build_args']['branch_url'])
126131
127 def test_getChroot(self):132 def test_getChroot(self):
128 # _getChroot produces the current chroot for the current Ubuntu133 # _getChroot produces the current chroot for the current Ubuntu
@@ -170,6 +175,38 @@
170 self.assertEqual(1, builder.cleanSlave.call_count)175 self.assertEqual(1, builder.cleanSlave.call_count)
171 self.assertEqual(0, behavior._uploadTarball.call_count)176 self.assertEqual(0, behavior._uploadTarball.call_count)
172177
178 def test_verifySlaveBuildID_success(self):
179 # TranslationTemplatesBuildJob.getName generates slave build ids
180 # that TranslationTemplatesBuildBehavior.verifySlaveBuildID
181 # accepts.
182 behavior = self._makeBehavior()
183 buildfarmjob = behavior.buildfarmjob
184 job = buildfarmjob.job
185
186 # The test is that this not raise CorruptBuildID (or anything
187 # else, for that matter).
188 behavior.verifySlaveBuildID(behavior.buildfarmjob.getName())
189
190 def test_verifySlaveBuildID_handles_dashes(self):
191 # TranslationTemplatesBuildBehavior.verifySlaveBuildID can deal
192 # with dashes in branch names.
193 behavior = self._makeBehavior()
194 buildfarmjob = behavior.buildfarmjob
195 job = buildfarmjob.job
196 buildfarmjob.branch.name = 'x-y-z--'
197
198 # The test is that this not raise CorruptBuildID (or anything
199 # else, for that matter).
200 behavior.verifySlaveBuildID(behavior.buildfarmjob.getName())
201
202 def test_verifySlaveBuildID_malformed(self):
203 behavior = self._makeBehavior()
204 self.assertRaises(CorruptBuildID, behavior.verifySlaveBuildID, 'huh?')
205
206 def test_verifySlaveBuildID_notfound(self):
207 behavior = self._makeBehavior()
208 self.assertRaises(CorruptBuildID, behavior.verifySlaveBuildID, '1-1')
209
173210
174def test_suite():211def test_suite():
175 return TestLoader().loadTestsFromName(__name__)212 return TestLoader().loadTestsFromName(__name__)

Subscribers

People subscribed via source and target branches

to status/vote changes: