Launchpad itself

Merge lp:~michael.nelson/launchpad/487009-db-generalise-ibuilder-1b into lp:launchpad/db-devel

  1. 487009-db-generalise-ibuilder-1b
  2. Merge into db-devel
Proposed by Michael Nelson
Status: Merged
Approved by: Michael Nelson
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~michael.nelson/launchpad/487009-db-generalise-ibuilder-1b
Merge into: lp:launchpad/db-devel
Diff against target: 481 lines (+296/-7)
11 files modified
lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py (+38/-0)
lib/lp/buildmaster/model/buildfarmjobbehavior.py (+50/-0)
lib/lp/soyuz/configure.zcml (+7/-0)
lib/lp/soyuz/doc/builder.txt (+12/-0)
lib/lp/soyuz/doc/buildqueue.txt (+10/-0)
lib/lp/soyuz/interfaces/builder.py (+8/-2)
lib/lp/soyuz/interfaces/buildqueue.py (+5/-1)
lib/lp/soyuz/model/binarypackagebuildbehavior.py (+35/-0)
lib/lp/soyuz/model/builder.py (+55/-4)
lib/lp/soyuz/model/buildqueue.py (+7/-0)
lib/lp/soyuz/tests/test_builder.py (+69/-0)
To merge this branch: bzr merge lp:~michael.nelson/launchpad/487009-db-generalise-ibuilder-1b
Reviewer Review Type Date Requested Status
Graham Binns (community) code Approve
Review via email: mp+15477@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Michael Nelson (michael.nelson) wrote :

= Summary =

This branch creates the infrastructure to allow IBuilder to support different behaviors (BinaryPackageBuildBehavior currently, but later SourcePackageRecipeBuildBehavior etc.).

== Proposed fix ==

Create a general interface for build farm job behaviors that can be implemented by various classes, and allow the behavior to be an attribute of the builder to which all relevant calls will be delegated.

This branch begins the extraction of soyuz-specific build behavior for the [Binary]BuildPackageJob into a BinaryPackageBuildBehavior. Subsequent branches (in a pipeline) will continue this.

I plan to re-base these to devel after the release when db-stable is merged back into devel.

== Pre-implementation notes ==

See the pre-implementation notes on bug 487009.

== Implementation details ==

== Tests ==

bin/test -vv -t doc/builder.txt -t doc/buildqueue.txt -t TestCurrentBuildBehavior.txt

== Demo and Q/A ==

We will need to Q/A this on dogfood.

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/soyuz/model/builder.py
  lib/lp/soyuz/doc/builder.txt
  lib/lp/soyuz/doc/buildqueue.txt
  lib/lp/soyuz/configure.zcml
  lib/lp/soyuz/model/buildqueue.py
  lib/lp/buildmaster/model/buildfarmjobbehavior.py
  lib/lp/soyuz/interfaces/buildqueue.py
  lib/lp/soyuz/tests/test_builder.py
  lib/lp/soyuz/model/binarypackagebuildbehavior.py
  lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py
  lib/lp/soyuz/interfaces/builder.py

== Pylint notices ==

lib/lp/soyuz/model/builder.py
    23: [F0401] Unable to import 'lazr.delegates' (No module named delegates)

lib/lp/soyuz/interfaces/buildqueue.py
    18: [F0401] Unable to import 'lazr.restful.fields' (No module named restful)

--
Michael

Revision history for this message
Michael Nelson (michael.nelson) wrote :

> == Tests ==
>
> bin/test -vv -t doc/builder.txt -t doc/buildqueue.txt -t
> TestCurrentBuildBehavior.txt

Also include doc/buildd-dispatching.txt which demonstrates that build dispatching still passes with the refactoring.

Revision history for this message
Graham Binns (gmb) wrote :
Download full text (19.3 KiB)

Hi Michael,

Nice branch. Good work. I've got some stylistic nitpicks - our
guidelines gleefully discard PEP8, hurrah - and I'd like you to do a
strip-trailing-whitespace run on the files you've changed (there's not
much, but enough for me to notice).

Other than that, r=me.

> === added file 'lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py'
> --- lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 1970-01-01 00:00:00 +0000
> +++ lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2009-12-01 10:00:36 +0000
> @@ -0,0 +1,38 @@
> +# Copyright 2009 Canonical Ltd. This software is licensed under the
> +# GNU Affero General Public License version 3 (see the file LICENSE).
> +
> +# pylint: disable-msg=E0211,E0213
> +
> +"""Interface for build farm job behaviors."""
> +
> +__metaclass__ = type
> +
> +__all__ = [
> + 'IBuildFarmJobBehavior',
> + ]
> +
> +from zope.interface import Interface
> +
> +
> +class BuildBehaviorMismatch(Exception):
> + """
> + A general exception that can be raised when the builder's current behavior
> + does not match the expected behavior.
> + """
> +
> +
> +class IBuildFarmJobBehavior(Interface):
> +
> + def set_builder(builder):
> + """Sets the associated builder reference for this instance."""
> +
> + def log_start_build(build_queue_item, logger):
> + """Log the start of a specific build queue item.
> +
> + The form of the log message will vary depending on the type of build.
> + :param build_queue_item: A BuildQueueItem to build.
> + :param logger: A logger to be used to log diagnostic information.
> + """
> + # A number of other methods to go here that can be customised for each of
> + # the different build types (branch build, recipe build, translation
> + # build etc.)
>
> === added file 'lib/lp/buildmaster/model/buildfarmjobbehavior.py'
> --- lib/lp/buildmaster/model/buildfarmjobbehavior.py 1970-01-01 00:00:00 +0000
> +++ lib/lp/buildmaster/model/buildfarmjobbehavior.py 2009-12-01 10:00:36 +0000
> @@ -0,0 +1,53 @@
> +# Copyright 2009 Canonical Ltd. This software is licensed under the
> +# GNU Affero General Public License version 3 (see the file LICENSE).
> +
> +# pylint: disable-msg=E0211,E0213
> +
> +"""Base and idle BuildFarmJobBehavior classes."""
> +
> +__metaclass__ = type
> +
> +__all__ = [
> + 'BuildFarmJobBehaviorBase',
> + 'IdleBuildBehavior'
> + ]
> +
> +from zope.interface import implements
> +
> +from lp.buildmaster.interfaces.buildfarmjobbehavior import (
> + BuildBehaviorMismatch, IBuildFarmJobBehavior)
> +
> +
> +class BuildFarmJobBehaviorBase:
> + """Ensures that all behaviors inherit the same initialisation.
> +
> + All build-farm job behaviors should inherit from this.
> + """
> +
> + def __init__(self, buildfarmjob):
> + """
> + Store a reference to the job_type with which we were created.
> + """
> + self.buildfarmjob = buildfarmjob
> + self._builder = None
> +
> + def set_builder(self, builder):

This should be setBuilder(), because who needs PEP8?

> + """The builder should be set once and not changed."""
> + self._builder = builde...

review: Approve (code)
Revision history for this message
Michael Nelson (michael.nelson) wrote :
Download full text (21.3 KiB)

On Tue, Dec 1, 2009 at 4:30 PM, Graham Binns <email address hidden> wrote:

> Review: Approve code
>
> Hi Michael,
>
> Nice branch. Good work. I've got some stylistic nitpicks - our
> guidelines gleefully discard PEP8, hurrah - and I'd like you to do a
>

gah, I'd thought I was cleaning up old code (too much lazrjs lately).

> strip-trailing-whitespace run on the files you've changed (there's not
> much, but enough for me to notice).
>
>
Sorry - switched editors recently and forgot to setup tailing whitespace
removal. I've updated my .rc file to automatically delete trailing spaces
and fixed all the files. Thanks.

> Other than that, r=me.
>
> > === added file 'lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py'
> > --- lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 1970-01-01
> 00:00:00 +0000
> > +++ lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2009-12-01
> 10:00:36 +0000
> > @@ -0,0 +1,38 @@
> > +# Copyright 2009 Canonical Ltd. This software is licensed under the
> > +# GNU Affero General Public License version 3 (see the file LICENSE).
> > +
> > +# pylint: disable-msg=E0211,E0213
> > +
> > +"""Interface for build farm job behaviors."""
> > +
> > +__metaclass__ = type
> > +
> > +__all__ = [
> > + 'IBuildFarmJobBehavior',
> > + ]
> > +
> > +from zope.interface import Interface
> > +
> > +
> > +class BuildBehaviorMismatch(Exception):
> > + """
> > + A general exception that can be raised when the builder's current
> behavior
> > + does not match the expected behavior.
> > + """
> > +
> > +
> > +class IBuildFarmJobBehavior(Interface):
> > +
> > + def set_builder(builder):
> > + """Sets the associated builder reference for this instance."""
> > +
> > + def log_start_build(build_queue_item, logger):
> > + """Log the start of a specific build queue item.
> > +
> > + The form of the log message will vary depending on the type of
> build.
> > + :param build_queue_item: A BuildQueueItem to build.
> > + :param logger: A logger to be used to log diagnostic
> information.
> > + """
> > + # A number of other methods to go here that can be customised for
> each of
> > + # the different build types (branch build, recipe build, translation
> > + # build etc.)
> >
> > === added file 'lib/lp/buildmaster/model/buildfarmjobbehavior.py'
> > --- lib/lp/buildmaster/model/buildfarmjobbehavior.py 1970-01-01 00:00:00
> +0000
> > +++ lib/lp/buildmaster/model/buildfarmjobbehavior.py 2009-12-01 10:00:36
> +0000
> > @@ -0,0 +1,53 @@
> > +# Copyright 2009 Canonical Ltd. This software is licensed under the
> > +# GNU Affero General Public License version 3 (see the file LICENSE).
> > +
> > +# pylint: disable-msg=E0211,E0213
> > +
> > +"""Base and idle BuildFarmJobBehavior classes."""
> > +
> > +__metaclass__ = type
> > +
> > +__all__ = [
> > + 'BuildFarmJobBehaviorBase',
> > + 'IdleBuildBehavior'
> > + ]
> > +
> > +from zope.interface import implements
> > +
> > +from lp.buildmaster.interfaces.buildfarmjobbehavior import (
> > + BuildBehaviorMismatch, IBuildFarmJobBehavior)
> > +
> > +
> > +class BuildFarmJobBehaviorBase:
> > + """Ensures that all ...

changes_gmb.diff
=== modified file 'lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py'
--- lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2009-11-27 13:50:29 +0000
+++ lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2009-12-01 16:40:02 +0000
@@ -23,10 +23,10 @@
2323
24class IBuildFarmJobBehavior(Interface):24class IBuildFarmJobBehavior(Interface):
2525
26 def set_builder(builder):26 def setBuilder(builder):
27 """Sets the associated builder reference for this instance."""27 """Sets the associated builder reference for this instance."""
2828
29 def log_start_build(build_queue_item, logger):29 def logStartBuild(build_queue_item, logger):
30 """Log the start of a specific build queue item.30 """Log the start of a specific build queue item.
3131
32 The form of the log message will vary depending on the type of build.32 The form of the log message will vary depending on the type of build.
3333
=== modified file 'lib/lp/buildmaster/model/buildfarmjobbehavior.py'
--- lib/lp/buildmaster/model/buildfarmjobbehavior.py 2009-12-01 09:43:41 +0000
+++ lib/lp/buildmaster/model/buildfarmjobbehavior.py 2009-12-01 16:38:56 +0000
@@ -25,13 +25,11 @@
25 """25 """
2626
27 def __init__(self, buildfarmjob):27 def __init__(self, buildfarmjob):
28 """28 """Store a reference to the job_type with which we were created."""
29 Store a reference to the job_type with which we were created.
30 """
31 self.buildfarmjob = buildfarmjob29 self.buildfarmjob = buildfarmjob
32 self._builder = None30 self._builder = None
3331
34 def set_builder(self, builder):32 def setBuilder(self, builder):
35 """The builder should be set once and not changed."""33 """The builder should be set once and not changed."""
36 self._builder = builder34 self._builder = builder
3735
@@ -41,9 +39,8 @@
41 implements(IBuildFarmJobBehavior)39 implements(IBuildFarmJobBehavior)
4240
43 def __init__(self):41 def __init__(self):
44 """42 """The idle behavior is special in that a buildfarmjob is not
45 The idle behavior is special in that a buildfarmjob is not specified43 specified during initialisation.
46 during initialisation.
47 """44 """
48 super(IdleBuildBehavior, self).__init__(None)45 super(IdleBuildBehavior, self).__init__(None)
4946
5047
=== modified file 'lib/lp/soyuz/model/binarypackagebuildbehavior.py'
--- lib/lp/soyuz/model/binarypackagebuildbehavior.py 2009-12-01 09:02:14 +0000
+++ lib/lp/soyuz/model/binarypackagebuildbehavior.py 2009-12-01 16:39:48 +0000
@@ -26,8 +26,8 @@
2626
27 implements(IBuildFarmJobBehavior)27 implements(IBuildFarmJobBehavior)
2828
29 def log_start_build(self, build_queue_item, logger):29 def logStartBuild(self, build_queue_item, logger):
30 """See `IBuildFarmJobBehavior`.""" 30 """See `IBuildFarmJobBehavior`."""
31 build = getUtility(IBuildSet).getByQueueEntry(build_queue_item)31 build = getUtility(IBuildSet).getByQueueEntry(build_queue_item)
32 spr = build.sourcepackagerelease32 spr = build.sourcepackagerelease
3333
3434
=== modified file 'lib/lp/soyuz/model/builder.py'
--- lib/lp/soyuz/model/builder.py 2009-12-01 09:38:05 +0000
+++ lib/lp/soyuz/model/builder.py 2009-12-01 16:37:20 +0000
@@ -142,7 +142,7 @@
142 vm_host = StringCol(dbName='vm_host')142 vm_host = StringCol(dbName='vm_host')
143 active = BoolCol(dbName='active', notNull=True, default=True)143 active = BoolCol(dbName='active', notNull=True, default=True)
144144
145 def _get_current_build_behavior(self):145 def _getCurrentBuildBehavior(self):
146 """Return the current build behavior."""146 """Return the current build behavior."""
147 if not safe_hasattr(self, '_current_build_behavior'):147 if not safe_hasattr(self, '_current_build_behavior'):
148 self._current_build_behavior = None148 self._current_build_behavior = None
@@ -156,14 +156,14 @@
156 # ...we'll set it based on our current job.156 # ...we'll set it based on our current job.
157 self._current_build_behavior = (157 self._current_build_behavior = (
158 currentjob.required_build_behavior)158 currentjob.required_build_behavior)
159 self._current_build_behavior.set_builder(self)159 self._current_build_behavior.setBuilder(self)
160 return self._current_build_behavior160 return self._current_build_behavior
161 elif self._current_build_behavior is None:161 elif self._current_build_behavior is None:
162 # If we don't have a current job or an idle behavior162 # If we don't have a current job or an idle behavior
163 # already set, then we just set the idle behavior163 # already set, then we just set the idle behavior
164 # before returning.164 # before returning.
165 self._current_build_behavior = IdleBuildBehavior()165 self._current_build_behavior = IdleBuildBehavior()
166 return self._current_build_behavior 166 return self._current_build_behavior
167167
168 else:168 else:
169 # We did have a current non-idle build behavior set, so169 # We did have a current non-idle build behavior set, so
@@ -171,7 +171,7 @@
171 return self._current_build_behavior171 return self._current_build_behavior
172172
173173
174 def _set_current_build_behavior(self, new_behavior):174 def _setCurrentBuildBehavior(self, new_behavior):
175 """Set the current build behavior."""175 """Set the current build behavior."""
176176
177 if self.currentjob is not None:177 if self.currentjob is not None:
@@ -182,10 +182,10 @@
182 "exists.")182 "exists.")
183 else:183 else:
184 self._current_build_behavior = new_behavior184 self._current_build_behavior = new_behavior
185 self._current_build_behavior.set_builder(self)185 self._current_build_behavior.setBuilder(self)
186186
187 current_build_behavior = property(187 current_build_behavior = property(
188 _get_current_build_behavior, _set_current_build_behavior)188 _getCurrentBuildBehavior, _setCurrentBuildBehavior)
189189
190 def cacheFileOnSlave(self, logger, libraryfilealias):190 def cacheFileOnSlave(self, logger, libraryfilealias):
191 """See `IBuilder`."""191 """See `IBuilder`."""
@@ -403,9 +403,9 @@
403403
404 def startBuild(self, build_queue_item, logger):404 def startBuild(self, build_queue_item, logger):
405 """See IBuilder."""405 """See IBuilder."""
406 # Set the build behavior depending on the provided build queue item. 406 # Set the build behavior depending on the provided build queue item.
407 self.current_build_behavior = build_queue_item.required_build_behavior407 self.current_build_behavior = build_queue_item.required_build_behavior
408 self.log_start_build(build_queue_item, logger)408 self.logStartBuild(build_queue_item, logger)
409409
410 # Make sure the request is valid; an exception is raised if it's not.410 # Make sure the request is valid; an exception is raised if it's not.
411 self._verifyBuildRequest(build_queue_item, logger)411 self._verifyBuildRequest(build_queue_item, logger)
412412
=== modified file 'lib/lp/soyuz/tests/test_builder.py'
--- lib/lp/soyuz/tests/test_builder.py 2009-12-01 09:43:41 +0000
+++ lib/lp/soyuz/tests/test_builder.py 2009-12-01 16:39:29 +0000
@@ -221,8 +221,8 @@
221221
222222
223class TestCurrentBuildBehavior(TestCaseWithFactory):223class TestCurrentBuildBehavior(TestCaseWithFactory):
224 """224 """This test ensures the get/set behavior of IBuilder's
225 This test ensures the get/set behavior of Builder.current_build_behavior.225 current_build_behavior property.
226 """226 """
227227
228 layer = LaunchpadZopelessLayer228 layer = LaunchpadZopelessLayer
@@ -231,7 +231,7 @@
231 """Create a new builder ready for testing."""231 """Create a new builder ready for testing."""
232 super(TestCurrentBuildBehavior, self).setUp()232 super(TestCurrentBuildBehavior, self).setUp()
233 self.builder = self.factory.makeBuilder(name='builder')233 self.builder = self.factory.makeBuilder(name='builder')
234 234
235 # Have a publisher and a ppa handy for some of the tests below.235 # Have a publisher and a ppa handy for some of the tests below.
236 self.publisher = SoyuzTestPublisher()236 self.publisher = SoyuzTestPublisher()
237 self.publisher.prepareBreezyAutotest()237 self.publisher.prepareBreezyAutotest()
@@ -244,8 +244,7 @@
244 self.buildfarmjob = self.build.buildqueue_record.specific_job244 self.buildfarmjob = self.build.buildqueue_record.specific_job
245245
246 def test_idle_behavior_when_no_current_build(self):246 def test_idle_behavior_when_no_current_build(self):
247 """247 """We return an idle behavior when there is no behavior specified
248 We return an idle behavior when there is no behavior specified
249 nor a current build.248 nor a current build.
250 """249 """
251 self.assertIsInstance(250 self.assertIsInstance(
@@ -254,7 +253,7 @@
254 def test_set_behavior_when_no_current_job(self):253 def test_set_behavior_when_no_current_job(self):
255 """If a builder is idle then it is possible to set the behavior."""254 """If a builder is idle then it is possible to set the behavior."""
256 self.builder.current_build_behavior = IBuildFarmJobBehavior(255 self.builder.current_build_behavior = IBuildFarmJobBehavior(
257 self.buildfarmjob) 256 self.buildfarmjob)
258257
259 self.assertIsInstance(258 self.assertIsInstance(
260 self.builder.current_build_behavior, BinaryPackageBuildBehavior)259 self.builder.current_build_behavior, BinaryPackageBuildBehavior)
@@ -269,9 +268,7 @@
269 self.builder.current_build_behavior, BinaryPackageBuildBehavior)268 self.builder.current_build_behavior, BinaryPackageBuildBehavior)
270269
271 def test_set_behavior_when_current_job(self):270 def test_set_behavior_when_current_job(self):
272 """271 """If a builder has a current job then it's behavior cannot be set.
273 If a builder has a current job then it's behavior cannot be
274 set.
275 """272 """
276 self.build.buildqueue_record.builder = self.builder273 self.build.buildqueue_record.builder = self.builder
277274
@@ -280,7 +277,7 @@
280 assertion_raised = False277 assertion_raised = False
281 try:278 try:
282 self.builder.current_build_behavior = IBuildFarmJobBehavior(279 self.builder.current_build_behavior = IBuildFarmJobBehavior(
283 self.buildfarmjob) 280 self.buildfarmjob)
284 except BuildBehaviorMismatch, e:281 except BuildBehaviorMismatch, e:
285 assertion_raised = True282 assertion_raised = True
286283

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== added file 'lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py'
--- lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 1970-01-01 00:00:00 +0000
+++ lib/lp/buildmaster/interfaces/buildfarmjobbehavior.py 2009-12-01 16:47:13 +0000
@@ -0,0 +1,38 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4# pylint: disable-msg=E0211,E0213
5
6"""Interface for build farm job behaviors."""
7
8__metaclass__ = type
9
10__all__ = [
11 'IBuildFarmJobBehavior',
12 ]
13
14from zope.interface import Interface
15
16
17class BuildBehaviorMismatch(Exception):
18 """
19 A general exception that can be raised when the builder's current behavior
20 does not match the expected behavior.
21 """
22
23
24class IBuildFarmJobBehavior(Interface):
25
26 def setBuilder(builder):
27 """Sets the associated builder reference for this instance."""
28
29 def logStartBuild(build_queue_item, logger):
30 """Log the start of a specific build queue item.
31
32 The form of the log message will vary depending on the type of build.
33 :param build_queue_item: A BuildQueueItem to build.
34 :param logger: A logger to be used to log diagnostic information.
35 """
36 # A number of other methods to go here that can be customised for each of
37 # the different build types (branch build, recipe build, translation
38 # build etc.)
039
=== added file 'lib/lp/buildmaster/model/buildfarmjobbehavior.py'
--- lib/lp/buildmaster/model/buildfarmjobbehavior.py 1970-01-01 00:00:00 +0000
+++ lib/lp/buildmaster/model/buildfarmjobbehavior.py 2009-12-01 16:47:13 +0000
@@ -0,0 +1,50 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4# pylint: disable-msg=E0211,E0213
5
6"""Base and idle BuildFarmJobBehavior classes."""
7
8__metaclass__ = type
9
10__all__ = [
11 'BuildFarmJobBehaviorBase',
12 'IdleBuildBehavior'
13 ]
14
15from zope.interface import implements
16
17from lp.buildmaster.interfaces.buildfarmjobbehavior import (
18 BuildBehaviorMismatch, IBuildFarmJobBehavior)
19
20
21class BuildFarmJobBehaviorBase:
22 """Ensures that all behaviors inherit the same initialisation.
23
24 All build-farm job behaviors should inherit from this.
25 """
26
27 def __init__(self, buildfarmjob):
28 """Store a reference to the job_type with which we were created."""
29 self.buildfarmjob = buildfarmjob
30 self._builder = None
31
32 def setBuilder(self, builder):
33 """The builder should be set once and not changed."""
34 self._builder = builder
35
36
37class IdleBuildBehavior(BuildFarmJobBehaviorBase):
38
39 implements(IBuildFarmJobBehavior)
40
41 def __init__(self):
42 """The idle behavior is special in that a buildfarmjob is not
43 specified during initialisation.
44 """
45 super(IdleBuildBehavior, self).__init__(None)
46
47 def logStartBuild(self, build_queue_item, logger):
48 """See `IBuildFarmJobBehavior`."""
49 raise BuildBehaviorMismatch(
50 "Builder was idle when asked to log the start of a build.")
051
=== modified file 'lib/lp/soyuz/configure.zcml'
--- lib/lp/soyuz/configure.zcml 2009-11-16 22:06:14 +0000
+++ lib/lp/soyuz/configure.zcml 2009-12-01 16:47:13 +0000
@@ -892,4 +892,11 @@
892 interface="lp.soyuz.interfaces.buildpackagejob.IBuildPackageJob"/>892 interface="lp.soyuz.interfaces.buildpackagejob.IBuildPackageJob"/>
893 </class>893 </class>
894894
895 <!-- BinaryPackageBuildBehavior -->
896 <adapter
897 for="lp.soyuz.interfaces.buildpackagejob.IBuildPackageJob"
898 provides="lp.buildmaster.interfaces.buildfarmjobbehavior.IBuildFarmJobBehavior"
899 factory="lp.soyuz.model.binarypackagebuildbehavior.BinaryPackageBuildBehavior"
900 permission="zope.Public" />
901
895</configure>902</configure>
896903
=== modified file 'lib/lp/soyuz/doc/builder.txt'
--- lib/lp/soyuz/doc/builder.txt 2009-11-12 12:52:23 +0000
+++ lib/lp/soyuz/doc/builder.txt 2009-12-01 16:47:13 +0000
@@ -40,6 +40,18 @@
40 >>> builder.builderok = True40 >>> builder.builderok = True
41 >>> builder.failnotes = None41 >>> builder.failnotes = None
4242
43A builder provides a current_build_behavior attribute to which all the
44build-type specific behavior for the current build is delegated. For
45our sample data, the behavior is specifically a behavior for dealing
46with binary packages
47
48 >>> from zope.security.proxy import isinstance
49 >>> from lp.soyuz.model.binarypackagebuildbehavior import (
50 ... BinaryPackageBuildBehavior)
51 >>> isinstance(
52 ... builder.current_build_behavior, BinaryPackageBuildBehavior)
53 True
54
43In case of copy archives the status string will show both the copy55In case of copy archives the status string will show both the copy
44archive owner as well as the copy archive name.56archive owner as well as the copy archive name.
4557
4658
=== modified file 'lib/lp/soyuz/doc/buildqueue.txt'
--- lib/lp/soyuz/doc/buildqueue.txt 2009-11-13 19:34:17 +0000
+++ lib/lp/soyuz/doc/buildqueue.txt 2009-12-01 16:47:13 +0000
@@ -98,6 +98,16 @@
98 (True, 1000)98 (True, 1000)
9999
100100
101The BuildQueue item is responsible for providing the required build behavior
102for the item.
103
104 >>> from zope.security.proxy import isinstance
105 >>> from lp.soyuz.model.binarypackagebuildbehavior import (
106 ... BinaryPackageBuildBehavior)
107 >>> isinstance(bq.required_build_behavior, BinaryPackageBuildBehavior)
108 True
109
110
101== Dispatching and Reseting jobs ==111== Dispatching and Reseting jobs ==
102112
103The sampledata contains an active job, being built by the 'bob'113The sampledata contains an active job, being built by the 'bob'
104114
=== modified file 'lib/lp/soyuz/interfaces/builder.py'
--- lib/lp/soyuz/interfaces/builder.py 2009-11-20 18:06:28 +0000
+++ lib/lp/soyuz/interfaces/builder.py 2009-12-01 16:47:13 +0000
@@ -19,11 +19,13 @@
19 ]19 ]
2020
21from zope.interface import Interface, Attribute21from zope.interface import Interface, Attribute
22from zope.schema import Choice, TextLine, Text, Bool22from zope.schema import Bool, Choice, Field, Text, TextLine
2323
24from canonical.launchpad import _24from canonical.launchpad import _
25from canonical.launchpad.fields import Title, Description25from canonical.launchpad.fields import Title, Description
26from lp.registry.interfaces.role import IHasOwner26from lp.registry.interfaces.role import IHasOwner
27from lp.buildmaster.interfaces.buildfarmjobbehavior import (
28 IBuildFarmJobBehavior)
27from canonical.launchpad.validators.name import name_validator29from canonical.launchpad.validators.name import name_validator
28from canonical.launchpad.validators.url import builder_url_validator30from canonical.launchpad.validators.url import builder_url_validator
2931
@@ -57,7 +59,7 @@
57 """The build slave has suffered an error and cannot be used."""59 """The build slave has suffered an error and cannot be used."""
5860
5961
60class IBuilder(IHasOwner):62class IBuilder(IHasOwner, IBuildFarmJobBehavior):
61 """Build-slave information and state.63 """Build-slave information and state.
6264
63 Builder instance represents a single builder slave machine within the65 Builder instance represents a single builder slave machine within the
@@ -138,6 +140,10 @@
138 "new jobs. "),140 "new jobs. "),
139 required=False)141 required=False)
140142
143 current_build_behavior = Field(
144 title=u"The current behavior of the builder for the current job.",
145 required=False)
146
141 def cacheFileOnSlave(logger, libraryfilealias):147 def cacheFileOnSlave(logger, libraryfilealias):
142 """Ask the slave to cache a librarian file to its local disk.148 """Ask the slave to cache a librarian file to its local disk.
143149
144150
=== modified file 'lib/lp/soyuz/interfaces/buildqueue.py'
--- lib/lp/soyuz/interfaces/buildqueue.py 2009-11-20 18:06:28 +0000
+++ lib/lp/soyuz/interfaces/buildqueue.py 2009-12-01 16:47:13 +0000
@@ -13,7 +13,7 @@
13 ]13 ]
1414
15from zope.interface import Interface, Attribute15from zope.interface import Interface, Attribute
16from zope.schema import Choice, Datetime, Timedelta16from zope.schema import Choice, Datetime, Field, Timedelta
1717
18from lazr.restful.fields import Reference18from lazr.restful.fields import Reference
1919
@@ -51,6 +51,10 @@
51 title=_('Job type'), required=True, vocabulary=BuildFarmJobType,51 title=_('Job type'), required=True, vocabulary=BuildFarmJobType,
52 description=_("The type of this job."))52 description=_("The type of this job."))
5353
54 required_build_behavior = Field(
55 title=_('The builder behavior required to run this job.'),
56 required=False, readonly=True)
57
54 estimated_duration = Timedelta(58 estimated_duration = Timedelta(
55 title=_("Estimated Job Duration"), required=True,59 title=_("Estimated Job Duration"), required=True,
56 description=_("Estimated job duration interval."))60 description=_("Estimated job duration interval."))
5761
=== added file 'lib/lp/soyuz/model/binarypackagebuildbehavior.py'
--- lib/lp/soyuz/model/binarypackagebuildbehavior.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/model/binarypackagebuildbehavior.py 2009-12-01 16:47:13 +0000
@@ -0,0 +1,35 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4# pylint: disable-msg=E0211,E0213
5
6"""Builder behavior for binary package builds."""
7
8__metaclass__ = type
9
10__all__ = [
11 'BinaryPackageBuildBehavior',
12 ]
13
14from lp.buildmaster.interfaces.buildfarmjobbehavior import (
15 IBuildFarmJobBehavior)
16from lp.buildmaster.model.buildfarmjobbehavior import (
17 BuildFarmJobBehaviorBase)
18from lp.soyuz.interfaces.build import IBuildSet
19
20from zope.component import getUtility
21from zope.interface import implements
22
23
24class BinaryPackageBuildBehavior(BuildFarmJobBehaviorBase):
25 """Define the behavior of binary package builds."""
26
27 implements(IBuildFarmJobBehavior)
28
29 def logStartBuild(self, build_queue_item, logger):
30 """See `IBuildFarmJobBehavior`."""
31 build = getUtility(IBuildSet).getByQueueEntry(build_queue_item)
32 spr = build.sourcepackagerelease
33
34 logger.info("startBuild(%s, %s, %s, %s)", self._builder.url,
35 spr.name, spr.version, build.pocket.title)
036
=== modified file 'lib/lp/soyuz/model/builder.py'
--- lib/lp/soyuz/model/builder.py 2009-11-20 18:06:28 +0000
+++ lib/lp/soyuz/model/builder.py 2009-12-01 16:47:13 +0000
@@ -20,6 +20,8 @@
20import urllib220import urllib2
21import xmlrpclib21import xmlrpclib
2222
23from lazr.delegates import delegates
24
23from zope.interface import implements25from zope.interface import implements
24from zope.component import getUtility26from zope.component import getUtility
2527
@@ -31,7 +33,10 @@
31from canonical.cachedproperty import cachedproperty33from canonical.cachedproperty import cachedproperty
32from canonical.config import config34from canonical.config import config
33from canonical.buildd.slave import BuilderStatus35from canonical.buildd.slave import BuilderStatus
36from lp.buildmaster.interfaces.buildfarmjobbehavior import (
37 BuildBehaviorMismatch, IBuildFarmJobBehavior)
34from lp.buildmaster.master import BuilddMaster38from lp.buildmaster.master import BuilddMaster
39from lp.buildmaster.model.buildfarmjobbehavior import IdleBuildBehavior
35from canonical.database.sqlbase import SQLBase, sqlvalues40from canonical.database.sqlbase import SQLBase, sqlvalues
36from lp.soyuz.adapters.archivedependencies import (41from lp.soyuz.adapters.archivedependencies import (
37 get_primary_current_component, get_sources_list_for_building)42 get_primary_current_component, get_sources_list_for_building)
@@ -54,6 +59,7 @@
54 PackagePublishingStatus)59 PackagePublishingStatus)
55from lp.soyuz.model.buildpackagejob import BuildPackageJob60from lp.soyuz.model.buildpackagejob import BuildPackageJob
56from canonical.launchpad.webapp import urlappend61from canonical.launchpad.webapp import urlappend
62from canonical.lazr.utils import safe_hasattr
57from canonical.librarian.utils import copy_and_close63from canonical.librarian.utils import copy_and_close
5864
5965
@@ -114,6 +120,7 @@
114class Builder(SQLBase):120class Builder(SQLBase):
115121
116 implements(IBuilder, IHasBuildRecords)122 implements(IBuilder, IHasBuildRecords)
123 delegates(IBuildFarmJobBehavior, context="current_build_behavior")
117 _table = 'Builder'124 _table = 'Builder'
118125
119 _defaultOrder = ['id']126 _defaultOrder = ['id']
@@ -135,6 +142,51 @@
135 vm_host = StringCol(dbName='vm_host')142 vm_host = StringCol(dbName='vm_host')
136 active = BoolCol(dbName='active', notNull=True, default=True)143 active = BoolCol(dbName='active', notNull=True, default=True)
137144
145 def _getCurrentBuildBehavior(self):
146 """Return the current build behavior."""
147 if not safe_hasattr(self, '_current_build_behavior'):
148 self._current_build_behavior = None
149
150 if (self._current_build_behavior is None or
151 isinstance(self._current_build_behavior, IdleBuildBehavior)):
152 # If we don't currently have a current build behavior set,
153 # or we are currently idle, then...
154 currentjob = self.currentjob
155 if currentjob is not None:
156 # ...we'll set it based on our current job.
157 self._current_build_behavior = (
158 currentjob.required_build_behavior)
159 self._current_build_behavior.setBuilder(self)
160 return self._current_build_behavior
161 elif self._current_build_behavior is None:
162 # If we don't have a current job or an idle behavior
163 # already set, then we just set the idle behavior
164 # before returning.
165 self._current_build_behavior = IdleBuildBehavior()
166 return self._current_build_behavior
167
168 else:
169 # We did have a current non-idle build behavior set, so
170 # we just return it.
171 return self._current_build_behavior
172
173
174 def _setCurrentBuildBehavior(self, new_behavior):
175 """Set the current build behavior."""
176
177 if self.currentjob is not None:
178 # We do not allow the current build behavior to be reset if we
179 # have a current job.
180 raise BuildBehaviorMismatch(
181 "Attempt to reset builder behavior while a current build "
182 "exists.")
183 else:
184 self._current_build_behavior = new_behavior
185 self._current_build_behavior.setBuilder(self)
186
187 current_build_behavior = property(
188 _getCurrentBuildBehavior, _setCurrentBuildBehavior)
189
138 def cacheFileOnSlave(self, logger, libraryfilealias):190 def cacheFileOnSlave(self, logger, libraryfilealias):
139 """See `IBuilder`."""191 """See `IBuilder`."""
140 url = libraryfilealias.http_url192 url = libraryfilealias.http_url
@@ -351,10 +403,9 @@
351403
352 def startBuild(self, build_queue_item, logger):404 def startBuild(self, build_queue_item, logger):
353 """See IBuilder."""405 """See IBuilder."""
354 build = getUtility(IBuildSet).getByQueueEntry(build_queue_item)406 # Set the build behavior depending on the provided build queue item.
355 spr = build.sourcepackagerelease407 self.current_build_behavior = build_queue_item.required_build_behavior
356 logger.info("startBuild(%s, %s, %s, %s)", self.url,408 self.logStartBuild(build_queue_item, logger)
357 spr.name, spr.version, build.pocket.title)
358409
359 # Make sure the request is valid; an exception is raised if it's not.410 # Make sure the request is valid; an exception is raised if it's not.
360 self._verifyBuildRequest(build_queue_item, logger)411 self._verifyBuildRequest(build_queue_item, logger)
361412
=== modified file 'lib/lp/soyuz/model/buildqueue.py'
--- lib/lp/soyuz/model/buildqueue.py 2009-11-20 18:06:28 +0000
+++ lib/lp/soyuz/model/buildqueue.py 2009-12-01 16:47:13 +0000
@@ -24,6 +24,8 @@
24from canonical.database.sqlbase import SQLBase, sqlvalues24from canonical.database.sqlbase import SQLBase, sqlvalues
25from canonical.launchpad.webapp.interfaces import NotFoundError25from canonical.launchpad.webapp.interfaces import NotFoundError
26from lp.buildmaster.interfaces.buildfarmjob import BuildFarmJobType26from lp.buildmaster.interfaces.buildfarmjob import BuildFarmJobType
27from lp.buildmaster.interfaces.buildfarmjobbehavior import (
28 IBuildFarmJobBehavior)
27from lp.services.job.interfaces.job import JobStatus29from lp.services.job.interfaces.job import JobStatus
28from lp.services.job.model.job import Job30from lp.services.job.model.job import Job
29from lp.soyuz.interfaces.build import BuildStatus, IBuildSet31from lp.soyuz.interfaces.build import BuildStatus, IBuildSet
@@ -49,6 +51,11 @@
49 estimated_duration = IntervalCol()51 estimated_duration = IntervalCol()
5052
51 @property53 @property
54 def required_build_behavior(self):
55 """See `IBuildQueue`."""
56 return IBuildFarmJobBehavior(self.specific_job)
57
58 @property
52 def specific_job(self):59 def specific_job(self):
53 """See `IBuildQueue`."""60 """See `IBuildQueue`."""
54 store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)61 store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
5562
=== modified file 'lib/lp/soyuz/tests/test_builder.py'
--- lib/lp/soyuz/tests/test_builder.py 2009-11-13 16:37:05 +0000
+++ lib/lp/soyuz/tests/test_builder.py 2009-12-01 16:47:13 +0000
@@ -8,10 +8,15 @@
8from zope.component import getUtility8from zope.component import getUtility
99
10from canonical.testing import LaunchpadZopelessLayer10from canonical.testing import LaunchpadZopelessLayer
11from lp.buildmaster.interfaces.buildfarmjobbehavior import (
12 BuildBehaviorMismatch, IBuildFarmJobBehavior)
13from lp.buildmaster.model.buildfarmjobbehavior import IdleBuildBehavior
11from lp.soyuz.interfaces.archive import ArchivePurpose14from lp.soyuz.interfaces.archive import ArchivePurpose
12from lp.soyuz.interfaces.build import BuildStatus, IBuildSet15from lp.soyuz.interfaces.build import BuildStatus, IBuildSet
13from lp.soyuz.interfaces.builder import IBuilderSet16from lp.soyuz.interfaces.builder import IBuilderSet
14from lp.soyuz.interfaces.publishing import PackagePublishingStatus17from lp.soyuz.interfaces.publishing import PackagePublishingStatus
18from lp.soyuz.model.binarypackagebuildbehavior import (
19 BinaryPackageBuildBehavior)
15from lp.soyuz.tests.test_publishing import SoyuzTestPublisher20from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
16from lp.testing import TestCaseWithFactory21from lp.testing import TestCaseWithFactory
1722
@@ -214,5 +219,69 @@
214 self.failUnlessEqual('primary', build.archive.name)219 self.failUnlessEqual('primary', build.archive.name)
215 self.failUnlessEqual('firefox', build.sourcepackagerelease.name)220 self.failUnlessEqual('firefox', build.sourcepackagerelease.name)
216221
222
223class TestCurrentBuildBehavior(TestCaseWithFactory):
224 """This test ensures the get/set behavior of IBuilder's
225 current_build_behavior property.
226 """
227
228 layer = LaunchpadZopelessLayer
229
230 def setUp(self):
231 """Create a new builder ready for testing."""
232 super(TestCurrentBuildBehavior, self).setUp()
233 self.builder = self.factory.makeBuilder(name='builder')
234
235 # Have a publisher and a ppa handy for some of the tests below.
236 self.publisher = SoyuzTestPublisher()
237 self.publisher.prepareBreezyAutotest()
238 self.ppa_joe = self.factory.makeArchive(name="joesppa")
239
240 self.build = self.publisher.getPubSource(
241 sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
242 archive=self.ppa_joe).createMissingBuilds()[0]
243
244 self.buildfarmjob = self.build.buildqueue_record.specific_job
245
246 def test_idle_behavior_when_no_current_build(self):
247 """We return an idle behavior when there is no behavior specified
248 nor a current build.
249 """
250 self.assertIsInstance(
251 self.builder.current_build_behavior, IdleBuildBehavior)
252
253 def test_set_behavior_when_no_current_job(self):
254 """If a builder is idle then it is possible to set the behavior."""
255 self.builder.current_build_behavior = IBuildFarmJobBehavior(
256 self.buildfarmjob)
257
258 self.assertIsInstance(
259 self.builder.current_build_behavior, BinaryPackageBuildBehavior)
260
261 def test_current_job_behavior(self):
262 """The current behavior is set automatically from the current job."""
263 # Set the builder attribute on the buildqueue record so that our
264 # builder will think it has a current build.
265 self.build.buildqueue_record.builder = self.builder
266
267 self.assertIsInstance(
268 self.builder.current_build_behavior, BinaryPackageBuildBehavior)
269
270 def test_set_behavior_when_current_job(self):
271 """If a builder has a current job then it's behavior cannot be set.
272 """
273 self.build.buildqueue_record.builder = self.builder
274
275 # As we can't use assertRaises for a property, we use a try-except
276 # instead.
277 assertion_raised = False
278 try:
279 self.builder.current_build_behavior = IBuildFarmJobBehavior(
280 self.buildfarmjob)
281 except BuildBehaviorMismatch, e:
282 assertion_raised = True
283
284 self.failUnless(assertion_raised)
285
217def test_suite():286def test_suite():
218 return unittest.TestLoader().loadTestsFromName(__name__)287 return unittest.TestLoader().loadTestsFromName(__name__)

Subscribers

People subscribed via source and target branches

to status/vote changes: