Merge lp:~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild-3 into lp:launchpad/db-devel
- 567922-binarypackagebuild-packagebuild-3
- Merge into db-devel
Status: | Merged | ||||
---|---|---|---|---|---|
Merged at revision: | 9405 | ||||
Proposed branch: | lp:~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild-3 | ||||
Merge into: | lp:launchpad/db-devel | ||||
Prerequisite: | lp:~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild-2 | ||||
Diff against target: |
786 lines (+263/-94) (has conflicts) 13 files modified
BRANCH.TODO (+4/-0) lib/lp/buildmaster/doc/buildfarmjob.txt (+8/-5) lib/lp/buildmaster/interfaces/buildbase.py (+1/-1) lib/lp/buildmaster/interfaces/packagebuild.py (+32/-0) lib/lp/buildmaster/model/buildbase.py (+103/-57) lib/lp/buildmaster/model/buildfarmjob.py (+9/-3) lib/lp/buildmaster/model/buildfarmjobbehavior.py (+3/-2) lib/lp/buildmaster/model/packagebuild.py (+53/-9) lib/lp/buildmaster/tests/test_buildbase.py (+3/-3) lib/lp/buildmaster/tests/test_packagebuild.py (+19/-6) lib/lp/code/model/sourcepackagerecipebuild.py (+9/-3) lib/lp/soyuz/model/buildpackagejob.py (+9/-4) lib/lp/translations/model/translationtemplatesbuildjob.py (+10/-1) Text conflict in lib/lp/buildmaster/interfaces/buildbase.py Text conflict in lib/lp/buildmaster/model/buildbase.py Text conflict in lib/lp/buildmaster/tests/test_buildbase.py |
||||
To merge this branch: | bzr merge lp:~michael.nelson/launchpad/567922-binarypackagebuild-packagebuild-3 | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jelmer Vernooij (community) | code* | Approve | |
Review via email: mp+24327@code.launchpad.net |
Commit message
Description of the change
Please read the description of the previous MP at:
This branch adds the last three interface methods to IPackageBuild: verifySuccessfu
The same tests to run.
The next branch steps back from all this re-shuffling and cleans up the results.
Jelmer Vernooij (jelmer) wrote : | # |
It'd be great if this branch can also use Store.flush rather than flush_database_
Michael Nelson (michael.nelson) wrote : | # |
On Thu, Apr 29, 2010 at 3:52 PM, Jelmer Vernooij <email address hidden> wrote:
> Review: Approve code*
> Sorry, still haven't sorted out my mailing list subscriptions. I promise I'll do that for the next one and reply inline...
>
> Making the BuildBase methods static is ugly, though I can understand why you're doing it and it seems like a reasonable thing to do for the moment.
Yep - it's temporary just so there's only one implementation while I
keep refactoring (and tests continue to pass). I've updated the
BRANCH.TODO to reflect that (which will ensure the code doesn't land
until it is fixed).
> Some minor issues:
>
> The various versions of _set_build_farm_job have an extra heading space in their docstring.
Fixed.
>
> The extra_info parameter to IPackageBuild.
Fixed.
>
> Also, this code will unfortunately conflict once my simplify-
Yeah, that's ok. I'm expecting to be resolving conflicts for the next while :)
Incremental diff attached. And I'd already replaced
flush_database_
Thanks!
1 | === modified file 'BRANCH.TODO' | |||
2 | --- BRANCH.TODO 2010-04-13 19:17:03 +0000 | |||
3 | +++ BRANCH.TODO 2010-04-29 15:10:20 +0000 | |||
4 | @@ -2,3 +2,7 @@ | |||
5 | 2 | # landing. There is a test to ensure it is empty in trunk. If there is | 2 | # landing. There is a test to ensure it is empty in trunk. If there is |
6 | 3 | # stuff still here when you are ready to land, the items should probably | 3 | # stuff still here when you are ready to land, the items should probably |
7 | 4 | # be converted to bugs so they can be scheduled. | 4 | # be converted to bugs so they can be scheduled. |
8 | 5 | |||
9 | 6 | The static methods on IBuildFarmJob/IPackageBuild should be reverted | ||
10 | 7 | to normal methods once all *Build classes have transitioned to the new | ||
11 | 8 | model and IBuildBase is removed. | ||
12 | 5 | 9 | ||
13 | === modified file 'lib/lp/buildmaster/interfaces/packagebuild.py' | |||
14 | --- lib/lp/buildmaster/interfaces/packagebuild.py 2010-04-29 10:56:03 +0000 | |||
15 | +++ lib/lp/buildmaster/interfaces/packagebuild.py 2010-04-29 15:18:45 +0000 | |||
16 | @@ -130,7 +130,13 @@ | |||
17 | 130 | """ | 130 | """ |
18 | 131 | 131 | ||
19 | 132 | def notify(extra_info=None): | 132 | def notify(extra_info=None): |
21 | 133 | """Notify current build state to related people via email.""" | 133 | """Notify current build state to related people via email. |
22 | 134 | |||
23 | 135 | :param extra_info: Optional extra information that will be included | ||
24 | 136 | in the notification email. If the notification is for a | ||
25 | 137 | failed-to-upload error then this must be the content of the | ||
26 | 138 | upload log. | ||
27 | 139 | """ | ||
28 | 134 | 140 | ||
29 | 135 | 141 | ||
30 | 136 | class IPackageBuildSource(Interface): | 142 | class IPackageBuildSource(Interface): |
31 | 137 | 143 | ||
32 | === modified file 'lib/lp/code/model/sourcepackagerecipebuild.py' | |||
33 | --- lib/lp/code/model/sourcepackagerecipebuild.py 2010-04-28 08:32:10 +0000 | |||
34 | +++ lib/lp/code/model/sourcepackagerecipebuild.py 2010-04-29 15:10:58 +0000 | |||
35 | @@ -223,7 +223,7 @@ | |||
36 | 223 | super(SourcePackageRecipeBuildJob, self).__init__() | 223 | super(SourcePackageRecipeBuildJob, self).__init__() |
37 | 224 | 224 | ||
38 | 225 | def _set_build_farm_job(self): | 225 | def _set_build_farm_job(self): |
40 | 226 | """ Setup the IBuildFarmJob delegate. | 226 | """Setup the IBuildFarmJob delegate. |
41 | 227 | 227 | ||
42 | 228 | We override this to provide a delegate specific to package builds.""" | 228 | We override this to provide a delegate specific to package builds.""" |
43 | 229 | self.build_farm_job = PackageBuild(self.build) | 229 | self.build_farm_job = PackageBuild(self.build) |
44 | 230 | 230 | ||
45 | === modified file 'lib/lp/soyuz/model/buildpackagejob.py' | |||
46 | --- lib/lp/soyuz/model/buildpackagejob.py 2010-04-28 08:32:10 +0000 | |||
47 | +++ lib/lp/soyuz/model/buildpackagejob.py 2010-04-29 15:11:32 +0000 | |||
48 | @@ -46,7 +46,7 @@ | |||
49 | 46 | super(BuildPackageJob, self).__init__() | 46 | super(BuildPackageJob, self).__init__() |
50 | 47 | 47 | ||
51 | 48 | def _set_build_farm_job(self): | 48 | def _set_build_farm_job(self): |
53 | 49 | """ Setup the IBuildFarmJob delegate. | 49 | """Setup the IBuildFarmJob delegate. |
54 | 50 | 50 | ||
55 | 51 | We override this to provide a delegate specific to package builds.""" | 51 | We override this to provide a delegate specific to package builds.""" |
56 | 52 | self.build_farm_job = PackageBuild(self.build) | 52 | self.build_farm_job = PackageBuild(self.build) |
57 | 53 | 53 | ||
58 | === modified file 'lib/lp/translations/model/translationtemplatesbuildjob.py' | |||
59 | --- lib/lp/translations/model/translationtemplatesbuildjob.py 2010-04-28 10:59:12 +0000 | |||
60 | +++ lib/lp/translations/model/translationtemplatesbuildjob.py 2010-04-29 15:11:07 +0000 | |||
61 | @@ -49,7 +49,7 @@ | |||
62 | 49 | super(TranslationTemplatesBuildJob, self).__init__(branch_job) | 49 | super(TranslationTemplatesBuildJob, self).__init__(branch_job) |
63 | 50 | 50 | ||
64 | 51 | def _set_build_farm_job(self): | 51 | def _set_build_farm_job(self): |
66 | 52 | """ Setup the IBuildFarmJob delegate. | 52 | """Setup the IBuildFarmJob delegate. |
67 | 53 | 53 | ||
68 | 54 | We override this to provide a non-database delegate that simply | 54 | We override this to provide a non-database delegate that simply |
69 | 55 | provides required functionality to the queue system.""" | 55 | provides required functionality to the queue system.""" |
Preview Diff
1 | === modified file 'BRANCH.TODO' | |||
2 | --- BRANCH.TODO 2010-04-13 19:17:03 +0000 | |||
3 | +++ BRANCH.TODO 2010-05-05 15:49:24 +0000 | |||
4 | @@ -2,3 +2,7 @@ | |||
5 | 2 | # landing. There is a test to ensure it is empty in trunk. If there is | 2 | # landing. There is a test to ensure it is empty in trunk. If there is |
6 | 3 | # stuff still here when you are ready to land, the items should probably | 3 | # stuff still here when you are ready to land, the items should probably |
7 | 4 | # be converted to bugs so they can be scheduled. | 4 | # be converted to bugs so they can be scheduled. |
8 | 5 | |||
9 | 6 | The static methods on IBuildFarmJob/IPackageBuild should be reverted | ||
10 | 7 | to normal methods once all *Build classes have transitioned to the new | ||
11 | 8 | model and IBuildBase is removed. | ||
12 | 5 | 9 | ||
13 | === modified file 'lib/lp/buildmaster/doc/buildfarmjob.txt' | |||
14 | --- lib/lp/buildmaster/doc/buildfarmjob.txt 2010-04-16 14:37:33 +0000 | |||
15 | +++ lib/lp/buildmaster/doc/buildfarmjob.txt 2010-05-05 15:49:24 +0000 | |||
16 | @@ -1,7 +1,8 @@ | |||
17 | 1 | BuildFarmJob | 1 | BuildFarmJob |
18 | 2 | ============ | 2 | ============ |
19 | 3 | 3 | ||
21 | 4 | >>> from lp.buildmaster.interfaces.buildfarmjob import IBuildFarmJob | 4 | >>> from lp.buildmaster.interfaces.buildfarmjob import ( |
22 | 5 | ... BuildFarmJobType, IBuildFarmJob) | ||
23 | 5 | >>> from lp.buildmaster.model.buildfarmjob import ( | 6 | >>> from lp.buildmaster.model.buildfarmjob import ( |
24 | 6 | ... BuildFarmJob, BuildFarmJobDerived) | 7 | ... BuildFarmJob, BuildFarmJobDerived) |
25 | 7 | 8 | ||
26 | @@ -9,12 +10,14 @@ | |||
27 | 9 | specific build farm job classes will automatically delegate to | 10 | specific build farm job classes will automatically delegate to |
28 | 10 | BuildFarmJob by inheriting BuildFarmJobDerived. | 11 | BuildFarmJob by inheriting BuildFarmJobDerived. |
29 | 11 | 12 | ||
31 | 12 | >>> buildfarmjob = BuildFarmJob() | 13 | >>> buildfarmjob = BuildFarmJob(BuildFarmJobType.PACKAGEBUILD) |
32 | 13 | >>> verifyObject(IBuildFarmJob, buildfarmjob) | 14 | >>> verifyObject(IBuildFarmJob, buildfarmjob) |
33 | 14 | True | 15 | True |
34 | 15 | 16 | ||
37 | 16 | The BuildFarmJob class does not (yet) do a lot itself, other than | 17 | The BuildFarmJob class provides many of the common attributes used by |
38 | 17 | providing default implementations for its methods. | 18 | different types of build farm jobs, as well as default implementations |
39 | 19 | for the common methods. The BuildFarmJob class provides default | ||
40 | 20 | implementations for many build-related methods. | ||
41 | 18 | 21 | ||
42 | 19 | >>> print buildfarmjob.getLogFileName() | 22 | >>> print buildfarmjob.getLogFileName() |
43 | 20 | buildlog.txt | 23 | buildlog.txt |
44 | @@ -24,6 +27,7 @@ | |||
45 | 24 | ... | 27 | ... |
46 | 25 | NotImplementedError | 28 | NotImplementedError |
47 | 26 | 29 | ||
48 | 30 | For more details, please refer to lp.buildmaster.tests.test_buildfarmjob. | ||
49 | 27 | 31 | ||
50 | 28 | BuildFarmJobDerived | 32 | BuildFarmJobDerived |
51 | 29 | =================== | 33 | =================== |
52 | @@ -40,4 +44,3 @@ | |||
53 | 40 | instance of a specific build-farm job implementation associated with a | 44 | instance of a specific build-farm job implementation associated with a |
54 | 41 | given Job instance which must be called in the context of a concrete | 45 | given Job instance which must be called in the context of a concrete |
55 | 42 | derived class. See lp.buildmaster.tests.test_buildqueue for examples. | 46 | derived class. See lp.buildmaster.tests.test_buildqueue for examples. |
56 | 43 | |||
57 | 44 | 47 | ||
58 | === modified file 'lib/lp/buildmaster/interfaces/buildbase.py' | |||
59 | --- lib/lp/buildmaster/interfaces/buildbase.py 2010-05-05 15:49:22 +0000 | |||
60 | +++ lib/lp/buildmaster/interfaces/buildbase.py 2010-05-05 15:49:24 +0000 | |||
61 | @@ -228,7 +228,7 @@ | |||
62 | 228 | >>>>>>> MERGE-SOURCE | 228 | >>>>>>> MERGE-SOURCE |
63 | 229 | """ | 229 | """ |
64 | 230 | 230 | ||
66 | 231 | def handleStatus(status, librarian, slave_status): | 231 | def handleStatus(build, status, librarian, slave_status): |
67 | 232 | """Handle a finished build status from a slave. | 232 | """Handle a finished build status from a slave. |
68 | 233 | 233 | ||
69 | 234 | :param status: Slave build status string with 'BuildStatus.' stripped. | 234 | :param status: Slave build status string with 'BuildStatus.' stripped. |
70 | 235 | 235 | ||
71 | === modified file 'lib/lp/buildmaster/interfaces/packagebuild.py' | |||
72 | --- lib/lp/buildmaster/interfaces/packagebuild.py 2010-05-05 15:49:22 +0000 | |||
73 | +++ lib/lp/buildmaster/interfaces/packagebuild.py 2010-05-05 15:49:24 +0000 | |||
74 | @@ -6,6 +6,7 @@ | |||
75 | 6 | __all__ = [ | 6 | __all__ = [ |
76 | 7 | 'IPackageBuild', | 7 | 'IPackageBuild', |
77 | 8 | 'IPackageBuildSource', | 8 | 'IPackageBuildSource', |
78 | 9 | 'IPackageBuildDerived', | ||
79 | 9 | ] | 10 | ] |
80 | 10 | 11 | ||
81 | 11 | 12 | ||
82 | @@ -118,6 +119,25 @@ | |||
83 | 118 | custom status handlers, but it should not be called externally. | 119 | custom status handlers, but it should not be called externally. |
84 | 119 | """ | 120 | """ |
85 | 120 | 121 | ||
86 | 122 | def verifySuccessfulUpload(): | ||
87 | 123 | """Verify that the upload of this build completed succesfully.""" | ||
88 | 124 | |||
89 | 125 | def storeUploadLog(content): | ||
90 | 126 | """Store the given content as the build upload_log. | ||
91 | 127 | |||
92 | 128 | :param content: string containing the upload-processor log output for | ||
93 | 129 | the binaries created in this build. | ||
94 | 130 | """ | ||
95 | 131 | |||
96 | 132 | def notify(extra_info=None): | ||
97 | 133 | """Notify current build state to related people via email. | ||
98 | 134 | |||
99 | 135 | :param extra_info: Optional extra information that will be included | ||
100 | 136 | in the notification email. If the notification is for a | ||
101 | 137 | failed-to-upload error then this must be the content of the | ||
102 | 138 | upload log. | ||
103 | 139 | """ | ||
104 | 140 | |||
105 | 121 | 141 | ||
106 | 122 | class IPackageBuildSource(Interface): | 142 | class IPackageBuildSource(Interface): |
107 | 123 | """A utility of this interface used to create _things_.""" | 143 | """A utility of this interface used to create _things_.""" |
108 | @@ -129,3 +149,15 @@ | |||
109 | 129 | :param pocket: An item of `PackagePublishingPocket`. | 149 | :param pocket: An item of `PackagePublishingPocket`. |
110 | 130 | :param dependencies: An optional debian-like dependency line. | 150 | :param dependencies: An optional debian-like dependency line. |
111 | 131 | """ | 151 | """ |
112 | 152 | |||
113 | 153 | |||
114 | 154 | class IPackageBuildDerived(Interface): | ||
115 | 155 | """Classes deriving from IPackageBuild inherit the default handleStatus. | ||
116 | 156 | """ | ||
117 | 157 | def handleStatus(status, librarian, slave_status): | ||
118 | 158 | """Handle a finished build status from a slave. | ||
119 | 159 | |||
120 | 160 | :param status: Slave build status string with 'BuildStatus.' stripped. | ||
121 | 161 | :param slave_status: A dict as returned by IBuilder.slaveStatus | ||
122 | 162 | """ | ||
123 | 163 | |||
124 | 132 | 164 | ||
125 | === modified file 'lib/lp/buildmaster/model/buildbase.py' | |||
126 | --- lib/lp/buildmaster/model/buildbase.py 2010-05-05 15:49:22 +0000 | |||
127 | +++ lib/lp/buildmaster/model/buildbase.py 2010-05-05 15:49:24 +0000 | |||
128 | @@ -125,20 +125,31 @@ | |||
129 | 125 | return None | 125 | return None |
130 | 126 | return self._getProxiedFileURL(self.upload_log) | 126 | return self._getProxiedFileURL(self.upload_log) |
131 | 127 | 127 | ||
133 | 128 | def handleStatus(self, status, librarian, slave_status): | 128 | @staticmethod |
134 | 129 | def handleStatus(build, status, librarian, slave_status): | ||
135 | 129 | """See `IBuildBase`.""" | 130 | """See `IBuildBase`.""" |
136 | 130 | logger = logging.getLogger(BUILDD_MANAGER_LOG_NAME) | 131 | logger = logging.getLogger(BUILDD_MANAGER_LOG_NAME) |
137 | 131 | 132 | ||
139 | 132 | method = getattr(self, '_handleStatus_' + status, None) | 133 | method = getattr(build, '_handleStatus_' + status, None) |
140 | 133 | 134 | ||
141 | 134 | if method is None: | 135 | if method is None: |
142 | 136 | <<<<<<< TREE | ||
143 | 135 | logger.critical("Unknown BuildStatus '%s' for builder '%s'" | 137 | logger.critical("Unknown BuildStatus '%s' for builder '%s'" |
144 | 136 | % (status, self.buildqueue_record.builder.url)) | 138 | % (status, self.buildqueue_record.builder.url)) |
145 | 139 | ======= | ||
146 | 140 | if build.buildqueue_record is not None: | ||
147 | 141 | logger.critical("Unknown BuildStatus '%s' for builder '%s'" | ||
148 | 142 | % (status, build.buildqueue_record.builder.url)) | ||
149 | 143 | else: | ||
150 | 144 | logger.critical("Unknown BuildStatus '%s' for %r" | ||
151 | 145 | % (status, build)) | ||
152 | 146 | >>>>>>> MERGE-SOURCE | ||
153 | 137 | return | 147 | return |
154 | 138 | 148 | ||
156 | 139 | method(librarian, slave_status, logger) | 149 | method(build, librarian, slave_status, logger) |
157 | 140 | 150 | ||
158 | 141 | <<<<<<< TREE | 151 | <<<<<<< TREE |
159 | 152 | def _handleStatus_OK(self, librarian, slave_status, logger): | ||
160 | 142 | ======= | 153 | ======= |
161 | 143 | def processUpload(self, leaf, root, logger): | 154 | def processUpload(self, leaf, root, logger): |
162 | 144 | """Process an upload. | 155 | """Process an upload. |
163 | @@ -168,8 +179,9 @@ | |||
164 | 168 | ztm=ZopelessTransactionManager, log=logger) | 179 | ztm=ZopelessTransactionManager, log=logger) |
165 | 169 | processor.processUploadQueue(leaf) | 180 | processor.processUploadQueue(leaf) |
166 | 170 | 181 | ||
167 | 182 | @staticmethod | ||
168 | 183 | def _handleStatus_OK(build, librarian, slave_status, logger): | ||
169 | 171 | >>>>>>> MERGE-SOURCE | 184 | >>>>>>> MERGE-SOURCE |
170 | 172 | def _handleStatus_OK(self, librarian, slave_status, logger): | ||
171 | 173 | """Handle a package that built successfully. | 185 | """Handle a package that built successfully. |
172 | 174 | 186 | ||
173 | 175 | Once built successfully, we pull the files, store them in a | 187 | Once built successfully, we pull the files, store them in a |
174 | @@ -179,36 +191,41 @@ | |||
175 | 179 | filemap = slave_status['filemap'] | 191 | filemap = slave_status['filemap'] |
176 | 180 | 192 | ||
177 | 181 | logger.info("Processing successful build %s from builder %s" % ( | 193 | logger.info("Processing successful build %s from builder %s" % ( |
180 | 182 | self.buildqueue_record.specific_job.build.title, | 194 | build.buildqueue_record.specific_job.build.title, |
181 | 183 | self.buildqueue_record.builder.name)) | 195 | build.buildqueue_record.builder.name)) |
182 | 184 | # Explode before collect a binary that is denied in this | 196 | # Explode before collect a binary that is denied in this |
183 | 185 | # distroseries/pocket | 197 | # distroseries/pocket |
186 | 186 | if not self.archive.allowUpdatesToReleasePocket(): | 198 | if not build.archive.allowUpdatesToReleasePocket(): |
187 | 187 | assert self.distroseries.canUploadToPocket(self.pocket), ( | 199 | assert build.distroseries.canUploadToPocket(build.pocket), ( |
188 | 188 | "%s (%s) can not be built for pocket %s: illegal status" | 200 | "%s (%s) can not be built for pocket %s: illegal status" |
190 | 189 | % (self.title, self.id, self.pocket.name)) | 201 | % (build.title, build.id, build.pocket.name)) |
191 | 190 | 202 | ||
192 | 191 | # ensure we have the correct build root as: | 203 | # ensure we have the correct build root as: |
193 | 192 | # <BUILDMASTER_ROOT>/incoming/<UPLOAD_LEAF>/<TARGET_PATH>/[FILES] | 204 | # <BUILDMASTER_ROOT>/incoming/<UPLOAD_LEAF>/<TARGET_PATH>/[FILES] |
194 | 193 | root = os.path.abspath(config.builddmaster.root) | 205 | root = os.path.abspath(config.builddmaster.root) |
195 | 194 | 206 | ||
196 | 195 | # create a single directory to store build result files | 207 | # create a single directory to store build result files |
200 | 196 | upload_leaf = self.getUploadDirLeaf( | 208 | upload_leaf = build.getUploadDirLeaf( |
201 | 197 | '%s-%s' % (self.id, self.buildqueue_record.id)) | 209 | '%s-%s' % (build.id, build.buildqueue_record.id)) |
202 | 198 | upload_dir = self.getUploadDir(upload_leaf) | 210 | upload_dir = build.getUploadDir(upload_leaf) |
203 | 199 | logger.debug("Storing build result at '%s'" % upload_dir) | 211 | logger.debug("Storing build result at '%s'" % upload_dir) |
204 | 200 | 212 | ||
205 | 201 | # Build the right UPLOAD_PATH so the distribution and archive | 213 | # Build the right UPLOAD_PATH so the distribution and archive |
206 | 202 | # can be correctly found during the upload: | 214 | # can be correctly found during the upload: |
207 | 203 | # <archive_id>/distribution_name | 215 | # <archive_id>/distribution_name |
208 | 204 | # for all destination archive types. | 216 | # for all destination archive types. |
209 | 217 | <<<<<<< TREE | ||
210 | 205 | archive = self.archive | 218 | archive = self.archive |
211 | 206 | distribution_name = self.distribution.name | 219 | distribution_name = self.distribution.name |
212 | 207 | target_path = '%s/%s' % (archive.id, distribution_name) | 220 | target_path = '%s/%s' % (archive.id, distribution_name) |
213 | 208 | upload_path = os.path.join(upload_dir, target_path) | 221 | upload_path = os.path.join(upload_dir, target_path) |
214 | 222 | ======= | ||
215 | 223 | upload_path = os.path.join(upload_dir, str(build.archive.id), | ||
216 | 224 | build.distribution.name) | ||
217 | 225 | >>>>>>> MERGE-SOURCE | ||
218 | 209 | os.makedirs(upload_path) | 226 | os.makedirs(upload_path) |
219 | 210 | 227 | ||
221 | 211 | slave = removeSecurityProxy(self.buildqueue_record.builder.slave) | 228 | slave = removeSecurityProxy(build.buildqueue_record.builder.slave) |
222 | 212 | successful_copy_from_slave = True | 229 | successful_copy_from_slave = True |
223 | 213 | for filename in filemap: | 230 | for filename in filemap: |
224 | 214 | logger.info("Grabbing file: %s" % filename) | 231 | logger.info("Grabbing file: %s" % filename) |
225 | @@ -220,7 +237,7 @@ | |||
226 | 220 | successful_copy_from_slave = False | 237 | successful_copy_from_slave = False |
227 | 221 | logger.warning( | 238 | logger.warning( |
228 | 222 | "A slave tried to upload the file '%s' " | 239 | "A slave tried to upload the file '%s' " |
230 | 223 | "for the build %d." % (filename, self.id)) | 240 | "for the build %d." % (filename, build.id)) |
231 | 224 | break | 241 | break |
232 | 225 | out_file = open(out_file_name, "wb") | 242 | out_file = open(out_file_name, "wb") |
233 | 226 | slave_file = slave.getFile(filemap[filename]) | 243 | slave_file = slave.getFile(filemap[filename]) |
234 | @@ -229,6 +246,7 @@ | |||
235 | 229 | # We only attempt the upload if we successfully copied all the | 246 | # We only attempt the upload if we successfully copied all the |
236 | 230 | # files from the slave. | 247 | # files from the slave. |
237 | 231 | if successful_copy_from_slave: | 248 | if successful_copy_from_slave: |
238 | 249 | <<<<<<< TREE | ||
239 | 232 | uploader_logfilename = os.path.join( | 250 | uploader_logfilename = os.path.join( |
240 | 233 | upload_dir, UPLOAD_LOG_FILENAME) | 251 | upload_dir, UPLOAD_LOG_FILENAME) |
241 | 234 | uploader_command = self.getUploaderCommand( | 252 | uploader_command = self.getUploaderCommand( |
242 | @@ -251,6 +269,14 @@ | |||
243 | 251 | # to not return error, it only happen when the code is broken). | 269 | # to not return error, it only happen when the code is broken). |
244 | 252 | uploader_result_code = uploader_process.returncode | 270 | uploader_result_code = uploader_process.returncode |
245 | 253 | logger.info("Uploader returned %d" % uploader_result_code) | 271 | logger.info("Uploader returned %d" % uploader_result_code) |
246 | 272 | ======= | ||
247 | 273 | logger.info("Invoking uploader on %s for %s" % (root, upload_leaf)) | ||
248 | 274 | upload_logger = BufferLogger() | ||
249 | 275 | upload_log = build.processUpload(upload_leaf, root, upload_logger) | ||
250 | 276 | uploader_log_content = upload_logger.buffer.getvalue() | ||
251 | 277 | else: | ||
252 | 278 | uploader_log_content = 'Copy from slave was unsuccessful.' | ||
253 | 279 | >>>>>>> MERGE-SOURCE | ||
254 | 254 | 280 | ||
255 | 255 | # Quick and dirty hack to carry on on process-upload failures | 281 | # Quick and dirty hack to carry on on process-upload failures |
256 | 256 | if os.path.exists(upload_dir): | 282 | if os.path.exists(upload_dir): |
257 | @@ -290,7 +316,7 @@ | |||
258 | 290 | 316 | ||
259 | 291 | # Store build information, build record was already updated during | 317 | # Store build information, build record was already updated during |
260 | 292 | # the binary upload. | 318 | # the binary upload. |
262 | 293 | self.storeBuildInfo(self, librarian, slave_status) | 319 | build.storeBuildInfo(build, librarian, slave_status) |
263 | 294 | 320 | ||
264 | 295 | # Retrive the up-to-date build record and perform consistency | 321 | # Retrive the up-to-date build record and perform consistency |
265 | 296 | # checks. The build record should be updated during the binary | 322 | # checks. The build record should be updated during the binary |
266 | @@ -305,56 +331,65 @@ | |||
267 | 305 | # uploader about this occurrence. The failure notification will | 331 | # uploader about this occurrence. The failure notification will |
268 | 306 | # also contain the information required to manually reprocess the | 332 | # also contain the information required to manually reprocess the |
269 | 307 | # binary upload when it was the case. | 333 | # binary upload when it was the case. |
271 | 308 | if (self.buildstate != BuildStatus.FULLYBUILT or | 334 | if (build.buildstate != BuildStatus.FULLYBUILT or |
272 | 309 | not successful_copy_from_slave or | 335 | not successful_copy_from_slave or |
273 | 336 | <<<<<<< TREE | ||
274 | 310 | not self.verifySuccessfulUpload()): | 337 | not self.verifySuccessfulUpload()): |
275 | 311 | logger.warning("Build %s upload failed." % self.id) | 338 | logger.warning("Build %s upload failed." % self.id) |
276 | 312 | self.buildstate = BuildStatus.FAILEDTOUPLOAD | 339 | self.buildstate = BuildStatus.FAILEDTOUPLOAD |
277 | 313 | uploader_log_content = self.getUploadLogContent(root, | 340 | uploader_log_content = self.getUploadLogContent(root, |
278 | 314 | upload_leaf) | 341 | upload_leaf) |
279 | 342 | ======= | ||
280 | 343 | not build.verifySuccessfulUpload()): | ||
281 | 344 | logger.warning("Build %s upload failed." % build.id) | ||
282 | 345 | build.buildstate = BuildStatus.FAILEDTOUPLOAD | ||
283 | 346 | >>>>>>> MERGE-SOURCE | ||
284 | 315 | # Store the upload_log_contents in librarian so it can be | 347 | # Store the upload_log_contents in librarian so it can be |
285 | 316 | # accessed by anyone with permission to see the build. | 348 | # accessed by anyone with permission to see the build. |
287 | 317 | self.storeUploadLog(uploader_log_content) | 349 | build.storeUploadLog(uploader_log_content) |
288 | 318 | # Notify the build failure. | 350 | # Notify the build failure. |
290 | 319 | self.notify(extra_info=uploader_log_content) | 351 | build.notify(extra_info=uploader_log_content) |
291 | 320 | else: | 352 | else: |
292 | 321 | logger.info( | 353 | logger.info( |
293 | 322 | "Gathered %s %d completely" % ( | 354 | "Gathered %s %d completely" % ( |
295 | 323 | self.__class__.__name__, self.id)) | 355 | build.__class__.__name__, build.id)) |
296 | 324 | 356 | ||
297 | 325 | # Release the builder for another job. | 357 | # Release the builder for another job. |
299 | 326 | self.buildqueue_record.builder.cleanSlave() | 358 | build.buildqueue_record.builder.cleanSlave() |
300 | 327 | # Remove BuildQueue record. | 359 | # Remove BuildQueue record. |
302 | 328 | self.buildqueue_record.destroySelf() | 360 | build.buildqueue_record.destroySelf() |
303 | 329 | 361 | ||
305 | 330 | def _handleStatus_PACKAGEFAIL(self, librarian, slave_status, logger): | 362 | @staticmethod |
306 | 363 | def _handleStatus_PACKAGEFAIL(build, librarian, slave_status, logger): | ||
307 | 331 | """Handle a package that had failed to build. | 364 | """Handle a package that had failed to build. |
308 | 332 | 365 | ||
309 | 333 | Build has failed when trying the work with the target package, | 366 | Build has failed when trying the work with the target package, |
310 | 334 | set the job status as FAILEDTOBUILD, store available info and | 367 | set the job status as FAILEDTOBUILD, store available info and |
311 | 335 | remove Buildqueue entry. | 368 | remove Buildqueue entry. |
312 | 336 | """ | 369 | """ |
318 | 337 | self.buildstate = BuildStatus.FAILEDTOBUILD | 370 | build.buildstate = BuildStatus.FAILEDTOBUILD |
319 | 338 | self.storeBuildInfo(self, librarian, slave_status) | 371 | build.storeBuildInfo(build, librarian, slave_status) |
320 | 339 | self.buildqueue_record.builder.cleanSlave() | 372 | build.buildqueue_record.builder.cleanSlave() |
321 | 340 | self.notify() | 373 | build.notify() |
322 | 341 | self.buildqueue_record.destroySelf() | 374 | build.buildqueue_record.destroySelf() |
323 | 342 | 375 | ||
325 | 343 | def _handleStatus_DEPFAIL(self, librarian, slave_status, logger): | 376 | @staticmethod |
326 | 377 | def _handleStatus_DEPFAIL(build, librarian, slave_status, logger): | ||
327 | 344 | """Handle a package that had missing dependencies. | 378 | """Handle a package that had missing dependencies. |
328 | 345 | 379 | ||
329 | 346 | Build has failed by missing dependencies, set the job status as | 380 | Build has failed by missing dependencies, set the job status as |
330 | 347 | MANUALDEPWAIT, store available information, remove BuildQueue | 381 | MANUALDEPWAIT, store available information, remove BuildQueue |
331 | 348 | entry and release builder slave for another job. | 382 | entry and release builder slave for another job. |
332 | 349 | """ | 383 | """ |
335 | 350 | self.buildstate = BuildStatus.MANUALDEPWAIT | 384 | build.buildstate = BuildStatus.MANUALDEPWAIT |
336 | 351 | self.storeBuildInfo(self, librarian, slave_status) | 385 | build.storeBuildInfo(build, librarian, slave_status) |
337 | 352 | logger.critical("***** %s is MANUALDEPWAIT *****" | 386 | logger.critical("***** %s is MANUALDEPWAIT *****" |
341 | 353 | % self.buildqueue_record.builder.name) | 387 | % build.buildqueue_record.builder.name) |
342 | 354 | self.buildqueue_record.builder.cleanSlave() | 388 | build.buildqueue_record.builder.cleanSlave() |
343 | 355 | self.buildqueue_record.destroySelf() | 389 | build.buildqueue_record.destroySelf() |
344 | 356 | 390 | ||
346 | 357 | def _handleStatus_CHROOTFAIL(self, librarian, slave_status, | 391 | @staticmethod |
347 | 392 | def _handleStatus_CHROOTFAIL(build, librarian, slave_status, | ||
348 | 358 | logger): | 393 | logger): |
349 | 359 | """Handle a package that had failed when unpacking the CHROOT. | 394 | """Handle a package that had failed when unpacking the CHROOT. |
350 | 360 | 395 | ||
351 | @@ -362,15 +397,16 @@ | |||
352 | 362 | job as CHROOTFAIL, store available information, remove BuildQueue | 397 | job as CHROOTFAIL, store available information, remove BuildQueue |
353 | 363 | and release the builder. | 398 | and release the builder. |
354 | 364 | """ | 399 | """ |
357 | 365 | self.buildstate = BuildStatus.CHROOTWAIT | 400 | build.buildstate = BuildStatus.CHROOTWAIT |
358 | 366 | self.storeBuildInfo(self, librarian, slave_status) | 401 | build.storeBuildInfo(build, librarian, slave_status) |
359 | 367 | logger.critical("***** %s is CHROOTWAIT *****" % | 402 | logger.critical("***** %s is CHROOTWAIT *****" % |
364 | 368 | self.buildqueue_record.builder.name) | 403 | build.buildqueue_record.builder.name) |
365 | 369 | self.buildqueue_record.builder.cleanSlave() | 404 | build.buildqueue_record.builder.cleanSlave() |
366 | 370 | self.notify() | 405 | build.notify() |
367 | 371 | self.buildqueue_record.destroySelf() | 406 | build.buildqueue_record.destroySelf() |
368 | 372 | 407 | ||
370 | 373 | def _handleStatus_BUILDERFAIL(self, librarian, slave_status, logger): | 408 | @staticmethod |
371 | 409 | def _handleStatus_BUILDERFAIL(build, librarian, slave_status, logger): | ||
372 | 374 | """Handle builder failures. | 410 | """Handle builder failures. |
373 | 375 | 411 | ||
374 | 376 | Build has been failed when trying to build the target package, | 412 | Build has been failed when trying to build the target package, |
375 | @@ -378,14 +414,15 @@ | |||
376 | 378 | and 'clean' the builder to do another jobs. | 414 | and 'clean' the builder to do another jobs. |
377 | 379 | """ | 415 | """ |
378 | 380 | logger.warning("***** %s has failed *****" | 416 | logger.warning("***** %s has failed *****" |
381 | 381 | % self.buildqueue_record.builder.name) | 417 | % build.buildqueue_record.builder.name) |
382 | 382 | self.buildqueue_record.builder.failBuilder( | 418 | build.buildqueue_record.builder.failBuilder( |
383 | 383 | "Builder returned BUILDERFAIL when asked for its status") | 419 | "Builder returned BUILDERFAIL when asked for its status") |
384 | 384 | # simply reset job | 420 | # simply reset job |
387 | 385 | self.storeBuildInfo(self, librarian, slave_status) | 421 | build.storeBuildInfo(build, librarian, slave_status) |
388 | 386 | self.buildqueue_record.reset() | 422 | build.buildqueue_record.reset() |
389 | 387 | 423 | ||
391 | 388 | def _handleStatus_GIVENBACK(self, librarian, slave_status, logger): | 424 | @staticmethod |
392 | 425 | def _handleStatus_GIVENBACK(build, librarian, slave_status, logger): | ||
393 | 389 | """Handle automatic retry requested by builder. | 426 | """Handle automatic retry requested by builder. |
394 | 390 | 427 | ||
395 | 391 | GIVENBACK pseudo-state represents a request for automatic retry | 428 | GIVENBACK pseudo-state represents a request for automatic retry |
396 | @@ -393,15 +430,15 @@ | |||
397 | 393 | ZERO. | 430 | ZERO. |
398 | 394 | """ | 431 | """ |
399 | 395 | logger.warning("***** %s is GIVENBACK by %s *****" | 432 | logger.warning("***** %s is GIVENBACK by %s *****" |
403 | 396 | % (self.buildqueue_record.specific_job.build.title, | 433 | % (build.buildqueue_record.specific_job.build.title, |
404 | 397 | self.buildqueue_record.builder.name)) | 434 | build.buildqueue_record.builder.name)) |
405 | 398 | self.storeBuildInfo(self, librarian, slave_status) | 435 | build.storeBuildInfo(build, librarian, slave_status) |
406 | 399 | # XXX cprov 2006-05-30: Currently this information is not | 436 | # XXX cprov 2006-05-30: Currently this information is not |
407 | 400 | # properly presented in the Web UI. We will discuss it in | 437 | # properly presented in the Web UI. We will discuss it in |
408 | 401 | # the next Paris Summit, infinity has some ideas about how | 438 | # the next Paris Summit, infinity has some ideas about how |
409 | 402 | # to use this content. For now we just ensure it's stored. | 439 | # to use this content. For now we just ensure it's stored. |
412 | 403 | self.buildqueue_record.builder.cleanSlave() | 440 | build.buildqueue_record.builder.cleanSlave() |
413 | 404 | self.buildqueue_record.reset() | 441 | build.buildqueue_record.reset() |
414 | 405 | 442 | ||
415 | 406 | @staticmethod | 443 | @staticmethod |
416 | 407 | def getLogFromSlave(build): | 444 | def getLogFromSlave(build): |
417 | @@ -428,25 +465,34 @@ | |||
418 | 428 | else: | 465 | else: |
419 | 429 | build.dependencies = None | 466 | build.dependencies = None |
420 | 430 | 467 | ||
423 | 431 | def storeUploadLog(self, content): | 468 | @staticmethod |
424 | 432 | """See `IBuildBase`.""" | 469 | def createUploadLog(build, content, filename=None): |
425 | 470 | """Creates a file on the librarian for the upload log. | ||
426 | 471 | |||
427 | 472 | :return: ILibraryFileAlias for the upload log file. | ||
428 | 473 | """ | ||
429 | 433 | # The given content is stored in the librarian, restricted as | 474 | # The given content is stored in the librarian, restricted as |
430 | 434 | # necessary according to the targeted archive's privacy. The content | 475 | # necessary according to the targeted archive's privacy. The content |
431 | 435 | # object's 'upload_log' attribute will point to the | 476 | # object's 'upload_log' attribute will point to the |
432 | 436 | # `LibrarianFileAlias`. | 477 | # `LibrarianFileAlias`. |
433 | 437 | 478 | ||
435 | 438 | assert self.upload_log is None, ( | 479 | assert build.upload_log is None, ( |
436 | 439 | "Upload log information already exists and cannot be overridden.") | 480 | "Upload log information already exists and cannot be overridden.") |
437 | 440 | 481 | ||
439 | 441 | filename = 'upload_%s_log.txt' % self.id | 482 | if filename is None: |
440 | 483 | filename = 'upload_%s_log.txt' % build.id | ||
441 | 442 | contentType = filenameToContentType(filename) | 484 | contentType = filenameToContentType(filename) |
442 | 443 | file_size = len(content) | 485 | file_size = len(content) |
443 | 444 | file_content = StringIO(content) | 486 | file_content = StringIO(content) |
445 | 445 | restricted = self.is_private | 487 | restricted = build.is_private |
446 | 446 | 488 | ||
448 | 447 | library_file = getUtility(ILibraryFileAliasSet).create( | 489 | return getUtility(ILibraryFileAliasSet).create( |
449 | 448 | filename, file_size, file_content, contentType=contentType, | 490 | filename, file_size, file_content, contentType=contentType, |
450 | 449 | restricted=restricted) | 491 | restricted=restricted) |
451 | 492 | |||
452 | 493 | def storeUploadLog(self, content): | ||
453 | 494 | """See `IBuildBase`.""" | ||
454 | 495 | library_file = self.createUploadLog(self, content) | ||
455 | 450 | self.upload_log = library_file | 496 | self.upload_log = library_file |
456 | 451 | 497 | ||
457 | 452 | def queueBuild(self, suspended=False): | 498 | def queueBuild(self, suspended=False): |
458 | 453 | 499 | ||
459 | === modified file 'lib/lp/buildmaster/model/buildfarmjob.py' | |||
460 | --- lib/lp/buildmaster/model/buildfarmjob.py 2010-05-05 15:49:22 +0000 | |||
461 | +++ lib/lp/buildmaster/model/buildfarmjob.py 2010-05-05 15:49:24 +0000 | |||
462 | @@ -201,9 +201,15 @@ | |||
463 | 201 | self._set_build_farm_job() | 201 | self._set_build_farm_job() |
464 | 202 | 202 | ||
465 | 203 | def _set_build_farm_job(self): | 203 | def _set_build_farm_job(self): |
469 | 204 | """Set the build farm job to which we will delegate. | 204 | """Set the default build farm job to which we will delegate. |
470 | 205 | 205 | ||
471 | 206 | Sub-classes can override as required. | 206 | Sub-classes should override as required. |
472 | 207 | |||
473 | 208 | XXX 2010-04-27 michael.nelson bug=570939 | ||
474 | 209 | This only exists because certain classes assume that | ||
475 | 210 | BuildFarmJob/PackageBuild are in-memory objects that simply | ||
476 | 211 | provide methods to update the associated builds. | ||
477 | 212 | We can remove it once the above bug is completed. | ||
478 | 207 | """ | 213 | """ |
479 | 208 | self.build_farm_job = BuildFarmJob( | 214 | self.build_farm_job = BuildFarmJob( |
480 | 209 | job_type=BuildFarmJobType.PACKAGEBUILD) | 215 | job_type=BuildFarmJobType.PACKAGEBUILD) |
481 | 210 | 216 | ||
482 | === modified file 'lib/lp/buildmaster/model/buildfarmjobbehavior.py' | |||
483 | --- lib/lp/buildmaster/model/buildfarmjobbehavior.py 2010-04-12 16:23:13 +0000 | |||
484 | +++ lib/lp/buildmaster/model/buildfarmjobbehavior.py 2010-05-05 15:49:24 +0000 | |||
485 | @@ -183,8 +183,9 @@ | |||
486 | 183 | 183 | ||
487 | 184 | # XXX: dsilvers 2005-03-02: Confirm the builder has the right build? | 184 | # XXX: dsilvers 2005-03-02: Confirm the builder has the right build? |
488 | 185 | 185 | ||
491 | 186 | queueItem.specific_job.build.handleStatus( | 186 | build = queueItem.specific_job.build |
492 | 187 | build_status, librarian, slave_status) | 187 | build.handleStatus( |
493 | 188 | build, build_status, librarian, slave_status) | ||
494 | 188 | 189 | ||
495 | 189 | 190 | ||
496 | 190 | class IdleBuildBehavior(BuildFarmJobBehaviorBase): | 191 | class IdleBuildBehavior(BuildFarmJobBehaviorBase): |
497 | 191 | 192 | ||
498 | === modified file 'lib/lp/buildmaster/model/packagebuild.py' | |||
499 | --- lib/lp/buildmaster/model/packagebuild.py 2010-05-05 15:49:22 +0000 | |||
500 | +++ lib/lp/buildmaster/model/packagebuild.py 2010-05-05 15:49:24 +0000 | |||
501 | @@ -22,7 +22,7 @@ | |||
502 | 22 | from lp.buildmaster.interfaces.buildbase import BuildStatus | 22 | from lp.buildmaster.interfaces.buildbase import BuildStatus |
503 | 23 | from lp.buildmaster.interfaces.buildfarmjob import IBuildFarmJobSource | 23 | from lp.buildmaster.interfaces.buildfarmjob import IBuildFarmJobSource |
504 | 24 | from lp.buildmaster.interfaces.packagebuild import ( | 24 | from lp.buildmaster.interfaces.packagebuild import ( |
506 | 25 | IPackageBuild, IPackageBuildSource) | 25 | IPackageBuild, IPackageBuildDerived, IPackageBuildSource) |
507 | 26 | from lp.buildmaster.model.buildbase import BuildBase | 26 | from lp.buildmaster.model.buildbase import BuildBase |
508 | 27 | from lp.buildmaster.model.buildfarmjob import BuildFarmJobDerived | 27 | from lp.buildmaster.model.buildfarmjob import BuildFarmJobDerived |
509 | 28 | from lp.registry.interfaces.pocket import PackagePublishingPocket | 28 | from lp.registry.interfaces.pocket import PackagePublishingPocket |
510 | @@ -62,7 +62,7 @@ | |||
511 | 62 | def __init__(self, build): | 62 | def __init__(self, build): |
512 | 63 | """Construct a PackageBuild. | 63 | """Construct a PackageBuild. |
513 | 64 | 64 | ||
515 | 65 | XXX 2010-04-21 michael.nelson bug=536700 | 65 | XXX 2010-04-21 michael.nelson bug=570939 |
516 | 66 | This initialiser is only used by IBuildFarmJobDerived classes | 66 | This initialiser is only used by IBuildFarmJobDerived classes |
517 | 67 | that are not yet expecting a concrete BuildFarmJob (and so are | 67 | that are not yet expecting a concrete BuildFarmJob (and so are |
518 | 68 | expecting to pass in the build to which they refer, such as | 68 | expecting to pass in the build to which they refer, such as |
519 | @@ -196,10 +196,54 @@ | |||
520 | 196 | """See `IPackageBuild`.""" | 196 | """See `IPackageBuild`.""" |
521 | 197 | return BuildBase.storeBuildInfo(self, librarian, slave_status) | 197 | return BuildBase.storeBuildInfo(self, librarian, slave_status) |
522 | 198 | 198 | ||
530 | 199 | 199 | def verifySuccessfulUpload(self): | |
531 | 200 | class PackageBuildDerived(BuildFarmJobDerived): | 200 | """See `IPackageBuild`.""" |
532 | 201 | """Override the base delegate to use a build farm job specific to | 201 | raise NotImplementedError |
533 | 202 | packages. | 202 | |
534 | 203 | """ | 203 | def storeUploadLog(self, content): |
535 | 204 | def _set_build_farm_job(self): | 204 | """See `IPackageBuild`.""" |
536 | 205 | self.build_farm_job = PackageBuild(self.build) | 205 | filename = "upload_%s_log.txt" % self.build_farm_job.id |
537 | 206 | library_file = BuildBase.createUploadLog( | ||
538 | 207 | self, content, filename=filename) | ||
539 | 208 | self.upload_log = library_file | ||
540 | 209 | |||
541 | 210 | def notify(self, extra_info=None): | ||
542 | 211 | """See `IPackageBuild`.""" | ||
543 | 212 | raise NotImplementedError | ||
544 | 213 | |||
545 | 214 | |||
546 | 215 | class PackageBuildDerived: | ||
547 | 216 | """See `IPackageBuildDerived`.""" | ||
548 | 217 | implements(IPackageBuildDerived) | ||
549 | 218 | |||
550 | 219 | def handleStatus(self, status, librarian, slave_status): | ||
551 | 220 | """See `IPackageBuildDerived`.""" | ||
552 | 221 | return BuildBase.handleStatus(self, status, librarian, slave_status) | ||
553 | 222 | |||
554 | 223 | # The following private handlers currently re-use the BuildBase | ||
555 | 224 | # implementation until it is no longer in use. If we find in the | ||
556 | 225 | # future that it would be useful to delegate these also, they can be | ||
557 | 226 | # added to IBuildFarmJob or IPackageBuild as necessary. | ||
558 | 227 | def _handleStatus_OK(self, librarian, slave_status, logger): | ||
559 | 228 | return BuildBase._handleStatus_OK( | ||
560 | 229 | self, librarian, slave_status, logger) | ||
561 | 230 | |||
562 | 231 | def _handleStatus_PACKAGEFAIL(self, librarian, slave_status, logger): | ||
563 | 232 | return BuildBase._handleStatus_PACKAGEFAIL( | ||
564 | 233 | self, librarian, slave_status, logger) | ||
565 | 234 | |||
566 | 235 | def _handleStatus_DEPFAIL(self, librarian, slave_status, logger): | ||
567 | 236 | return BuildBase._handleStatus_DEPFAIL( | ||
568 | 237 | self, librarian, slave_status, logger) | ||
569 | 238 | |||
570 | 239 | def _handleStatus_CHROOTFAIL(self, librarian, slave_status, logger): | ||
571 | 240 | return BuildBase._handleStatus_CHROOTFAIL( | ||
572 | 241 | self, librarian, slave_status, logger) | ||
573 | 242 | |||
574 | 243 | def _handleStatus_BUILDERFAIL(self, librarian, slave_status, logger): | ||
575 | 244 | return BuildBase._handleStatus_BUILDERFAIL( | ||
576 | 245 | self, librarian, slave_status, logger) | ||
577 | 246 | |||
578 | 247 | def _handleStatus_GIVENBACK(self, librarian, slave_status, logger): | ||
579 | 248 | return BuildBase._handleStatus_GIVENBACK( | ||
580 | 249 | self, librarian, slave_status, logger) | ||
581 | 206 | 250 | ||
582 | === modified file 'lib/lp/buildmaster/tests/test_buildbase.py' | |||
583 | --- lib/lp/buildmaster/tests/test_buildbase.py 2010-05-05 15:49:22 +0000 | |||
584 | +++ lib/lp/buildmaster/tests/test_buildbase.py 2010-05-05 15:49:24 +0000 | |||
585 | @@ -192,7 +192,7 @@ | |||
586 | 192 | # A filemap with plain filenames should not cause a problem. | 192 | # A filemap with plain filenames should not cause a problem. |
587 | 193 | # The call to handleStatus will attempt to get the file from | 193 | # The call to handleStatus will attempt to get the file from |
588 | 194 | # the slave resulting in a URL error in this test case. | 194 | # the slave resulting in a URL error in this test case. |
590 | 195 | self.build.handleStatus('OK', None, { | 195 | self.build.handleStatus(self.build, 'OK', None, { |
591 | 196 | 'filemap': { 'myfile.py': 'test_file_hash'}, | 196 | 'filemap': { 'myfile.py': 'test_file_hash'}, |
592 | 197 | }) | 197 | }) |
593 | 198 | 198 | ||
594 | @@ -202,7 +202,7 @@ | |||
595 | 202 | def test_handleStatus_OK_absolute_filepath(self): | 202 | def test_handleStatus_OK_absolute_filepath(self): |
596 | 203 | # A filemap that tries to write to files outside of | 203 | # A filemap that tries to write to files outside of |
597 | 204 | # the upload directory will result in a failed upload. | 204 | # the upload directory will result in a failed upload. |
599 | 205 | self.build.handleStatus('OK', None, { | 205 | self.build.handleStatus(self.build, 'OK', None, { |
600 | 206 | 'filemap': { '/tmp/myfile.py': 'test_file_hash'}, | 206 | 'filemap': { '/tmp/myfile.py': 'test_file_hash'}, |
601 | 207 | }) | 207 | }) |
602 | 208 | self.assertEqual(BuildStatus.FAILEDTOUPLOAD, self.build.buildstate) | 208 | self.assertEqual(BuildStatus.FAILEDTOUPLOAD, self.build.buildstate) |
603 | @@ -211,7 +211,7 @@ | |||
604 | 211 | def test_handleStatus_OK_relative_filepath(self): | 211 | def test_handleStatus_OK_relative_filepath(self): |
605 | 212 | # A filemap that tries to write to files outside of | 212 | # A filemap that tries to write to files outside of |
606 | 213 | # the upload directory will result in a failed upload. | 213 | # the upload directory will result in a failed upload. |
608 | 214 | self.build.handleStatus('OK', None, { | 214 | self.build.handleStatus(self.build, 'OK', None, { |
609 | 215 | 'filemap': { '../myfile.py': 'test_file_hash'}, | 215 | 'filemap': { '../myfile.py': 'test_file_hash'}, |
610 | 216 | }) | 216 | }) |
611 | 217 | self.assertEqual(BuildStatus.FAILEDTOUPLOAD, self.build.buildstate) | 217 | self.assertEqual(BuildStatus.FAILEDTOUPLOAD, self.build.buildstate) |
612 | 218 | 218 | ||
613 | === modified file 'lib/lp/buildmaster/tests/test_packagebuild.py' | |||
614 | --- lib/lp/buildmaster/tests/test_packagebuild.py 2010-05-05 15:49:22 +0000 | |||
615 | +++ lib/lp/buildmaster/tests/test_packagebuild.py 2010-05-05 15:49:24 +0000 | |||
616 | @@ -6,12 +6,12 @@ | |||
617 | 6 | __metaclass__ = type | 6 | __metaclass__ = type |
618 | 7 | 7 | ||
619 | 8 | import unittest | 8 | import unittest |
620 | 9 | import hashlib | ||
621 | 9 | 10 | ||
622 | 10 | from storm.store import Store | 11 | from storm.store import Store |
623 | 11 | from zope.component import getUtility | 12 | from zope.component import getUtility |
624 | 12 | from zope.security.proxy import removeSecurityProxy | 13 | from zope.security.proxy import removeSecurityProxy |
625 | 13 | 14 | ||
626 | 14 | from canonical.database.sqlbase import flush_database_updates | ||
627 | 15 | from canonical.testing.layers import LaunchpadFunctionalLayer | 15 | from canonical.testing.layers import LaunchpadFunctionalLayer |
628 | 16 | 16 | ||
629 | 17 | from lp.buildmaster.interfaces.buildfarmjob import BuildFarmJobType | 17 | from lp.buildmaster.interfaces.buildfarmjob import BuildFarmJobType |
630 | @@ -74,8 +74,8 @@ | |||
631 | 74 | 74 | ||
632 | 75 | def test_saves_record(self): | 75 | def test_saves_record(self): |
633 | 76 | # A package build can be stored in the database. | 76 | # A package build can be stored in the database. |
634 | 77 | flush_database_updates() | ||
635 | 78 | store = Store.of(self.package_build) | 77 | store = Store.of(self.package_build) |
636 | 78 | store.flush() | ||
637 | 79 | retrieved_build = store.find( | 79 | retrieved_build = store.find( |
638 | 80 | PackageBuild, | 80 | PackageBuild, |
639 | 81 | PackageBuild.id == self.package_build.id).one() | 81 | PackageBuild.id == self.package_build.id).one() |
640 | @@ -86,6 +86,9 @@ | |||
641 | 86 | self.assertRaises(NotImplementedError, self.package_build.getTitle) | 86 | self.assertRaises(NotImplementedError, self.package_build.getTitle) |
642 | 87 | self.assertRaises( | 87 | self.assertRaises( |
643 | 88 | NotImplementedError, self.package_build.estimateDuration) | 88 | NotImplementedError, self.package_build.estimateDuration) |
644 | 89 | self.assertRaises( | ||
645 | 90 | NotImplementedError, self.package_build.verifySuccessfulUpload) | ||
646 | 91 | self.assertRaises(NotImplementedError, self.package_build.notify) | ||
647 | 89 | 92 | ||
648 | 90 | def test_default_values(self): | 93 | def test_default_values(self): |
649 | 91 | # PackageBuild has a number of default values. | 94 | # PackageBuild has a number of default values. |
650 | @@ -105,15 +108,25 @@ | |||
651 | 105 | self.package_build.build_farm_job.id), | 108 | self.package_build.build_farm_job.id), |
652 | 106 | log_url) | 109 | log_url) |
653 | 107 | 110 | ||
654 | 111 | def test_storeUploadLog(self): | ||
655 | 112 | # The given content is uploaded to the librarian and linked as | ||
656 | 113 | # the upload log. | ||
657 | 114 | self.package_build.storeUploadLog("Some content") | ||
658 | 115 | self.failIfEqual(None, self.package_build.upload_log) | ||
659 | 116 | self.failUnlessEqual( | ||
660 | 117 | hashlib.sha1("Some content").hexdigest(), | ||
661 | 118 | self.package_build.upload_log.content.sha1) | ||
662 | 119 | |||
663 | 108 | def test_upload_log_url(self): | 120 | def test_upload_log_url(self): |
664 | 109 | # The url of the upload log file is determined by the PackageBuild. | 121 | # The url of the upload log file is determined by the PackageBuild. |
667 | 110 | lfa = self.factory.makeLibraryFileAlias('myuploadlog.txt') | 122 | Store.of(self.package_build).flush() |
668 | 111 | removeSecurityProxy(self.package_build).upload_log = lfa | 123 | build_id = self.package_build.build_farm_job.id |
669 | 124 | self.package_build.storeUploadLog("Some content") | ||
670 | 112 | log_url = self.package_build.upload_log_url | 125 | log_url = self.package_build.upload_log_url |
671 | 113 | self.failUnlessEqual( | 126 | self.failUnlessEqual( |
672 | 114 | 'http://launchpad.dev/~joe/' | 127 | 'http://launchpad.dev/~joe/' |
675 | 115 | '+archive/ppa/+build/%d/+files/myuploadlog.txt' % ( | 128 | '+archive/ppa/+build/%d/+files/upload_%d_log.txt' % ( |
676 | 116 | self.package_build.build_farm_job.id), | 129 | build_id, build_id), |
677 | 117 | log_url) | 130 | log_url) |
678 | 118 | 131 | ||
679 | 119 | 132 | ||
680 | 120 | 133 | ||
681 | === modified file 'lib/lp/code/model/sourcepackagerecipebuild.py' | |||
682 | --- lib/lp/code/model/sourcepackagerecipebuild.py 2010-05-05 15:49:22 +0000 | |||
683 | +++ lib/lp/code/model/sourcepackagerecipebuild.py 2010-05-05 15:49:24 +0000 | |||
684 | @@ -27,8 +27,8 @@ | |||
685 | 27 | from lp.buildmaster.interfaces.buildfarmjob import BuildFarmJobType | 27 | from lp.buildmaster.interfaces.buildfarmjob import BuildFarmJobType |
686 | 28 | from lp.buildmaster.model.buildbase import BuildBase | 28 | from lp.buildmaster.model.buildbase import BuildBase |
687 | 29 | from lp.buildmaster.model.buildqueue import BuildQueue | 29 | from lp.buildmaster.model.buildqueue import BuildQueue |
690 | 30 | from lp.buildmaster.model.packagebuild import ( | 30 | from lp.buildmaster.model.buildfarmjob import BuildFarmJobDerived |
691 | 31 | PackageBuildDerived) | 31 | from lp.buildmaster.model.packagebuild import PackageBuild |
692 | 32 | from lp.code.interfaces.sourcepackagerecipebuild import ( | 32 | from lp.code.interfaces.sourcepackagerecipebuild import ( |
693 | 33 | ISourcePackageRecipeBuildJob, ISourcePackageRecipeBuildJobSource, | 33 | ISourcePackageRecipeBuildJob, ISourcePackageRecipeBuildJobSource, |
694 | 34 | ISourcePackageRecipeBuild, ISourcePackageRecipeBuildSource) | 34 | ISourcePackageRecipeBuild, ISourcePackageRecipeBuildSource) |
695 | @@ -199,7 +199,7 @@ | |||
696 | 199 | return | 199 | return |
697 | 200 | 200 | ||
698 | 201 | 201 | ||
700 | 202 | class SourcePackageRecipeBuildJob(PackageBuildDerived, Storm): | 202 | class SourcePackageRecipeBuildJob(BuildFarmJobDerived, Storm): |
701 | 203 | classProvides(ISourcePackageRecipeBuildJobSource) | 203 | classProvides(ISourcePackageRecipeBuildJobSource) |
702 | 204 | implements(ISourcePackageRecipeBuildJob) | 204 | implements(ISourcePackageRecipeBuildJob) |
703 | 205 | 205 | ||
704 | @@ -222,6 +222,12 @@ | |||
705 | 222 | self.job = job | 222 | self.job = job |
706 | 223 | super(SourcePackageRecipeBuildJob, self).__init__() | 223 | super(SourcePackageRecipeBuildJob, self).__init__() |
707 | 224 | 224 | ||
708 | 225 | def _set_build_farm_job(self): | ||
709 | 226 | """Setup the IBuildFarmJob delegate. | ||
710 | 227 | |||
711 | 228 | We override this to provide a delegate specific to package builds.""" | ||
712 | 229 | self.build_farm_job = PackageBuild(self.build) | ||
713 | 230 | |||
714 | 225 | @classmethod | 231 | @classmethod |
715 | 226 | def new(cls, build, job): | 232 | def new(cls, build, job): |
716 | 227 | """See `ISourcePackageRecipeBuildJobSource`.""" | 233 | """See `ISourcePackageRecipeBuildJobSource`.""" |
717 | 228 | 234 | ||
718 | === modified file 'lib/lp/soyuz/model/buildpackagejob.py' | |||
719 | --- lib/lp/soyuz/model/buildpackagejob.py 2010-05-05 15:49:22 +0000 | |||
720 | +++ lib/lp/soyuz/model/buildpackagejob.py 2010-05-05 15:49:24 +0000 | |||
721 | @@ -18,8 +18,8 @@ | |||
722 | 18 | from canonical.database.sqlbase import sqlvalues | 18 | from canonical.database.sqlbase import sqlvalues |
723 | 19 | 19 | ||
724 | 20 | from lp.buildmaster.interfaces.buildbase import BuildStatus | 20 | from lp.buildmaster.interfaces.buildbase import BuildStatus |
727 | 21 | from lp.buildmaster.model.packagebuild import ( | 21 | from lp.buildmaster.model.buildfarmjob import BuildFarmJobDerived |
728 | 22 | PackageBuildDerived) | 22 | from lp.buildmaster.model.packagebuild import PackageBuild |
729 | 23 | from lp.registry.interfaces.sourcepackage import SourcePackageUrgency | 23 | from lp.registry.interfaces.sourcepackage import SourcePackageUrgency |
730 | 24 | from lp.registry.interfaces.pocket import PackagePublishingPocket | 24 | from lp.registry.interfaces.pocket import PackagePublishingPocket |
731 | 25 | from lp.soyuz.interfaces.archive import ArchivePurpose | 25 | from lp.soyuz.interfaces.archive import ArchivePurpose |
732 | @@ -28,7 +28,7 @@ | |||
733 | 28 | from lp.soyuz.interfaces.publishing import PackagePublishingStatus | 28 | from lp.soyuz.interfaces.publishing import PackagePublishingStatus |
734 | 29 | 29 | ||
735 | 30 | 30 | ||
737 | 31 | class BuildPackageJob(PackageBuildDerived, Storm): | 31 | class BuildPackageJob(BuildFarmJobDerived, Storm): |
738 | 32 | """See `IBuildPackageJob`.""" | 32 | """See `IBuildPackageJob`.""" |
739 | 33 | implements(IBuildPackageJob) | 33 | implements(IBuildPackageJob) |
740 | 34 | 34 | ||
741 | @@ -42,10 +42,15 @@ | |||
742 | 42 | build = Reference(build_id, 'BinaryPackageBuild.id') | 42 | build = Reference(build_id, 'BinaryPackageBuild.id') |
743 | 43 | 43 | ||
744 | 44 | def __init__(self, build, job): | 44 | def __init__(self, build, job): |
745 | 45 | """ Setup the IBuildFarmJob delegation when new items are created.""" | ||
746 | 46 | self.build, self.job = build, job | 45 | self.build, self.job = build, job |
747 | 47 | super(BuildPackageJob, self).__init__() | 46 | super(BuildPackageJob, self).__init__() |
748 | 48 | 47 | ||
749 | 48 | def _set_build_farm_job(self): | ||
750 | 49 | """Setup the IBuildFarmJob delegate. | ||
751 | 50 | |||
752 | 51 | We override this to provide a delegate specific to package builds.""" | ||
753 | 52 | self.build_farm_job = PackageBuild(self.build) | ||
754 | 53 | |||
755 | 49 | def score(self): | 54 | def score(self): |
756 | 50 | """See `IBuildPackageJob`.""" | 55 | """See `IBuildPackageJob`.""" |
757 | 51 | score_pocketname = { | 56 | score_pocketname = { |
758 | 52 | 57 | ||
759 | === modified file 'lib/lp/translations/model/translationtemplatesbuildjob.py' | |||
760 | --- lib/lp/translations/model/translationtemplatesbuildjob.py 2010-04-28 08:24:54 +0000 | |||
761 | +++ lib/lp/translations/model/translationtemplatesbuildjob.py 2010-05-05 15:49:24 +0000 | |||
762 | @@ -22,7 +22,8 @@ | |||
763 | 22 | 22 | ||
764 | 23 | from lp.buildmaster.interfaces.buildfarmjob import BuildFarmJobType | 23 | from lp.buildmaster.interfaces.buildfarmjob import BuildFarmJobType |
765 | 24 | from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet | 24 | from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet |
767 | 25 | from lp.buildmaster.model.buildfarmjob import BuildFarmJobDerived | 25 | from lp.buildmaster.model.buildfarmjob import ( |
768 | 26 | BuildFarmJob, BuildFarmJobDerived) | ||
769 | 26 | from lp.buildmaster.model.buildqueue import BuildQueue | 27 | from lp.buildmaster.model.buildqueue import BuildQueue |
770 | 27 | from lp.code.interfaces.branchjob import IRosettaUploadJobSource | 28 | from lp.code.interfaces.branchjob import IRosettaUploadJobSource |
771 | 28 | from lp.buildmaster.interfaces.buildfarmbranchjob import IBuildFarmBranchJob | 29 | from lp.buildmaster.interfaces.buildfarmbranchjob import IBuildFarmBranchJob |
772 | @@ -49,6 +50,14 @@ | |||
773 | 49 | def __init__(self, branch_job): | 50 | def __init__(self, branch_job): |
774 | 50 | super(TranslationTemplatesBuildJob, self).__init__(branch_job) | 51 | super(TranslationTemplatesBuildJob, self).__init__(branch_job) |
775 | 51 | 52 | ||
776 | 53 | def _set_build_farm_job(self): | ||
777 | 54 | """Setup the IBuildFarmJob delegate. | ||
778 | 55 | |||
779 | 56 | We override this to provide a non-database delegate that simply | ||
780 | 57 | provides required functionality to the queue system.""" | ||
781 | 58 | self.build_farm_job = BuildFarmJob( | ||
782 | 59 | job_type=BuildFarmJobType.TRANSLATIONTEMPLATESBUILD) | ||
783 | 60 | |||
784 | 52 | def score(self): | 61 | def score(self): |
785 | 53 | """See `IBuildFarmJob`.""" | 62 | """See `IBuildFarmJob`.""" |
786 | 54 | # Hard-code score for now; anything other than 1000 is probably | 63 | # Hard-code score for now; anything other than 1000 is probably |
Sorry, still haven't sorted out my mailing list subscriptions. I promise I'll do that for the next one and reply inline...
Making the BuildBase methods static is ugly, though I can understand why you're doing it and it seems like a reasonable thing to do for the moment.
Some minor issues:
The various versions of _set_build_farm_job have an extra heading space in their docstring.
The extra_info parameter to IPackageBuild. verify is not documented in its interface.
Also, this code will unfortunately conflict once my simplify- uploadprocess branch lands.