Launchpad itself

Merge lp:~wgrant/launchpad/bug-241129-queue-binary-scaling into lp:launchpad

  1. bug-241129-queue-binary-scaling
  2. Merge into devel
Proposed by William Grant
Status: Merged
Approved by: Māris Fogels
Approved revision: no longer in the source branch.
Merged at revision: 11296
Proposed branch: lp:~wgrant/launchpad/bug-241129-queue-binary-scaling
Merge into: lp:launchpad
Diff against target: 82 lines (+16/-3)
2 files modified
lib/lp/soyuz/scripts/queue.py (+7/-2)
lib/lp/soyuz/scripts/tests/test_queue.py (+9/-1)
To merge this branch: bzr merge lp:~wgrant/launchpad/bug-241129-queue-binary-scaling
Reviewer Review Type Date Requested Status
Māris Fogels (community) code Approve
Review via email: mp+31604@code.launchpad.net

Commit message

'queue' no longer overrides each specified binary n^2 times.

Description of the change

The 'queue' archive admin commandline tool lets users specify multiple binary names to override in a single invocation.

However, the current implementation fetches the PackageUpload matching each binary, and overrides them all without removing duplicates. This is inefficient when overriding multiple binaries from the same build: each binary is overridden n² times (bug #241129)!

This branch fixes the scaling problem by only considering each PackageUpload once. It also adds tests to confirm the number of overridden packages does not increase.

To post a comment you must log in.
Revision history for this message
Māris Fogels (mars) wrote :

Hi William,

This change looks good. r=mars

Maris

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/soyuz/scripts/queue.py'
2--- lib/lp/soyuz/scripts/queue.py 2010-08-02 02:13:52 +0000
3+++ lib/lp/soyuz/scripts/queue.py 2010-08-03 04:16:03 +0000
4@@ -189,7 +189,8 @@
5 self.distroseries.distribution.name,
6 self.distroseries.name, self.pocket.name))
7
8- self.items.append(item)
9+ if item not in self.items:
10+ self.items.append(item)
11 self.explicit_ids_specified = True
12 else:
13 # retrieve PackageUpload item by name/version key
14@@ -201,7 +202,8 @@
15 for item in self.distroseries.getQueueItems(
16 status=self.queue, name=term, version=version,
17 exact_match=self.exact_match, pocket=self.pocket):
18- self.items.append(item)
19+ if item not in self.items:
20+ self.items.append(item)
21 self.package_names.append(term)
22
23 self.items_size = len(self.items)
24@@ -529,6 +531,7 @@
25 priority_name, announcelist, display,
26 no_mail=True, exact_match=False, log=log)
27 self.terms_start_index = 1
28+ self.overrides_performed = 0
29
30 def run(self):
31 """Perform Override action."""
32@@ -575,6 +578,7 @@
33 component,
34 queue_item.sourcepackagerelease.component
35 ])
36+ self.overrides_performed += 1
37 self.displayInfo(queue_item)
38
39 def _override_binary(self):
40@@ -614,6 +618,7 @@
41 binary.priority.name))
42 binary.override(component=component, section=section,
43 priority=priority)
44+ self.overrides_performed += 1
45 self.displayInfo(queue_item, only=binary.name)
46 # See if the new component requires a new archive on the
47 # build:
48
49=== modified file 'lib/lp/soyuz/scripts/tests/test_queue.py'
50--- lib/lp/soyuz/scripts/tests/test_queue.py 2010-02-09 00:17:40 +0000
51+++ lib/lp/soyuz/scripts/tests/test_queue.py 2010-08-03 04:16:03 +0000
52@@ -719,6 +719,7 @@
53 component_name='universe', section_name='editors')
54 # 'netapplet' appears 3 times, alsa-utils once.
55 self.assertEqual(4, queue_action.items_size)
56+ self.assertEqual(2, queue_action.overrides_performed)
57 # Check results.
58 queue_items = list(breezy_autotest.getQueueItems(
59 status=PackageUploadStatus.NEW, name='alsa-utils'))
60@@ -814,6 +815,7 @@
61 priority_name='optional')
62 # Check results.
63 self.assertEqual(2, queue_action.items_size)
64+ self.assertEqual(2, queue_action.overrides_performed)
65 queue_items = list(breezy_autotest.getQueueItems(
66 status=PackageUploadStatus.NEW, name='pmount'))
67 queue_items.extend(list(breezy_autotest.getQueueItems(
68@@ -861,7 +863,13 @@
69 component_name='restricted', section_name='editors',
70 priority_name='optional')
71
72- self.assertEqual(2, queue_action.items_size)
73+ # There are three binaries to override on this PackageUpload:
74+ # - mozilla-firefox in breezy-autotest
75+ # - mozilla-firefox and mozilla-firefox-data in warty
76+ # Each should be overridden exactly once.
77+ self.assertEqual(1, queue_action.items_size)
78+ self.assertEqual(3, queue_action.overrides_performed)
79+
80 queue_items = list(breezy_autotest.getQueueItems(
81 status=PackageUploadStatus.NEW, name='mozilla-firefox-data'))
82 queue_items.extend(list(breezy_autotest.getQueueItems(