Launchpad itself

Merge lp:~adeuring/launchpad/bug-438734-parse-kernel-release-nnode into lp:launchpad

  1. bug-438734-parse-kernel-release-nnode
  2. Merge into devel
Proposed by Abel Deuring
Status: Merged
Approved by: Brad Crittenden
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~adeuring/launchpad/bug-438734-parse-kernel-release-nnode
Merge into: lp:launchpad
Diff against target: 387 lines
3 files modified
lib/canonical/launchpad/scripts/hwdbsubmissions.py (+78/-0)
lib/canonical/launchpad/scripts/tests/hardwaretest-udev.xml (+13/-4)
lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py (+216/-2)
To merge this branch: bzr merge lp:~adeuring/launchpad/bug-438734-parse-kernel-release-nnode
Reviewer Review Type Date Requested Status
Brad Crittenden (community) code Approve
Review via email: mp+12589@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Abel Deuring (adeuring) wrote :
Download full text (4.3 KiB)

This branch makes the method l.c.l.scripts.hwsdbsubmissions.SubmissionParser._parseSummary() aware of the new optional node <kernel-release> in HWDB submissions. Data submitted by the HWDB client in Karmic will contain this data.

test: ./bin/test-t test_hwdb_submission_parser

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/canonical/launchpad/scripts/hwdbsubmissions.py
  lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py

== Pyflakes notices ==

lib/canonical/launchpad/scripts/hwdbsubmissions.py
    22: redefinition of unused 'etree' from line 20

lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py
    10: redefinition of unused 'etree' from line 8

== Pylint notices ==

lib/canonical/launchpad/scripts/hwdbsubmissions.py
    20: [F0401] Unable to import 'xml.etree.cElementTree' (No module named etree)

This message is not related to my changes.

This branch is based on lp:~adeuring/launchpad/bug-438169-parse-sysfs-attr, reviewed by gmb, but not yet landed. The diff against this branch:

=== modified file 'lib/canonical/launchpad/scripts/hwdbsubmissions.py'
--- lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-09-29 10:11:50 +0000
+++ lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-09-29 15:02:47 +0000
@@ -260,6 +260,7 @@
         'contactable': _getValueAttributeAsBoolean,
         'date_created': _getValueAttributeAsDateTime,
         'client': _getClientData,
+ 'kernel-release': _getValueAttributeAsString,
         }

     def _parseSummary(self, summary_node):

=== modified file 'lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.p
y'
--- lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-0
9-29 11:23:04 +0000
+++ lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-0
9-29 15:03:15 +0000
@@ -209,6 +209,58 @@
             summary, expected_data,
             'SubmissionParser.parseSummary returned an unexpected result')

+ def testSummaryNodeWithKernelRelease(self):
+ """The <smmary> node may contain the sub-node <kernel-release>."""
+ node = etree.fromstring("""
+ <summary>
+ <live_cd value="False"/>
+ <system_id value="f982bb1ab536469cebfd6eaadcea0ffc"/>
+ <distribution value="Ubuntu"/>
+ <distroseries value="7.04"/>
+ <architecture value="amd64"/>
+ <private value="False"/>
+ <contactable value="False"/>
+ <date_created value="2007-09-28T16:09:20.126842"/>
+ <client name="hwtest" version="0.9">
+ <plugin name="architecture_info" version="1.1"/>
+ <plugin name="find_network_controllers" version="2.34"/>
+ </client>
+ <kernel-release value="2.6.28-15-generic"/>
+ </summary>
+ """)
+ parser = SubmissionParser(self.log)
+ summary = parser._parseSummary(node)
+ utc_tz = pytz.timezone('UTC')
+ expected_data...

Read more...

Revision history for this message
Brad Crittenden (bac) wrote :

Hi Abel,

Thanks for making this change in support of Karmic. I found a few little issues:

typo: s/<smmary>/<summary>

The pytz package already has an instance of UTC we can use. Please replace utc_tz with just pytz.UTC. It's more readable and less work.

Thanks,
Brad

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/canonical/launchpad/scripts/hwdbsubmissions.py'
2--- lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-09-28 11:33:57 +0000
3+++ lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-09-29 15:36:18 +0000
4@@ -260,6 +260,7 @@
5 'contactable': _getValueAttributeAsBoolean,
6 'date_created': _getValueAttributeAsDateTime,
7 'client': _getClientData,
8+ 'kernel-release': _getValueAttributeAsString,
9 }
10
11 def _parseSummary(self, summary_node):
12@@ -556,6 +557,82 @@
13 dmi_data[record[0]] = record[1]
14 return dmi_data
15
16+ def _parseSysfsAttributes(self, sysfs_node):
17+ """Parse the <sysfs-attributes> node.
18+
19+ :return: A dictionary {path: attrs, ...} where path is the
20+ path is the path of a sysfs directory, and where attrs
21+ is a dictionary containing attribute names and values.
22+
23+ A sample of the input data:
24+
25+ P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
26+ A: modalias=input:b0019v0000p0001e0000-e0,1,k74,ramlsfw
27+ A: uniq=
28+ A: phys=LNXPWRBN/button/input0
29+ A: name=Power Button
30+
31+ P: /devices/LNXSYSTM:00/device:00/PNP0A08:00/device:03/input/input8
32+ A: modalias=input:b0019v0000p0006e0000-e0,1,kE0,E1,E3,F0,F1
33+ A: uniq=
34+ A: phys=/video/input0
35+ A: name=Video Bus
36+
37+ Data for different devices is separated by empty lines. The data
38+ for each device starts with a line 'P: /devices/LNXSYSTM...',
39+ specifying the sysfs path of a device, followed by zero or more
40+ lines of the form 'A: key=value'
41+ """
42+ sysfs_lines = sysfs_node.text.split('\n')
43+ sysfs_data = {}
44+ attributes = None
45+
46+ for line_number, line in enumerate(sysfs_lines):
47+ if len(line) == 0:
48+ attributes = None
49+ continue
50+ record = line.split(': ', 1)
51+ if len(record) != 2:
52+ self._logError(
53+ 'Line %i in <sysfs-attributes>: No valid key:value data: '
54+ '%r' % (line_number, line),
55+ self.submission_key)
56+ return None
57+
58+ key, value = record
59+ if key == 'P':
60+ if attributes is not None:
61+ self._logError(
62+ "Line %i in <sysfs-attributes>: duplicate 'P' line "
63+ "found: %r" % (line_number, line),
64+ self.submission_key)
65+ return None
66+ attributes = {}
67+ sysfs_data[value] = attributes
68+ elif key == 'A':
69+ if attributes is None:
70+ self._logError(
71+ "Line %i in <sysfs-attributes>: Block for a device "
72+ "does not start with 'P:': %r" % (line_number, line),
73+ self.submission_key)
74+ return None
75+ attribute_data = value.split('=', 1)
76+ if len(attribute_data) != 2:
77+ self._logError(
78+ 'Line %i in <sysfs-attributes>: Attribute line does '
79+ 'not contain key=value data: %r'
80+ % (line_number, line),
81+ self.submission_key)
82+ return None
83+ attributes[attribute_data[0]] = attribute_data[1]
84+ else:
85+ self._logError(
86+ 'Line %i in <sysfs-attributes>: Unexpected key: %r'
87+ % (line_number, line),
88+ self.submission_key)
89+ return None
90+ return sysfs_data
91+
92 def _setHardwareSectionParsers(self):
93 self._parse_hardware_section = {
94 'hal': self._parseHAL,
95@@ -563,6 +640,7 @@
96 'aliases': self._parseAliases,
97 'udev': self._parseUdev,
98 'dmi': self._parseDmi,
99+ 'sysfs-attributes': self._parseSysfsAttributes,
100 }
101
102 def _parseHardware(self, hardware_node):
103
104=== modified file 'lib/canonical/launchpad/scripts/tests/hardwaretest-udev.xml'
105--- lib/canonical/launchpad/scripts/tests/hardwaretest-udev.xml 2009-09-14 09:16:45 +0000
106+++ lib/canonical/launchpad/scripts/tests/hardwaretest-udev.xml 2009-09-29 15:36:18 +0000
107@@ -184,10 +184,19 @@
108 is stored in the same format as the DMI data:
109 /path/to/file:filecontent
110 -->
111- <sysfs-attributes>/sys/devices/pci0000:00/0000:00:1f.1/host3/target3:0:0/3:0:0:0/vendor:HL-DT-ST
112-/sys/devices/pci0000:00/0000:00:1f.1/host3/target3:0:0/3:0:0:0/model:DVDRAM GSA-4083N
113-/sys/devices/pci0000:00/0000:00:1f.1/host3/target3:0:0/3:0:0:0/type:5
114- </sysfs-attributes>
115+ <sysfs-attributes>
116+P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
117+A: modalias=input:b0019v0000p0001e0000-e0,1,k74,ramlsfw
118+A: uniq=
119+A: phys=LNXPWRBN/button/input0
120+A: name=Power Button
121+
122+P: /devices/LNXSYSTM:00/device:00/PNP0A08:00/device:03/input/input8
123+A: modalias=input:b0019v0000p0006e0000-e0,1,kE0,E1,E3,F0,F1,F2,F3,F4,F5,ramlsfw
124+A: uniq=
125+A: phys=/video/input0
126+A: name=Video Bus
127+</sysfs-attributes>
128
129 <!-- processors: Data about processors installed in a system.
130 The data is retrieved from /proc/cpuinfo.
131
132=== modified file 'lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py'
133--- lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-09-28 12:35:40 +0000
134+++ lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-09-29 15:36:18 +0000
135@@ -11,6 +11,7 @@
136 from datetime import datetime
137 import logging
138 import os
139+from textwrap import dedent
140 from unittest import TestCase, TestLoader
141
142 import pytz
143@@ -184,7 +185,6 @@
144 """)
145 parser = SubmissionParser(self.log)
146 summary = parser._parseSummary(node)
147- utc_tz = pytz.timezone('UTC')
148 expected_data = {
149 'live_cd': False,
150 'system_id': 'f982bb1ab536469cebfd6eaadcea0ffc',
151@@ -194,7 +194,7 @@
152 'private': False,
153 'contactable': False,
154 'date_created': datetime(2007, 9, 28, 16, 9, 20, 126842,
155- tzinfo=utc_tz),
156+ tzinfo=pytz.UTC),
157 'client': {
158 'name': 'hwtest',
159 'version': '0.9',
160@@ -208,6 +208,57 @@
161 summary, expected_data,
162 'SubmissionParser.parseSummary returned an unexpected result')
163
164+ def testSummaryNodeWithKernelRelease(self):
165+ """The <summary> node may contain the sub-node <kernel-release>."""
166+ node = etree.fromstring("""
167+ <summary>
168+ <live_cd value="False"/>
169+ <system_id value="f982bb1ab536469cebfd6eaadcea0ffc"/>
170+ <distribution value="Ubuntu"/>
171+ <distroseries value="7.04"/>
172+ <architecture value="amd64"/>
173+ <private value="False"/>
174+ <contactable value="False"/>
175+ <date_created value="2007-09-28T16:09:20.126842"/>
176+ <client name="hwtest" version="0.9">
177+ <plugin name="architecture_info" version="1.1"/>
178+ <plugin name="find_network_controllers" version="2.34"/>
179+ </client>
180+ <kernel-release value="2.6.28-15-generic"/>
181+ </summary>
182+ """)
183+ parser = SubmissionParser(self.log)
184+ summary = parser._parseSummary(node)
185+ expected_data = {
186+ 'live_cd': False,
187+ 'system_id': 'f982bb1ab536469cebfd6eaadcea0ffc',
188+ 'distribution': 'Ubuntu',
189+ 'distroseries': '7.04',
190+ 'architecture': 'amd64',
191+ 'private': False,
192+ 'contactable': False,
193+ 'date_created': datetime(2007, 9, 28, 16, 9, 20, 126842,
194+ tzinfo=pytz.UTC),
195+ 'client': {
196+ 'name': 'hwtest',
197+ 'version': '0.9',
198+ 'plugins': [
199+ {
200+ 'name': 'architecture_info',
201+ 'version': '1.1'
202+ },
203+ {
204+ 'name': 'find_network_controllers',
205+ 'version': '2.34'
206+ }
207+ ]
208+ },
209+ 'kernel-release': '2.6.28-15-generic',
210+ }
211+ self.assertEqual(
212+ summary, expected_data,
213+ 'SubmissionParser.parseSummary returned an unexpected result')
214+
215 def _runPropertyTest(self, xml):
216 parser = SubmissionParser(self.log)
217 node = etree.fromstring(xml)
218@@ -649,6 +700,169 @@
219 parser.submission_key,
220 "Line 1 in <dmi>: No valid key:value data: 'invalid line'")
221
222+ def testSysfsAttributes(self):
223+ """Test of SubmissionParser._parseSysfsAttributes().
224+
225+ The content of the <sys-attributes> node is converted into
226+ a dictionary.
227+ """
228+ parser = SubmissionParser(self.log)
229+ node = etree.fromstring(dedent("""
230+ <sysfs-attributes>
231+ P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
232+ A: modalias=input:b0019v0000p0001e0000-e0,1,k74
233+ A: uniq=
234+ A: phys=LNXPWRBN/button/input0
235+ A: name=Power Button
236+
237+ P: /devices/LNXSYSTM:00/device:00/PNP0A08:00/device:03
238+ A: uniq=
239+ A: phys=/video/input0
240+ A: name=Video Bus
241+ </sysfs-attributes>
242+ """))
243+ result = parser._parseSysfsAttributes(node)
244+ self.assertEqual(
245+ {
246+ '/devices/LNXSYSTM:00/LNXPWRBN:00/input/input0': {
247+ 'modalias': 'input:b0019v0000p0001e0000-e0,1,k74',
248+ 'uniq': '',
249+ 'phys': 'LNXPWRBN/button/input0',
250+ 'name': 'Power Button',
251+ },
252+ '/devices/LNXSYSTM:00/device:00/PNP0A08:00/device:03': {
253+ 'uniq': '',
254+ 'phys': '/video/input0',
255+ 'name': 'Video Bus',
256+ },
257+
258+ },
259+ result,
260+ 'Invalid parsing result of <sysfs-attributes> node.')
261+
262+ def testSysfsAttributesLineWithoutKeyValueData(self):
263+ """Test of SubmissionParser._parseSysfsAttributes().
264+
265+ Lines not in key: value format are rejected.
266+ """
267+ parser = SubmissionParser(self.log)
268+ parser.submission_key = (
269+ 'Detect <sysfs-attributes> lines not in key:value format')
270+ node = etree.fromstring(dedent("""
271+ <sysfs-attributes>
272+ P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
273+ A: modalias=input:b0019v0000p0001e0000-e0,1,k74
274+ invalid line
275+ </sysfs-attributes>
276+ """))
277+ result = parser._parseSysfsAttributes(node)
278+ self.assertEqual(
279+ None, result,
280+ 'Invalid parsing result of a <sysfs-attributes> node containing '
281+ 'a line not in key:value format.')
282+ self.assertErrorMessage(
283+ parser.submission_key,
284+ "Line 3 in <sysfs-attributes>: No valid key:value data: "
285+ "'invalid line'")
286+
287+ def testSysfsAttributesDuplicatePLine(self):
288+ """Test of SubmissionParser._parseSysfsAttributes().
289+
290+ A line starting with "P:" must be the first line of a device block.
291+ """
292+ parser = SubmissionParser(self.log)
293+ parser.submission_key = (
294+ 'Detect <sysfs-attributes> node with duplicate P: line')
295+ node = etree.fromstring(dedent("""
296+ <sysfs-attributes>
297+ P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
298+ A: modalias=input:b0019v0000p0001e0000-e0,1,k74
299+ P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
300+ </sysfs-attributes>
301+ """))
302+ result = parser._parseSysfsAttributes(node)
303+ self.assertEqual(
304+ None, result,
305+ 'Invalid parsing result of a <sysfs-attributes> node containing '
306+ 'a duplicate P: line.')
307+ self.assertErrorMessage(
308+ parser.submission_key,
309+ "Line 3 in <sysfs-attributes>: duplicate 'P' line found: "
310+ "'P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0'")
311+
312+ def testSysfsAttributesNoPLineAtDeviceStart(self):
313+ """Test of SubmissionParser._parseSysfsAttributes().
314+
315+ The data for a device must start with a "P:" line.
316+ """
317+ parser = SubmissionParser(self.log)
318+ parser.submission_key = (
319+ 'Detect <sysfs-attributes> node without leading P: line')
320+ node = etree.fromstring(dedent("""
321+ <sysfs-attributes>
322+ A: modalias=input:b0019v0000p0001e0000-e0,1,k74
323+ </sysfs-attributes>
324+ """))
325+ result = parser._parseSysfsAttributes(node)
326+ self.assertEqual(
327+ None, result,
328+ 'Invalid parsing result of a <sysfs-attributes> node where a '
329+ 'device block does not start with a "P": line.')
330+ self.assertErrorMessage(
331+ parser.submission_key,
332+ "Line 1 in <sysfs-attributes>: Block for a device does not "
333+ "start with 'P:': "
334+ "'A: modalias=input:b0019v0000p0001e0000-e0,1,k74'")
335+
336+ def testSysfsAttributesNoAttributeKeyValue(self):
337+ """Test of SubmissionParser._parseSysfsAttributes().
338+
339+ A line starting with "A:" must be in key=value format.
340+ """
341+ parser = SubmissionParser(self.log)
342+ parser.submission_key = (
343+ 'Detect <sysfs-attributes> node with A: line not in key=value '
344+ 'format')
345+ node = etree.fromstring(dedent("""
346+ <sysfs-attributes>
347+ P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
348+ A: equal sign is missing
349+ </sysfs-attributes>
350+ """))
351+ result = parser._parseSysfsAttributes(node)
352+ self.assertEqual(
353+ None, result,
354+ 'Invalid parsing result of a <sysfs-attributes> node with A: '
355+ 'line not in key=value format.')
356+ self.assertErrorMessage(
357+ parser.submission_key,
358+ "Line 2 in <sysfs-attributes>: Attribute line does not contain "
359+ "key=value data: 'A: equal sign is missing'")
360+
361+ def testSysfsAttributesInvalidMainKey(self):
362+ """Test of SubmissionParser._parseSysfsAttributes().
363+
364+ All lines must start with "P:" or "A:".
365+ """
366+ parser = SubmissionParser(self.log)
367+ parser.submission_key = (
368+ 'Detect <sysfs-attributes> node with invalid main key.')
369+ node = etree.fromstring(dedent("""
370+ <sysfs-attributes>
371+ P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
372+ X: an invalid line
373+ </sysfs-attributes>
374+ """))
375+ result = parser._parseSysfsAttributes(node)
376+ self.assertEqual(
377+ None, result,
378+ 'Invalid parsing result of a <sysfs-attributes> node containg '
379+ 'a line that does not start with "A:" or "P:".')
380+ self.assertErrorMessage(
381+ parser.submission_key,
382+ "Line 2 in <sysfs-attributes>: Unexpected key: "
383+ "'X: an invalid line'")
384+
385 def testHardware(self):
386 """The <hardware> tag is converted into a dictionary."""
387 test = self