Launchpad itself

Merge lp:~adeuring/launchpad/bug-438734-parse-kernel-release-nnode into lp:launchpad

  1. bug-438734-parse-kernel-release-nnode
  2. Merge into devel
Proposed by Abel Deuring
Status: Merged
Approved by: Brad Crittenden
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~adeuring/launchpad/bug-438734-parse-kernel-release-nnode
Merge into: lp:launchpad
Diff against target: 387 lines
3 files modified
lib/canonical/launchpad/scripts/hwdbsubmissions.py (+78/-0)
lib/canonical/launchpad/scripts/tests/hardwaretest-udev.xml (+13/-4)
lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py (+216/-2)
To merge this branch: bzr merge lp:~adeuring/launchpad/bug-438734-parse-kernel-release-nnode
Reviewer Review Type Date Requested Status
Brad Crittenden (community) code Approve
Review via email: mp+12589@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Abel Deuring (adeuring) wrote :
Download full text (4.3 KiB)

This branch makes the method l.c.l.scripts.hwsdbsubmissions.SubmissionParser._parseSummary() aware of the new optional node <kernel-release> in HWDB submissions. Data submitted by the HWDB client in Karmic will contain this data.

test: ./bin/test-t test_hwdb_submission_parser

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/canonical/launchpad/scripts/hwdbsubmissions.py
  lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py

== Pyflakes notices ==

lib/canonical/launchpad/scripts/hwdbsubmissions.py
    22: redefinition of unused 'etree' from line 20

lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py
    10: redefinition of unused 'etree' from line 8

== Pylint notices ==

lib/canonical/launchpad/scripts/hwdbsubmissions.py
    20: [F0401] Unable to import 'xml.etree.cElementTree' (No module named etree)

This message is not related to my changes.

This branch is based on lp:~adeuring/launchpad/bug-438169-parse-sysfs-attr, reviewed by gmb, but not yet landed. The diff against this branch:

=== modified file 'lib/canonical/launchpad/scripts/hwdbsubmissions.py'
--- lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-09-29 10:11:50 +0000
+++ lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-09-29 15:02:47 +0000
@@ -260,6 +260,7 @@
         'contactable': _getValueAttributeAsBoolean,
         'date_created': _getValueAttributeAsDateTime,
         'client': _getClientData,
+ 'kernel-release': _getValueAttributeAsString,
         }

     def _parseSummary(self, summary_node):

=== modified file 'lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.p
y'
--- lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-0
9-29 11:23:04 +0000
+++ lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-0
9-29 15:03:15 +0000
@@ -209,6 +209,58 @@
             summary, expected_data,
             'SubmissionParser.parseSummary returned an unexpected result')

+ def testSummaryNodeWithKernelRelease(self):
+ """The <smmary> node may contain the sub-node <kernel-release>."""
+ node = etree.fromstring("""
+ <summary>
+ <live_cd value="False"/>
+ <system_id value="f982bb1ab536469cebfd6eaadcea0ffc"/>
+ <distribution value="Ubuntu"/>
+ <distroseries value="7.04"/>
+ <architecture value="amd64"/>
+ <private value="False"/>
+ <contactable value="False"/>
+ <date_created value="2007-09-28T16:09:20.126842"/>
+ <client name="hwtest" version="0.9">
+ <plugin name="architecture_info" version="1.1"/>
+ <plugin name="find_network_controllers" version="2.34"/>
+ </client>
+ <kernel-release value="2.6.28-15-generic"/>
+ </summary>
+ """)
+ parser = SubmissionParser(self.log)
+ summary = parser._parseSummary(node)
+ utc_tz = pytz.timezone('UTC')
+ expected_data...

Read more...

Revision history for this message
Brad Crittenden (bac) wrote :

Hi Abel,

Thanks for making this change in support of Karmic. I found a few little issues:

typo: s/<smmary>/<summary>

The pytz package already has an instance of UTC we can use. Please replace utc_tz with just pytz.UTC. It's more readable and less work.

Thanks,
Brad

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/canonical/launchpad/scripts/hwdbsubmissions.py'
--- lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-09-28 11:33:57 +0000
+++ lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-09-29 15:36:18 +0000
@@ -260,6 +260,7 @@
260 'contactable': _getValueAttributeAsBoolean,260 'contactable': _getValueAttributeAsBoolean,
261 'date_created': _getValueAttributeAsDateTime,261 'date_created': _getValueAttributeAsDateTime,
262 'client': _getClientData,262 'client': _getClientData,
263 'kernel-release': _getValueAttributeAsString,
263 }264 }
264265
265 def _parseSummary(self, summary_node):266 def _parseSummary(self, summary_node):
@@ -556,6 +557,82 @@
556 dmi_data[record[0]] = record[1]557 dmi_data[record[0]] = record[1]
557 return dmi_data558 return dmi_data
558559
560 def _parseSysfsAttributes(self, sysfs_node):
561 """Parse the <sysfs-attributes> node.
562
563 :return: A dictionary {path: attrs, ...} where path is the
564 path is the path of a sysfs directory, and where attrs
565 is a dictionary containing attribute names and values.
566
567 A sample of the input data:
568
569 P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
570 A: modalias=input:b0019v0000p0001e0000-e0,1,k74,ramlsfw
571 A: uniq=
572 A: phys=LNXPWRBN/button/input0
573 A: name=Power Button
574
575 P: /devices/LNXSYSTM:00/device:00/PNP0A08:00/device:03/input/input8
576 A: modalias=input:b0019v0000p0006e0000-e0,1,kE0,E1,E3,F0,F1
577 A: uniq=
578 A: phys=/video/input0
579 A: name=Video Bus
580
581 Data for different devices is separated by empty lines. The data
582 for each device starts with a line 'P: /devices/LNXSYSTM...',
583 specifying the sysfs path of a device, followed by zero or more
584 lines of the form 'A: key=value'
585 """
586 sysfs_lines = sysfs_node.text.split('\n')
587 sysfs_data = {}
588 attributes = None
589
590 for line_number, line in enumerate(sysfs_lines):
591 if len(line) == 0:
592 attributes = None
593 continue
594 record = line.split(': ', 1)
595 if len(record) != 2:
596 self._logError(
597 'Line %i in <sysfs-attributes>: No valid key:value data: '
598 '%r' % (line_number, line),
599 self.submission_key)
600 return None
601
602 key, value = record
603 if key == 'P':
604 if attributes is not None:
605 self._logError(
606 "Line %i in <sysfs-attributes>: duplicate 'P' line "
607 "found: %r" % (line_number, line),
608 self.submission_key)
609 return None
610 attributes = {}
611 sysfs_data[value] = attributes
612 elif key == 'A':
613 if attributes is None:
614 self._logError(
615 "Line %i in <sysfs-attributes>: Block for a device "
616 "does not start with 'P:': %r" % (line_number, line),
617 self.submission_key)
618 return None
619 attribute_data = value.split('=', 1)
620 if len(attribute_data) != 2:
621 self._logError(
622 'Line %i in <sysfs-attributes>: Attribute line does '
623 'not contain key=value data: %r'
624 % (line_number, line),
625 self.submission_key)
626 return None
627 attributes[attribute_data[0]] = attribute_data[1]
628 else:
629 self._logError(
630 'Line %i in <sysfs-attributes>: Unexpected key: %r'
631 % (line_number, line),
632 self.submission_key)
633 return None
634 return sysfs_data
635
559 def _setHardwareSectionParsers(self):636 def _setHardwareSectionParsers(self):
560 self._parse_hardware_section = {637 self._parse_hardware_section = {
561 'hal': self._parseHAL,638 'hal': self._parseHAL,
@@ -563,6 +640,7 @@
563 'aliases': self._parseAliases,640 'aliases': self._parseAliases,
564 'udev': self._parseUdev,641 'udev': self._parseUdev,
565 'dmi': self._parseDmi,642 'dmi': self._parseDmi,
643 'sysfs-attributes': self._parseSysfsAttributes,
566 }644 }
567645
568 def _parseHardware(self, hardware_node):646 def _parseHardware(self, hardware_node):
569647
=== modified file 'lib/canonical/launchpad/scripts/tests/hardwaretest-udev.xml'
--- lib/canonical/launchpad/scripts/tests/hardwaretest-udev.xml 2009-09-14 09:16:45 +0000
+++ lib/canonical/launchpad/scripts/tests/hardwaretest-udev.xml 2009-09-29 15:36:18 +0000
@@ -184,10 +184,19 @@
184 is stored in the same format as the DMI data:184 is stored in the same format as the DMI data:
185 /path/to/file:filecontent185 /path/to/file:filecontent
186 -->186 -->
187 <sysfs-attributes>/sys/devices/pci0000:00/0000:00:1f.1/host3/target3:0:0/3:0:0:0/vendor:HL-DT-ST187 <sysfs-attributes>
188/sys/devices/pci0000:00/0000:00:1f.1/host3/target3:0:0/3:0:0:0/model:DVDRAM GSA-4083N188P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
189/sys/devices/pci0000:00/0000:00:1f.1/host3/target3:0:0/3:0:0:0/type:5189A: modalias=input:b0019v0000p0001e0000-e0,1,k74,ramlsfw
190 </sysfs-attributes>190A: uniq=
191A: phys=LNXPWRBN/button/input0
192A: name=Power Button
193
194P: /devices/LNXSYSTM:00/device:00/PNP0A08:00/device:03/input/input8
195A: modalias=input:b0019v0000p0006e0000-e0,1,kE0,E1,E3,F0,F1,F2,F3,F4,F5,ramlsfw
196A: uniq=
197A: phys=/video/input0
198A: name=Video Bus
199</sysfs-attributes>
191200
192 <!-- processors: Data about processors installed in a system.201 <!-- processors: Data about processors installed in a system.
193 The data is retrieved from /proc/cpuinfo.202 The data is retrieved from /proc/cpuinfo.
194203
=== modified file 'lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py'
--- lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-09-28 12:35:40 +0000
+++ lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-09-29 15:36:18 +0000
@@ -11,6 +11,7 @@
11from datetime import datetime11from datetime import datetime
12import logging12import logging
13import os13import os
14from textwrap import dedent
14from unittest import TestCase, TestLoader15from unittest import TestCase, TestLoader
1516
16import pytz17import pytz
@@ -184,7 +185,6 @@
184 """)185 """)
185 parser = SubmissionParser(self.log)186 parser = SubmissionParser(self.log)
186 summary = parser._parseSummary(node)187 summary = parser._parseSummary(node)
187 utc_tz = pytz.timezone('UTC')
188 expected_data = {188 expected_data = {
189 'live_cd': False,189 'live_cd': False,
190 'system_id': 'f982bb1ab536469cebfd6eaadcea0ffc',190 'system_id': 'f982bb1ab536469cebfd6eaadcea0ffc',
@@ -194,7 +194,7 @@
194 'private': False,194 'private': False,
195 'contactable': False,195 'contactable': False,
196 'date_created': datetime(2007, 9, 28, 16, 9, 20, 126842,196 'date_created': datetime(2007, 9, 28, 16, 9, 20, 126842,
197 tzinfo=utc_tz),197 tzinfo=pytz.UTC),
198 'client': {198 'client': {
199 'name': 'hwtest',199 'name': 'hwtest',
200 'version': '0.9',200 'version': '0.9',
@@ -208,6 +208,57 @@
208 summary, expected_data,208 summary, expected_data,
209 'SubmissionParser.parseSummary returned an unexpected result')209 'SubmissionParser.parseSummary returned an unexpected result')
210210
211 def testSummaryNodeWithKernelRelease(self):
212 """The <summary> node may contain the sub-node <kernel-release>."""
213 node = etree.fromstring("""
214 <summary>
215 <live_cd value="False"/>
216 <system_id value="f982bb1ab536469cebfd6eaadcea0ffc"/>
217 <distribution value="Ubuntu"/>
218 <distroseries value="7.04"/>
219 <architecture value="amd64"/>
220 <private value="False"/>
221 <contactable value="False"/>
222 <date_created value="2007-09-28T16:09:20.126842"/>
223 <client name="hwtest" version="0.9">
224 <plugin name="architecture_info" version="1.1"/>
225 <plugin name="find_network_controllers" version="2.34"/>
226 </client>
227 <kernel-release value="2.6.28-15-generic"/>
228 </summary>
229 """)
230 parser = SubmissionParser(self.log)
231 summary = parser._parseSummary(node)
232 expected_data = {
233 'live_cd': False,
234 'system_id': 'f982bb1ab536469cebfd6eaadcea0ffc',
235 'distribution': 'Ubuntu',
236 'distroseries': '7.04',
237 'architecture': 'amd64',
238 'private': False,
239 'contactable': False,
240 'date_created': datetime(2007, 9, 28, 16, 9, 20, 126842,
241 tzinfo=pytz.UTC),
242 'client': {
243 'name': 'hwtest',
244 'version': '0.9',
245 'plugins': [
246 {
247 'name': 'architecture_info',
248 'version': '1.1'
249 },
250 {
251 'name': 'find_network_controllers',
252 'version': '2.34'
253 }
254 ]
255 },
256 'kernel-release': '2.6.28-15-generic',
257 }
258 self.assertEqual(
259 summary, expected_data,
260 'SubmissionParser.parseSummary returned an unexpected result')
261
211 def _runPropertyTest(self, xml):262 def _runPropertyTest(self, xml):
212 parser = SubmissionParser(self.log)263 parser = SubmissionParser(self.log)
213 node = etree.fromstring(xml)264 node = etree.fromstring(xml)
@@ -649,6 +700,169 @@
649 parser.submission_key,700 parser.submission_key,
650 "Line 1 in <dmi>: No valid key:value data: 'invalid line'")701 "Line 1 in <dmi>: No valid key:value data: 'invalid line'")
651702
703 def testSysfsAttributes(self):
704 """Test of SubmissionParser._parseSysfsAttributes().
705
706 The content of the <sys-attributes> node is converted into
707 a dictionary.
708 """
709 parser = SubmissionParser(self.log)
710 node = etree.fromstring(dedent("""
711 <sysfs-attributes>
712 P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
713 A: modalias=input:b0019v0000p0001e0000-e0,1,k74
714 A: uniq=
715 A: phys=LNXPWRBN/button/input0
716 A: name=Power Button
717
718 P: /devices/LNXSYSTM:00/device:00/PNP0A08:00/device:03
719 A: uniq=
720 A: phys=/video/input0
721 A: name=Video Bus
722 </sysfs-attributes>
723 """))
724 result = parser._parseSysfsAttributes(node)
725 self.assertEqual(
726 {
727 '/devices/LNXSYSTM:00/LNXPWRBN:00/input/input0': {
728 'modalias': 'input:b0019v0000p0001e0000-e0,1,k74',
729 'uniq': '',
730 'phys': 'LNXPWRBN/button/input0',
731 'name': 'Power Button',
732 },
733 '/devices/LNXSYSTM:00/device:00/PNP0A08:00/device:03': {
734 'uniq': '',
735 'phys': '/video/input0',
736 'name': 'Video Bus',
737 },
738
739 },
740 result,
741 'Invalid parsing result of <sysfs-attributes> node.')
742
743 def testSysfsAttributesLineWithoutKeyValueData(self):
744 """Test of SubmissionParser._parseSysfsAttributes().
745
746 Lines not in key: value format are rejected.
747 """
748 parser = SubmissionParser(self.log)
749 parser.submission_key = (
750 'Detect <sysfs-attributes> lines not in key:value format')
751 node = etree.fromstring(dedent("""
752 <sysfs-attributes>
753 P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
754 A: modalias=input:b0019v0000p0001e0000-e0,1,k74
755 invalid line
756 </sysfs-attributes>
757 """))
758 result = parser._parseSysfsAttributes(node)
759 self.assertEqual(
760 None, result,
761 'Invalid parsing result of a <sysfs-attributes> node containing '
762 'a line not in key:value format.')
763 self.assertErrorMessage(
764 parser.submission_key,
765 "Line 3 in <sysfs-attributes>: No valid key:value data: "
766 "'invalid line'")
767
768 def testSysfsAttributesDuplicatePLine(self):
769 """Test of SubmissionParser._parseSysfsAttributes().
770
771 A line starting with "P:" must be the first line of a device block.
772 """
773 parser = SubmissionParser(self.log)
774 parser.submission_key = (
775 'Detect <sysfs-attributes> node with duplicate P: line')
776 node = etree.fromstring(dedent("""
777 <sysfs-attributes>
778 P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
779 A: modalias=input:b0019v0000p0001e0000-e0,1,k74
780 P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
781 </sysfs-attributes>
782 """))
783 result = parser._parseSysfsAttributes(node)
784 self.assertEqual(
785 None, result,
786 'Invalid parsing result of a <sysfs-attributes> node containing '
787 'a duplicate P: line.')
788 self.assertErrorMessage(
789 parser.submission_key,
790 "Line 3 in <sysfs-attributes>: duplicate 'P' line found: "
791 "'P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0'")
792
793 def testSysfsAttributesNoPLineAtDeviceStart(self):
794 """Test of SubmissionParser._parseSysfsAttributes().
795
796 The data for a device must start with a "P:" line.
797 """
798 parser = SubmissionParser(self.log)
799 parser.submission_key = (
800 'Detect <sysfs-attributes> node without leading P: line')
801 node = etree.fromstring(dedent("""
802 <sysfs-attributes>
803 A: modalias=input:b0019v0000p0001e0000-e0,1,k74
804 </sysfs-attributes>
805 """))
806 result = parser._parseSysfsAttributes(node)
807 self.assertEqual(
808 None, result,
809 'Invalid parsing result of a <sysfs-attributes> node where a '
810 'device block does not start with a "P": line.')
811 self.assertErrorMessage(
812 parser.submission_key,
813 "Line 1 in <sysfs-attributes>: Block for a device does not "
814 "start with 'P:': "
815 "'A: modalias=input:b0019v0000p0001e0000-e0,1,k74'")
816
817 def testSysfsAttributesNoAttributeKeyValue(self):
818 """Test of SubmissionParser._parseSysfsAttributes().
819
820 A line starting with "A:" must be in key=value format.
821 """
822 parser = SubmissionParser(self.log)
823 parser.submission_key = (
824 'Detect <sysfs-attributes> node with A: line not in key=value '
825 'format')
826 node = etree.fromstring(dedent("""
827 <sysfs-attributes>
828 P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
829 A: equal sign is missing
830 </sysfs-attributes>
831 """))
832 result = parser._parseSysfsAttributes(node)
833 self.assertEqual(
834 None, result,
835 'Invalid parsing result of a <sysfs-attributes> node with A: '
836 'line not in key=value format.')
837 self.assertErrorMessage(
838 parser.submission_key,
839 "Line 2 in <sysfs-attributes>: Attribute line does not contain "
840 "key=value data: 'A: equal sign is missing'")
841
842 def testSysfsAttributesInvalidMainKey(self):
843 """Test of SubmissionParser._parseSysfsAttributes().
844
845 All lines must start with "P:" or "A:".
846 """
847 parser = SubmissionParser(self.log)
848 parser.submission_key = (
849 'Detect <sysfs-attributes> node with invalid main key.')
850 node = etree.fromstring(dedent("""
851 <sysfs-attributes>
852 P: /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
853 X: an invalid line
854 </sysfs-attributes>
855 """))
856 result = parser._parseSysfsAttributes(node)
857 self.assertEqual(
858 None, result,
859 'Invalid parsing result of a <sysfs-attributes> node containg '
860 'a line that does not start with "A:" or "P:".')
861 self.assertErrorMessage(
862 parser.submission_key,
863 "Line 2 in <sysfs-attributes>: Unexpected key: "
864 "'X: an invalid line'")
865
652 def testHardware(self):866 def testHardware(self):
653 """The <hardware> tag is converted into a dictionary."""867 """The <hardware> tag is converted into a dictionary."""
654 test = self868 test = self