Launchpad itself

Merge lp:~adeuring/launchpad/bug-460976-noise-reduction-for-hwdb-processing-log into lp:launchpad/db-devel

  1. bug-460976-noise-reduction-for-hwdb-processing-log
  2. Merge into db-devel
Proposed by Abel Deuring
Status: Merged
Merged at revision: not available
Proposed branch: lp:~adeuring/launchpad/bug-460976-noise-reduction-for-hwdb-processing-log
Merge into: lp:launchpad/db-devel
Diff against target: 671 lines
5 files modified
lib/canonical/launchpad/scripts/hwdbsubmissions.py (+69/-14)
lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py (+121/-96)
lib/canonical/launchpad/scripts/tests/test_hwdb_submission_processing.py (+14/-5)
lib/lp/testing/__init__.py (+67/-0)
lib/lp/testing/tests/test_inlinetests.py (+20/-0)
To merge this branch: bzr merge lp:~adeuring/launchpad/bug-460976-noise-reduction-for-hwdb-processing-log
Reviewer Review Type Date Requested Status
Eleanor Berger (community) code Approve
Review via email: mp+13950@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Abel Deuring (adeuring) wrote :
Download full text (8.5 KiB)

The property BaseDevice.raw_bus is used to determine if a udev/HAL node represents a real, physical device or if it defines only an "aspect" of a physical device (property BaseDevice.is_real_device). It is also used to determine if the submitted data contains enough details, like a vendor and product ID, for the given kind of device so that we can store it in our database (property BaseDevice.has_reliable_data).

In submissions with udev data we have a number of raw_bus values not used in HAL for nodes that either represent physical devices for which we don't have enough data or that represent only "aspects" or real devices. This leads to lots of warnings in the processing log.

This branch update BaseDevice.has_reliable_data and BaseDevice.is_real_device so that udev nodes with the "new" values of baseDevice.raw_bus are treated properly and do not produced warnings in the processing log.

test: ./bin/test --test=test_hwdb_submission_processing

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/canonical/launchpad/scripts/hwdbsubmissions.py
  lib/canonical/launchpad/scripts/tests/test_hwdb_submission_processing.py

== Pyflakes notices ==

lib/canonical/launchpad/scripts/hwdbsubmissions.py
    22: redefinition of unused 'etree' from line 20

== Pylint notices ==

lib/canonical/launchpad/scripts/hwdbsubmissions.py
    20: [F0401] Unable to import 'xml.etree.cElementTree' (No module named etree)

This branch is based on lp:~adeuring/launchpad/bug-460935-hwdb-better-consistency-check-udev-usb-devices, which is reviewed but is mot yet merged. The diff against this brnach:

=== modified file 'lib/canonical/launchpad/scripts/hwdbsubmissions.py'
--- lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-10-26 10:01:51 +0000
+++ lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-10-26 11:35:15 +0000
@@ -1969,6 +1969,40 @@

             info.bus == 'video4linux' is used for the "input aspect"
             of video devices.
+
+ 'ac97' is used in submissions with udev data for a sub-node
+ of sound devices.
+
+ 'hid' is used in submissions with udev data for a sub-node
+ of USB input devices.
+
+ 'drm_minor', 'pci_express', 'tifm_adapter', 'gameport',
+ 'spi_host', 'tifm', 'wlan' are used in submissions with
+ udev data for sub-nodes of PCI devices.
+
+ 'pcmcia_socket' is used in submissions with udev data for
+ a sub-node of PC Card and PCMCIA bridges.
+
+ 'ieee80211' is used in submissions with udev data for
+ sub-nodes IEEE 802.11 WLAN devices.
+
+ 'host', 'link' are used in submissions with udev data for
+ sub.nodes of bluetooth devices.
+
+ 'usb_host' and 'usbmon' are used in submissions with udev
+ data for sub-nodes of USB controllers.
+
+ 'usb_endpoint', 'usb-serial', 'lirc' are used in
+ submissions with udev data for sub-nodes of USB devices.
+
+ 'enclosure' is used in submissions with udev data for a
+ ...

Read more...

Revision history for this message
Eleanor Berger (intellectronica) :
review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/canonical/launchpad/scripts/hwdbsubmissions.py'
--- lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-10-22 11:08:18 +0000
+++ lib/canonical/launchpad/scripts/hwdbsubmissions.py 2009-10-26 12:05:33 +0000
@@ -1228,9 +1228,11 @@
1228 def checkUdevUsbProperties(self, udev_data):1228 def checkUdevUsbProperties(self, udev_data):
1229 """Validation of udev USB devices.1229 """Validation of udev USB devices.
12301230
1231 USB devices must have the properties DEVTYPE (value1231 USB devices must either have the three properties DEVTYPE
1232 'usb_device' or 'usb_interface'), PRODUCT and TYPE. PRODUCT1232 (value 'usb_device' or 'usb_interface'), PRODUCT and TYPE,
1233 must be a tuple of three integers in hexadecimal1233 or they must have none of them.
1234
1235 PRODUCT must be a tuple of three integers in hexadecimal
1234 representation, separates by '/'. TYPE must be a a tuple of1236 representation, separates by '/'. TYPE must be a a tuple of
1235 three integers in decimal representation, separated by '/'.1237 three integers in decimal representation, separated by '/'.
1236 usb_interface nodes must additionally have a property1238 usb_interface nodes must additionally have a property
@@ -1245,6 +1247,10 @@
1245 property_names = set(properties)1247 property_names = set(properties)
1246 existing_usb_properties = property_names.intersection(1248 existing_usb_properties = property_names.intersection(
1247 UDEV_USB_DEVICE_PROPERTIES)1249 UDEV_USB_DEVICE_PROPERTIES)
1250
1251 if len(existing_usb_properties) == 0:
1252 continue
1253
1248 if existing_usb_properties != UDEV_USB_DEVICE_PROPERTIES:1254 if existing_usb_properties != UDEV_USB_DEVICE_PROPERTIES:
1249 missing_properties = UDEV_USB_DEVICE_PROPERTIES.difference(1255 missing_properties = UDEV_USB_DEVICE_PROPERTIES.difference(
1250 existing_usb_properties)1256 existing_usb_properties)
@@ -1365,7 +1371,8 @@
1365 if ('udev' in parsed_data['hardware']1371 if ('udev' in parsed_data['hardware']
1366 and not self.checkConsistentUdevDeviceData(1372 and not self.checkConsistentUdevDeviceData(
1367 parsed_data['hardware']['udev'],1373 parsed_data['hardware']['udev'],
1368 parsed_data['hardware']['sysfs-attributes'])):1374 parsed_data['hardware']['sysfs-attributes'],
1375 parsed_data['hardware']['dmi'],)):
1369 return False1376 return False
1370 duplicate_ids = self.findDuplicateIDs(parsed_data)1377 duplicate_ids = self.findDuplicateIDs(parsed_data)
1371 if duplicate_ids:1378 if duplicate_ids:
@@ -1962,6 +1969,40 @@
19621969
1963 info.bus == 'video4linux' is used for the "input aspect"1970 info.bus == 'video4linux' is used for the "input aspect"
1964 of video devices.1971 of video devices.
1972
1973 'ac97' is used in submissions with udev data for a sub-node
1974 of sound devices.
1975
1976 'hid' is used in submissions with udev data for a sub-node
1977 of USB input devices.
1978
1979 'drm_minor', 'pci_express', 'tifm_adapter', 'gameport',
1980 'spi_host', 'tifm', 'wlan' are used in submissions with
1981 udev data for sub-nodes of PCI devices.
1982
1983 'pcmcia_socket' is used in submissions with udev data for
1984 a sub-node of PC Card and PCMCIA bridges.
1985
1986 'ieee80211' is used in submissions with udev data for
1987 sub-nodes IEEE 802.11 WLAN devices.
1988
1989 'host', 'link' are used in submissions with udev data for
1990 sub.nodes of bluetooth devices.
1991
1992 'usb_host' and 'usbmon' are used in submissions with udev
1993 data for sub-nodes of USB controllers.
1994
1995 'usb_endpoint', 'usb-serial', 'lirc' are used in
1996 submissions with udev data for sub-nodes of USB devices.
1997
1998 'enclosure' is used in submissions with udev data for a
1999 sub.node of SCSI devices.
2000
2001 'graphics' is used in submissions with udev data for a
2002 sub-node of graphics cards.
2003
2004 'hwmon' is is used in submissions with udev data in
2005 many sub-nodes.
1965 """2006 """
1966 # The root node is always a real device, but its raw_bus2007 # The root node is always a real device, but its raw_bus
1967 # property can have different values: None or 'Unknown' in2008 # property can have different values: None or 'Unknown' in
@@ -1974,13 +2015,15 @@
1974 # This set of buses is only used once; it's easier to have it2015 # This set of buses is only used once; it's easier to have it
1975 # here than to put it elsewhere and have to document its2016 # here than to put it elsewhere and have to document its
1976 # location and purpose.2017 # location and purpose.
1977 if bus in (None, 'disk', 'drm', 'dvb', 'memstick_host', 'net',2018 if bus in (None, 'ac97', 'disk', 'drm', 'drm_minor', 'dvb',
1978 'partition', 'scsi_disk', 'scsi_generic', 'scsi_host',2019 'enclosure', 'gameport', 'graphics', 'hid', 'host',
1979 'scsi_target', 'sound', 'spi_transport', 'ssb', 'tty',2020 'hwmon', 'ieee80211', 'link', 'lirc', 'memstick_host',
1980 'usb', 'usb_interface', 'video4linux', ):2021 'net', 'partition', 'pci_express', 'pcmcia_socket',
1981 #2022 'scsi_disk', 'scsi_generic', 'scsi_host', 'scsi_target',
1982 # The computer itself is the only HAL device without the2023 'sound', 'spi_host', 'spi_transport', 'ssb', 'tifm',
1983 # info.bus property that we treat as a real device.2024 'tifm_adapter', 'tty', 'usb', 'usb-serial', 'usb_endpoint',
2025 'usb_host', 'usb_interface', 'usbmon', 'video4linux',
2026 'wlan'):
1984 return False2027 return False
1985 elif bus == 'usb_device':2028 elif bus == 'usb_device':
1986 vendor_id = self.usb_vendor_id2029 vendor_id = self.usb_vendor_id
@@ -2088,6 +2131,16 @@
2088 for CPUs, power supply etc. Except for the main sytsem, none2131 for CPUs, power supply etc. Except for the main sytsem, none
2089 of them provides a vendor or product id, so we ignore them.2132 of them provides a vendor or product id, so we ignore them.
20902133
2134 raw_bus == 'video_output', 'thermal', 'vtconsole', 'bdi',
2135 'mem', 'ppp', 'vc', 'dmi', 'hidraw', 'hwmon', 'heci', 'rfkill',
2136 'i2c-adapter', 'ttm', 'ppdev', 'printer' is used in submissions
2137 with udev data for virtual devices.
2138
2139 'pci_bus' is used in submissions with udev data for a node
2140 describing a PCI bus.
2141
2142 'leds' is used in submissions with udev data to describe LEDs.
2143
2091 XXX Abel Deuring 2008-05-06: IEEE1394 devices are a bit2144 XXX Abel Deuring 2008-05-06: IEEE1394 devices are a bit
2092 nasty: The standard does not define any specification2145 nasty: The standard does not define any specification
2093 for product IDs or product names, hence HAL often uses2146 for product IDs or product names, hence HAL often uses
@@ -2115,9 +2168,11 @@
2115 # The root node is course a real device; storing data2168 # The root node is course a real device; storing data
2116 # about other devices with the bus "unkown" is pointless.2169 # about other devices with the bus "unkown" is pointless.
2117 return False2170 return False
2118 if bus in ('backlight', 'bluetooth', 'ieee1394', 'input', 'misc',2171 if bus in ('backlight', 'bdi', 'bluetooth', 'dmi', 'heci', 'hidraw',
2119 'mmc', 'mmc_host', 'pcmcia', 'platform', 'pnp',2172 'hwmon', 'i2c-adapter', 'ieee1394', 'input', 'leds', 'mem',
2120 'power_supply'):2173 'misc', 'mmc', 'mmc_host', 'pci_bus', 'pcmcia', 'platform',
2174 'pnp', 'power_supply', 'ppdev', 'ppp', 'printer', 'rfkill',
2175 'thermal', 'ttm', 'vc', 'video_output', 'vtconsole'):
2121 return False2176 return False
21222177
2123 # We identify devices by bus, vendor ID and product ID;2178 # We identify devices by bus, vendor ID and product ID;
21242179
=== modified file 'lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py'
--- lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-10-21 16:49:03 +0000
+++ lib/canonical/launchpad/scripts/tests/test_hwdb_submission_parser.py 2009-10-26 12:05:33 +0000
@@ -23,6 +23,7 @@
23 ROOT_UDI)23 ROOT_UDI)
24from canonical.testing import BaseLayer24from canonical.testing import BaseLayer
2525
26from lp.testing import validate_mock_class
2627
27class SubmissionParserTestParseSoftware(SubmissionParser):28class SubmissionParserTestParseSoftware(SubmissionParser):
28 """A Variant used to test SubmissionParser._parseSoftware.29 """A Variant used to test SubmissionParser._parseSoftware.
@@ -1928,27 +1929,41 @@
1928 "'/devices/pci0000:00/0000:00:1f.2'")1929 "'/devices/pci0000:00/0000:00:1f.2'")
19291930
1930 def testCheckUdevUsbProperties(self):1931 def testCheckUdevUsbProperties(self):
1931 """Test of SubmissionParser.checkUdevUsbProperties()."""1932 """Test of SubmissionParser.checkUdevUsbProperties().
1933
1934 udev nodes for USB devices must define the three properties
1935 DEVTYPE, PRODUCT, TYPE or none of them.
1936 """
1932 parser = SubmissionParser()1937 parser = SubmissionParser()
1933 self.assertTrue(parser.checkUdevUsbProperties(1938 self.assertTrue(parser.checkUdevUsbProperties(
1934 [self.udev_root_device, self.udev_usb_device,1939 [self.udev_root_device, self.udev_usb_device,
1935 self.udev_usb_interface]))1940 self.udev_usb_interface]))
19361941
1942 for property_name in ('DEVTYPE', 'PRODUCT', 'TYPE'):
1943 del self.udev_usb_device['E'][property_name]
1944 self.assertTrue(parser.checkUdevUsbProperties(
1945 [self.udev_root_device, self.udev_usb_device,
1946 self.udev_usb_interface]))
1947
1937 def testCheckUdevUsbProperties_missing_required_property(self):1948 def testCheckUdevUsbProperties_missing_required_property(self):
1938 """Test of SubmissionParser.checkUdevUsbProperties().1949 """Test of SubmissionParser.checkUdevUsbProperties().
19391950
1940 A USB device that does not have a required property makes a1951 A USB device where some but not all of the properties DEVTYPE,
1941 submission invalid.1952 PRODUCT, TYPE are defined makes a submission invalid.
1942 """1953 """
1943 del self.udev_usb_device['E']['DEVTYPE']1954 for property_name in ('DEVTYPE', 'PRODUCT', 'TYPE'):
1944 parser = SubmissionParser(self.log)1955 saved_property = self.udev_usb_device['E'].pop(property_name)
1945 parser.submission_key = 'USB device without DEVTYPE property'1956 parser = SubmissionParser(self.log)
1946 self.assertFalse(parser.checkUdevUsbProperties(1957 parser.submission_key = (
1947 [self.udev_root_device, self.udev_usb_device]))1958 'USB device without %s property' % property_name)
1948 self.assertErrorMessage(1959 self.assertFalse(parser.checkUdevUsbProperties(
1949 parser.submission_key,1960 [self.udev_root_device, self.udev_usb_device]))
1950 "USB udev device found without required properties: "1961 self.assertErrorMessage(
1951 "set(['DEVTYPE']) '/devices/pci0000:00/0000:00:1d.1/usb3/3-2'")1962 parser.submission_key,
1963 "USB udev device found without required properties: "
1964 "set(['%s']) '/devices/pci0000:00/0000:00:1d.1/usb3/3-2'"
1965 % property_name)
1966 self.udev_usb_device['E'][property_name] = saved_property
19521967
1953 def testCheckUdevUsbProperties_with_invalid_product_id(self):1968 def testCheckUdevUsbProperties_with_invalid_product_id(self):
1954 """Test of SubmissionParser.checkUdevUsbProperties().1969 """Test of SubmissionParser.checkUdevUsbProperties().
@@ -2102,7 +2117,7 @@
21022117
2103 All shortcut methods return True.2118 All shortcut methods return True.
2104 """2119 """
2105 def checkUdevDictsHavePathKey(self, udev_data):2120 def checkUdevDictsHavePathKey(self, udev_nodes):
2106 """See `SubmissionParser`."""2121 """See `SubmissionParser`."""
2107 return True2122 return True
21082123
@@ -2114,7 +2129,7 @@
2114 """See `SubmissionParser`."""2129 """See `SubmissionParser`."""
2115 return True2130 return True
21162131
2117 def checkUdevScsiProperties(self, udev_data, syfs_data):2132 def checkUdevScsiProperties(self, udev_data, sysfs_data):
2118 """See `SubmissionParser`."""2133 """See `SubmissionParser`."""
2119 return True2134 return True
21202135
@@ -2122,6 +2137,8 @@
2122 """See `SubmissionParser`."""2137 """See `SubmissionParser`."""
2123 return True2138 return True
21242139
2140 validate_mock_class(UdevTestSubmissionParser)
2141
2125 def testCheckConsistentUdevDeviceData(self):2142 def testCheckConsistentUdevDeviceData(self):
2126 """Test of SubmissionParser.checkConsistentUdevDeviceData(),"""2143 """Test of SubmissionParser.checkConsistentUdevDeviceData(),"""
2127 parser = self.UdevTestSubmissionParser()2144 parser = self.UdevTestSubmissionParser()
@@ -2137,10 +2154,12 @@
2137 self.UdevTestSubmissionParser):2154 self.UdevTestSubmissionParser):
2138 """A SubmissionPaser where checkUdevDictsHavePathKey() fails."""2155 """A SubmissionPaser where checkUdevDictsHavePathKey() fails."""
21392156
2140 def checkUdevDictsHavePathKey(self, udev_data):2157 def checkUdevDictsHavePathKey(self, udev_nodes):
2141 """See `SubmissionParser`."""2158 """See `SubmissionParser`."""
2142 return False2159 return False
21432160
2161 validate_mock_class(SubmissionParserUdevPathCheckFails)
2162
2144 parser = SubmissionParserUdevPathCheckFails()2163 parser = SubmissionParserUdevPathCheckFails()
2145 self.assertFalse(parser.checkConsistentUdevDeviceData(2164 self.assertFalse(parser.checkConsistentUdevDeviceData(
2146 None, None, None))2165 None, None, None))
@@ -2158,6 +2177,8 @@
2158 """See `SubmissionParser`."""2177 """See `SubmissionParser`."""
2159 return False2178 return False
21602179
2180 validate_mock_class(SubmissionParserUdevPciCheckFails)
2181
2161 parser = SubmissionParserUdevPciCheckFails()2182 parser = SubmissionParserUdevPciCheckFails()
2162 self.assertFalse(parser.checkConsistentUdevDeviceData(2183 self.assertFalse(parser.checkConsistentUdevDeviceData(
2163 None, None, None))2184 None, None, None))
@@ -2175,6 +2196,8 @@
2175 """See `SubmissionParser`."""2196 """See `SubmissionParser`."""
2176 return False2197 return False
21772198
2199 validate_mock_class(SubmissionParserUdevUsbCheckFails)
2200
2178 parser = SubmissionParserUdevUsbCheckFails()2201 parser = SubmissionParserUdevUsbCheckFails()
2179 self.assertFalse(parser.checkConsistentUdevDeviceData(2202 self.assertFalse(parser.checkConsistentUdevDeviceData(
2180 None, None, None))2203 None, None, None))
@@ -2192,6 +2215,8 @@
2192 """See `SubmissionParser`."""2215 """See `SubmissionParser`."""
2193 return False2216 return False
21942217
2218 validate_mock_class(SubmissionParserUdevUsbCheckFails)
2219
2195 parser = SubmissionParserUdevUsbCheckFails()2220 parser = SubmissionParserUdevUsbCheckFails()
2196 self.assertFalse(parser.checkConsistentUdevDeviceData(2221 self.assertFalse(parser.checkConsistentUdevDeviceData(
2197 None, None, None))2222 None, None, None))
@@ -2209,55 +2234,38 @@
2209 """See `SubmissionParser`."""2234 """See `SubmissionParser`."""
2210 return False2235 return False
22112236
2237 validate_mock_class(SubmissionParserUdevUsbCheckFails)
2238
2212 parser = SubmissionParserUdevUsbCheckFails()2239 parser = SubmissionParserUdevUsbCheckFails()
2213 self.assertFalse(parser.checkConsistentUdevDeviceData(2240 self.assertFalse(parser.checkConsistentUdevDeviceData(
2214 None, None, None))2241 None, None, None))
22152242
2216 def _setupConsistencyCheckParser(self):2243 class MockSubmissionParser(SubmissionParser):
2217 """Prepare and return a SubmissionParser instance.2244 """A SubmissionParser variant for testing checkCOnsistentData()
22182245
2219 All "method substitutes" return a valid result.2246 All "method substitutes" return a valid result.
2220 """2247 """
2221 test = self2248
2222 def findDuplicateIDs(self, parsed_data):2249 def findDuplicateIDs(self, parsed_data):
2223 test.assertTrue(isinstance(self, SubmissionParser))
2224 return set()2250 return set()
22252251
2226 def findInvalidIDReferences(self, parsed_data):2252 def findInvalidIDReferences(self, parsed_data):
2227 test.assertTrue(isinstance(self, SubmissionParser))
2228 return set()2253 return set()
22292254
2230 def getUDIDeviceMap(self, devices):2255 def getUDIDeviceMap(self, devices):
2231 test.assertTrue(isinstance(self, SubmissionParser))
2232 return {}2256 return {}
22332257
2234 def getUDIChildren(self, udi_device_map):2258 def getUDIChildren(self, udi_device_map):
2235 test.assertTrue(isinstance(self, SubmissionParser))
2236 return {}2259 return {}
22372260
2238 def checkHALDevicesParentChildConsistency(self, devices):2261 def checkHALDevicesParentChildConsistency(self, udi_children):
2239 test.assertTrue(isinstance(self, SubmissionParser))
2240 return []2262 return []
22412263
2242 def checkConsistentUdevDeviceData(self, udev_data, sysfs_data):2264 def checkConsistentUdevDeviceData(
2265 self, udev_data, sysfs_data, dmi_data):
2243 return True2266 return True
22442267
2245 parser = SubmissionParser(self.log)2268 validate_mock_class(MockSubmissionParser)
2246 parser.findDuplicateIDs = (
2247 lambda parsed_data: findDuplicateIDs(parser, parsed_data))
2248 parser.findInvalidIDReferences = (
2249 lambda parsed_data: findInvalidIDReferences(parser, parsed_data))
2250 parser.getUDIDeviceMap = (
2251 lambda devices: getUDIDeviceMap(parser, devices))
2252 parser.getUDIChildren = (
2253 lambda udi_device_map: getUDIChildren(parser, udi_device_map))
2254 parser.checkHALDevicesParentChildConsistency = (
2255 lambda udi_children: checkHALDevicesParentChildConsistency(
2256 parser, udi_children))
2257 parser.checkConsistentUdevDeviceData = (
2258 lambda udev_data, sysfs_data: checkConsistentUdevDeviceData(
2259 parser, udev_data, sysfs_data))
2260 return parser
22612269
2262 def assertErrorMessage(self, submission_key, log_message):2270 def assertErrorMessage(self, submission_key, log_message):
2263 """Search for message in the log entries for submission_key.2271 """Search for message in the log entries for submission_key.
@@ -2309,7 +2317,7 @@
23092317
2310 def testConsistencyCheck(self):2318 def testConsistencyCheck(self):
2311 """Test of SubmissionParser.checkConsistency."""2319 """Test of SubmissionParser.checkConsistency."""
2312 parser = self._setupConsistencyCheckParser()2320 parser = self.MockSubmissionParser()
2313 result = parser.checkConsistency({'hardware':2321 result = parser.checkConsistency({'hardware':
2314 {'hal': {'devices': []}}})2322 {'hal': {'devices': []}}})
2315 self.assertEqual(result, True,2323 self.assertEqual(result, True,
@@ -2318,45 +2326,53 @@
23182326
2319 def testConsistencyCheckValidUdevData(self):2327 def testConsistencyCheckValidUdevData(self):
2320 """Test of SubmissionParser.checkConsistency."""2328 """Test of SubmissionParser.checkConsistency."""
2321 parser = self._setupConsistencyCheckParser()2329 parser = self.MockSubmissionParser()
2322 self.assertTrue(parser.checkConsistency(2330 self.assertTrue(parser.checkConsistency(
2323 {2331 {
2324 'hardware': {2332 'hardware': {
2325 'udev': [],2333 'udev': None,
2326 'sysfs-attributes': []2334 'sysfs-attributes': None,
2335 'dmi': None,
2327 }2336 }
2328 }2337 }
2329 ))2338 ))
23302339
2331 def testConsistencyCheck_invalid_udev_data(self):2340 def testConsistencyCheck_invalid_udev_data(self):
2332 """Test of SubmissionParser.checkConsistency."""2341 """Test of SubmissionParser.checkConsistency."""
2333 def checkConsistentUdevDeviceData(self, udev_data, sysfs_data):2342 class MockSubmissionParserBadUdevDeviceData(
2334 return False2343 self.MockSubmissionParser):
23352344 """A parser where checkConsistentUdevDeviceData() fails."""
2336 parser = self._setupConsistencyCheckParser()2345
2337 parser.checkConsistentUdevDeviceData = (2346 def checkConsistentUdevDeviceData(self, udev_data, sysfs_data,
2338 lambda udev_data, sysfs_data: checkConsistentUdevDeviceData(2347 dmi_data):
2339 parser, udev_data, sysfs_data))2348 return False
2349
2350 validate_mock_class(MockSubmissionParserBadUdevDeviceData)
2351
2352 parser = MockSubmissionParserBadUdevDeviceData()
2340 self.assertFalse(parser.checkConsistency(2353 self.assertFalse(parser.checkConsistency(
2341 {2354 {
2342 'hardware': {2355 'hardware': {
2343 'udev': [{}],2356 'udev': None,
2344 'sysfs-attributes': []2357 'sysfs-attributes': None,
2358 'dmi': None,
2345 }2359 }
2346 }2360 }
2347 ))2361 ))
23482362
2349 def testConsistencyCheckWithDuplicateIDs(self):2363 def testConsistencyCheckWithDuplicateIDs(self):
2350 """SubmissionParser.checkConsistency detects duplicate IDs."""2364 """SubmissionParser.checkConsistency detects duplicate IDs."""
2351 test = self2365 class MockSubmissionParserDuplicateIds(
2352 def findDuplicateIDs(self, parsed_data):2366 self.MockSubmissionParser):
2353 test.assertTrue(isinstance(self, SubmissionParser))2367 """A parser where findDuplicateIDs() fails."""
2354 return set([1])2368
23552369 def findDuplicateIDs(self, parsed_data):
2356 parser = self._setupConsistencyCheckParser()2370 return set([1])
2371
2372 validate_mock_class(MockSubmissionParserDuplicateIds)
2373
2374 parser = MockSubmissionParserDuplicateIds(self.log)
2357 parser.submission_key = 'Consistency check detects duplicate IDs'2375 parser.submission_key = 'Consistency check detects duplicate IDs'
2358 parser.findDuplicateIDs = (
2359 lambda parsed_data: findDuplicateIDs(parser, parsed_data))
2360 result = parser.checkConsistency({'hardware':2376 result = parser.checkConsistency({'hardware':
2361 {'hal': {'devices': []}}})2377 {'hal': {'devices': []}}})
2362 self.assertEqual(result, False,2378 self.assertEqual(result, False,
@@ -2366,15 +2382,16 @@
23662382
2367 def testConsistencyCheckWithInvalidIDReferences(self):2383 def testConsistencyCheckWithInvalidIDReferences(self):
2368 """SubmissionParser.checkConsistency detects invalid ID references."""2384 """SubmissionParser.checkConsistency detects invalid ID references."""
2369 test = self2385 class MockSubmissionParserInvalidIDReferences(
2370 def findInvalidIDReferences(self, parsed_data):2386 self.MockSubmissionParser):
2371 test.assertTrue(isinstance(self, SubmissionParser))2387 """A parser where findInvalidIDReferences() fails."""
2372 return set([1])2388 def findInvalidIDReferences(self, parsed_data):
23732389 return set([1])
2374 parser = self._setupConsistencyCheckParser()2390
2391 validate_mock_class(MockSubmissionParserInvalidIDReferences)
2392
2393 parser = MockSubmissionParserInvalidIDReferences(self.log)
2375 parser.submission_key = 'Consistency check detects invalid ID refs'2394 parser.submission_key = 'Consistency check detects invalid ID refs'
2376 parser.findInvalidIDReferences = (
2377 lambda parsed_data: findInvalidIDReferences(parser, parsed_data))
2378 result = parser.checkConsistency({'hardware':2395 result = parser.checkConsistency({'hardware':
2379 {'hal': {'devices': []}}})2396 {'hal': {'devices': []}}})
2380 self.assertEqual(result, False,2397 self.assertEqual(result, False,
@@ -2384,16 +2401,18 @@
23842401
2385 def testConsistencyCheckWithDuplicateUDI(self):2402 def testConsistencyCheckWithDuplicateUDI(self):
2386 """SubmissionParser.checkConsistency detects duplicate UDIs."""2403 """SubmissionParser.checkConsistency detects duplicate UDIs."""
2387 test = self2404 class MockSubmissionParserUDIDeviceMapFails(
2388 def getUDIDeviceMap(self, parsed_data):2405 self.MockSubmissionParser):
2389 test.assertTrue(isinstance(self, SubmissionParser))2406 """A parser where getUDIDeviceMap() fails."""
2390 raise ValueError(2407
2391 'Duplicate UDI: /org/freedesktop/Hal/devices/computer')2408 def getUDIDeviceMap(self, devices):
23922409 raise ValueError(
2393 parser = self._setupConsistencyCheckParser()2410 'Duplicate UDI: /org/freedesktop/Hal/devices/computer')
2411
2412 validate_mock_class(MockSubmissionParserUDIDeviceMapFails)
2413
2414 parser = MockSubmissionParserUDIDeviceMapFails(self.log)
2394 parser.submission_key = 'Consistency check detects invalid ID refs'2415 parser.submission_key = 'Consistency check detects invalid ID refs'
2395 parser.getUDIDeviceMap = (
2396 lambda devices: getUDIDeviceMap(parser, devices))
2397 result = parser.checkConsistency({'hardware':2416 result = parser.checkConsistency({'hardware':
2398 {'hal': {'devices': []}}})2417 {'hal': {'devices': []}}})
2399 self.assertEqual(result, False,2418 self.assertEqual(result, False,
@@ -2404,15 +2423,17 @@
24042423
2405 def testConsistencyCheckChildUDIWithoutParent(self):2424 def testConsistencyCheckChildUDIWithoutParent(self):
2406 """SubmissionParser.checkConsistency detects "orphaned" devices."""2425 """SubmissionParser.checkConsistency detects "orphaned" devices."""
2407 test = self2426 class MockSubmissionParserUDIChildrenFails(
2408 def getUDIChildren(self, udi_device_map):2427 self.MockSubmissionParser):
2409 test.assertTrue(isinstance(self, SubmissionParser))2428 """A parser where getUDIChildren() fails."""
2410 raise ValueError('Unknown parent UDI /foo in <device id="3">')2429
24112430 def getUDIChildren(self, udi_device_map):
2412 parser = self._setupConsistencyCheckParser()2431 raise ValueError('Unknown parent UDI /foo in <device id="3">')
2432
2433 validate_mock_class(MockSubmissionParserUDIChildrenFails)
2434
2435 parser = MockSubmissionParserUDIChildrenFails(self.log)
2413 parser.submission_key = 'Consistency check detects invalid ID refs'2436 parser.submission_key = 'Consistency check detects invalid ID refs'
2414 parser.getUDIChildren = (
2415 lambda udi_device_map: getUDIChildren(parser, udi_device_map))
2416 result = parser.checkConsistency({'hardware':2437 result = parser.checkConsistency({'hardware':
2417 {'hal': {'devices': []}}})2438 {'hal': {'devices': []}}})
2418 self.assertEqual(result, False,2439 self.assertEqual(result, False,
@@ -2423,17 +2444,21 @@
24232444
2424 def testConsistencyCheckCircularParentChildRelation(self):2445 def testConsistencyCheckCircularParentChildRelation(self):
2425 """SubmissionParser.checkConsistency detects "orphaned" devices."""2446 """SubmissionParser.checkConsistency detects "orphaned" devices."""
2426 test = self2447 class MockSubmissionParserHALDevicesParentChildConsistency(
2427 def checkHALDevicesParentChildConsistency(self, devices):2448 self.MockSubmissionParser):
2428 test.assertTrue(isinstance(self, SubmissionParser))2449 """A parser where checkHALDevicesParentChildConsistency() fails.
2429 return ['/foo', '/bar']2450 """
24302451
2431 parser = self._setupConsistencyCheckParser()2452 def checkHALDevicesParentChildConsistency(self, udi_children):
2453 return ['/foo', '/bar']
2454
2455 validate_mock_class(
2456 MockSubmissionParserHALDevicesParentChildConsistency)
2457
2458 parser = MockSubmissionParserHALDevicesParentChildConsistency(
2459 self.log)
2432 parser.submission_key = ('Consistency check detects circular '2460 parser.submission_key = ('Consistency check detects circular '
2433 'parent-child relationships')2461 'parent-child relationships')
2434 parser.checkHALDevicesParentChildConsistency = (
2435 lambda devices: checkHALDevicesParentChildConsistency(
2436 parser, devices))
2437 result = parser.checkConsistency({'hardware':2462 result = parser.checkConsistency({'hardware':
2438 {'hal': {'devices': []}}})2463 {'hal': {'devices': []}}})
2439 self.assertEqual(result, False,2464 self.assertEqual(result, False,
24402465
=== modified file 'lib/canonical/launchpad/scripts/tests/test_hwdb_submission_processing.py'
--- lib/canonical/launchpad/scripts/tests/test_hwdb_submission_processing.py 2009-10-22 11:08:18 +0000
+++ lib/canonical/launchpad/scripts/tests/test_hwdb_submission_processing.py 2009-10-26 12:05:33 +0000
@@ -1672,8 +1672,14 @@
1672 parser = SubmissionParser(self.log)1672 parser = SubmissionParser(self.log)
16731673
1674 ignored_buses = (1674 ignored_buses = (
1675 'drm', 'dvb', 'memstick_host', 'net', 'scsi_generic', 'scsi_host',1675 'ac97', 'disk', 'drm', 'drm_minor', 'dvb', 'enclosure',
1676 'sound', 'ssb', 'tty', 'usb', 'video4linux', )1676 'gameport', 'hid', 'host', 'ieee80211', 'link', 'lirc',
1677 'memstick_host', 'net', 'partition', 'pci_express',
1678 'pcmcia_socket', 'scsi_disk', 'scsi_generic', 'scsi_host',
1679 'scsi_target', 'sound', 'spi_host', 'spi_transport', 'ssb',
1680 'tifm', 'tifm_adapter', 'tty', 'usb', 'usb-serial',
1681 'usb_endpoint', 'usb_host', 'usb_interface', 'video4linux',
1682 'wlan')
1677 for tested_bus in ignored_buses:1683 for tested_bus in ignored_buses:
1678 properties['info.bus'] = (tested_bus, 'str')1684 properties['info.bus'] = (tested_bus, 'str')
1679 parser.buildHalDeviceList(parsed_data)1685 parser.buildHalDeviceList(parsed_data)
@@ -2004,9 +2010,12 @@
2004 }2010 }
2005 parser = SubmissionParser(self.log)2011 parser = SubmissionParser(self.log)
2006 properties = devices[0]['properties']2012 properties = devices[0]['properties']
2007 for bus in ('backlight', 'bluetooth', 'ieee1394', 'input', 'misc',2013 for bus in ('backlight', 'bdi', 'bluetooth', 'dmi', 'heci', 'hidraw',
2008 'mmc', 'mmc_host', 'pcmcia', 'platform', 'pnp',2014 'hwmon', 'i2c-adapter', 'ieee1394', 'input', 'leds', 'mem',
2009 'power_supply', 'unknown'):2015 'misc', 'mmc', 'mmc_host', 'pci_bus', 'pcmcia', 'platform',
2016 'pnp', 'power_supply', 'ppdev', 'ppp', 'printer', 'rfkill',
2017 'thermal', 'ttm', 'unknown', 'vc', 'video_output',
2018 'vtconsole'):
2010 properties['info.bus'] = (bus, 'str')2019 properties['info.bus'] = (bus, 'str')
2011 parser.buildHalDeviceList(parsed_data)2020 parser.buildHalDeviceList(parsed_data)
2012 device = parser.devices[self.UDI_SATA_CONTROLLER]2021 device = parser.devices[self.UDI_SATA_CONTROLLER]
20132022
=== modified file 'lib/lp/testing/__init__.py'
--- lib/lp/testing/__init__.py 2009-10-23 11:07:32 +0000
+++ lib/lp/testing/__init__.py 2009-10-26 12:05:33 +0000
@@ -8,6 +8,7 @@
8from datetime import datetime, timedelta8from datetime import datetime, timedelta
9from pprint import pformat9from pprint import pformat
10import copy10import copy
11from inspect import getargspec, getmembers, getmro, isclass, ismethod
11import os12import os
12import shutil13import shutil
13import subprocess14import subprocess
@@ -793,3 +794,69 @@
793 tree.unlock()794 tree.unlock()
794795
795 return contents796 return contents
797
798def validate_mock_class(mock_class):
799 """Validate method signatures in mock classes derived from real classes.
800
801 We often use mock classes in tests which are derived from real
802 classes.
803
804 This function ensures that methods redefined in the mock
805 class have the same signature as the corresponding methods of
806 the base class.
807
808 >>> class A:
809 ...
810 ... def method_one(self, a):
811 ... pass
812
813 >>>
814 >>> class B(A):
815 ... def method_one(self, a):
816 ... pass
817 >>> validate_mock_class(B)
818
819 If a class derived from A defines method_one with a different
820 signature, we get an AssertionError.
821
822 >>> class C(A):
823 ... def method_one(self, a, b):
824 ... pass
825 >>> validate_mock_class(C)
826 Traceback (most recent call last):
827 ...
828 AssertionError: Different method signature for method_one:...
829
830 Even a parameter name must not be modified.
831
832 >>> class D(A):
833 ... def method_one(self, b):
834 ... pass
835 >>> validate_mock_class(D)
836 Traceback (most recent call last):
837 ...
838 AssertionError: Different method signature for method_one:...
839
840 If validate_mock_class() for anything but a class, we get an
841 AssertionError.
842
843 >>> validate_mock_class('a string')
844 Traceback (most recent call last):
845 ...
846 AssertionError: validate_mock_class() must be called for a class
847 """
848 assert isclass(mock_class), (
849 "validate_mock_class() must be called for a class")
850 base_classes = getmro(mock_class)
851 for name, obj in getmembers(mock_class):
852 if ismethod(obj):
853 for base_class in base_classes[1:]:
854 if name in base_class.__dict__:
855 mock_args = getargspec(obj)
856 real_args = getargspec(base_class.__dict__[name])
857 if mock_args != real_args:
858 raise AssertionError(
859 'Different method signature for %s: %r %r' % (
860 name, mock_args, real_args))
861 else:
862 break
796863
=== added file 'lib/lp/testing/tests/test_inlinetests.py'
--- lib/lp/testing/tests/test_inlinetests.py 1970-01-01 00:00:00 +0000
+++ lib/lp/testing/tests/test_inlinetests.py 2009-10-26 12:05:33 +0000
@@ -0,0 +1,20 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Run the doc string tests."""
5
6import doctest
7
8from zope.testing.doctest import NORMALIZE_WHITESPACE, ELLIPSIS
9
10from canonical.launchpad.testing.systemdocs import LayeredDocFileSuite
11from canonical.testing import BaseLayer
12from lp import testing
13
14def test_suite():
15 suite = LayeredDocFileSuite(
16 layer=BaseLayer)
17 suite.addTest(doctest.DocTestSuite(
18 testing, optionflags=NORMALIZE_WHITESPACE|ELLIPSIS))
19 return suite
20

Subscribers

People subscribed via source and target branches

to status/vote changes: