subunit

Merge lp:~lifeless/subunit/filter into lp:~subunit/subunit/trunk

  1. filter
  2. Merge into trunk
Proposed by Jonathan Lange
Status: Merged
Approved by: Jonathan Lange
Approved revision: 56
Merged at revision: not available
Proposed branch: lp:~lifeless/subunit/filter
Merge into: lp:~subunit/subunit/trunk
Diff against target: None lines
To merge this branch: bzr merge lp:~lifeless/subunit/filter
Reviewer Review Type Date Requested Status
Jonathan Lange Approve
Review via email: mp+4006@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Jonathan Lange (jml) wrote :

Add filters for listing a stream and selecting only pass|failed|errored tests.

lp:~lifeless/subunit/filter updated
57. By Robert Collins

Update skip support in python to be in line with testtools.

58. By Robert Collins

subunit-filter can now filter skips too.

Revision history for this message
Jonathan Lange (jml) wrote :

Two things!
* Rename FilterResult in subunit-ls so the name indicates what kind of filtering it does. ListingFilterResult, for example.
* Spurious blank line at python/subunit/tests/test_subunit_filter.py:307

review: Approve
Revision history for this message
Jonathan Lange (jml) wrote :

I meant line 307 of the diff, sorry.

lp:~lifeless/subunit/filter updated
59. By Robert Collins

Review feedback on filters.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'README'
2--- README 2009-02-15 11:55:00 +0000
3+++ README 2009-02-22 13:18:04 +0000
4@@ -40,6 +40,8 @@
5 stream on-the-fly. Currently subunit provides:
6 * tap2subunit - convert perl's TestAnythingProtocol to subunit.
7 * subunit2pyunit - convert a subunit stream to pyunit test results.
8+ * subunit-filter - filter out tests from a subunit stream.
9+ * subunit-ls - list the tests present in a subunit stream.
10 * subunit-stats - generate a summary of a subunit stream.
11 * subunit-tags - add or remove tags from a stream.
12
13
14=== added file 'filters/subunit-filter'
15--- filters/subunit-filter 1970-01-01 00:00:00 +0000
16+++ filters/subunit-filter 2009-02-22 07:51:04 +0000
17@@ -0,0 +1,50 @@
18+#!/usr/bin/env python
19+# subunit: extensions to python unittest to get test results from subprocesses.
20+# Copyright (C) 2008 Robert Collins <robertc@robertcollins.net>
21+#
22+# This program is free software; you can redistribute it and/or modify
23+# it under the terms of the GNU General Public License as published by
24+# the Free Software Foundation; either version 2 of the License, or
25+# (at your option) any later version.
26+#
27+# This program is distributed in the hope that it will be useful,
28+# but WITHOUT ANY WARRANTY; without even the implied warranty of
29+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
30+# GNU General Public License for more details.
31+#
32+# You should have received a copy of the GNU General Public License
33+# along with this program; if not, write to the Free Software
34+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
35+#
36+
37+"""Filter a subunit stream to include/exclude tests.
38+
39+The default is to strip successful tests.
40+"""
41+
42+from optparse import OptionParser
43+import sys
44+import unittest
45+
46+from subunit import ProtocolTestCase, TestResultFilter, TestProtocolClient
47+
48+parser = OptionParser(description=__doc__)
49+parser.add_option("--error", action="store_false",
50+ help="include errors", default=False, dest="error")
51+parser.add_option("-e", "--no-error", action="store_true",
52+ help="exclude errors", dest="error")
53+parser.add_option("--failure", action="store_false",
54+ help="include failures", default=False, dest="failure")
55+parser.add_option("-f", "--no-failure", action="store_true",
56+ help="include failures", dest="failure")
57+parser.add_option("-s", "--success", action="store_false",
58+ help="include successes", dest="success")
59+parser.add_option("--no-success", action="store_true",
60+ help="exclude successes", default=True, dest="success")
61+(options, args) = parser.parse_args()
62+result = TestProtocolClient(sys.stdout)
63+result = TestResultFilter(result, filter_error=options.error, filter_failure=options.failure,
64+ filter_success=options.success)
65+test = ProtocolTestCase(sys.stdin)
66+test.run(result)
67+sys.exit(0)
68
69=== added file 'filters/subunit-ls'
70--- filters/subunit-ls 1970-01-01 00:00:00 +0000
71+++ filters/subunit-ls 2009-02-23 10:54:28 +0000
72@@ -0,0 +1,62 @@
73+#!/usr/bin/env python
74+# subunit: extensions to python unittest to get test results from subprocesses.
75+# Copyright (C) 2008 Robert Collins <robertc@robertcollins.net>
76+#
77+# This program is free software; you can redistribute it and/or modify
78+# it under the terms of the GNU General Public License as published by
79+# the Free Software Foundation; either version 2 of the License, or
80+# (at your option) any later version.
81+#
82+# This program is distributed in the hope that it will be useful,
83+# but WITHOUT ANY WARRANTY; without even the implied warranty of
84+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
85+# GNU General Public License for more details.
86+#
87+# You should have received a copy of the GNU General Public License
88+# along with this program; if not, write to the Free Software
89+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
90+#
91+
92+"""List tests in a subunit stream."""
93+
94+import sys
95+import unittest
96+
97+from subunit import ProtocolTestCase
98+
99+class FilterResult(unittest.TestResult):
100+ """Filter test objects for display."""
101+
102+ def __init__(self, stream):
103+ """Create a FilterResult object outputting to stream."""
104+ unittest.TestResult.__init__(self)
105+ self._stream = stream
106+ self.failed_tests = 0
107+
108+ def addError(self, test, err):
109+ self.failed_tests += 1
110+ self.reportTest(test)
111+
112+ def addFailure(self, test, err):
113+ self.failed_tests += 1
114+ self.reportTest(test)
115+
116+ def addSuccess(self, test):
117+ self.reportTest(test)
118+
119+ def reportTest(self, test):
120+ self._stream.write(test.id() + '\n')
121+
122+ def wasSuccessful(self):
123+ "Tells whether or not this result was a success"
124+ return self.failed_tests == 0
125+
126+
127+result = FilterResult(sys.stdout)
128+test = ProtocolTestCase(sys.stdin)
129+test.run(result)
130+if result.wasSuccessful():
131+ exit_code = 0
132+else:
133+ exit_code = 1
134+sys.exit(exit_code)
135
136=== modified file 'python/subunit/__init__.py'
137--- python/subunit/__init__.py 2009-02-15 11:55:00 +0000
138+++ python/subunit/__init__.py 2009-02-23 10:54:28 +0000
139@@ -363,7 +363,7 @@
140 return self.__description
141
142 def id(self):
143- return "%s.%s" % (self._strclass(), self.__description)
144+ return "%s" % (self.__description,)
145
146 def __str__(self):
147 return "%s (%s)" % (self.__description, self._strclass())
148@@ -681,3 +681,52 @@
149 def wasSuccessful(self):
150 """Tells whether or not this result was a success"""
151 return self.failed_tests == 0
152+
153+
154+class TestResultFilter(unittest.TestResult):
155+ """A pyunit TestResult interface implementation which filters tests.
156+
157+ Tests that pass the filter are handed on to another TestResult instance
158+ for further processing/reporting. To obtain the filtered results,
159+ the other instance must be interrogated.
160+
161+ :ivar result: The result that tests are passed to after filtering.
162+ """
163+
164+ def __init__(self, result, filter_error=False, filter_failure=False,
165+ filter_success=True):
166+ """Create a FilterResult object filtering to result.
167+
168+ :param filter_error: Filter out errors.
169+ :param filter_failure: Filter out failures.
170+ :param filter_success: Filter out successful tests.
171+ """
172+ unittest.TestResult.__init__(self)
173+ self.result = result
174+ self._filter_error = filter_error
175+ self._filter_failure = filter_failure
176+ self._filter_success = filter_success
177+
178+ def addError(self, test, err):
179+ if not self._filter_error:
180+ self.result.startTest(test)
181+ self.result.addError(test, err)
182+ self.result.stopTest(test)
183+
184+ def addFailure(self, test, err):
185+ if not self._filter_failure:
186+ self.result.startTest(test)
187+ self.result.addFailure(test, err)
188+ self.result.stopTest(test)
189+
190+ def addSuccess(self, test):
191+ if not self._filter_success:
192+ self.result.startTest(test)
193+ self.result.addSuccess(test)
194+ self.result.stopTest(test)
195+
196+ def id_to_orig_id(self, id):
197+ if id.startswith("subunit.RemotedTestCase."):
198+ return id[len("subunit.RemotedTestCase."):]
199+ return id
200+
201
202=== modified file 'python/subunit/tests/__init__.py'
203--- python/subunit/tests/__init__.py 2008-12-09 01:00:03 +0000
204+++ python/subunit/tests/__init__.py 2009-02-22 06:28:08 +0000
205@@ -19,6 +19,7 @@
206
207 from subunit.tests import (
208 TestUtil,
209+ test_subunit_filter,
210 test_subunit_stats,
211 test_subunit_tags,
212 test_tap2subunit,
213@@ -29,6 +30,7 @@
214 result = TestUtil.TestSuite()
215 result.addTest(test_test_protocol.test_suite())
216 result.addTest(test_tap2subunit.test_suite())
217+ result.addTest(test_subunit_filter.test_suite())
218 result.addTest(test_subunit_tags.test_suite())
219 result.addTest(test_subunit_stats.test_suite())
220 return result
221
222=== added file 'python/subunit/tests/test_subunit_filter.py'
223--- python/subunit/tests/test_subunit_filter.py 1970-01-01 00:00:00 +0000
224+++ python/subunit/tests/test_subunit_filter.py 2009-02-23 10:54:28 +0000
225@@ -0,0 +1,115 @@
226+#
227+# subunit: extensions to python unittest to get test results from subprocesses.
228+# Copyright (C) 2005 Robert Collins <robertc@robertcollins.net>
229+#
230+# This program is free software; you can redistribute it and/or modify
231+# it under the terms of the GNU General Public License as published by
232+# the Free Software Foundation; either version 2 of the License, or
233+# (at your option) any later version.
234+#
235+# This program is distributed in the hope that it will be useful,
236+# but WITHOUT ANY WARRANTY; without even the implied warranty of
237+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
238+# GNU General Public License for more details.
239+#
240+# You should have received a copy of the GNU General Public License
241+# along with this program; if not, write to the Free Software
242+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
243+#
244+
245+"""Tests for subunit.TestResultFilter."""
246+
247+import unittest
248+from StringIO import StringIO
249+
250+import subunit
251+
252+
253+class TestTestResultFilter(unittest.TestCase):
254+ """Test for TestResultFilter, a TestResult object which filters tests."""
255+
256+ def _setUp(self):
257+ self.output = StringIO()
258+
259+ def test_default(self):
260+ """The default is to exclude success and include everything else."""
261+ self.filtered_result = unittest.TestResult()
262+ self.filter = subunit.TestResultFilter(self.filtered_result)
263+ self.run_tests()
264+ self.assertEqual(['error'],
265+ [error[0].id() for error in self.filtered_result.errors])
266+ self.assertEqual(['failed'],
267+ [failure[0].id() for failure in
268+ self.filtered_result.failures])
269+ self.assertEqual(2, self.filtered_result.testsRun)
270+
271+ def test_exclude_errors(self):
272+ self.filtered_result = unittest.TestResult()
273+ self.filter = subunit.TestResultFilter(self.filtered_result,
274+ filter_error=True)
275+ self.run_tests()
276+ self.assertEqual([],
277+ [error[0].id() for error in self.filtered_result.errors])
278+ self.assertEqual(['failed'],
279+ [failure[0].id() for failure in
280+ self.filtered_result.failures])
281+ self.assertEqual(1, self.filtered_result.testsRun)
282+
283+ def test_exclude_failure(self):
284+ self.filtered_result = unittest.TestResult()
285+ self.filter = subunit.TestResultFilter(self.filtered_result,
286+ filter_failure=True)
287+ self.run_tests()
288+ self.assertEqual(['error'],
289+ [error[0].id() for error in self.filtered_result.errors])
290+ self.assertEqual([],
291+ [failure[0].id() for failure in
292+ self.filtered_result.failures])
293+ self.assertEqual(1, self.filtered_result.testsRun)
294+
295+ def test_include_success(self):
296+ """Success's can be included if requested."""
297+ self.filtered_result = unittest.TestResult()
298+ self.filter = subunit.TestResultFilter(self.filtered_result,
299+ filter_success=False)
300+ self.run_tests()
301+ self.assertEqual(['error'],
302+ [error[0].id() for error in self.filtered_result.errors])
303+ self.assertEqual(['failed'],
304+ [failure[0].id() for failure in
305+ self.filtered_result.failures])
306+ self.assertEqual(5, self.filtered_result.testsRun)
307+
308+
309+ def run_tests(self):
310+ self.setUpTestStream()
311+ self.test = subunit.ProtocolTestCase(self.input_stream)
312+ self.test.run(self.filter)
313+
314+ def setUpTestStream(self):
315+ # While TestResultFilter works on python objects, using a subunit
316+ # stream is an easy pithy way of getting a series of test objects to
317+ # call into the TestResult, and as TestResultFilter is intended for use
318+ # with subunit also has the benefit of detecting any interface skew issues.
319+ self.input_stream = StringIO()
320+ self.input_stream.write("""tags: global
321+test passed
322+success passed
323+test failed
324+tags: local
325+failure failed
326+test error
327+error error
328+test skipped
329+skip skipped
330+test todo
331+xfail todo
332+""")
333+ self.input_stream.seek(0)
334+
335+
336+
337+def test_suite():
338+ loader = subunit.tests.TestUtil.TestLoader()
339+ result = loader.loadTestsFromName(__name__)
340+ return result
341
342=== modified file 'python/subunit/tests/test_test_protocol.py'
343--- python/subunit/tests/test_test_protocol.py 2008-12-14 18:37:08 +0000
344+++ python/subunit/tests/test_test_protocol.py 2009-02-23 10:54:28 +0000
345@@ -772,7 +772,7 @@
346 self.assertRaises(NotImplementedError, test.tearDown)
347 self.assertEqual("A test description",
348 test.shortDescription())
349- self.assertEqual("subunit.RemotedTestCase.A test description",
350+ self.assertEqual("A test description",
351 test.id())
352 self.assertEqual("A test description (subunit.RemotedTestCase)", "%s" % test)
353 self.assertEqual("<subunit.RemotedTestCase description="

Subscribers

People subscribed via source and target branches