subunit

Merge lp:~lifeless/subunit/filter into lp:~subunit/subunit/trunk

  1. filter
  2. Merge into trunk
Proposed by Jonathan Lange
Status: Merged
Approved by: Jonathan Lange
Approved revision: 56
Merged at revision: not available
Proposed branch: lp:~lifeless/subunit/filter
Merge into: lp:~subunit/subunit/trunk
Diff against target: None lines
To merge this branch: bzr merge lp:~lifeless/subunit/filter
Reviewer Review Type Date Requested Status
Jonathan Lange Approve
Review via email: mp+4006@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Jonathan Lange (jml) wrote :

Add filters for listing a stream and selecting only pass|failed|errored tests.

lp:~lifeless/subunit/filter updated
57. By Robert Collins

Update skip support in python to be in line with testtools.

58. By Robert Collins

subunit-filter can now filter skips too.

Revision history for this message
Jonathan Lange (jml) wrote :

Two things!
* Rename FilterResult in subunit-ls so the name indicates what kind of filtering it does. ListingFilterResult, for example.
* Spurious blank line at python/subunit/tests/test_subunit_filter.py:307

review: Approve
Revision history for this message
Jonathan Lange (jml) wrote :

I meant line 307 of the diff, sorry.

lp:~lifeless/subunit/filter updated
59. By Robert Collins

Review feedback on filters.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'README'
--- README 2009-02-15 11:55:00 +0000
+++ README 2009-02-22 13:18:04 +0000
@@ -40,6 +40,8 @@
40stream on-the-fly. Currently subunit provides:40stream on-the-fly. Currently subunit provides:
41 * tap2subunit - convert perl's TestAnythingProtocol to subunit.41 * tap2subunit - convert perl's TestAnythingProtocol to subunit.
42 * subunit2pyunit - convert a subunit stream to pyunit test results.42 * subunit2pyunit - convert a subunit stream to pyunit test results.
43 * subunit-filter - filter out tests from a subunit stream.
44 * subunit-ls - list the tests present in a subunit stream.
43 * subunit-stats - generate a summary of a subunit stream.45 * subunit-stats - generate a summary of a subunit stream.
44 * subunit-tags - add or remove tags from a stream.46 * subunit-tags - add or remove tags from a stream.
4547
4648
=== added file 'filters/subunit-filter'
--- filters/subunit-filter 1970-01-01 00:00:00 +0000
+++ filters/subunit-filter 2009-02-22 07:51:04 +0000
@@ -0,0 +1,50 @@
1#!/usr/bin/env python
2# subunit: extensions to python unittest to get test results from subprocesses.
3# Copyright (C) 2008 Robert Collins <robertc@robertcollins.net>
4#
5# This program is free software; you can redistribute it and/or modify
6# it under the terms of the GNU General Public License as published by
7# the Free Software Foundation; either version 2 of the License, or
8# (at your option) any later version.
9#
10# This program is distributed in the hope that it will be useful,
11# but WITHOUT ANY WARRANTY; without even the implied warranty of
12# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13# GNU General Public License for more details.
14#
15# You should have received a copy of the GNU General Public License
16# along with this program; if not, write to the Free Software
17# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
18#
19
20"""Filter a subunit stream to include/exclude tests.
21
22The default is to strip successful tests.
23"""
24
25from optparse import OptionParser
26import sys
27import unittest
28
29from subunit import ProtocolTestCase, TestResultFilter, TestProtocolClient
30
31parser = OptionParser(description=__doc__)
32parser.add_option("--error", action="store_false",
33 help="include errors", default=False, dest="error")
34parser.add_option("-e", "--no-error", action="store_true",
35 help="exclude errors", dest="error")
36parser.add_option("--failure", action="store_false",
37 help="include failures", default=False, dest="failure")
38parser.add_option("-f", "--no-failure", action="store_true",
39 help="include failures", dest="failure")
40parser.add_option("-s", "--success", action="store_false",
41 help="include successes", dest="success")
42parser.add_option("--no-success", action="store_true",
43 help="exclude successes", default=True, dest="success")
44(options, args) = parser.parse_args()
45result = TestProtocolClient(sys.stdout)
46result = TestResultFilter(result, filter_error=options.error, filter_failure=options.failure,
47 filter_success=options.success)
48test = ProtocolTestCase(sys.stdin)
49test.run(result)
50sys.exit(0)
051
=== added file 'filters/subunit-ls'
--- filters/subunit-ls 1970-01-01 00:00:00 +0000
+++ filters/subunit-ls 2009-02-23 10:54:28 +0000
@@ -0,0 +1,62 @@
1#!/usr/bin/env python
2# subunit: extensions to python unittest to get test results from subprocesses.
3# Copyright (C) 2008 Robert Collins <robertc@robertcollins.net>
4#
5# This program is free software; you can redistribute it and/or modify
6# it under the terms of the GNU General Public License as published by
7# the Free Software Foundation; either version 2 of the License, or
8# (at your option) any later version.
9#
10# This program is distributed in the hope that it will be useful,
11# but WITHOUT ANY WARRANTY; without even the implied warranty of
12# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13# GNU General Public License for more details.
14#
15# You should have received a copy of the GNU General Public License
16# along with this program; if not, write to the Free Software
17# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
18#
19
20"""List tests in a subunit stream."""
21
22import sys
23import unittest
24
25from subunit import ProtocolTestCase
26
27class FilterResult(unittest.TestResult):
28 """Filter test objects for display."""
29
30 def __init__(self, stream):
31 """Create a FilterResult object outputting to stream."""
32 unittest.TestResult.__init__(self)
33 self._stream = stream
34 self.failed_tests = 0
35
36 def addError(self, test, err):
37 self.failed_tests += 1
38 self.reportTest(test)
39
40 def addFailure(self, test, err):
41 self.failed_tests += 1
42 self.reportTest(test)
43
44 def addSuccess(self, test):
45 self.reportTest(test)
46
47 def reportTest(self, test):
48 self._stream.write(test.id() + '\n')
49
50 def wasSuccessful(self):
51 "Tells whether or not this result was a success"
52 return self.failed_tests == 0
53
54
55result = FilterResult(sys.stdout)
56test = ProtocolTestCase(sys.stdin)
57test.run(result)
58if result.wasSuccessful():
59 exit_code = 0
60else:
61 exit_code = 1
62sys.exit(exit_code)
063
=== modified file 'python/subunit/__init__.py'
--- python/subunit/__init__.py 2009-02-15 11:55:00 +0000
+++ python/subunit/__init__.py 2009-02-23 10:54:28 +0000
@@ -363,7 +363,7 @@
363 return self.__description363 return self.__description
364364
365 def id(self):365 def id(self):
366 return "%s.%s" % (self._strclass(), self.__description)366 return "%s" % (self.__description,)
367367
368 def __str__(self):368 def __str__(self):
369 return "%s (%s)" % (self.__description, self._strclass())369 return "%s (%s)" % (self.__description, self._strclass())
@@ -681,3 +681,52 @@
681 def wasSuccessful(self):681 def wasSuccessful(self):
682 """Tells whether or not this result was a success"""682 """Tells whether or not this result was a success"""
683 return self.failed_tests == 0683 return self.failed_tests == 0
684
685
686class TestResultFilter(unittest.TestResult):
687 """A pyunit TestResult interface implementation which filters tests.
688
689 Tests that pass the filter are handed on to another TestResult instance
690 for further processing/reporting. To obtain the filtered results,
691 the other instance must be interrogated.
692
693 :ivar result: The result that tests are passed to after filtering.
694 """
695
696 def __init__(self, result, filter_error=False, filter_failure=False,
697 filter_success=True):
698 """Create a FilterResult object filtering to result.
699
700 :param filter_error: Filter out errors.
701 :param filter_failure: Filter out failures.
702 :param filter_success: Filter out successful tests.
703 """
704 unittest.TestResult.__init__(self)
705 self.result = result
706 self._filter_error = filter_error
707 self._filter_failure = filter_failure
708 self._filter_success = filter_success
709
710 def addError(self, test, err):
711 if not self._filter_error:
712 self.result.startTest(test)
713 self.result.addError(test, err)
714 self.result.stopTest(test)
715
716 def addFailure(self, test, err):
717 if not self._filter_failure:
718 self.result.startTest(test)
719 self.result.addFailure(test, err)
720 self.result.stopTest(test)
721
722 def addSuccess(self, test):
723 if not self._filter_success:
724 self.result.startTest(test)
725 self.result.addSuccess(test)
726 self.result.stopTest(test)
727
728 def id_to_orig_id(self, id):
729 if id.startswith("subunit.RemotedTestCase."):
730 return id[len("subunit.RemotedTestCase."):]
731 return id
732
684733
=== modified file 'python/subunit/tests/__init__.py'
--- python/subunit/tests/__init__.py 2008-12-09 01:00:03 +0000
+++ python/subunit/tests/__init__.py 2009-02-22 06:28:08 +0000
@@ -19,6 +19,7 @@
1919
20from subunit.tests import (20from subunit.tests import (
21 TestUtil,21 TestUtil,
22 test_subunit_filter,
22 test_subunit_stats,23 test_subunit_stats,
23 test_subunit_tags,24 test_subunit_tags,
24 test_tap2subunit,25 test_tap2subunit,
@@ -29,6 +30,7 @@
29 result = TestUtil.TestSuite()30 result = TestUtil.TestSuite()
30 result.addTest(test_test_protocol.test_suite())31 result.addTest(test_test_protocol.test_suite())
31 result.addTest(test_tap2subunit.test_suite())32 result.addTest(test_tap2subunit.test_suite())
33 result.addTest(test_subunit_filter.test_suite())
32 result.addTest(test_subunit_tags.test_suite())34 result.addTest(test_subunit_tags.test_suite())
33 result.addTest(test_subunit_stats.test_suite())35 result.addTest(test_subunit_stats.test_suite())
34 return result36 return result
3537
=== added file 'python/subunit/tests/test_subunit_filter.py'
--- python/subunit/tests/test_subunit_filter.py 1970-01-01 00:00:00 +0000
+++ python/subunit/tests/test_subunit_filter.py 2009-02-23 10:54:28 +0000
@@ -0,0 +1,115 @@
1#
2# subunit: extensions to python unittest to get test results from subprocesses.
3# Copyright (C) 2005 Robert Collins <robertc@robertcollins.net>
4#
5# This program is free software; you can redistribute it and/or modify
6# it under the terms of the GNU General Public License as published by
7# the Free Software Foundation; either version 2 of the License, or
8# (at your option) any later version.
9#
10# This program is distributed in the hope that it will be useful,
11# but WITHOUT ANY WARRANTY; without even the implied warranty of
12# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13# GNU General Public License for more details.
14#
15# You should have received a copy of the GNU General Public License
16# along with this program; if not, write to the Free Software
17# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
18#
19
20"""Tests for subunit.TestResultFilter."""
21
22import unittest
23from StringIO import StringIO
24
25import subunit
26
27
28class TestTestResultFilter(unittest.TestCase):
29 """Test for TestResultFilter, a TestResult object which filters tests."""
30
31 def _setUp(self):
32 self.output = StringIO()
33
34 def test_default(self):
35 """The default is to exclude success and include everything else."""
36 self.filtered_result = unittest.TestResult()
37 self.filter = subunit.TestResultFilter(self.filtered_result)
38 self.run_tests()
39 self.assertEqual(['error'],
40 [error[0].id() for error in self.filtered_result.errors])
41 self.assertEqual(['failed'],
42 [failure[0].id() for failure in
43 self.filtered_result.failures])
44 self.assertEqual(2, self.filtered_result.testsRun)
45
46 def test_exclude_errors(self):
47 self.filtered_result = unittest.TestResult()
48 self.filter = subunit.TestResultFilter(self.filtered_result,
49 filter_error=True)
50 self.run_tests()
51 self.assertEqual([],
52 [error[0].id() for error in self.filtered_result.errors])
53 self.assertEqual(['failed'],
54 [failure[0].id() for failure in
55 self.filtered_result.failures])
56 self.assertEqual(1, self.filtered_result.testsRun)
57
58 def test_exclude_failure(self):
59 self.filtered_result = unittest.TestResult()
60 self.filter = subunit.TestResultFilter(self.filtered_result,
61 filter_failure=True)
62 self.run_tests()
63 self.assertEqual(['error'],
64 [error[0].id() for error in self.filtered_result.errors])
65 self.assertEqual([],
66 [failure[0].id() for failure in
67 self.filtered_result.failures])
68 self.assertEqual(1, self.filtered_result.testsRun)
69
70 def test_include_success(self):
71 """Success's can be included if requested."""
72 self.filtered_result = unittest.TestResult()
73 self.filter = subunit.TestResultFilter(self.filtered_result,
74 filter_success=False)
75 self.run_tests()
76 self.assertEqual(['error'],
77 [error[0].id() for error in self.filtered_result.errors])
78 self.assertEqual(['failed'],
79 [failure[0].id() for failure in
80 self.filtered_result.failures])
81 self.assertEqual(5, self.filtered_result.testsRun)
82
83
84 def run_tests(self):
85 self.setUpTestStream()
86 self.test = subunit.ProtocolTestCase(self.input_stream)
87 self.test.run(self.filter)
88
89 def setUpTestStream(self):
90 # While TestResultFilter works on python objects, using a subunit
91 # stream is an easy pithy way of getting a series of test objects to
92 # call into the TestResult, and as TestResultFilter is intended for use
93 # with subunit also has the benefit of detecting any interface skew issues.
94 self.input_stream = StringIO()
95 self.input_stream.write("""tags: global
96test passed
97success passed
98test failed
99tags: local
100failure failed
101test error
102error error
103test skipped
104skip skipped
105test todo
106xfail todo
107""")
108 self.input_stream.seek(0)
109
110
111
112def test_suite():
113 loader = subunit.tests.TestUtil.TestLoader()
114 result = loader.loadTestsFromName(__name__)
115 return result
0116
=== modified file 'python/subunit/tests/test_test_protocol.py'
--- python/subunit/tests/test_test_protocol.py 2008-12-14 18:37:08 +0000
+++ python/subunit/tests/test_test_protocol.py 2009-02-23 10:54:28 +0000
@@ -772,7 +772,7 @@
772 self.assertRaises(NotImplementedError, test.tearDown)772 self.assertRaises(NotImplementedError, test.tearDown)
773 self.assertEqual("A test description",773 self.assertEqual("A test description",
774 test.shortDescription())774 test.shortDescription())
775 self.assertEqual("subunit.RemotedTestCase.A test description",775 self.assertEqual("A test description",
776 test.id())776 test.id())
777 self.assertEqual("A test description (subunit.RemotedTestCase)", "%s" % test)777 self.assertEqual("A test description (subunit.RemotedTestCase)", "%s" % test)
778 self.assertEqual("<subunit.RemotedTestCase description="778 self.assertEqual("<subunit.RemotedTestCase description="

Subscribers

People subscribed via source and target branches