subunit

Merge lp:~james-w/subunit/discovery into lp:~subunit/subunit/trunk

  1. discovery
  2. Merge into trunk
Proposed by James Westby
Status: Merged
Merged at revision: 129
Proposed branch: lp:~james-w/subunit/discovery
Merge into: lp:~subunit/subunit/trunk
Diff against target: 69 lines (+35/-6)
2 files modified
NEWS (+7/-0)
python/subunit/run.py (+28/-6)
To merge this branch: bzr merge lp:~james-w/subunit/discovery
Reviewer Review Type Date Requested Status
Robert Collins Needs Fixing
Review via email: mp+26893@code.launchpad.net

Commit message

Support test discovery.

Description of the change

Hi,

Here's a hacky first pass at implementing discovery in the way you suggested.

The biggest thing is that there is no way to pass a testsuite to TestProgram,
so I implemented some of that class inline to get the job done.

I tested this with testr

mkdir foo
touch foo/__init__.py
create foo/test_foo.py with
from testtools import TestCase

class foo(TestCase):
    def test_pass(self): pass
    def test_fail(self): self.fail("foo")

and .testr.conf with
[DEFAULT]
test_command=python -m subunit.run $IDLIST
test_id_list_default=--discover .

as I said on IRC the --discover . was a little unexpected to me, and I
don't think it's ideal that it is required to be that.

This is clearly quite hacky, so I'm looking for suggestions on cleaning it
up.

Thanks,

James

To post a comment you must log in.
Revision history for this message
Robert Collins (lifeless) wrote :

As discussed, this is best done in testtools.run and subunit.run changed to layer on that; the testtools part has landed.

Revision history for this message
James Westby (james-w) wrote :

How does it look now?

Thanks,

James

Revision history for this message
Robert Collins (lifeless) wrote :

Very nice except that the help is now the testtools help, and it really should mention that it will be outputting subunit ;).

review: Needs Fixing
lp:~james-w/subunit/discovery updated
129. By James Westby

Modify the usage message from subunit.run.

Revision history for this message
James Westby (james-w) wrote :

On Mon, 14 Jun 2010 08:58:22 -0000, Robert Collins <email address hidden> wrote:
> Review: Needs Fixing
> Very nice except that the help is now the testtools help, and it really should mention that it will be outputting subunit ;).

Have a look at it now.

python -m subunit.run --asdf
option --asdf not recognized
Usage: run.py [options] [tests]

Options:
  -h, --help Show this message
  -v, --verbose Verbose output
  -q, --quiet Minimal output
  -f, --failfast Stop on first failure
  -c, --catch Catch control-C and display results
  -b, --buffer Buffer stdout and stderr during test runs

Examples:
  run.py test_module - run tests from test_module
  run.py module.TestClass - run tests from module.TestClass
  run.py module.Class.test_method - run specified test method

[tests] can be a list of any number of test modules, classes and test
methods.

Alternative Usage: run.py discover [options]

Options:
  -v, --verbose Verbose output
  -f, --failfast Stop on first failure
  -c, --catch Catch control-C and display results
  -b, --buffer Buffer stdout and stderr during test runs
  -s directory Directory to start discovery ('.' default)
  -p pattern Pattern to match test files ('test*.py' default)
  -t directory Top level directory of project (default to
                   start directory)

For test discovery all test modules must be importable from the top
level directory of the project.

The output will be in subunit format.

Thanks,

James

Revision history for this message
Robert Collins (lifeless) wrote :

I think I'd like to see something like this:

python -m subunit.run --asdf
option --asdf not recognized

Execute a python test suite with subunit reporting.

Usage: python -m subunit.run [options] [tests]

.. the stock stuff here.

And in testtools it should say

python -m testtools.run --asdf
option --asdf not recognized

Execute a python test suite.

Usage: testtools.run [options] [tests]

...

----

I note that the thing to run is being misdetected for you; I thought I
fixed that, perhaps you didn't grab testtools trunk - or my fix was
incomplete.

Revision history for this message
James Westby (james-w) wrote :

On Tue, 22 Jun 2010 19:10:49 -0000, Robert Collins <email address hidden> wrote:
> I think I'd like to see something like this:
>
> python -m subunit.run --asdf
> option --asdf not recognized
>
> Execute a python test suite with subunit reporting.
>
> Usage: python -m subunit.run [options] [tests]
>
> .. the stock stuff here.
>
> And in testtools it should say
>
> python -m testtools.run --asdf
> option --asdf not recognized
>
> Execute a python test suite.
>
> Usage: testtools.run [options] [tests]
>
> ...
>
> ----
>
> I note that the thing to run is being misdetected for you; I thought I
> fixed that, perhaps you didn't grab testtools trunk - or my fix was
> incomplete.

I wasn't using trunk.

Doing what you ask is possible, but means we move further from 2.7's
TestProgram, and don't pick up extra options in the usage output
that easily. That at least would go away once 2.7 was the minimum.

I'm happy to do that if you like.

Thanks,

James

Revision history for this message
Robert Collins (lifeless) wrote :

On Wed, Jun 23, 2010 at 7:34 AM, James Westby <email address hidden> wrote:
> I wasn't using trunk.
>
>
> Doing what you ask is possible, but means we move further from 2.7's
> TestProgram, and don't pick up extra options in the usage output
> that easily. That at least would go away once 2.7 was the minimum.
>
> I'm happy to do that if you like

We should file patches for changes we make that are relevant to the
reusability of TestProgram in python upstream; simple porting changes
aren't relevant because - well that obvious :).

So far, I checked and our changes didn't need upstream patches; we may
at this point need to send one in, and thats fine IMO.

We don't need to block on sending such a patch in, we should get
something we're happy with, then send in a patch.

-Rob

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'NEWS'
2--- NEWS 2010-06-11 01:35:58 +0000
3+++ NEWS 2010-06-22 09:04:24 +0000
4@@ -23,6 +23,13 @@
5 mangled by the default line ending translation on that platform.
6 (Robert Collins, Martin [gz], #579296)
7
8+IMPROVEMENTS
9+~~~~~~~~~~~~
10+
11+* subunit now supports test discovery by building on the testtools support for
12+ it. You can take advantage of it with "python -m subunit.run discover [path]"
13+ and see "python -m subunit.run discover --help" for more options.
14+
15 0.0.5
16 -----
17
18
19=== modified file 'python/subunit/run.py'
20--- python/subunit/run.py 2010-01-15 04:16:50 +0000
21+++ python/subunit/run.py 2010-06-22 09:04:24 +0000
22@@ -23,6 +23,13 @@
23 import sys
24
25 from subunit import TestProtocolClient, get_default_formatter
26+from testtools.run import (
27+ BUFFEROUTPUT,
28+ CATCHBREAK,
29+ FAILFAST,
30+ TestProgram,
31+ USAGE_AS_MAIN,
32+ )
33
34
35 class SubunitTestRunner(object):
36@@ -36,12 +43,27 @@
37 return result
38
39
40+class SubunitTestProgram(TestProgram):
41+
42+ USAGE = USAGE_AS_MAIN
43+
44+ def usageExit(self, msg=None):
45+ if msg:
46+ print msg
47+ usage = {'progName': self.progName, 'catchbreak': '', 'failfast': '',
48+ 'buffer': ''}
49+ if self.failfast != False:
50+ usage['failfast'] = FAILFAST
51+ if self.catchbreak != False:
52+ usage['catchbreak'] = CATCHBREAK
53+ if self.buffer != False:
54+ usage['buffer'] = BUFFEROUTPUT
55+ print self.USAGE % usage
56+ print "The output will be in subunit format.\n"
57+ sys.exit(2)
58+
59+
60 if __name__ == '__main__':
61- import optparse
62- from unittest import TestProgram
63- parser = optparse.OptionParser(__doc__)
64- args = parser.parse_args()[1]
65 stream = get_default_formatter()
66 runner = SubunitTestRunner(stream)
67- program = TestProgram(module=None, argv=[sys.argv[0]] + args,
68- testRunner=runner)
69+ SubunitTestProgram(module=None, argv=sys.argv, testRunner=runner)

Subscribers

People subscribed via source and target branches