Bazaar

Merge lp:~vila/bzr/shell-like-tests into lp:bzr

shell-like-tests
Merge into bzr.dev

Proposed by Vincent Ladeuil on 2009-09-04

Status:	Rejected
Rejected by:	Vincent Ladeuil on 2009-09-17
Proposed branch:	lp:~vila/bzr/shell-like-tests
Merge into:	lp:bzr
Diff against target:	None lines
To merge this branch:	bzr merge lp:~vila/bzr/shell-like-tests
Related bugs:	Link a bug report

Reviewer	Review Type	Date Requested	Status
Martin Pool		2009-09-04	Needs Fixing on 2009-09-16
Review via email: mp+11204@code.launchpad.net

Revision history for this message

Vincent Ladeuil (vila) wrote on 2009-09-04:

This patch address some points raised during the review of the first proposal:
- document the ways the scripts can be used, including some assertions,
- stop the script execution on unexpected errors
- allow the '...' doctest notation
- separate the script runner from the test cases

It doesn't address:
- running the scripts without touching the disk

But I really like the idea that it could be done :)
Thinking a bit about that from John remark, I came to the conclusion that:
1) I didn't intend to make it happen *now*
2) It is certainly possible to get there as long as we find a way to:
   a) Use a local transport in WorkingTree and TreeTransform objects
   b) Trap and override all calls to file() open() and os. calls (osutils is a good start)
   c) Manage to trick dirstate to work in memory too :)

Given that the above is non-trivial, I'd like to land at least a usable version of this shell-like
tests feature and file bugs about all missing bits.

The initial intent was to address a UI hole, let's focus on that first and optimize the implementation later.

Revision history for this message

Martin Pool (mbp) wrote on 2009-09-16:

Download full text (23.4 KiB)

This is looking pretty cool.

=== modified file 'bzrlib/tests/__init__.py'
--- bzrlib/tests/__init__.py 2009-08-28 21:05:31 +0000
+++ bzrlib/tests/__init__.py 2009-09-01 08:24:44 +0000
@@ -3618,6 +3618,7 @@
         'bzrlib.tests.test_rio',
         'bzrlib.tests.test_rules',
         'bzrlib.tests.test_sampler',
+ 'bzrlib.tests.test_script',
         'bzrlib.tests.test_selftest',
         'bzrlib.tests.test_serializer',
         'bzrlib.tests.test_setup',

=== added file 'bzrlib/tests/script.py'
--- bzrlib/tests/script.py 1970-01-01 00:00:00 +0000
+++ bzrlib/tests/script.py 2009-09-04 14:55:59 +0000
@@ -0,0 +1,389 @@
+# Copyright (C) 2009 Canonical Ltd
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+"""Shell-like test scripts.

There should be two blank lines above that.

+
+This allows users to write tests in a syntax very close to a shell session,
+using a restricted and limited set of commands that should be enough to mimic
+most of the behaviours.

This is a nice description. However, I think it's probably going to be
more discoverable and more point-to-able if it's in the developer
documentation rather than in the module docstring.

It would be kinda nice if pydoc handled this differently but
realistically I think the developer docs are much more likely to be read
before reading the code than the pydocs are.

+
+A script is a set of commands, each command is composed of:
+- one mandatory command line,
+- one optional set of input lines to feed the command,
+- one optional set of output expected lines,
+- one optional set of error expected lines.

.. in that order?

+
+The optional lines starts with a special string (mnemonic: shell redirection):
+- '<' for input,
+- '>' for output,
+- '2>' for errors,

Hm, I guess we'll see how the examples look but I think this may not be
the clearest description of it. How about instead

  $ the command
  < the input
  output by default
  2> errors

and I'm not so convinced people will find the errors bit obvious. Do we
really need to distinguish them from stdout?

+
+The execution stops as soon as an expected output or an expected error is not
+matched.
+
+When no output is specified, any ouput from the command is accepted
+and let the execution continue.

'output', 'is accepted and execution continues'.

Is this a good idea though? I feel like I'd like at least '...' so that
there's some indication that stuff will happen but we don't care.

+
+If an error occurs and no expected error is specified, the execution stops.
+
+The ...

This is looking pretty cool.

=== modified file 'bzrlib/tests/__init__.py'
--- bzrlib/tests/__init__.py	2009-08-28 21:05:31 +0000
+++ bzrlib/tests/__init__.py	2009-09-01 08:24:44 +0000
@@ -3618,6 +3618,7 @@
         'bzrlib.tests.test_rio',
         'bzrlib.tests.test_rules',
         'bzrlib.tests.test_sampler',
+        'bzrlib.tests.test_script',
         'bzrlib.tests.test_selftest',
         'bzrlib.tests.test_serializer',
         'bzrlib.tests.test_setup',

=== added file 'bzrlib/tests/script.py'
--- bzrlib/tests/script.py	1970-01-01 00:00:00 +0000
+++ bzrlib/tests/script.py	2009-09-04 14:55:59 +0000
@@ -0,0 +1,389 @@
+# Copyright (C) 2009 Canonical Ltd
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+"""Shell-like test scripts.

There should be two blank lines above that.

+
+This allows users to write tests in a syntax very close to a shell session,
+using a restricted and limited set of commands that should be enough to mimic
+most of the behaviours.

This is a nice description.  However, I think it's probably going to be
more discoverable and more point-to-able if it's in the developer
documentation rather than in the module docstring.

It would be kinda nice if pydoc handled this differently but
realistically I think the developer docs are much more likely to be read
before reading the code than the pydocs are.

.. in that order?

+
+The optional lines starts with a special string (mnemonic: shell redirection):
+- '<' for input,
+- '>' for output,
+- '2>' for errors,

Hm, I guess we'll see how the examples look but I think this may not be
the clearest description of it.  How about instead

$ the command
  < the input
  output by default
  2> errors

and I'm not so convinced people will find the errors bit obvious.  Do we
really need to distinguish them from stdout?

+
+The execution stops as soon as an expected output or an expected error is not
+matched. 
+
+When no output is specified, any ouput from the command is accepted
+and let the execution continue.

'output', 'is accepted and execution continues'.

Is this a good idea though?  I feel like I'd like at least '...' so that
there's some indication that stuff will happen but we don't care.

+
+If an error occurs and no expected error is specified, the execution stops.
+
+The matching is done on a full string comparison basis unless '...' is used, in
+which case expected output/errors can be lees precise.

'less'

+
+Examples:
+
+The following will succeeds only if 'bzr add' outputs 'adding file'.
+
+  bzr add file
+  >adding file

As ReST style you should put a double colon at the end of the line ::

like so::
    
    thus

+You can check the content of a file with cat:
+
+  cat <file
+  >expected content

Is the redirection needed?

+You can also check the existence of a file with cat, the following will fail if
+the file doesn't exist:
+
+  cat file

+
+"""
+
+import doctest
+import os
+import shlex
+from cStringIO import StringIO
+
+from bzrlib import (
+    osutils,
+    tests,
+    )
+
+
+def split(s):
+    """Split a command line respecting quotes."""

Better name?

+    scanner = shlex.shlex(s)
+    scanner.quotes = '\'"`'
+    scanner.whitespace_split = True
+    for t in list(scanner):
+        # Strip the simple and double quotes since we don't care about them.
+        # We leave the backquotes in place though since they have a different
+        # semantic.
+        if t[0] in  ('"', "'") and t[0] == t[-1]:
+            yield t[1:-1]
+        else:
+            yield t
+
+
+def _script_to_commands(text, file_name=None):
+    """Turn a script into a list of commands with their associated IOs.
+
+    Each command appears on a line by itself. It can be associated with an
+    input that will feed it and an expected output.
+    Comments starts with '#' until the end of line.

Comments should be documented?

How about rather than returning a command list, we have a factory for
Script objects that can then be run?

+    Empty lines are ignored.
+    Input and output are full lines terminated by a '\n'.
+    Input lines start with '<'.
+    Output lines start with '>'.
+    Error lines start with '2>'.
+    """
+
+    commands = []
+
+    def add_command(cmd, input, output, error):
+        if cmd is not None:
+            if input is not None:
+                input = ''.join(input)
+            if output is not None:
+                output = ''.join(output)
+            if error is not None:
+                error = ''.join(error)
+            commands.append((cmd, input, output, error))
+
+    cmd_cur = None
+    cmd_line = 1
+    lineno = 0
+    input, output, error = None, None, None
+    for line in text.split('\n'):
+        lineno += 1
+        # Keep a copy for error reporting
+        orig = line
+        comment =  line.find('#')
+        if comment >= 0:
+            # Delete comments
+            line = line[0:comment]
+            line = line.rstrip()
+        if line == '':
+            # Ignore empty lines
+            continue
+        if line.startswith('<'):
+            if input is None:
+                if cmd_cur is None:
+                    raise SyntaxError('No command for that input',
+                                      (file_name, lineno, 1, orig))
+                input = []
+            input.append(line[1:] + '\n')
+            continue
+        elif line.startswith('>'):
+            if output is None:
+                if cmd_cur is None:
+                    raise SyntaxError('No command for that output',
+                                      (file_name, lineno, 1, orig))
+                output = []
+            output.append(line[1:] + '\n')
+            continue
+        elif line.startswith('2>'):
+            if error is None:
+                if cmd_cur is None:
+                    raise SyntaxError('No command for that error',
+                                      (file_name, lineno, 1, orig))
+                error = []
+            error.append(line[2:] + '\n')
+            continue
+        else:
+            # Time to output the current command
+            add_command(cmd_cur, input, output, error)
+            # And start a new one
+            cmd_cur = list(split(line))
+            cmd_line = lineno
+            input, output, error = None, None, None
+    # Add the last seen command
+    add_command(cmd_cur, input, output, error)
+    return commands
+
+
+def _scan_redirection_options(args):
+    """Recognize and process input and output redirections.
+
+    :param args: The command line arguments
+
+    :return: A tuple containing: 
+        - The file name redirected from or None
+        - The file name redirected to or None
+        - The mode to open the output file or None
+        - The reamining arguments
+    """
+    remaining = []
+    in_name = None
+    out_name, out_mode = None, None
+    for arg in  args:
+        if arg.startswith('<'):
+            in_name = arg[1:]
+        elif arg.startswith('>>'):
+            out_name = arg[2:]
+            out_mode = 'ab+'
+        elif arg.startswith('>'):
+            out_name = arg[1:]
+            out_mode = 'wb+'
+        else:
+            remaining.append(arg)
+    return in_name, out_name, out_mode, remaining
+
+
+class ScriptRunner(object):
+
+    def __init__(self, test_case):
+        self.test_case = test_case
+        self.output_checker = doctest.OutputChecker()
+        self.check_options = doctest.ELLIPSIS
+
+    def run_script(self, text):
+        for cmd, input, output, error in _script_to_commands(text):
+            out, err = self.run_command(cmd, input, output, error)
+
+    def _check_output(self, expected, actual):
+        if expected is None:
+            # Specifying None means: any output is accepted
+            return
+        if actual is None:
+            self.test_case.fail('Unexpected: %s' % actual)
+        matching = self.output_checker.check_output(
+            expected, actual, self.check_options)
+        if not matching:
+            # Note that we can't use output_checker.output_difference() here
+            # because... the API is boken (expected must be a doctest specific
+            # object of whicha 'want' attribute will be our 'expected'
+            # parameter. So we just fallbacl to our good old assertEqualDiff
+            # since we know there are differences and the output should be
+            # decently readable.
+            self.test_case.assertEqualDiff(expected, actual)
+
+    def run_command(self, cmd, input, output, error):
+        mname = 'do_' + cmd[0]

Maybe we should put the specific command implementations for bzr in a
subclass? I guess it's easy to do later on if desired.

+        method = getattr(self, mname, None)
+        if method is None:
+            raise SyntaxError('Command not found "%s"' % (cmd[0],),
+                              None, 1, ' '.join(cmd))
+        if input is None:
+            str_input = ''
+        else:
+            str_input = ''.join(input)
+        actual_output, actual_error = method(str_input, cmd[1:])
+
+        self._check_output(output, actual_output)
+        self._check_output(error, actual_error)
+        if not error and actual_error:
+            self.test_case.fail('Unexpected error: %s' % actual_error)
+        return actual_output, actual_error
+
+    def _read_input(self, input, in_name):
+        if in_name is not None:
+            infile = open(in_name, 'rb')
+            try:
+                # Command redirection takes precedence over provided input
+                input = infile.read()
+            finally:
+                infile.close()
+        return input
+
+    def _write_output(self, output, out_name, out_mode):
+        if out_name is not None:
+            outfile = open(out_name, out_mode)
+            try:
+                outfile.write(output)
+            finally:
+                outfile.close()
+            output = None
+        return output
+
+    def do_bzr(self, input, args):
+        out, err = self.test_case._run_bzr_core(
+            args, retcode=None, encoding=None, stdin=input, working_dir=None)
+        return out, err
+
+    def do_cat(self, input, args):
+        (in_name, out_name, out_mode, args) = _scan_redirection_options(args)
+        if len(args) > 1:
+            raise SyntaxError('Usage: cat [file1]')
+        if args:
+            if in_name is not None:
+                raise SyntaxError('Specify a file OR use redirection')
+            in_name = args[0]
+        input = self._read_input(input, in_name)
+        # Basically cat copy input to output
+        output = input
+        # Handle output redirections
+        output = self._write_output(output, out_name, out_mode)
+        return output, None
+
+    def do_echo(self, input, args):
+        (in_name, out_name, out_mode, args) = _scan_redirection_options(args)
+        if input and args:
+                raise SyntaxError('Specify parameters OR use redirection')
+        if args:
+            input = ''.join(args)
+        input = self._read_input(input, in_name)
+        # Always append a \n'
+        input += '\n'
+        # Process output
+        output = input
+        # Handle output redirections
+        output = self._write_output(output, out_name, out_mode)
+        return output, None
+
+    def _ensure_in_jail(self, path):
+        jail_root = self.test_case.get_jail_root()
+        if not osutils.is_inside(jail_root, osutils.normalizepath(path)):
+            raise ValueError('%s is not inside %s' % (path, jail_root))
+
+    def do_cd(self, input, args):
+        if len(args) > 1:
+            raise SyntaxError('Usage: cd [dir]')
+        if len(args) == 1:
+            d = args[0]
+            self._ensure_in_jail(d)
+        else:
+            d = self.test_case.get_jail_root()
+        os.chdir(d)
+        return None, None
+
+    def do_mkdir(self, input, args):
+        if not args or len(args) != 1:
+            raise SyntaxError('Usage: mkdir dir')
+        d = args[0]
+        self._ensure_in_jail(d)
+        os.mkdir(d)
+        return None, None

How about if we use a transport in here, not direct os calls?  Then we
can more easily move it to a MemoryTransport in some cases later on.

+
+

+class TestCaseWithMemoryTransportAndScript(tests.TestCaseWithMemoryTransport):

This is a bit like what I was talking about with Robert on the list
today: I don't like the idea of entangling tests with base classes so
that you have to inherit from a particular test.  I'd like someone to be
able to just use this style within a particular test without having to
move it to a different class or to change the whole TestCase class to a
different parent.

Would it be feasible to just have an object that tests can create and
pass scripts to?

It seems like the only dependency is calling jail_root...

+
+    def setUp(self):
+        super(TestCaseWithMemoryTransportAndScript, self).setUp()
+        self.script_runner = ScriptRunner(self)
+        # Break the circular dependency
+        def break_dependency():
+            self.script_runner = None
+        self.addCleanup(break_dependency)
+
+    def get_jail_root(self):
+        raise NotImplementedError(self.get_jail_root)
+
+    def run_script(self, script):
+        return self.script_runner.run_script(script)
+
+    def run_command(self, cmd, input, output, error):
+        return self.script_runner.run_command(cmd, input, output, error)
+
+
+class TestCaseWithTransportAndScript(tests.TestCaseWithTransport):
+
+    def setUp(self):
+        super(TestCaseWithTransportAndScript, self).setUp()
+        self.script_runner = ScriptRunner(self)
+        # Break the circular dependency
+        def break_dependency():
+            self.script_runner = None
+        self.addCleanup(break_dependency)
+
+    def get_jail_root(self):
+        return self.test_dir
+
+    def run_script(self, script):
+        return self.script_runner.run_script(script)
+
+    def run_command(self, cmd, input, output, error):
+        return self.script_runner.run_command(cmd, input, output, error)

=== added file 'bzrlib/tests/test_script.py'
--- bzrlib/tests/test_script.py	1970-01-01 00:00:00 +0000
+++ bzrlib/tests/test_script.py	2009-09-04 14:55:59 +0000
@@ -0,0 +1,257 @@
+# Copyright (C) 2009 Canonical Ltd
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+
+from bzrlib import (
+    osutils,
+    tests,
+    )
+from bzrlib.tests import script
+
+
+class TestScriptSyntax(tests.TestCase):
+
+    def test_comment_is_ignored(self):
+        self.assertEquals([], script._script_to_commands('#comment\n'))
+
+    def test_empty_line_is_ignored(self):
+        self.assertEquals([], script._script_to_commands('\n'))
+
+    def test_simple_command(self):
+        self.assertEquals([(['cd', 'trunk'], None, None, None)],
+                           script._script_to_commands('cd trunk'))
+
+    def test_command_with_single_quoted_param(self):
+        story = """bzr commit -m 'two words'"""
+        self.assertEquals([(['bzr', 'commit', '-m', 'two words'],
+                            None, None, None)],
+                           script._script_to_commands(story))
+
+    def test_command_with_double_quoted_param(self):
+        story = """bzr commit -m "two words" """
+        self.assertEquals([(['bzr', 'commit', '-m', 'two words'],
+                            None, None, None)],
+                           script._script_to_commands(story))
+
+    def test_command_with_input(self):
+        self.assertEquals([(['cat', '>file'], 'content\n', None, None)],
+                           script._script_to_commands('cat >file\n<content\n'))
+
+    def test_command_with_output(self):
+        story = """
+bzr add
+>adding file
+>adding file2
+"""
+        self.assertEquals([(['bzr', 'add'], None,
+                            'adding file\nadding file2\n', None)],
+                          script._script_to_commands(story))
+
+    def test_command_with_error(self):
+        story = """
+bzr branch foo
+2>bzr: ERROR: Not a branch: "foo"
+"""
+        self.assertEquals([(['bzr', 'branch', 'foo'],
+                            None, None, 'bzr: ERROR: Not a branch: "foo"\n')],
+                          script._script_to_commands(story))
+    def test_input_without_command(self):
+        self.assertRaises(SyntaxError, script._script_to_commands, '<input')
+
+    def test_output_without_command(self):
+        self.assertRaises(SyntaxError, script._script_to_commands, '>input')
+
+    def test_command_with_backquotes(self):
+        story = """
+foo = `bzr file-id toto`
+"""
+        self.assertEquals([(['foo', '=', '`bzr file-id toto`'],
+                            None, None, None)],
+                          script._script_to_commands(story))
+
+
+class TestScriptExecution(script.TestCaseWithTransportAndScript):
+
+    def test_unknown_command(self):
+        self.assertRaises(SyntaxError, self.run_script, 'foo')
+
+    def test_stops_on_unexpected_output(self):
+        story = """
+mkdir dir
+cd dir
+>Hello, I have just cd into dir !
+"""
+        self.assertRaises(AssertionError, self.run_script, story)
+
+
+    def test_stops_on_unexpected_error(self):
+        story = """
+cat
+<Hello
+bzr not-a-command
+"""
+        self.assertRaises(AssertionError, self.run_script, story)
+
+    def test_continue_on_expected_error(self):
+        story = """
+bzr not-a-command
+2>..."not-a-command"
+"""
+        self.run_script(story)
+
+    def test_ellipsis_output(self):
+        story = """
+cat
+<first line
+<second line
+<last line
+>first line
+>...
+>last line
+"""
+        self.run_script(story)
+        story = """
+bzr not-a-command
+2>..."not-a-command"
+"""
+        self.run_script(story)
+
+        story = """
+bzr branch not-a-branch
+2>bzr: ERROR: Not a branch...not-a-branch/".
+"""
+        self.run_script(story)
+
+
+class TestCat(script.TestCaseWithTransportAndScript):
+
+    def test_cat_usage(self):
+        self.assertRaises(SyntaxError, self.run_script, 'cat foo bar baz')
+        self.assertRaises(SyntaxError, self.run_script, 'cat foo <bar')
+
+    def test_cat_input_to_output(self):
+        out, err = self.run_command(['cat'], 'content\n', 'content\n', None)
+        self.assertEquals('content\n', out)
+        self.assertEquals(None, err)
+
+    def test_cat_file_to_output(self):
+        self.build_tree_contents([('file', 'content\n')])
+        out, err = self.run_command(['cat', 'file'], None, 'content\n', None)
+        self.assertEquals('content\n', out)
+        self.assertEquals(None, err)
+
+    def test_cat_input_to_file(self):
+        out, err = self.run_command(['cat', '>file'], 'content\n', None, None)
+        self.assertFileEqual('content\n', 'file')
+        self.assertEquals(None, out)
+        self.assertEquals(None, err)
+        out, err = self.run_command(['cat', '>>file'], 'more\n', None, None)
+        self.assertFileEqual('content\nmore\n', 'file')
+        self.assertEquals(None, out)
+        self.assertEquals(None, err)
+
+    def test_cat_file_to_file(self):
+        self.build_tree_contents([('file', 'content\n')])
+        out, err = self.run_command(['cat', 'file', '>file2'], None, None, None)
+        self.assertFileEqual('content\n', 'file2')
+
+
+class TestMkdir(script.TestCaseWithTransportAndScript):
+
+    def test_mkdir_usage(self):
+        self.assertRaises(SyntaxError, self.run_script, 'mkdir')
+        self.assertRaises(SyntaxError, self.run_script, 'mkdir foo bar')
+
+    def test_mkdir_jailed(self):
+        self.assertRaises(ValueError, self.run_script, 'mkdir /out-of-jail')
+        self.assertRaises(ValueError, self.run_script, 'mkdir ../out-of-jail')
+
+    def test_mkdir_in_jail(self):
+        self.run_script("""
+mkdir dir
+cd dir
+mkdir ../dir2
+cd ..
+""")
+        self.failUnlessExists('dir')
+        self.failUnlessExists('dir2')
+
+
+class TestCd(script.TestCaseWithTransportAndScript):
+
+    def test_cd_usage(self):
+        self.assertRaises(SyntaxError, self.run_script, 'cd foo bar')
+
+    def test_cd_out_of_jail(self):
+        self.assertRaises(ValueError, self.run_script, 'cd /out-of-jail')
+        self.assertRaises(ValueError, self.run_script, 'cd ..')
+
+    def test_cd_dir_and_back_home(self):
+        self.assertEquals(self.test_dir, osutils.getcwd())
+        self.run_script("""
+mkdir dir
+cd dir
+""")
+        self.assertEquals(osutils.pathjoin(self.test_dir, 'dir'),
+                          osutils.getcwd())
+
+        self.run_script('cd')
+        self.assertEquals(self.test_dir, osutils.getcwd())
+
+
+class TestBzr(script.TestCaseWithTransportAndScript):
+
+    def test_bzr_smoke(self):
+        self.run_script('bzr init branch')
+        self.failUnlessExists('branch')
+
+
+class TestEcho(script.TestCaseWithMemoryTransportAndScript):
+
+    def test_echo_usage(self):
+        story = """
+echo foo
+<bar
+"""
+        self.assertRaises(SyntaxError, self.run_script, story)
+
+    def test_echo_to_output(self):
+        out, err = self.run_command(['echo'], None, '\n', None)
+        self.assertEquals('\n', out)
+        self.assertEquals(None, err)
+
+    def test_echo_some_to_output(self):
+        out, err = self.run_command(['echo', 'hello'], None, 'hello\n', None)
+        self.assertEquals('hello\n', out)
+        self.assertEquals(None, err)
+
+    def test_echo_more_output(self):
+        out, err = self.run_command(['echo', 'hello', 'happy', 'world'],
+                                    None, 'hellohappyworld\n', None)
+        self.assertEquals('hellohappyworld\n', out)
+        self.assertEquals(None, err)

Ah, that's a bit different from regular unix echo, which would print
them with spaces in between.

But these tests puzzle me a bit because I don't see why they're not
using scripts directly?

+
+    def test_echo_appended(self):
+        out, err = self.run_command(['echo', 'hello', '>file'],
+                                    None, None, None)
+        self.assertEquals(None, out)
+        self.assertEquals(None, err)
+        self.assertFileEqual('hello\n', 'file')
+        out, err = self.run_command(['echo', 'happy', '>>file'],
+                                    None, None, None)
+        self.assertEquals(None, out)
+        self.assertEquals(None, err)
+        self.assertFileEqual('hello\nhappy\n', 'file')

review: Needs Fixing

Revision history for this message

Vincent Ladeuil (vila) wrote on 2009-09-17:

Download full text (7.7 KiB)

>>>>> "martin" == Martin Pool <email address hidden> writes:

martin> Review: Needs Fixing
martin> This is looking pretty cool.

Hey, there was so much comments I missed that introductory one
:-D

<snip/>

    martin> This is a nice description. However, I think it's
    martin> probably going to be more discoverable and more
    martin> point-to-able if it's in the developer documentation
    martin> rather than in the module docstring.

Done.

    martin> +
    martin> +A script is a set of commands, each command is composed of:
    martin> +- one mandatory command line,
    martin> +- one optional set of input lines to feed the command,
    martin> +- one optional set of output expected lines,
    martin> +- one optional set of error expected lines.

martin> .. in that order?

No, there is no constraints on the order, in fact the lines can
even be mixed if you wish.

    martin> +
    martin> +The optional lines starts with a special string (mnemonic: shell redirection):
    martin> +- '<' for input,
    martin> +- '>' for output,
    martin> +- '2>' for errors,

martin> Hm, I guess we'll see how the examples look but I think this may not be
martin> the clearest description of it. How about instead

    martin> $ the command
    martin> < the input
    martin> output by default
    2> errors

Done.

    martin> and I'm not so convinced people will find the errors
    martin> bit obvious. Do we really need to distinguish them
    martin> from stdout?

Yes. I didn't at first, and then I realize that:

- most of the time you don't want to check the output and you
don't care about it.

- most of the time you don't want to check stderr and you don't
care about it,

- you want the script to stop if an error occurs (which is not
related to stderr being empty or not), note that this has been
added in a later revision that you haven't reviewed yet,

- separating output and errors allows the script to match against
one OR the other OR both (but I've yet to use the later).

<snip/>

martin> 'output', 'is accepted and execution continues'.

    martin> Is this a good idea though? I feel like I'd like at
    martin> least '...' so that there's some indication that
    martin> stuff will happen but we don't care.

I thought so first too, but then each time you use 'init', 'add',
'st', well, any command with output you quickly don't want to
check neither output nor errors :D

In fact expected output and expected errors are mostly used as
assertions in conventional tests, so mainly you use them at the
end of the script and sometimes you embed some. At least that's
my limited experience so far.

<snip/>

martin> As ReST style you should put a double colon at the end of the line ::

Done.

<snip/>

    martin> +You can check the content of a file with cat:
    martin> +
    martin> + cat <file
    martin> + >expected content

martin> Is the redirection needed?

Not anymore, it was when I first wrote the example, but then I
needed to enhance cat anyway.

<snip/>

martin> +def split(s):
martin> + """Split a command line respecting quotes."""

martin> Better name?

It's the same that the c...

>>>>> "martin" == Martin Pool <mbp@sourcefrog.net> writes:

martin> Review: Needs Fixing
    martin> This is looking pretty cool.

Hey, there was so much comments I missed that introductory one
:-D

<snip/>

martin> This is a nice description.  However, I think it's
    martin> probably going to be more discoverable and more
    martin> point-to-able if it's in the developer documentation
    martin> rather than in the module docstring.

Done.

martin> .. in that order?

No, there is no constraints on the order, in fact the lines can
even be mixed if you wish.

martin> +
    martin> +The optional lines starts with a special string (mnemonic: shell redirection):
    martin> +- '<' for input,
    martin> +- '>' for output,
    martin> +- '2>' for errors,

martin> Hm, I guess we'll see how the examples look but I think this may not be
    martin> the clearest description of it.  How about instead

martin>   $ the command
    martin>   < the input
    martin>   output by default
    2> errors

Done.

martin> and I'm not so convinced people will find the errors
    martin> bit obvious.  Do we really need to distinguish them
    martin> from stdout?

Yes. I didn't at first, and then I realize that:

- most of the time you don't want to check the output and you
  don't care about it.

- most of the time you don't want to check stderr and you don't
  care about it,

- you want the script to stop if an error occurs (which is not
  related to stderr being empty or not), note that this has been
  added in a later revision that you haven't reviewed yet,

- separating output and errors allows the script to match against
  one OR the other OR both (but I've yet to use the later).

<snip/>

martin> 'output', 'is accepted and execution continues'.

martin> Is this a good idea though?  I feel like I'd like at
    martin> least '...' so that there's some indication that
    martin> stuff will happen but we don't care.

I thought so first too, but then each time you use 'init', 'add',
'st', well, any command with output you quickly don't want to
check neither output nor errors :D

<snip/>

martin> As ReST style you should put a double colon at the end of the line ::

Done.

<snip/>

martin> +You can check the content of a file with cat:
    martin> +
    martin> +  cat <file
    martin> +  >expected content

martin> Is the redirection needed?

Not anymore, it was when I first wrote the example, but then I
needed to enhance cat anyway.

<snip/>

martin> +def split(s):
    martin> +    """Split a command line respecting quotes."""

martin> Better name?

It's the same that the corresponding shlex function, I'd prefer
to keep it like that.

<snip/>

martin> +def _script_to_commands(text, file_name=None):
    martin> +    """Turn a script into a list of commands with their associated IOs.
    martin> +
    martin> +    Each command appears on a line by itself. It can be associated with an
    martin> +    input that will feed it and an expected output.
    martin> +    Comments starts with '#' until the end of line.

martin> Comments should be documented?

<Grumble> . o 0 (comments are documentation what does he mean ?)

OOhh, of course, yes, we should encourage users to add comments
and that wasn't documented ! Done :D

martin> How about rather than returning a command list, we
    martin> have a factory for Script objects that can then be
    martin> run?

You mean a factory for Command objects that can be run ? Yes, I
more or less plan to do that because I'm currently limited to
report errors precisely and that once you write a couple of buggy
script you quickly want to have errors with line numbers (even if
they will be relative to the beginning of the script and not line
numbers in the python source file (unless I find a simple enough
yet clever trick to provide that)).

Not done yet though.

<snip/>

martin> +    def run_command(self, cmd, input, output, error):
    martin> +        mname = 'do_' + cmd[0]

martin> Maybe we should put the specific command
    martin> implementations for bzr in a subclass? I guess it's
    martin> easy to do later on if desired.

<snip/>

martin> +    def do_mkdir(self, input, args):
    martin> +        if not args or len(args) != 1:
    martin> +            raise SyntaxError('Usage: mkdir dir')
    martin> +        d = args[0]
    martin> +        self._ensure_in_jail(d)
    martin> +        os.mkdir(d)
    martin> +        return None, None

martin> How about if we use a transport in here, not direct
    martin> os calls?  Then we can more easily move it to a
    martin> MemoryTransport in some cases later on.

Sure, that's the idea, as you can see the implementation is
pretty trivial so..

<snip/>
    martin> +class TestCaseWithMemoryTransportAndScript(tests.TestCaseWithMemoryTransport):

martin> This is a bit like what I was talking about with
    martin> Robert on the list today: I don't like the idea of
    martin> entangling tests with base classes so that you have
    martin> to inherit from a particular test.  I'd like someone
    martin> to be able to just use this style within a particular
    martin> test without having to move it to a different class
    martin> or to change the whole TestCase class to a different
    martin> parent.

I have the opposite position, I think we should write more test
classes and put our tests in more specialized classes.

Many of the refactorings I've done these last months have been in
that direction:

- bring a bunch of related tests,
- put them in the same class,
- add a common setUp(),
- remove the duplication at the beginning of all the tests.

And from that I got the strong feeling that defining *more* test
classes help to write tests with better defect localization :D

martin> Would it be feasible to just have an object that
    martin> tests can create and pass scripts to?

And create it in the setUp ?

martin> It seems like the only dependency is calling jail_root...

Yes, triggered by an early review by jam, I separate the script
engine from the test class as much as I could (if only to
demonstrate that we can't, yet, implement the script engine
without a true backing file system[1]).

But a test class is still needed in the end, and forcing users to
pass an additional parameter to run_script... is not appealing so
far.

<snip/>

martin> +    def test_echo_more_output(self):
    martin> +        out, err = self.run_command(['echo', 'hello', 'happy', 'world'],
    martin> +                                    None, 'hellohappyworld\n', None)
    martin> +        self.assertEquals('hellohappyworld\n', out)
    martin> +        self.assertEquals(None, err)

martin> Ah, that's a bit different from regular unix echo,
    martin> which would print them with spaces in between.

Bug ! Nice catch ! Fixed.

martin> But these tests puzzle me a bit because I don't see
    martin> why they're not using scripts directly?

Because the output of a script is not specified, only commands
can specify their expected outputs and errors.

Courtesy new submission review link:
https://code.edge.launchpad.net/~vila/bzr/shell-like-tests/+merge/11976

Vincent

[1]: And don't make me implement one before accepting that patch
     or it will never land :D

Revision history for this message

Martin Pool (mbp) wrote on 2009-09-18:

Download full text (3.8 KiB)

2009/9/18 Vincent Ladeuil <email address hidden>:

> martin> +
> martin> +A script is a set of commands, each command is composed of:
> martin> +- one mandatory command line,
> martin> +- one optional set of input lines to feed the command,
> martin> +- one optional set of output expected lines,
> martin> +- one optional set of error expected lines.
>
> martin> .. in that order?
>
> No, there is no constraints on the order, in fact the lines can
> even be mixed if you wish.

I think you should say so in the docs.

> martin> 'output', 'is accepted and execution continues'.
>
> martin> Is this a good idea though? I feel like I'd like at
> martin> least '...' so that there's some indication that
> martin> stuff will happen but we don't care.
>
> I thought so first too, but then each time you use 'init', 'add',
> 'st', well, any command with output you quickly don't want to
> check neither output nor errors :D
>
> In fact expected output and expected errors are mostly used as
> assertions in conventional tests, so mainly you use them at the
> end of the script and sometimes you embed some. At least that's
> my limited experience so far.

OK, but would it be too bad to say:

  $ bzr init
  ...
  $ bzr add foo
  ...

> <snip/>
> martin> +class TestCaseWithMemoryTransportAndScript(tests.TestCaseWithMemoryTransport):
>
> martin> This is a bit like what I was talking about with
> martin> Robert on the list today: I don't like the idea of
> martin> entangling tests with base classes so that you have
> martin> to inherit from a particular test. I'd like someone
> martin> to be able to just use this style within a particular
> martin> test without having to move it to a different class
> martin> or to change the whole TestCase class to a different
> martin> parent.
>
> I have the opposite position, I think we should write more test
> classes and put our tests in more specialized classes.

I don't object to that.

> Many of the refactorings I've done these last months have been in
> that direction:
>
> - bring a bunch of related tests,
> - put them in the same class,
> - add a common setUp(),
> - remove the duplication at the beginning of all the tests.

I think that's fine too, except that I would tend to put the common
code in an explicitly named method, not in setUp() (see the other
thread.)

But none of this is convincing me that you need a specific parent
class for the particular case of using scripts and a memory transport.
Why not just let people create scripts when they need them? It
doesn't need to be a static property of the test class. Or at the
very most, this could be a mixin superclass.

> And from that I got the strong feeling that defining *more* test
> classes help to write tests with better defect localization :D
>
> martin> Would it be feasible to just have an object that
> martin> tests can create and pass scripts to?
>
> And create it in the setUp ?

Why not just

def test_add(self):
run_script("""$bzr add foo
...
""")

> martin> + def test_echo_more_output(self):
> martin> + out, err = self.run_command(['echo', 'hello', 'happy', 'world'],
>...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk

Subscribers

People subscribed via source and target branches

to all changes:

Alejandro Cornejo2

Bazaar Codereview Subscribers

Benoit Pierre

Gmood

Karl Bielefeldt

Mahmoud Hassan

Matt Nordhoff

Mohd Fikri Mohd Amin

MrJOHN

Vincent Ladeuil

Václav Haisman

bzr PQM

vincenzo

to status/vote changes:

Alexander Belchenko

amandla2023

Bazaar

Merge lp:~vila/bzr/shell-like-tests into lp:bzr

Commit message

Description of the change

Preview Diff

Subscribers

 === modified file 'bzrlib/tests/__init__.py'
 --- bzrlib/tests/__init__.py	2009-08-28 21:05:31 +0000
 +++ bzrlib/tests/__init__.py	2009-09-01 08:24:44 +0000
@@ -3618,6 +3618,7 @@
          'bzrlib.tests.test_rio',
          'bzrlib.tests.test_rules',
          'bzrlib.tests.test_sampler',
++        'bzrlib.tests.test_script',
          'bzrlib.tests.test_selftest',
          'bzrlib.tests.test_serializer',
          'bzrlib.tests.test_setup',
 === added file 'bzrlib/tests/script.py'
 --- bzrlib/tests/script.py	1970-01-01 00:00:00 +0000
 +++ bzrlib/tests/script.py	2009-09-04 14:55:59 +0000
@@ -0,0 +1,389 @@
++# Copyright (C) 2009 Canonical Ltd
++#
++# This program is free software; you can redistribute it and/or modify
++# it under the terms of the GNU General Public License as published by
++# the Free Software Foundation; either version 2 of the License, or
++# (at your option) any later version.
++#
++# This program is distributed in the hope that it will be useful,
++# but WITHOUT ANY WARRANTY; without even the implied warranty of
++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
++# GNU General Public License for more details.
++#
++# You should have received a copy of the GNU General Public License
++# along with this program; if not, write to the Free Software
++# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
++"""Shell-like test scripts.
++
++This allows users to write tests in a syntax very close to a shell session,
++using a restricted and limited set of commands that should be enough to mimic
++most of the behaviours.
++
++A script is a set of commands, each command is composed of:
++- one mandatory command line,
++- one optional set of input lines to feed the command,
++- one optional set of output expected lines,
++- one optional set of error expected lines.
++
++The optional lines starts with a special string (mnemonic: shell redirection):
++- '<' for input,
++- '>' for output,
++- '2>' for errors,
++
++The execution stops as soon as an expected output or an expected error is not
++matched.
++
++When no output is specified, any ouput from the command is accepted
++and let the execution continue.
++
++If an error occurs and no expected error is specified, the execution stops.
++
++The matching is done on a full string comparison basis unless '...' is used, in
++which case expected output/errors can be lees precise.
++
++Examples:
++
++The following will succeeds only if 'bzr add' outputs 'adding file'.
++
++  bzr add file
++  >adding file
++
++If you want the command to succeed for any output, just use:
++
++  bzr add file
++
++The following will stop with an error:
++
++  bzr not-a-command
++
++If you want it to succeed, use:
++
++  bzr not-a-command
++  2> bzr: ERROR: unknown command "not-a-command"
++
++You can use ellipsis (...) to replace any piece of text you don't want to be
++matched exactly:
++
++  bzr branch not-a-branch
++  2>bzr: ERROR: Not a branch...not-a-branch/".
++
++
++This can be used to ignore entire lines too:
++
++cat
++<first line
++<second line
++<third line
++<fourth line
++<last line
++>first line
++>...
++>last line
++
++You can check the content of a file with cat:
++
++  cat <file
++  >expected content
++
++You can also check the existence of a file with cat, the following will fail if
++the file doesn't exist:
++
++  cat file
++
++"""
++
++import doctest
++import os
++import shlex
++from cStringIO import StringIO
++
++from bzrlib import (
++    osutils,
++    tests,
++    )
++
++
++def split(s):
++    """Split a command line respecting quotes."""
++    scanner = shlex.shlex(s)
++    scanner.quotes = '\'"`'
++    scanner.whitespace_split = True
++    for t in list(scanner):
++        # Strip the simple and double quotes since we don't care about them.
++        # We leave the backquotes in place though since they have a different
++        # semantic.
++        if t[0] in  ('"', "'") and t[0] == t[-1]:
++            yield t[1:-1]
++        else:
++            yield t
++
++
++def _script_to_commands(text, file_name=None):
++    """Turn a script into a list of commands with their associated IOs.
++
++    Each command appears on a line by itself. It can be associated with an
++    input that will feed it and an expected output.
++    Comments starts with '#' until the end of line.
++    Empty lines are ignored.
++    Input and output are full lines terminated by a '\n'.
++    Input lines start with '<'.
++    Output lines start with '>'.
++    Error lines start with '2>'.
++    """
++
++    commands = []
++
++    def add_command(cmd, input, output, error):
++        if cmd is not None:
++            if input is not None:
++                input = ''.join(input)
++            if output is not None:
++                output = ''.join(output)
++            if error is not None:
++                error = ''.join(error)
++            commands.append((cmd, input, output, error))
++
++    cmd_cur = None
++    cmd_line = 1
++    lineno = 0
++    input, output, error = None, None, None
++    for line in text.split('\n'):
++        lineno += 1
++        # Keep a copy for error reporting
++        orig = line
++        comment =  line.find('#')
++        if comment >= 0:
++            # Delete comments
++            line = line[0:comment]
++            line = line.rstrip()
++        if line == '':
++            # Ignore empty lines
++            continue
++        if line.startswith('<'):
++            if input is None:
++                if cmd_cur is None:
++                    raise SyntaxError('No command for that input',
++                                      (file_name, lineno, 1, orig))
++                input = []
++            input.append(line[1:] + '\n')
++            continue
++        elif line.startswith('>'):
++            if output is None:
++                if cmd_cur is None:
++                    raise SyntaxError('No command for that output',
++                                      (file_name, lineno, 1, orig))
++                output = []
++            output.append(line[1:] + '\n')
++            continue
++        elif line.startswith('2>'):
++            if error is None:
++                if cmd_cur is None:
++                    raise SyntaxError('No command for that error',
++                                      (file_name, lineno, 1, orig))
++                error = []
++            error.append(line[2:] + '\n')
++            continue
++        else:
++            # Time to output the current command
++            add_command(cmd_cur, input, output, error)
++            # And start a new one
++            cmd_cur = list(split(line))
++            cmd_line = lineno
++            input, output, error = None, None, None
++    # Add the last seen command
++    add_command(cmd_cur, input, output, error)
++    return commands
++
++
++def _scan_redirection_options(args):
++    """Recognize and process input and output redirections.
++
++    :param args: The command line arguments
++
++    :return: A tuple containing:
++        - The file name redirected from or None
++        - The file name redirected to or None
++        - The mode to open the output file or None
++        - The reamining arguments
++    """
++    remaining = []
++    in_name = None
++    out_name, out_mode = None, None
++    for arg in  args:
++        if arg.startswith('<'):
++            in_name = arg[1:]
++        elif arg.startswith('>>'):
++            out_name = arg[2:]
++            out_mode = 'ab+'
++        elif arg.startswith('>'):
++            out_name = arg[1:]
++            out_mode = 'wb+'
++        else:
++            remaining.append(arg)
++    return in_name, out_name, out_mode, remaining
++
++
++class ScriptRunner(object):
++
++    def __init__(self, test_case):
++        self.test_case = test_case
++        self.output_checker = doctest.OutputChecker()
++        self.check_options = doctest.ELLIPSIS
++
++    def run_script(self, text):
++        for cmd, input, output, error in _script_to_commands(text):
++            out, err = self.run_command(cmd, input, output, error)
++
++    def _check_output(self, expected, actual):
++        if expected is None:
++            # Specifying None means: any output is accepted
++            return
++        if actual is None:
++            self.test_case.fail('Unexpected: %s' % actual)
++        matching = self.output_checker.check_output(
++            expected, actual, self.check_options)
++        if not matching:
++            # Note that we can't use output_checker.output_difference() here
++            # because... the API is boken (expected must be a doctest specific
++            # object of whicha 'want' attribute will be our 'expected'
++            # parameter. So we just fallbacl to our good old assertEqualDiff
++            # since we know there are differences and the output should be
++            # decently readable.
++            self.test_case.assertEqualDiff(expected, actual)
++
++    def run_command(self, cmd, input, output, error):
++        mname = 'do_' + cmd[0]
++        method = getattr(self, mname, None)
++        if method is None:
++            raise SyntaxError('Command not found "%s"' % (cmd[0],),
++                              None, 1, ' '.join(cmd))
++        if input is None:
++            str_input = ''
++        else:
++            str_input = ''.join(input)
++        actual_output, actual_error = method(str_input, cmd[1:])
++
++        self._check_output(output, actual_output)
++        self._check_output(error, actual_error)
++        if not error and actual_error:
++            self.test_case.fail('Unexpected error: %s' % actual_error)
++        return actual_output, actual_error
++
++    def _read_input(self, input, in_name):
++        if in_name is not None:
++            infile = open(in_name, 'rb')
++            try:
++                # Command redirection takes precedence over provided input
++                input = infile.read()
++            finally:
++                infile.close()
++        return input
++
++    def _write_output(self, output, out_name, out_mode):
++        if out_name is not None:
++            outfile = open(out_name, out_mode)
++            try:
++                outfile.write(output)
++            finally:
++                outfile.close()
++            output = None
++        return output
++
++    def do_bzr(self, input, args):
++        out, err = self.test_case._run_bzr_core(
++            args, retcode=None, encoding=None, stdin=input, working_dir=None)
++        return out, err
++
++    def do_cat(self, input, args):
++        (in_name, out_name, out_mode, args) = _scan_redirection_options(args)
++        if len(args) > 1:
++            raise SyntaxError('Usage: cat [file1]')
++        if args:
++            if in_name is not None:
++                raise SyntaxError('Specify a file OR use redirection')
++            in_name = args[0]
++        input = self._read_input(input, in_name)
++        # Basically cat copy input to output
++        output = input
++        # Handle output redirections
++        output = self._write_output(output, out_name, out_mode)
++        return output, None
++
++    def do_echo(self, input, args):
++        (in_name, out_name, out_mode, args) = _scan_redirection_options(args)
++        if input and args:
++                raise SyntaxError('Specify parameters OR use redirection')
++        if args:
++            input = ''.join(args)
++        input = self._read_input(input, in_name)
++        # Always append a \n'
++        input += '\n'
++        # Process output
++        output = input
++        # Handle output redirections
++        output = self._write_output(output, out_name, out_mode)
++        return output, None
++
++    def _ensure_in_jail(self, path):
++        jail_root = self.test_case.get_jail_root()
++        if not osutils.is_inside(jail_root, osutils.normalizepath(path)):
++            raise ValueError('%s is not inside %s' % (path, jail_root))
++
++    def do_cd(self, input, args):
++        if len(args) > 1:
++            raise SyntaxError('Usage: cd [dir]')
++        if len(args) == 1:
++            d = args[0]
++            self._ensure_in_jail(d)
++        else:
++            d = self.test_case.get_jail_root()
++        os.chdir(d)
++        return None, None
++
++    def do_mkdir(self, input, args):
++        if not args or len(args) != 1:
++            raise SyntaxError('Usage: mkdir dir')
++        d = args[0]
++        self._ensure_in_jail(d)
++        os.mkdir(d)
++        return None, None
++
++
++class TestCaseWithMemoryTransportAndScript(tests.TestCaseWithMemoryTransport):
++
++    def setUp(self):
++        super(TestCaseWithMemoryTransportAndScript, self).setUp()
++        self.script_runner = ScriptRunner(self)
++        # Break the circular dependency
++        def break_dependency():
++            self.script_runner = None
++        self.addCleanup(break_dependency)
++
++    def get_jail_root(self):
++        raise NotImplementedError(self.get_jail_root)
++
++    def run_script(self, script):
++        return self.script_runner.run_script(script)
++
++    def run_command(self, cmd, input, output, error):
++        return self.script_runner.run_command(cmd, input, output, error)
++
++
++class TestCaseWithTransportAndScript(tests.TestCaseWithTransport):
++
++    def setUp(self):
++        super(TestCaseWithTransportAndScript, self).setUp()
++        self.script_runner = ScriptRunner(self)
++        # Break the circular dependency
++        def break_dependency():
++            self.script_runner = None
++        self.addCleanup(break_dependency)
++
++    def get_jail_root(self):
++        return self.test_dir
++
++    def run_script(self, script):
++        return self.script_runner.run_script(script)
++
++    def run_command(self, cmd, input, output, error):
++        return self.script_runner.run_command(cmd, input, output, error)
 === added file 'bzrlib/tests/test_script.py'
 --- bzrlib/tests/test_script.py	1970-01-01 00:00:00 +0000
 +++ bzrlib/tests/test_script.py	2009-09-04 14:55:59 +0000
@@ -0,0 +1,257 @@
++# Copyright (C) 2009 Canonical Ltd
++#
++# This program is free software; you can redistribute it and/or modify
++# it under the terms of the GNU General Public License as published by
++# the Free Software Foundation; either version 2 of the License, or
++# (at your option) any later version.
++#
++# This program is distributed in the hope that it will be useful,
++# but WITHOUT ANY WARRANTY; without even the implied warranty of
++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
++# GNU General Public License for more details.
++#
++# You should have received a copy of the GNU General Public License
++# along with this program; if not, write to the Free Software
++# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
++
++
++from bzrlib import (
++    osutils,
++    tests,
++    )
++from bzrlib.tests import script
++
++
++class TestScriptSyntax(tests.TestCase):
++
++    def test_comment_is_ignored(self):
++        self.assertEquals([], script._script_to_commands('#comment\n'))
++
++    def test_empty_line_is_ignored(self):
++        self.assertEquals([], script._script_to_commands('\n'))
++
++    def test_simple_command(self):
++        self.assertEquals([(['cd', 'trunk'], None, None, None)],
++                           script._script_to_commands('cd trunk'))
++
++    def test_command_with_single_quoted_param(self):
++        story = """bzr commit -m 'two words'"""
++        self.assertEquals([(['bzr', 'commit', '-m', 'two words'],
++                            None, None, None)],
++                           script._script_to_commands(story))
++
++    def test_command_with_double_quoted_param(self):
++        story = """bzr commit -m "two words" """
++        self.assertEquals([(['bzr', 'commit', '-m', 'two words'],
++                            None, None, None)],
++                           script._script_to_commands(story))
++
++    def test_command_with_input(self):
++        self.assertEquals([(['cat', '>file'], 'content\n', None, None)],
++                           script._script_to_commands('cat >file\n<content\n'))
++
++    def test_command_with_output(self):
++        story = """
++bzr add
++>adding file
++>adding file2
++"""
++        self.assertEquals([(['bzr', 'add'], None,
++                            'adding file\nadding file2\n', None)],
++                          script._script_to_commands(story))
++
++    def test_command_with_error(self):
++        story = """
++bzr branch foo
++2>bzr: ERROR: Not a branch: "foo"
++"""
++        self.assertEquals([(['bzr', 'branch', 'foo'],
++                            None, None, 'bzr: ERROR: Not a branch: "foo"\n')],
++                          script._script_to_commands(story))
++    def test_input_without_command(self):
++        self.assertRaises(SyntaxError, script._script_to_commands, '<input')
++
++    def test_output_without_command(self):
++        self.assertRaises(SyntaxError, script._script_to_commands, '>input')
++
++    def test_command_with_backquotes(self):
++        story = """
++foo = `bzr file-id toto`
++"""
++        self.assertEquals([(['foo', '=', '`bzr file-id toto`'],
++                            None, None, None)],
++                          script._script_to_commands(story))
++
++
++class TestScriptExecution(script.TestCaseWithTransportAndScript):
++
++    def test_unknown_command(self):
++        self.assertRaises(SyntaxError, self.run_script, 'foo')
++
++    def test_stops_on_unexpected_output(self):
++        story = """
++mkdir dir
++cd dir
++>Hello, I have just cd into dir !
++"""
++        self.assertRaises(AssertionError, self.run_script, story)
++
++
++    def test_stops_on_unexpected_error(self):
++        story = """
++cat
++<Hello
++bzr not-a-command
++"""
++        self.assertRaises(AssertionError, self.run_script, story)
++
++    def test_continue_on_expected_error(self):
++        story = """
++bzr not-a-command
++2>..."not-a-command"
++"""
++        self.run_script(story)
++
++    def test_ellipsis_output(self):
++        story = """
++cat
++<first line
++<second line
++<last line
++>first line
++>...
++>last line
++"""
++        self.run_script(story)
++        story = """
++bzr not-a-command
++2>..."not-a-command"
++"""
++        self.run_script(story)
++
++        story = """
++bzr branch not-a-branch
++2>bzr: ERROR: Not a branch...not-a-branch/".
++"""
++        self.run_script(story)
++
++
++class TestCat(script.TestCaseWithTransportAndScript):
++
++    def test_cat_usage(self):
++        self.assertRaises(SyntaxError, self.run_script, 'cat foo bar baz')
++        self.assertRaises(SyntaxError, self.run_script, 'cat foo <bar')
++
++    def test_cat_input_to_output(self):
++        out, err = self.run_command(['cat'], 'content\n', 'content\n', None)
++        self.assertEquals('content\n', out)
++        self.assertEquals(None, err)
++
++    def test_cat_file_to_output(self):
++        self.build_tree_contents([('file', 'content\n')])
++        out, err = self.run_command(['cat', 'file'], None, 'content\n', None)
++        self.assertEquals('content\n', out)
++        self.assertEquals(None, err)
++
++    def test_cat_input_to_file(self):
++        out, err = self.run_command(['cat', '>file'], 'content\n', None, None)
++        self.assertFileEqual('content\n', 'file')
++        self.assertEquals(None, out)
++        self.assertEquals(None, err)
++        out, err = self.run_command(['cat', '>>file'], 'more\n', None, None)
++        self.assertFileEqual('content\nmore\n', 'file')
++        self.assertEquals(None, out)
++        self.assertEquals(None, err)
++
++    def test_cat_file_to_file(self):
++        self.build_tree_contents([('file', 'content\n')])
++        out, err = self.run_command(['cat', 'file', '>file2'], None, None, None)
++        self.assertFileEqual('content\n', 'file2')
++
++
++class TestMkdir(script.TestCaseWithTransportAndScript):
++
++    def test_mkdir_usage(self):
++        self.assertRaises(SyntaxError, self.run_script, 'mkdir')
++        self.assertRaises(SyntaxError, self.run_script, 'mkdir foo bar')
++
++    def test_mkdir_jailed(self):
++        self.assertRaises(ValueError, self.run_script, 'mkdir /out-of-jail')
++        self.assertRaises(ValueError, self.run_script, 'mkdir ../out-of-jail')
++
++    def test_mkdir_in_jail(self):
++        self.run_script("""
++mkdir dir
++cd dir
++mkdir ../dir2
++cd ..
++""")
++        self.failUnlessExists('dir')
++        self.failUnlessExists('dir2')
++
++
++class TestCd(script.TestCaseWithTransportAndScript):
++
++    def test_cd_usage(self):
++        self.assertRaises(SyntaxError, self.run_script, 'cd foo bar')
++
++    def test_cd_out_of_jail(self):
++        self.assertRaises(ValueError, self.run_script, 'cd /out-of-jail')
++        self.assertRaises(ValueError, self.run_script, 'cd ..')
++
++    def test_cd_dir_and_back_home(self):
++        self.assertEquals(self.test_dir, osutils.getcwd())
++        self.run_script("""
++mkdir dir
++cd dir
++""")
++        self.assertEquals(osutils.pathjoin(self.test_dir, 'dir'),
++                          osutils.getcwd())
++
++        self.run_script('cd')
++        self.assertEquals(self.test_dir, osutils.getcwd())
++
++
++class TestBzr(script.TestCaseWithTransportAndScript):
++
++    def test_bzr_smoke(self):
++        self.run_script('bzr init branch')
++        self.failUnlessExists('branch')
++
++
++class TestEcho(script.TestCaseWithMemoryTransportAndScript):
++
++    def test_echo_usage(self):
++        story = """
++echo foo
++<bar
++"""
++        self.assertRaises(SyntaxError, self.run_script, story)
++
++    def test_echo_to_output(self):
++        out, err = self.run_command(['echo'], None, '\n', None)
++        self.assertEquals('\n', out)
++        self.assertEquals(None, err)
++
++    def test_echo_some_to_output(self):
++        out, err = self.run_command(['echo', 'hello'], None, 'hello\n', None)
++        self.assertEquals('hello\n', out)
++        self.assertEquals(None, err)
++
++    def test_echo_more_output(self):
++        out, err = self.run_command(['echo', 'hello', 'happy', 'world'],
++                                    None, 'hellohappyworld\n', None)
++        self.assertEquals('hellohappyworld\n', out)
++        self.assertEquals(None, err)
++
++    def test_echo_appended(self):
++        out, err = self.run_command(['echo', 'hello', '>file'],
++                                    None, None, None)
++        self.assertEquals(None, out)
++        self.assertEquals(None, err)
++        self.assertFileEqual('hello\n', 'file')
++        out, err = self.run_command(['echo', 'happy', '>>file'],
++                                    None, None, None)
++        self.assertEquals(None, out)
++        self.assertEquals(None, err)
++        self.assertFileEqual('hello\nhappy\n', 'file')