Merge lp:~jml/launchpad/sftp-poppy into lp:launchpad

Proposed by Jonathan Lange
Status: Merged
Approved by: Jonathan Lange
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~jml/launchpad/sftp-poppy
Merge into: lp:launchpad
Diff against target: 485 lines (+125/-146)
8 files modified
lib/canonical/launchpad/daemons/tachandler.py (+83/-47)
lib/lp/poppy/filesystem.py (+0/-1)
lib/lp/poppy/tests/__init__.py (+5/-0)
lib/lp/poppy/tests/helpers.py (+8/-16)
lib/lp/poppy/tests/test_poppy.py (+17/-23)
lib/lp/services/osutils.py (+8/-54)
lib/lp/soyuz/doc/soyuz-upload.txt.disabled (+1/-1)
lib/lp/testing/__init__.py (+3/-4)
To merge this branch: bzr merge lp:~jml/launchpad/sftp-poppy
Reviewer Review Type Date Requested Status
Gavin Panella (community) Approve
Michael Nelson Pending
Review via email: mp+21627@code.launchpad.net

Commit message

Add two_stage_kill, speed up tachandler and kill by pidfile. Clean up poppy tests.

Description of the change

Contrary to what you might expect, this branch actually has nothing to do with SFTP. Instead, it does a bunch of cleanups and fixes to testing code that I had to touch in order to start thinking about with Poppy.

 canonical/launchpad/daemons/tachandler.py | 36 -----------------

Use osutils.kill_by_pidfile, which appeared to a direct copy and paste from this file.

 lp/services/osutils.py | 61 +++++++++++++++++++-----------

Extract the two-stage kill part of osutils.kill_by_pidfile into a separate function called two_stage_kill.

Fix a bug in the two-stage kill algorithm. If you terminate a process and don't give it the chance to report its results back to the parent process, the terminated process remains as a zombie. Without the waitpid, the polling loop will *always* run to completion and then do the SIGKILL, thus making the call take over 5s guaranteed.

Fixing this may well make some layers tear down faster.

 lp/poppy/filesystem.py | 1

Remove flakes.

 lp/poppy/tests/__init__.py | 5 ++
 lp/poppy/tests/helpers.py | 24 +++--------

Move all of the helper code out of __init__ and into helpers.py. Add a stub __init__.py.

Fix up some of the formatting in helpers, and re-use two_stage_kill, getting rid of the slow (wait for 2s) two-stage kill implementation here.

 lp/poppy/tests/test_poppy.py | 40 ++++++++-----------

Bits and pieces of cleanup:
 * Use lp.testing.TestCase
 * Fix some flakes
 * Rename tests to test_foo from testFoo, according to the standard
 * Delete the useless layer
 * Fix spelling mistakes
 * Use addCleanup instead of tearDown
 * Use makeTemporaryDirectory
 * Whitespace cleanups

 lp/soyuz/doc/soyuz-upload.txt.disabled | 2

The return value of killPoppy isn't useful and isn't used.

 lp/testing/__init__.py | 7 +--

Avoid using lambdas in addCleanup -- this is a throwback to an old addCleanup API. Re-use makeTemporaryDirectory in useTempDir.

Thanks,
jml

To post a comment you must log in.
Revision history for this message
Gavin Panella (allenap) wrote :

Looks good. Two comments:

* It's not necessary to do waitpid(pid, os.WNOHANG) followed by
  kill(pid, 0); the return value from waitpid() will be (0, 0) when
  the process has not exited yet, and it will error the same as kill()
  if the process does not exist.

* It might be nice to be able to pass a subprocess.Popen into
  two_stage_kill, so that its poll() method is used, and so the
  returncode attribute is set correctly to avoid confusion. But,
  really, this is probably not worth the effort.

review: Approve
Revision history for this message
Michael Nelson (michael.nelson) wrote :
Download full text (8.7 KiB)

Review: Approve
Nothing I can add to Gavin's (which you're investigating).

On Thu, Mar 18, 2010 at 11:35 AM, Jonathan Lange <email address hidden> wrote:
> Jonathan Lange has proposed merging lp:~jml/launchpad/sftp-poppy into lp:launchpad/devel.
>
> Requested reviews:
>  Canonical Launchpad Engineering (launchpad)
>
>
> Contrary to what you might expect, this branch actually has nothing to do with SFTP. Instead, it does a bunch of cleanups and fixes to testing code that I had to touch in order to start thinking about with Poppy.
>
>  canonical/launchpad/daemons/tachandler.py |   36 -----------------
>
> Use osutils.kill_by_pidfile, which appeared to a direct copy and paste from this file.
>
>  lp/services/osutils.py                    |   61 +++++++++++++++++++-----------
>
> Extract the two-stage kill part of osutils.kill_by_pidfile into a separate function called two_stage_kill.
>
> Fix a bug in the two-stage kill algorithm. If you terminate a process and don't give it the chance to report its results back to the parent process, the terminated process remains as a zombie. Without the waitpid, the polling loop will *always* run to completion and then do the SIGKILL, thus making the call take over 5s guaranteed.

Nice.

>
> Fixing this may well make some layers tear down faster.
>
>  lp/poppy/filesystem.py                    |    1
>
> Remove flakes.
>
>  lp/poppy/tests/__init__.py                |    5 ++
>  lp/poppy/tests/helpers.py                 |   24 +++--------
>
> Move all of the helper code out of __init__ and into helpers.py. Add a stub __init__.py.
>
> Fix up some of the formatting in helpers, and re-use two_stage_kill, getting rid of the slow (wait for 2s) two-stage kill implementation here.
>
>  lp/poppy/tests/test_poppy.py              |   40 ++++++++-----------
>
> Bits and pieces of cleanup:
>  * Use lp.testing.TestCase
>  * Fix some flakes
>  * Rename tests to test_foo from testFoo, according to the standard
>  * Delete the useless layer
>  * Fix spelling mistakes
>  * Use addCleanup instead of tearDown
>  * Use makeTemporaryDirectory
>  * Whitespace cleanups
>
>  lp/soyuz/doc/soyuz-upload.txt.disabled    |    2
>
> The return value of killPoppy isn't useful and isn't used.
>
>  lp/testing/__init__.py                    |    7 +--
>
> Avoid using lambdas in addCleanup -- this is a throwback to an old addCleanup API. Re-use makeTemporaryDirectory in useTempDir.
>
> Thanks,
> jml
> --
> https://code.launchpad.net/~jml/launchpad/sftp-poppy/+merge/21627
> Your team Launchpad code reviewers from Canonical is subscribed to branch lp:launchpad/devel.
>
> === modified file 'lib/canonical/launchpad/daemons/tachandler.py'
> --- lib/canonical/launchpad/daemons/tachandler.py       2010-02-19 13:27:29 +0000
> +++ lib/canonical/launchpad/daemons/tachandler.py       2010-03-18 11:35:30 +0000
> @@ -16,12 +16,12 @@
>  import sys
>  import os
>  import time
> -from signal import SIGTERM, SIGKILL
>  import subprocess
>
>  from twisted.application import service
>  from twisted.python import log
>
> +from lp.services.osutils import kill_by_pidfile
>
>  twistd_script = os.path.abspath(os.path.join(
>     os.path.dirname(__file__),
> @@ -118,39 +11...

Read more...

Revision history for this message
Jonathan Lange (jml) wrote :

On Thu, Mar 18, 2010 at 11:57 AM, Gavin Panella
<email address hidden> wrote:
> Review: Approve
> Looks good. Two comments:
>
> * It's not necessary to do waitpid(pid, os.WNOHANG) followed by
>  kill(pid, 0); the return value from waitpid() will be (0, 0) when
>  the process has not exited yet, and it will error the same as kill()
>  if the process does not exist.
>

Hmm. I see. I wasn't quite sure what the behaviour actually is, so I
did Science.

$ ps ax | grep 3711
 7702 pts/2 R+ 0:00 grep 3711
$ python
Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os

Spawn a long-running process

>>> pid = os.spawnv(os.P_NOWAIT, '/bin/sleep', ['/bin/sleep', '10'])
>>> pid
7820

waitpid on-existent process raises errors

>>> os.waitpid(3711, os.WNOHANG)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 10] No child processes

waitpid returns (0, 0) while it's running for (pid, result)
>>> os.waitpid(7820, os.WNOHANG)
(0, 0)

Once it's terminated, returns (pid, result) with a non-zero pid.
>>> os.waitpid(7820, os.WNOHANG)
(7820, 0)

Now that it's terminated and we've reaped it, it'll raise an exception
>>> os.waitpid(7820, os.WNOHANG)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 10] No child processes

I'll fix the code accordingly, making sure that we don't force a 0.1s
sleep if the SIGTERM was successful.

> * It might be nice to be able to pass a subprocess.Popen into
>  two_stage_kill, so that its poll() method is used, and so the
>  returncode attribute is set correctly to avoid confusion. But,
>  really, this is probably not worth the effort.

It's valuable having a low-level function that takes a pid. I agree
that what you describe would also be useful, but maybe not right now.
I wonder what we have to do to encourage others who might need this to
stick code into this module and re-use two-stage kill.

Thanks!
jml

Revision history for this message
Gavin Panella (allenap) wrote :

jml wrote:
> I wonder what we have to do to encourage others who might need this to
> stick code into this module and re-use two-stage kill.

Get it into the standard library! :)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/canonical/launchpad/daemons/tachandler.py'
--- lib/canonical/launchpad/daemons/tachandler.py 2010-02-19 13:27:29 +0000
+++ lib/canonical/launchpad/daemons/tachandler.py 2010-03-18 19:22:29 +0000
@@ -7,11 +7,16 @@
77
8__metaclass__ = type8__metaclass__ = type
99
10__all__ = ['TacTestSetup', 'ReadyService', 'TacException']10__all__ = [
1111 'TacTestSetup',
1212 'ReadyService',
13# This file is used by launchpad-buildd, so it cannot import any13 'TacException',
14# Launchpad code!14 'kill_by_pidfile',
15 'remove_if_exists',
16 'two_stage_kill',
17 ]
18
19
15import errno20import errno
16import sys21import sys
17import os22import os
@@ -22,6 +27,77 @@
22from twisted.application import service27from twisted.application import service
23from twisted.python import log28from twisted.python import log
2429
30# This file is used by launchpad-buildd, so it cannot import any
31# Launchpad code!
32
33
34def two_stage_kill(pid, poll_interval=0.1, num_polls=50):
35 """Kill process 'pid' with SIGTERM. If it doesn't die, SIGKILL it.
36
37 :param pid: The pid of the process to kill.
38 :param poll_interval: The polling interval used to check if the
39 process is still around.
40 :param num_polls: The number of polls to do before doing a SIGKILL.
41 """
42 # Kill the process.
43 try:
44 os.kill(pid, SIGTERM)
45 except OSError, e:
46 if e.errno in (errno.ESRCH, errno.ECHILD):
47 # Process has already been killed.
48 return
49
50 # Poll until the process has ended.
51 for i in range(num_polls):
52 try:
53 # Reap the child process and get its return value. If it's not
54 # gone yet, continue.
55 new_pid, result = os.waitpid(pid, os.WNOHANG)
56 if new_pid:
57 return result
58 time.sleep(poll_interval)
59 except OSError, e:
60 # Raised if the process is gone by the time we try to get the
61 # return value.
62 return
63
64 # The process is still around, so terminate it violently.
65 try:
66 os.kill(pid, SIGKILL)
67 except OSError:
68 # Already terminated
69 pass
70
71
72def kill_by_pidfile(pidfile_path):
73 """Kill a process identified by the pid stored in a file.
74
75 The pid file is removed from disk.
76 """
77 if not os.path.exists(pidfile_path):
78 return
79 try:
80 # Get the pid.
81 pid = open(pidfile_path, 'r').read().split()[0]
82 try:
83 pid = int(pid)
84 except ValueError:
85 # pidfile contains rubbish
86 return
87
88 two_stage_kill(pid)
89 finally:
90 remove_if_exists(pidfile_path)
91
92
93def remove_if_exists(path):
94 """Remove the given file if it exists."""
95 try:
96 os.remove(path)
97 except OSError, e:
98 if e.errno != errno.ENOENT:
99 raise
100
25101
26twistd_script = os.path.abspath(os.path.join(102twistd_script = os.path.abspath(os.path.join(
27 os.path.dirname(__file__),103 os.path.dirname(__file__),
@@ -48,7 +124,7 @@
48 # started. If it sees an old logfile, then it will find the LOG_MAGIC124 # started. If it sees an old logfile, then it will find the LOG_MAGIC
49 # string and return immediately, provoking hard-to-diagnose race125 # string and return immediately, provoking hard-to-diagnose race
50 # conditions. Delete the logfile to make sure this does not happen.126 # conditions. Delete the logfile to make sure this does not happen.
51 self._removeFile(self.logfile)127 remove_if_exists(self.logfile)
52128
53 self.setUpRoot()129 self.setUpRoot()
54 args = [sys.executable, twistd_script, '-o', '-y', self.tacfile,130 args = [sys.executable, twistd_script, '-o', '-y', self.tacfile,
@@ -107,50 +183,10 @@
107 def tearDown(self):183 def tearDown(self):
108 self.killTac()184 self.killTac()
109185
110 def _removeFile(self, filename):
111 """Remove the given file if it exists."""
112 try:
113 os.remove(filename)
114 except OSError, e:
115 if e.errno != errno.ENOENT:
116 raise
117
118 def killTac(self):186 def killTac(self):
119 """Kill the TAC file if it is running."""187 """Kill the TAC file if it is running."""
120 pidfile = self.pidfile188 pidfile = self.pidfile
121 if not os.path.exists(pidfile):189 kill_by_pidfile(pidfile)
122 return
123
124 # Get the pid.
125 pid = open(pidfile, 'r').read().strip()
126 try:
127 pid = int(pid)
128 except ValueError:
129 # pidfile contains rubbish
130 return
131
132 # Kill the process.
133 try:
134 os.kill(pid, SIGTERM)
135 except OSError, e:
136 if e.errno in (errno.ESRCH, errno.ECHILD):
137 # Process has already been killed.
138 return
139
140 # Poll until the process has ended.
141 for i in range(50):
142 try:
143 os.kill(pid, 0)
144 time.sleep(0.1)
145 except OSError, e:
146 break
147 else:
148 # The process is still around, so terminate it violently.
149 try:
150 os.kill(pid, SIGKILL)
151 except OSError:
152 # Already terminated
153 pass
154190
155 def setUpRoot(self):191 def setUpRoot(self):
156 """Override this.192 """Override this.
157193
=== modified file 'lib/lp/poppy/filesystem.py'
--- lib/lp/poppy/filesystem.py 2009-06-25 05:30:52 +0000
+++ lib/lp/poppy/filesystem.py 2010-03-18 19:22:29 +0000
@@ -6,7 +6,6 @@
6import datetime6import datetime
7import os7import os
8import stat8import stat
9import shutil
10from zope.security.interfaces import Unauthorized9from zope.security.interfaces import Unauthorized
11from zope.server.interfaces.ftp import IFileSystem10from zope.server.interfaces.ftp import IFileSystem
12from zope.interface import implements11from zope.interface import implements
1312
=== added file 'lib/lp/poppy/tests/__init__.py'
--- lib/lp/poppy/tests/__init__.py 1970-01-01 00:00:00 +0000
+++ lib/lp/poppy/tests/__init__.py 2010-03-18 19:22:29 +0000
@@ -0,0 +1,5 @@
1# Copyright 2010 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Tests for Poppy."""
5
06
=== renamed file 'lib/lp/poppy/tests/__init__.py' => 'lib/lp/poppy/tests/helpers.py'
--- lib/lp/poppy/tests/__init__.py 2009-06-25 05:39:50 +0000
+++ lib/lp/poppy/tests/helpers.py 2010-03-18 19:22:29 +0000
@@ -5,16 +5,18 @@
55
6import os6import os
7import select7import select
8import signal
9import subprocess8import subprocess
10import sys9import sys
11import time10import time
1211
13from canonical.config import config12from canonical.config import config
13from lp.services.osutils import two_stage_kill
14
1415
15class SoyuzUploadError(Exception):16class SoyuzUploadError(Exception):
16 """Used in the soyuz-upload test."""17 """Used in the soyuz-upload test."""
1718
19
18class PoppyTestSetup:20class PoppyTestSetup:
19 """Provides a setup and teardown mechanism for a poppy subprocess instance.21 """Provides a setup and teardown mechanism for a poppy subprocess instance.
2022
@@ -34,28 +36,18 @@
34 def startPoppy(self):36 def startPoppy(self):
35 """Start the poppy instance."""37 """Start the poppy instance."""
36 script = os.path.join(config.root, "daemons/poppy-upload.py")38 script = os.path.join(config.root, "daemons/poppy-upload.py")
37 self.process = subprocess.Popen([sys.executable, script,39 self.process = subprocess.Popen(
38 "--allow-user", self.user,40 [sys.executable, script, "--allow-user", self.user,
39 "--cmd", self.cmd,41 "--cmd", self.cmd, self.fsroot, self.port],
40 self.fsroot, self.port],42 stdout=subprocess.PIPE)
41 stdout=subprocess.PIPE)
42 self.running = True43 self.running = True
4344
44 def killPoppy(self):45 def killPoppy(self):
45 """Kill the poppy instance dead."""46 """Kill the poppy instance dead."""
46 if not self.alive:47 if not self.alive:
47 return48 return
48 os.kill(self.process.pid, signal.SIGTERM)49 two_stage_kill(self.process.pid)
49 # Give poppy a chance to die
50 time.sleep(2)
51 # Look to see if it has died yet
52 ret = self.process.poll()
53 if ret is None:
54 # Poppy had not died, so send it SIGKILL
55 os.kill(self.process.pid, signal.SIGKILL)
56 # Finally return the result.
57 self.running = False50 self.running = False
58 return self.process.wait()
5951
60 @property52 @property
61 def alive(self):53 def alive(self):
6254
=== modified file 'lib/lp/poppy/tests/test_poppy.py'
--- lib/lp/poppy/tests/test_poppy.py 2010-03-17 11:04:20 +0000
+++ lib/lp/poppy/tests/test_poppy.py 2010-03-18 19:22:29 +0000
@@ -7,45 +7,38 @@
77
8import ftplib8import ftplib
9import os9import os
10import shutil
11import socket10import socket
12import tempfile
13import unittest11import unittest
14import StringIO12import StringIO
1513
16from canonical.testing import LaunchpadZopelessLayer14from lp.poppy.tests.helpers import PoppyTestSetup
17from lp.poppy.tests import PoppyTestSetup15from lp.testing import TestCase
1816
1917
20class TestPoppy(unittest.TestCase):18class TestPoppy(TestCase):
21 """Test if poppy.py daemon works properly."""19 """Test if poppy.py daemon works properly."""
2220
23 layer = LaunchpadZopelessLayer
24
25 def setUp(self):21 def setUp(self):
26 """Set up poppy in a temp dir."""22 """Set up poppy in a temp dir."""
27 self.root_dir = tempfile.mkdtemp()23 super(TestPoppy, self).setUp()
24 self.root_dir = self.makeTemporaryDirectory()
28 self.port = 342125 self.port = 3421
29 self.poppy = PoppyTestSetup(self.root_dir, port=self.port,26 self.poppy = PoppyTestSetup(self.root_dir, port=self.port,
30 cmd='echo CLOSED')27 cmd='echo CLOSED')
31 self.poppy.startPoppy()28 self.poppy.startPoppy()
3229 self.addCleanup(self.poppy.killPoppy)
33 def tearDown(self):
34 """Purge poppy root directory."""
35 self.poppy.killPoppy()
36 shutil.rmtree(self.root_dir)
3730
38 def getFTPConnection(self, login=True, user="ubuntu", password=""):31 def getFTPConnection(self, login=True, user="ubuntu", password=""):
39 """Build and return a FTP connection to the current poppy.32 """Build and return a FTP connection to the current poppy.
4033
41 Optionally log in with as 'annonymous' & empty password, or passed34 Optionally log in with as 'anonymous' & empty password, or passed
42 user/password.35 user/password.
43 """36 """
44 conn = ftplib.FTP()37 conn = ftplib.FTP()
45 # poppy usually takes sometime to come up, we need to wait, or insist.38 # poppy usually takes sometime to come up, we need to wait, or insist.
46 while True:39 while True:
47 try:40 try:
48 reply = conn.connect("localhost", self.port)41 conn.connect("localhost", self.port)
49 except socket.error:42 except socket.error:
50 if not self.poppy.alive:43 if not self.poppy.alive:
51 raise44 raise
@@ -75,7 +68,7 @@
75 upload_dir = contents[1]68 upload_dir = contents[1]
76 return os.path.join(self.root_dir, upload_dir, path)69 return os.path.join(self.root_dir, upload_dir, path)
7770
78 def testLOGIN(self):71 def test_LOGIN(self):
79 """Check login procedure and the state of the toplevel lockfile.72 """Check login procedure and the state of the toplevel lockfile.
8073
81 The toplevel lockfile should allow the process-upload runner (lp_queue,74 The toplevel lockfile should allow the process-upload runner (lp_queue,
@@ -90,7 +83,7 @@
90 lockfile_path = os.path.join(self.root_dir, '.lock')83 lockfile_path = os.path.join(self.root_dir, '.lock')
91 self.assertEqual(os.stat(lockfile_path).st_mode, 0100664)84 self.assertEqual(os.stat(lockfile_path).st_mode, 0100664)
9285
93 def testCWD(self):86 def test_CWD(self):
94 """Check automatic creation of directories 'cwd'ed in.87 """Check automatic creation of directories 'cwd'ed in.
9588
96 Also ensure they are created with proper permission (g+rwxs)89 Also ensure they are created with proper permission (g+rwxs)
@@ -107,7 +100,7 @@
107 self.assertTrue(os.path.exists(wanted_path))100 self.assertTrue(os.path.exists(wanted_path))
108 self.assertEqual(os.stat(wanted_path).st_mode, 042775)101 self.assertEqual(os.stat(wanted_path).st_mode, 042775)
109102
110 def testMKD(self):103 def test_MKD(self):
111 """Check recursive MKD (aka mkdir -p).104 """Check recursive MKD (aka mkdir -p).
112105
113 Also ensure they are created with proper permission (g+rwxs)106 Also ensure they are created with proper permission (g+rwxs)
@@ -128,7 +121,7 @@
128 self.assertTrue(os.path.exists(wanted_path))121 self.assertTrue(os.path.exists(wanted_path))
129 self.assertEqual(os.stat(wanted_path).st_mode, 042775)122 self.assertEqual(os.stat(wanted_path).st_mode, 042775)
130123
131 def testRMD(self):124 def test_RMD(self):
132 """Check recursive RMD (aka rmdir)"""125 """Check recursive RMD (aka rmdir)"""
133 conn = self.getFTPConnection()126 conn = self.getFTPConnection()
134 self.assertEqual(127 self.assertEqual(
@@ -143,7 +136,7 @@
143 wanted_path = self._uploadPath('foo')136 wanted_path = self._uploadPath('foo')
144 self.assertFalse(os.path.exists(wanted_path))137 self.assertFalse(os.path.exists(wanted_path))
145138
146 def testSTOR(self):139 def test_STOR(self):
147 """Check if the parent directories are created during file upload.140 """Check if the parent directories are created during file upload.
148141
149 The uploaded file permissions are also special (g+rwxs).142 The uploaded file permissions are also special (g+rwxs).
@@ -160,7 +153,7 @@
160 self.assertEqual(fs_content, "fake contents")153 self.assertEqual(fs_content, "fake contents")
161 self.assertEqual(os.stat(wanted_path).st_mode, 0102674)154 self.assertEqual(os.stat(wanted_path).st_mode, 0102674)
162155
163 def testUploadIsolation(self):156 def test_uploadIsolation(self):
164 """Check if poppy isolates the uploads properly.157 """Check if poppy isolates the uploads properly.
165158
166 Upload should be done atomically, i.e., poppy should isolate the159 Upload should be done atomically, i.e., poppy should isolate the
@@ -216,5 +209,6 @@
216 self.root_dir, upload_dirs[index], "test")).read()209 self.root_dir, upload_dirs[index], "test")).read()
217 self.assertEqual(content, expected_contents[index])210 self.assertEqual(content, expected_contents[index])
218211
212
219def test_suite():213def test_suite():
220 return unittest.TestLoader().loadTestsFromName(__name__)214 return unittest.TestLoader().loadTestsFromName(__name__)
221215
=== modified file 'lib/lp/services/osutils.py'
--- lib/lp/services/osutils.py 2009-12-21 02:50:08 +0000
+++ lib/lp/services/osutils.py 2010-03-18 19:22:29 +0000
@@ -8,67 +8,21 @@
8 'remove_tree',8 'remove_tree',
9 'kill_by_pidfile',9 'kill_by_pidfile',
10 'remove_if_exists',10 'remove_if_exists',
11 'two_stage_kill',
11 ]12 ]
1213
13import errno
14import os.path14import os.path
15from signal import SIGTERM, SIGKILL
16import shutil15import shutil
17import time16
17
18from canonical.launchpad.daemons.tachandler import (
19 kill_by_pidfile,
20 remove_if_exists,
21 two_stage_kill,
22 )
1823
1924
20def remove_tree(path):25def remove_tree(path):
21 """Remove the tree at 'path' from disk."""26 """Remove the tree at 'path' from disk."""
22 if os.path.exists(path):27 if os.path.exists(path):
23 shutil.rmtree(path)28 shutil.rmtree(path)
24
25
26def kill_by_pidfile(pidfile_path):
27 """Kill a process identified by the pid stored in a file.
28
29 The pid file is removed from disk.
30 """
31 if not os.path.exists(pidfile_path):
32 return
33 try:
34 # Get the pid.
35 pid = open(pidfile_path, 'r').read().split()[0]
36 try:
37 pid = int(pid)
38 except ValueError:
39 # pidfile contains rubbish
40 return
41
42 # Kill the process.
43 try:
44 os.kill(pid, SIGTERM)
45 except OSError, e:
46 if e.errno in (errno.ESRCH, errno.ECHILD):
47 # Process has already been killed.
48 return
49
50 # Poll until the process has ended.
51 for i in range(50):
52 try:
53 os.kill(pid, 0)
54 time.sleep(0.1)
55 except OSError, e:
56 break
57 else:
58 # The process is still around, so terminate it violently.
59 try:
60 os.kill(pid, SIGKILL)
61 except OSError:
62 # Already terminated
63 pass
64 finally:
65 remove_if_exists(pidfile_path)
66
67
68def remove_if_exists(path):
69 """Remove the given file if it exists."""
70 try:
71 os.remove(path)
72 except OSError, e:
73 if e.errno != errno.ENOENT:
74 raise
7529
=== modified file 'lib/lp/soyuz/doc/soyuz-upload.txt.disabled'
--- lib/lp/soyuz/doc/soyuz-upload.txt.disabled 2010-03-17 11:04:20 +0000
+++ lib/lp/soyuz/doc/soyuz-upload.txt.disabled 2010-03-18 19:22:29 +0000
@@ -186,7 +186,7 @@
186Right, that's all we need from the FTP server. We don't need it anymore,186Right, that's all we need from the FTP server. We don't need it anymore,
187so we'll just kill the process.187so we'll just kill the process.
188188
189 >>> status = poppy.killPoppy()189 >>> poppy.killPoppy()
190190
191Finally, we'll just create an entirely empty upload folder. We rely for191Finally, we'll just create an entirely empty upload folder. We rely for
192our tests on a poppy-like naming system, ie. that the upload folder192our tests on a poppy-like naming system, ie. that the upload folder
193193
=== modified file 'lib/lp/testing/__init__.py'
--- lib/lp/testing/__init__.py 2010-03-16 18:03:34 +0000
+++ lib/lp/testing/__init__.py 2010-03-18 19:22:29 +0000
@@ -215,7 +215,7 @@
215 def makeTemporaryDirectory(self):215 def makeTemporaryDirectory(self):
216 """Create a temporary directory, and return its path."""216 """Create a temporary directory, and return its path."""
217 tempdir = tempfile.mkdtemp()217 tempdir = tempfile.mkdtemp()
218 self.addCleanup(lambda: shutil.rmtree(tempdir))218 self.addCleanup(shutil.rmtree, tempdir)
219 return tempdir219 return tempdir
220220
221 def assertProvides(self, obj, interface):221 def assertProvides(self, obj, interface):
@@ -380,11 +380,10 @@
380380
381 def useTempDir(self):381 def useTempDir(self):
382 """Use a temporary directory for this test."""382 """Use a temporary directory for this test."""
383 tempdir = tempfile.mkdtemp()383 tempdir = self.makeTemporaryDirectory()
384 self.addCleanup(lambda: shutil.rmtree(tempdir))
385 cwd = os.getcwd()384 cwd = os.getcwd()
386 os.chdir(tempdir)385 os.chdir(tempdir)
387 self.addCleanup(lambda: os.chdir(cwd))386 self.addCleanup(os.chdir, cwd)
388387
389388
390class TestCaseWithFactory(TestCase):389class TestCaseWithFactory(TestCase):