Merge lp:~justin-fathomdb/nova/check-subprocess-exit-code into lp:~hudson-openstack/nova/trunk

Proposed by justinsb
Status: Merged
Approved by: Jay Pipes
Approved revision: 166
Merged at revision: 251
Proposed branch: lp:~justin-fathomdb/nova/check-subprocess-exit-code
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 543 lines (+140/-135)
12 files modified
bin/nova-import-canonical-imagestore (+3/-3)
bin/nova-manage (+2/-1)
nova/cloudpipe/bootscript.sh (+2/-2)
nova/objectstore/image.py (+12/-3)
nova/process.py (+85/-100)
nova/tests/network_unittest.py (+0/-1)
nova/tests/process_unittest.py (+1/-1)
nova/utils.py (+14/-8)
nova/virt/images.py (+1/-1)
nova/virt/libvirt_conn.py (+3/-3)
nova/volume/service.py (+13/-7)
tools/install_venv.py (+4/-5)
To merge this branch: bzr merge lp:~justin-fathomdb/nova/check-subprocess-exit-code
Reviewer Review Type Date Requested Status
termie (community) Approve
justinsb (community) Approve
Vish Ishaya (community) Approve
Review via email: mp+30707@code.launchpad.net

Commit message

Check exit codes when spawning processes by default
Also pass --fail to curl so that it sets exit code when download fails

Description of the change

Check exit codes when spawning processes by default
Also pass --fail to curl so that it sets exit code when download fails

To post a comment you must log in.
Revision history for this message
Vish Ishaya (vishvananda) wrote :

Any reason why you didn't make the change in process.py as well? A lot of the shell commands are called with process.simple_execute

review: Needs Information
Revision history for this message
Vish Ishaya (vishvananda) wrote :

In fact, I'm hoping we can get rid of runthis in favor of process.execute at least. After my refactors I think it is only used in a few places, perhaps utils.execute will disappear eventually as well

Revision history for this message
justinsb (justin-fathomdb) wrote :

Didn't make the changes in process.py because I didn't realize that processes were called from there as well... I'll make the similar change to process.py

Revision history for this message
justinsb (justin-fathomdb) wrote :

Now checking the exit code in the twisted code also.

Also incorporated passing the --fail flag to curl so that it sets the exit code. Seems to make sense here, in that we're now checking error codes, so we want curl to set them!

review: Needs Resubmitting
Revision history for this message
Vish Ishaya (vishvananda) wrote :

looks good. I requested a review from Termie, since he wrote the process.py code. Also, i noticed a few problems with nova-volume's use of simple_execute. I had to add an error_ok or two because some of the shell calls were returning output on stderr. We might need to run through creating a few volumes with my branch lp:~vishvananda/nova/fix-volumes and see if we need check_error_code=False on any of the commands. I only noticed this stuff when actually creating volumes (i.e. not using fakes).

Revision history for this message
justinsb (justin-fathomdb) wrote :

Yes - it definitely needs an Twisted expert's eye on it. I certainly haven't run through every code path, so odds are the error checking will surface more issues and break things, but I think that at this stage of the project that's probably the right thing to do - we at least want to know when things are failing! I found some problems with the virtualenv stuff, for example, which otherwise were going unnoticed, and probably resulting in mixed versions of python libs etc...

Revision history for this message
termie (termie) wrote :

* Please change all variable and parameter names to match Twisted's (camelCase)

* When you write a note about something (you have a comment in an exception below) please use

NOTE(justinsb): comment text

rather than

comment text (justinsb)

* Please make the lines fit in 80 characters.

* Your comment at the beginning of BackRelay has odd spacing.

* Technically the pip change (good catch) should be a different branch, it is fine now that it is already in there but please keep changes as small and specific as possible.

In general I don't think we should change the behavior of how "errortoo" works.

Specifically I don't think the majority of the changes in the twisted code are necessary, it looks like simply making a failure state in processEnded if checkExitCode is true and the exit code is non-zero is enough. I agree that we should also always store the stderr separately in addition to what twisted is doing so that we can give it back when the exit code is non-zero, but that should be an add-on to existing behavior, not a modification.

review: Needs Fixing
Revision history for this message
justinsb (justin-fathomdb) wrote :

Agree that the pip comment should be in a different branch, as probably should the --fail argument to curl; just trying to minimize the overhead involved in patch submission.

On the changes to the twisted code, there were intended to fix the problems with checking stderr & exit code, that were documented in the code itself. It also felt cleaner to have one class than two, particularly as some methods in the base class appeared to be broken, but were never called as the derived class overrides those method. I'm happy to reapply my fixes against a branch if you want to submit something more along your lines of thinking.

The other comments seem to be about formatting. Can you please point me to the formatting rules we're using? Ideally (from my point of view) we could just all agree on a vim configuration (or some other editor) and post that for Python noobs such as myself.

Not sure I agree on use of camel case because Twisted does it, given that the rest of the code seems to use underscores, and Twisted might well be ripped out anyway. But this is the sort of thing that will be covered by the project formatting rules, so we don't need to debate it :-)

If someone can point me to the formatting rules for vim that we're using, I'll auto-format the code and resubmit.

Revision history for this message
justinsb (justin-fathomdb) wrote :

Ping!

review: Needs Resubmitting
Revision history for this message
justinsb (justin-fathomdb) wrote :

Fixes for greater compliance with pep8/pylint style checks.

In places where nova's code style was incompatible with Twisted's code style (camelCase), I used camelCase only where we had to for interactions with Twisted (e.g. method overrides, specifying arguments), and this style is confined to the process.py module and mostly to the BackRelayWithInput class.

review: Needs Resubmitting
Revision history for this message
Vish Ishaya (vishvananda) wrote :

Cleanup here is good enough for me. Be nice if termie approved as well. There have been a few additions/changes to commands since this was written, so it may be that this will need to be updated before merging (specifically I'm thinking about volume/service.py)

review: Approve
Revision history for this message
termie (termie) wrote :

I still don't agree that these changes are necessary, can you point out a use case where the existing code is not functional? I suspect there is a way to solve your issues that don't involve modifying this code.

review: Disapprove
Revision history for this message
justinsb (justin-fathomdb) wrote :

Sure - the primary case I was trying to fix when messing with the Twisted code (which is what I think you're questioning), is the one in this comment:

# NOTE(termie): current behavior means if error_ok is True
# we won't throw an error even if the process
# exited with a non-0 status, so you can't be
# okay with stderr output and not with bad exit
# codes.

I cleaned up the logic so that we can still check the exit code, whether or not the process outputs on stderr.

Revision history for this message
Eric Day (eday) wrote :

Any more feedback from termie on the last comment? We should merge this if it's still possible, or reject/remove from the merge queue if not.

Revision history for this message
termie (termie) wrote :

weird, I got Eric's message but not the message from Justin. Sorry for the late reply, I expected to get an email about it.

Justin: That is a NOTE not a TODO, I don't see any use cases where this behavior is a problem, what I am asking you for is a use case that we have right now where we can't do what we want because of this behavior. I think the change you are proposing increases the complexity of the code significantly and would rather it not happen unless there is a good reason for it.

Revision history for this message
justinsb (justin-fathomdb) wrote :

I don't believe a use-case should be needed for merging in a patch that adds correct error checking, but anyway...

From nova/volume/service.py:
        # NOTE(vish): these commands sometimes sends output to stderr for warnings
        yield process.simple_execute("sudo vblade-persist auto all", error_ok=1)
        yield process.simple_execute("sudo vblade-persist start all", error_ok=1)

In other words, there was crummy stderr/exit code checking code (I think it was grabbed out of Twisted?) Though this was known, there was a multi-line note instead of a (shorter) fix for it. As inevitably happens, the bug eventually caused problems - in this case, for vish in the storage service, but the only way to work around it was to mark stderr as being OK, or to fix the problem and risk a conflict with this merge-in-progress. Which means we now probably have to fix nova/volume/service.py as well (to state that stderr is OK but a non-zero exit code is not).

When I see technical debt in code I'm working on, I clean it up.

I believe the new code is much cleaner, so I don't know what you can point to that makes you believe it to be more complex. The old code used dynamic class modification (errReceivedIsBad, errReceivedIsGood), two classes where only the derived class was actually used, had some probable syntax errors flagged by pylint (but the methods were overridden in the derived class so were never actually called), and did a questionable dynamic import of twisted.internet.reactor if the reactor parameter to getProcessOutput was None. I don't believe there are any 'tricks' at all in the new code - it should be surprise-free. I think I've also reduced lines of code and have much greater pylint compliance.

Finally, given that it seems unlikely that Twisted will survive for long in Nova, why are we nit-picking? Isn't proper error checking more important, so we can start getting nova working solidly and get more features working? I have a huge backlog of patches (not least a mostly working abstraction of the Redis data store and a SQL alternative implementation), but everything depends on this branch because - frankly - I find it really difficult to make progress when there's no error checking when spawning processes.

Revision history for this message
termie (termie) wrote :
Download full text (3.6 KiB)

On Wed, Aug 18, 2010 at 12:24 AM, justinsb <email address hidden> wrote:

> I don't believe a use-case should be needed for merging in a patch that
> adds correct error checking, but anyway...
>

Correct is what does what we need, without a use case it is just an opinion.

>
> >From nova/volume/service.py:
> # NOTE(vish): these commands sometimes sends output to stderr for
> warnings
> yield process.simple_execute("sudo vblade-persist auto all",
> error_ok=1)
> yield process.simple_execute("sudo vblade-persist start all",
> error_ok=1)

In other words, there was crummy stderr/exit code checking code (I think it
> was grabbed out of Twisted?)

Thanks this was what I was asking for. I'd also add that it is bad form to
call code 'crummy' but I sense that you are frustrated at the moment, so I
hope that if whoever wrote it sees this they won't take personal offense.

> Though this was known, there was a multi-line note instead of a (shorter)
> fix for it. As inevitably happens, the bug eventually caused problems - in
> this case, for vish in the storage service, but the only way to work around
> it was to mark stderr as being OK, or to fix the problem and risk a conflict
> with this merge-in-progress. Which means we now probably have to fix
> nova/volume/service.py as well (to state that stderr is OK but a non-zero
> exit code is not).
>
> When I see technical debt in code I'm working on, I clean it up.
>
> I believe the new code is much cleaner, so I don't know what you can point
> to that makes you believe it to be more complex. The old code used dynamic
> class modification (errReceivedIsBad, errReceivedIsGood), two classes where
> only the derived class was actually used, had some probable syntax errors
> flagged by pylint (but the methods were overridden in the derived class so
> were never actually called), and did a questionable dynamic import of
> twisted.internet.reactor if the reactor parameter to getProcessOutput was
> None. I don't believe there are any 'tricks' at all in the new code - it
> should be surprise-free. I think I've also reduced lines of code and have
> much greater pylint compliance.
>
> Finally, given that it seems unlikely that Twisted will survive for long in
> Nova, why are we nit-picking? Isn't proper error checking more important,
> so we can start getting nova working solidly and get more features working?
> I have a huge backlog of patches (not least a mostly working abstraction of
> the Redis data store and a SQL alternative implementation), but everything
> depends on this branch because - frankly - I find it really difficult to
> make progress when there's no error checking when spawning processes.
>

No need to get upset with me Justin. I apologize that I was slow on the
review, I haven't been receiving emails from any of your updates until this
one and Launchpad's workflow has yet to sink in with me.

In summary what I've understood from your comment:

1. We have a specific use case where an external tool is outputting on
stderr even though it has succeeded and therefore there is need to change
the behavior of this code.

2. You feel that the change is a general improvement in...

Read more...

Revision history for this message
termie (termie) :
review: Approve
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Attempt to merge lp:~justin-fathomdb/nova/check-subprocess-exit-code into lp:nova failed due to merge conflicts:

text conflict in bin/nova-manage
text conflict in nova/utils.py
text conflict in nova/volume/service.py
text conflict in tools/install_venv.py

Revision history for this message
justinsb (justin-fathomdb) wrote :

Merged with trunk; changed error_ok to terminate_on_stderr=False because this is what I believe is wanted (this affects a shell to vgcreate/lvcreate, so this patch would probably have helped with Bug 620027).

Also fixed a unit test that was calling release_ip it didn't get from issue_ip. Though the unit test was wrong, we weren't detecting the failure. As Ewan noticed in the most recent mailing list thread, the network unit tests are unusually tolerant of being FUBAR, this is probably why - they weren't checking exit codes!

Revision history for this message
justinsb (justin-fathomdb) :
review: Approve
Revision history for this message
termie (termie) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'bin/nova-import-canonical-imagestore'
--- bin/nova-import-canonical-imagestore 2010-08-08 02:51:17 +0000
+++ bin/nova-import-canonical-imagestore 2010-08-18 21:36:47 +0000
@@ -56,21 +56,21 @@
56 for f in img['files']:56 for f in img['files']:
57 if f['kind'] == 'kernel':57 if f['kind'] == 'kernel':
58 dest = os.path.join(tempdir, 'kernel')58 dest = os.path.join(tempdir, 'kernel')
59 subprocess.call(['curl', f['url'], '-o', dest])59 subprocess.call(['curl', '--fail', f['url'], '-o', dest])
60 kernel_id = image.Image.add(dest,60 kernel_id = image.Image.add(dest,
61 description='kernel/' + img['title'], kernel=True)61 description='kernel/' + img['title'], kernel=True)
6262
63 for f in img['files']:63 for f in img['files']:
64 if f['kind'] == 'ramdisk':64 if f['kind'] == 'ramdisk':
65 dest = os.path.join(tempdir, 'ramdisk')65 dest = os.path.join(tempdir, 'ramdisk')
66 subprocess.call(['curl', f['url'], '-o', dest])66 subprocess.call(['curl', '--fail', f['url'], '-o', dest])
67 ramdisk_id = image.Image.add(dest,67 ramdisk_id = image.Image.add(dest,
68 description='ramdisk/' + img['title'], ramdisk=True)68 description='ramdisk/' + img['title'], ramdisk=True)
6969
70 for f in img['files']:70 for f in img['files']:
71 if f['kind'] == 'image':71 if f['kind'] == 'image':
72 dest = os.path.join(tempdir, 'image')72 dest = os.path.join(tempdir, 'image')
73 subprocess.call(['curl', f['url'], '-o', dest])73 subprocess.call(['curl', '--fail', f['url'], '-o', dest])
74 ramdisk_id = image.Image.add(dest,74 ramdisk_id = image.Image.add(dest,
75 description=img['title'], kernel=kernel_id, ramdisk=ramdisk_id)75 description=img['title'], kernel=kernel_id, ramdisk=ramdisk_id)
7676
7777
=== modified file 'bin/nova-manage'
--- bin/nova-manage 2010-08-13 21:09:30 +0000
+++ bin/nova-manage 2010-08-18 21:36:47 +0000
@@ -56,7 +56,8 @@
56 vpn = self._vpn_for(project.id)56 vpn = self._vpn_for(project.id)
57 if vpn:57 if vpn:
58 command = "ping -c1 -w1 %s > /dev/null; echo $?"58 command = "ping -c1 -w1 %s > /dev/null; echo $?"
59 out, _err = utils.execute(command % vpn['private_dns_name'])59 out, _err = utils.execute( command % vpn['private_dns_name'],
60 check_exit_code=False)
60 if out.strip() == '0':61 if out.strip() == '0':
61 net = 'up'62 net = 'up'
62 else:63 else:
6364
=== modified file 'nova/cloudpipe/bootscript.sh'
--- nova/cloudpipe/bootscript.sh 2010-07-20 22:05:02 +0000
+++ nova/cloudpipe/bootscript.sh 2010-08-18 21:36:47 +0000
@@ -44,8 +44,8 @@
4444
45# SIGN the csr and save as server.crt45# SIGN the csr and save as server.crt
46# CURL fetch to the supervisor, POSTing the CSR text, saving the result as the CRT file46# CURL fetch to the supervisor, POSTing the CSR text, saving the result as the CRT file
47curl $SUPERVISOR -d "cert=$CSRTEXT" > /etc/openvpn/server.crt47curl --fail $SUPERVISOR -d "cert=$CSRTEXT" > /etc/openvpn/server.crt
48curl $SUPERVISOR/getca/ > /etc/openvpn/ca.crt48curl --fail $SUPERVISOR/getca/ > /etc/openvpn/ca.crt
4949
50# Customize the server.conf.template50# Customize the server.conf.template
51cd /etc/openvpn51cd /etc/openvpn
5252
=== modified file 'nova/objectstore/image.py'
--- nova/objectstore/image.py 2010-08-16 12:16:21 +0000
+++ nova/objectstore/image.py 2010-08-18 21:36:47 +0000
@@ -232,13 +232,22 @@
232232
233 @staticmethod233 @staticmethod
234 def decrypt_image(encrypted_filename, encrypted_key, encrypted_iv, cloud_private_key, decrypted_filename):234 def decrypt_image(encrypted_filename, encrypted_key, encrypted_iv, cloud_private_key, decrypted_filename):
235 key, err = utils.execute('openssl rsautl -decrypt -inkey %s' % cloud_private_key, encrypted_key)235 key, err = utils.execute(
236 'openssl rsautl -decrypt -inkey %s' % cloud_private_key,
237 process_input=encrypted_key,
238 check_exit_code=False)
236 if err:239 if err:
237 raise exception.Error("Failed to decrypt private key: %s" % err)240 raise exception.Error("Failed to decrypt private key: %s" % err)
238 iv, err = utils.execute('openssl rsautl -decrypt -inkey %s' % cloud_private_key, encrypted_iv)241 iv, err = utils.execute(
242 'openssl rsautl -decrypt -inkey %s' % cloud_private_key,
243 process_input=encrypted_iv,
244 check_exit_code=False)
239 if err:245 if err:
240 raise exception.Error("Failed to decrypt initialization vector: %s" % err)246 raise exception.Error("Failed to decrypt initialization vector: %s" % err)
241 out, err = utils.execute('openssl enc -d -aes-128-cbc -in %s -K %s -iv %s -out %s' % (encrypted_filename, key, iv, decrypted_filename))247 _out, err = utils.execute(
248 'openssl enc -d -aes-128-cbc -in %s -K %s -iv %s -out %s'
249 % (encrypted_filename, key, iv, decrypted_filename),
250 check_exit_code=False)
242 if err:251 if err:
243 raise exception.Error("Failed to decrypt image file %s : %s" % (encrypted_filename, err))252 raise exception.Error("Failed to decrypt image file %s : %s" % (encrypted_filename, err))
244253
245254
=== modified file 'nova/process.py'
--- nova/process.py 2010-08-16 12:16:21 +0000
+++ nova/process.py 2010-08-18 21:36:47 +0000
@@ -2,6 +2,7 @@
22
3# Copyright 2010 United States Government as represented by the3# Copyright 2010 United States Government as represented by the
4# Administrator of the National Aeronautics and Space Administration.4# Administrator of the National Aeronautics and Space Administration.
5# Copyright 2010 FathomDB Inc.
5# All Rights Reserved.6# All Rights Reserved.
6#7#
7# Licensed under the Apache License, Version 2.0 (the "License"); you may8# Licensed under the Apache License, Version 2.0 (the "License"); you may
@@ -20,17 +21,12 @@
20Process pool, still buggy right now.21Process pool, still buggy right now.
21"""22"""
2223
23import logging
24import multiprocessing
25import StringIO24import StringIO
2625
27from twisted.internet import defer26from twisted.internet import defer
28from twisted.internet import error27from twisted.internet import error
29from twisted.internet import process
30from twisted.internet import protocol28from twisted.internet import protocol
31from twisted.internet import reactor29from twisted.internet import reactor
32from twisted.internet import threads
33from twisted.python import failure
3430
35from nova import flags31from nova import flags
3632
@@ -55,111 +51,100 @@
55 IOError.__init__(self, "got stdout: %r\nstderr: %r" % (stdout, stderr))51 IOError.__init__(self, "got stdout: %r\nstderr: %r" % (stdout, stderr))
5652
5753
58# NOTE(termie): this too54# This is based on _BackRelay from twister.internal.utils, but modified to
59class _BackRelay(protocol.ProcessProtocol):55# capture both stdout and stderr, without odd stderr handling, and also to
56# handle stdin
57class BackRelayWithInput(protocol.ProcessProtocol):
60 """58 """
61 Trivial protocol for communicating with a process and turning its output59 Trivial protocol for communicating with a process and turning its output
62 into the result of a L{Deferred}.60 into the result of a L{Deferred}.
6361
64 @ivar deferred: A L{Deferred} which will be called back with all of stdout62 @ivar deferred: A L{Deferred} which will be called back with all of stdout
65 and, if C{errortoo} is true, all of stderr as well (mixed together in63 and all of stderr as well (as a tuple). C{terminate_on_stderr} is true
66 one string). If C{errortoo} is false and any bytes are received over64 and any bytes are received over stderr, this will fire with an
67 stderr, this will fire with an L{_UnexpectedErrorOutput} instance and65 L{_UnexpectedErrorOutput} instance and the attribute will be set to
68 the attribute will be set to C{None}.66 C{None}.
6967
70 @ivar onProcessEnded: If C{errortoo} is false and bytes are received over68 @ivar onProcessEnded: If C{terminate_on_stderr} is false and bytes are
71 stderr, this attribute will refer to a L{Deferred} which will be called69 received over stderr, this attribute will refer to a L{Deferred} which
72 back when the process ends. This C{Deferred} is also associated with70 will be called back when the process ends. This C{Deferred} is also
73 the L{_UnexpectedErrorOutput} which C{deferred} fires with earlier in71 associated with the L{_UnexpectedErrorOutput} which C{deferred} fires
74 this case so that users can determine when the process has actually72 with earlier in this case so that users can determine when the process
75 ended, in addition to knowing when bytes have been received via stderr.73 has actually ended, in addition to knowing when bytes have been received
74 via stderr.
76 """75 """
7776
78 def __init__(self, deferred, errortoo=0):77 def __init__(self, deferred, started_deferred=None,
78 terminate_on_stderr=False, check_exit_code=True,
79 process_input=None):
79 self.deferred = deferred80 self.deferred = deferred
80 self.s = StringIO.StringIO()81 self.stdout = StringIO.StringIO()
81 if errortoo:82 self.stderr = StringIO.StringIO()
82 self.errReceived = self.errReceivedIsGood83 self.started_deferred = started_deferred
83 else:84 self.terminate_on_stderr = terminate_on_stderr
84 self.errReceived = self.errReceivedIsBad85 self.check_exit_code = check_exit_code
8586 self.process_input = process_input
86 def errReceivedIsBad(self, text):87 self.on_process_ended = None
87 if self.deferred is not None:88
88 self.onProcessEnded = defer.Deferred()89 def errReceived(self, text):
89 err = UnexpectedErrorOutput(text, self.onProcessEnded)90 self.stderr.write(text)
90 self.deferred.errback(failure.Failure(err))91 if self.terminate_on_stderr and (self.deferred is not None):
92 self.on_process_ended = defer.Deferred()
93 self.deferred.errback(UnexpectedErrorOutput(
94 stdout=self.stdout.getvalue(),
95 stderr=self.stderr.getvalue()))
91 self.deferred = None96 self.deferred = None
92 self.transport.loseConnection()97 self.transport.loseConnection()
9398
94 def errReceivedIsGood(self, text):
95 self.s.write(text)
96
97 def outReceived(self, text):99 def outReceived(self, text):
98 self.s.write(text)100 self.stdout.write(text)
99101
100 def processEnded(self, reason):102 def processEnded(self, reason):
101 if self.deferred is not None:103 if self.deferred is not None:
102 self.deferred.callback(self.s.getvalue())104 stdout, stderr = self.stdout.getvalue(), self.stderr.getvalue()
103 elif self.onProcessEnded is not None:
104 self.onProcessEnded.errback(reason)
105
106
107class BackRelayWithInput(_BackRelay):
108 def __init__(self, deferred, startedDeferred=None, error_ok=0,
109 input=None):
110 # Twisted doesn't use new-style classes in most places :(
111 _BackRelay.__init__(self, deferred, errortoo=error_ok)
112 self.error_ok = error_ok
113 self.input = input
114 self.stderr = StringIO.StringIO()
115 self.startedDeferred = startedDeferred
116
117 def errReceivedIsBad(self, text):
118 self.stderr.write(text)
119 self.transport.loseConnection()
120
121 def errReceivedIsGood(self, text):
122 self.stderr.write(text)
123
124 def connectionMade(self):
125 if self.startedDeferred:
126 self.startedDeferred.callback(self)
127 if self.input:
128 self.transport.write(self.input)
129 self.transport.closeStdin()
130
131 def processEnded(self, reason):
132 if self.deferred is not None:
133 stdout, stderr = self.s.getvalue(), self.stderr.getvalue()
134 try:105 try:
135 # NOTE(termie): current behavior means if error_ok is True106 if self.check_exit_code:
136 # we won't throw an error even if the process
137 # exited with a non-0 status, so you can't be
138 # okay with stderr output and not with bad exit
139 # codes.
140 if not self.error_ok:
141 reason.trap(error.ProcessDone)107 reason.trap(error.ProcessDone)
142 self.deferred.callback((stdout, stderr))108 self.deferred.callback((stdout, stderr))
143 except:109 except:
110 # NOTE(justinsb): This logic is a little suspicious to me...
111 # If the callback throws an exception, then errback will be
112 # called also. However, this is what the unit tests test for...
144 self.deferred.errback(UnexpectedErrorOutput(stdout, stderr))113 self.deferred.errback(UnexpectedErrorOutput(stdout, stderr))
145114 elif self.on_process_ended is not None:
146115 self.on_process_ended.errback(reason)
147def getProcessOutput(executable, args=None, env=None, path=None, reactor=None,116
148 error_ok=0, input=None, startedDeferred=None):117
149 if reactor is None:118 def connectionMade(self):
150 from twisted.internet import reactor119 if self.started_deferred:
120 self.started_deferred.callback(self)
121 if self.process_input:
122 self.transport.write(self.process_input)
123 self.transport.closeStdin()
124
125def get_process_output(executable, args=None, env=None, path=None,
126 process_reactor=None, check_exit_code=True,
127 process_input=None, started_deferred=None,
128 terminate_on_stderr=False):
129 if process_reactor is None:
130 process_reactor = reactor
151 args = args and args or ()131 args = args and args or ()
152 env = env and env and {}132 env = env and env and {}
153 d = defer.Deferred()133 deferred = defer.Deferred()
154 p = BackRelayWithInput(134 process_handler = BackRelayWithInput(
155 d, startedDeferred=startedDeferred, error_ok=error_ok, input=input)135 deferred,
136 started_deferred=started_deferred,
137 check_exit_code=check_exit_code,
138 process_input=process_input,
139 terminate_on_stderr=terminate_on_stderr)
156 # NOTE(vish): commands come in as unicode, but self.executes needs140 # NOTE(vish): commands come in as unicode, but self.executes needs
157 # strings or process.spawn raises a deprecation warning141 # strings or process.spawn raises a deprecation warning
158 executable = str(executable)142 executable = str(executable)
159 if not args is None:143 if not args is None:
160 args = [str(x) for x in args]144 args = [str(x) for x in args]
161 reactor.spawnProcess(p, executable, (executable,)+tuple(args), env, path)145 process_reactor.spawnProcess( process_handler, executable,
162 return d146 (executable,)+tuple(args), env, path)
147 return deferred
163148
164149
165class ProcessPool(object):150class ProcessPool(object):
@@ -185,26 +170,26 @@
185 return self.execute(executable, args, **kw)170 return self.execute(executable, args, **kw)
186171
187 def execute(self, *args, **kw):172 def execute(self, *args, **kw):
188 d = self._pool.acquire()173 deferred = self._pool.acquire()
189174
190 def _associateProcess(proto):175 def _associate_process(proto):
191 d.process = proto.transport176 deferred.process = proto.transport
192 return proto.transport177 return proto.transport
193178
194 started = defer.Deferred()179 started = defer.Deferred()
195 started.addCallback(_associateProcess)180 started.addCallback(_associate_process)
196 kw.setdefault('startedDeferred', started)181 kw.setdefault('started_deferred', started)
197182
198 d.process = None183 deferred.process = None
199 d.started = started184 deferred.started = started
200185
201 d.addCallback(lambda _: getProcessOutput(*args, **kw))186 deferred.addCallback(lambda _: get_process_output(*args, **kw))
202 d.addBoth(self._release)187 deferred.addBoth(self._release)
203 return d188 return deferred
204189
205 def _release(self, rv=None):190 def _release(self, retval=None):
206 self._pool.release()191 self._pool.release()
207 return rv192 return retval
208193
209194
210class SharedPool(object):195class SharedPool(object):
211196
=== modified file 'nova/tests/network_unittest.py'
--- nova/tests/network_unittest.py 2010-08-18 15:44:24 +0000
+++ nova/tests/network_unittest.py 2010-08-18 21:36:47 +0000
@@ -166,7 +166,6 @@
166 release_ip(mac3, address3, hostname, net.bridge_name)166 release_ip(mac3, address3, hostname, net.bridge_name)
167 net = model.get_project_network(self.projects[0].id, "default")167 net = model.get_project_network(self.projects[0].id, "default")
168 self.service.deallocate_fixed_ip(firstaddress)168 self.service.deallocate_fixed_ip(firstaddress)
169 release_ip(mac, firstaddress, hostname, net.bridge_name)
170169
171 def test_vpn_ip_and_port_looks_valid(self):170 def test_vpn_ip_and_port_looks_valid(self):
172 """Ensure the vpn ip and port are reasonable"""171 """Ensure the vpn ip and port are reasonable"""
173172
=== modified file 'nova/tests/process_unittest.py'
--- nova/tests/process_unittest.py 2010-07-29 08:08:31 +0000
+++ nova/tests/process_unittest.py 2010-08-18 21:36:47 +0000
@@ -48,7 +48,7 @@
4848
49 def test_execute_stderr(self):49 def test_execute_stderr(self):
50 pool = process.ProcessPool(2)50 pool = process.ProcessPool(2)
51 d = pool.simple_execute('cat BAD_FILE', error_ok=1)51 d = pool.simple_execute('cat BAD_FILE', check_exit_code=False)
52 def _check(rv):52 def _check(rv):
53 self.assertEqual(rv[0], '')53 self.assertEqual(rv[0], '')
54 self.assert_('No such file' in rv[1])54 self.assert_('No such file' in rv[1])
5555
=== modified file 'nova/utils.py'
--- nova/utils.py 2010-08-16 12:16:21 +0000
+++ nova/utils.py 2010-08-18 21:36:47 +0000
@@ -56,23 +56,25 @@
56# c.perform()56# c.perform()
57# c.close()57# c.close()
58# fp.close()58# fp.close()
59 execute("curl %s -o %s" % (url, target))59 execute("curl --fail %s -o %s" % (url, target))
6060
6161def execute(cmd, process_input=None, addl_env=None, check_exit_code=True):
62def execute(cmd, input=None, addl_env=None):
63 env = os.environ.copy()62 env = os.environ.copy()
64 if addl_env:63 if addl_env:
65 env.update(addl_env)64 env.update(addl_env)
66 obj = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE,65 obj = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE,
67 stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)66 stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
68 result = None67 result = None
69 if input != None:68 if process_input != None:
70 result = obj.communicate(input)69 result = obj.communicate(process_input)
71 else:70 else:
72 result = obj.communicate()71 result = obj.communicate()
73 obj.stdin.close()72 obj.stdin.close()
74 if obj.returncode:73 if obj.returncode:
75 logging.debug("Result was %s" % (obj.returncode))74 logging.debug("Result was %s" % (obj.returncode))
75 if check_exit_code and obj.returncode <> 0:
76 raise Exception( "Unexpected exit code: %s. result=%s"
77 % (obj.returncode, result))
76 return result78 return result
7779
7880
@@ -98,9 +100,13 @@
98 return arg100 return arg
99101
100102
101def runthis(prompt, cmd):103def runthis(prompt, cmd, check_exit_code = True):
102 logging.debug("Running %s" % (cmd))104 logging.debug("Running %s" % (cmd))
103 logging.debug(prompt % (subprocess.call(cmd.split(" "))))105 exit_code = subprocess.call(cmd.split(" "))
106 logging.debug(prompt % (exit_code))
107 if check_exit_code and exit_code <> 0:
108 raise Exception( "Unexpected exit code: %s from cmd: %s"
109 % (exit_code, cmd))
104110
105111
106def generate_uid(topic, size=8):112def generate_uid(topic, size=8):
107113
=== modified file 'nova/virt/images.py'
--- nova/virt/images.py 2010-08-17 11:53:30 +0000
+++ nova/virt/images.py 2010-08-18 21:36:47 +0000
@@ -60,7 +60,7 @@
60 url_path)60 url_path)
61 headers['Authorization'] = 'AWS %s:%s' % (access, signature)61 headers['Authorization'] = 'AWS %s:%s' % (access, signature)
6262
63 cmd = ['/usr/bin/curl', '--silent', url]63 cmd = ['/usr/bin/curl', '--fail', '--silent', url]
64 for (k,v) in headers.iteritems():64 for (k,v) in headers.iteritems():
65 cmd += ['-H', '%s: %s' % (k,v)]65 cmd += ['-H', '%s: %s' % (k,v)]
6666
6767
=== modified file 'nova/virt/libvirt_conn.py'
--- nova/virt/libvirt_conn.py 2010-08-17 11:53:30 +0000
+++ nova/virt/libvirt_conn.py 2010-08-18 21:36:47 +0000
@@ -227,10 +227,10 @@
227 if not os.path.exists(basepath('ramdisk')):227 if not os.path.exists(basepath('ramdisk')):
228 yield images.fetch(data['ramdisk_id'], basepath('ramdisk'), user, project)228 yield images.fetch(data['ramdisk_id'], basepath('ramdisk'), user, project)
229229
230 execute = lambda cmd, input=None: \230 execute = lambda cmd, process_input=None: \
231 process.simple_execute(cmd=cmd,231 process.simple_execute(cmd=cmd,
232 input=input,232 process_input=process_input,
233 error_ok=1)233 check_exit_code=True)
234234
235 key = data['key_data']235 key = data['key_data']
236 net = None236 net = None
237237
=== modified file 'nova/volume/service.py'
--- nova/volume/service.py 2010-08-16 12:16:21 +0000
+++ nova/volume/service.py 2010-08-18 21:36:47 +0000
@@ -131,8 +131,10 @@
131 if FLAGS.fake_storage:131 if FLAGS.fake_storage:
132 return132 return
133 # NOTE(vish): these commands sometimes sends output to stderr for warnings133 # NOTE(vish): these commands sometimes sends output to stderr for warnings
134 yield process.simple_execute("sudo vblade-persist auto all", error_ok=1)134 yield process.simple_execute( "sudo vblade-persist auto all",
135 yield process.simple_execute("sudo vblade-persist start all", error_ok=1)135 terminate_on_stderr=False)
136 yield process.simple_execute( "sudo vblade-persist start all",
137 terminate_on_stderr=False)
136138
137 @defer.inlineCallbacks139 @defer.inlineCallbacks
138 def _init_volume_group(self):140 def _init_volume_group(self):
@@ -247,13 +249,14 @@
247 "sudo lvcreate -L %s -n %s %s" % (sizestr,249 "sudo lvcreate -L %s -n %s %s" % (sizestr,
248 self['volume_id'],250 self['volume_id'],
249 FLAGS.volume_group),251 FLAGS.volume_group),
250 error_ok=1)252 terminate_on_stderr=False)
251253
252 @defer.inlineCallbacks254 @defer.inlineCallbacks
253 def _delete_lv(self):255 def _delete_lv(self):
254 yield process.simple_execute(256 yield process.simple_execute(
255 "sudo lvremove -f %s/%s" % (FLAGS.volume_group,257 "sudo lvremove -f %s/%s" % (FLAGS.volume_group,
256 self['volume_id']), error_ok=1)258 self['volume_id']),
259 terminate_on_stderr=False)
257260
258 @property261 @property
259 def __devices_key(self):262 def __devices_key(self):
@@ -281,7 +284,8 @@
281 self['blade_id'],284 self['blade_id'],
282 FLAGS.aoe_eth_dev,285 FLAGS.aoe_eth_dev,
283 FLAGS.volume_group,286 FLAGS.volume_group,
284 self['volume_id']), error_ok=1)287 self['volume_id']),
288 terminate_on_stderr=False)
285289
286 @defer.inlineCallbacks290 @defer.inlineCallbacks
287 def _remove_export(self):291 def _remove_export(self):
@@ -294,10 +298,12 @@
294 def _exec_remove_export(self):298 def _exec_remove_export(self):
295 yield process.simple_execute(299 yield process.simple_execute(
296 "sudo vblade-persist stop %s %s" % (self['shelf_id'],300 "sudo vblade-persist stop %s %s" % (self['shelf_id'],
297 self['blade_id']), error_ok=1)301 self['blade_id']),
302 terminate_on_stderr=False)
298 yield process.simple_execute(303 yield process.simple_execute(
299 "sudo vblade-persist destroy %s %s" % (self['shelf_id'],304 "sudo vblade-persist destroy %s %s" % (self['shelf_id'],
300 self['blade_id']), error_ok=1)305 self['blade_id']),
306 terminate_on_stderr=False)
301307
302308
303class FakeVolume(Volume):309class FakeVolume(Volume):
304310
=== modified file 'tools/install_venv.py'
--- tools/install_venv.py 2010-08-17 15:03:15 +0000
+++ tools/install_venv.py 2010-08-18 21:36:47 +0000
@@ -37,7 +37,7 @@
37 sys.exit(1)37 sys.exit(1)
3838
3939
40def run_command(cmd, redirect_output=True, error_ok=False):40def run_command(cmd, redirect_output=True, check_exit_code=True):
41 """41 """
42 Runs a command in an out-of-process shell, returning the42 Runs a command in an out-of-process shell, returning the
43 output of that command. Working directory is ROOT.43 output of that command. Working directory is ROOT.
@@ -49,19 +49,18 @@
4949
50 proc = subprocess.Popen(cmd, cwd=ROOT, stdout=stdout)50 proc = subprocess.Popen(cmd, cwd=ROOT, stdout=stdout)
51 output = proc.communicate()[0]51 output = proc.communicate()[0]
52 if not error_ok and proc.returncode != 0:52 if check_exit_code and proc.returncode != 0:
53 die('Command "%s" failed.\n%s', ' '.join(cmd), output)53 die('Command "%s" failed.\n%s', ' '.join(cmd), output)
54 return output54 return output
5555
5656
57HAS_EASY_INSTALL = bool(run_command(['which', 'easy_install']).strip())57HAS_EASY_INSTALL = bool(run_command(['which', 'easy_install'], check_exit_code=False).strip())
58HAS_VIRTUALENV = bool(run_command(['which', 'virtualenv']).strip())58HAS_VIRTUALENV = bool(run_command(['which', 'virtualenv'], check_exit_code=False).strip())
5959
6060
61def check_dependencies():61def check_dependencies():
62 """Make sure virtualenv is in the path."""62 """Make sure virtualenv is in the path."""
6363
64 print 'Checking for virtualenv...',
65 if not HAS_VIRTUALENV:64 if not HAS_VIRTUALENV:
66 print 'not found.'65 print 'not found.'
67 # Try installing it via easy_install...66 # Try installing it via easy_install...