Merge lp:~lifeless/bzr/fix-terminal-spew into lp:bzr

Proposed by Robert Collins
Status: Merged
Approved by: Robert Collins
Approved revision: no longer in the source branch.
Merged at revision: 5310
Proposed branch: lp:~lifeless/bzr/fix-terminal-spew
Merge into: lp:bzr
Diff against target: 414 lines (+204/-59)
7 files modified
NEWS (+17/-2)
bzr (+8/-5)
bzrlib/__init__.py (+120/-43)
bzrlib/smart/medium.py (+5/-5)
bzrlib/trace.py (+3/-3)
bzrlib/ui/__init__.py (+19/-1)
doc/developers/overview.txt (+32/-0)
To merge this branch: bzr merge lp:~lifeless/bzr/fix-terminal-spew
Reviewer Review Type Date Requested Status
Andrew Bennetts Needs Fixing
Review via email: mp+28024@code.launchpad.net

Commit message

Aaron's output fix for bzr from the UDS sprint, tweaked to use a context manager for the whole library. (Aaron Bentley)

Description of the change

Aaron's output fix for bzr from the UDS sprint, tweaked to use a context
manager for the whole library.

To post a comment you must log in.
Revision history for this message
Andrew Bennetts (spiv) wrote :

It is a little bit odd to have an __exit__ that ignores the exception info passed to it, in the same way that "except: pass" tends to look a bit odd. As you say on IRC, "until we'd behave differently in an __exit__ call" then there's nothing much to do here. Certainly for the 'bzr' front-end we'd expect exceptions to be caught at this point... although if someone did use this in a 'with' statement in 2.5 it would stop e.g. KeyboardInterrupt propagating. So if the exc_* args are not None, I think BzrLibraryState.__exit__ should "return False" to tell 'with' statements to propagate the exception. And pass these arguments to the UI factory's __exit__, obviously... hmm, maybe should return it's true/false value?

I don't think that needs to be fixed before this can land — it's not a regression, and an overall improvement. But there is a small trap lurking here for advanced API users, so if it's not fixed then an XXX would be warranted.

The only other issue is documentation: bzrlib.initialize's docstring needs to describe its return value, and BzrLibraryState's docstring probably ought to mention that it implements the context manager introduced in Python 2.5. Either that, or you should add docstrings to __enter__ or __exit__, but I know which one is easier ;) It's a bit pedantic, but this is now a front door to the bzrlib API so it should be documented thoroughly and clearly.

Finally, I don't think you've addressed Martin[gz]'s concern about moving the UI cleanup earlier: in particular, if the call to sys.exitfunc() (i.e. to run the atexit hooks) causes something to use the UI, what happens? I think some of the -D flags that output end-of-run summaries (e.g. of HPSS call counts, or transport activity), might cause trouble here. Or maybe not, but I'd some reassurance about it :)

review: Needs Fixing
Revision history for this message
Robert Collins (lifeless) wrote :

Re return codes - I'll make it return False everywhere; I would have thought that None == False ?

I'll document it too.

I was going to say that with this change we don't use atexit, but thats a lie.

We use it in medium.py, and I think thats actually a little unsafe. Concretely its _DebugCounter, and it registers atexit so that when gc hasn't happened, when atexit triggers, it can report.

That medium in particular will go boom if its atexit code triggers, because it exists to call into trace.note. (And thats not safe either, because trace.note can change when tests change, so this particular code is very unsafe for unittests - debug counts from one test can report into a different test or even the global ui).

I propose instead that the library state have a cleanup stack (\o/), that we make the library state also get stored to a global by initialise (stopgaps R us), that the test suite set an appropriate library state as part of its log file shenanigans and finally that DebugCounter register with the cleanup stack of the librarystate that it finds.

Revision history for this message
Robert Collins (lifeless) wrote :

Further to this:
 - I'm going to cleanup the test log support stuff; it all fits in neatly.

Exception propogation return values - argh. Seems like cleanups :). If there are N contexts, which ones should we honour the return value of; should we OR/NOT/XOR/AND them?

Revision history for this message
Robert Collins (lifeless) wrote :

sent to pqm by email

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'NEWS'
2--- NEWS 2010-06-20 22:55:07 +0000
3+++ NEWS 2010-06-21 04:18:27 +0000
4@@ -14,11 +14,25 @@
5 Compatibility Breaks
6 ********************
7
8+* bzrlib library users now need to call ``__enter__`` and ``__exit__`` on
9+ the result of ``bzrlib.initialize``. This change was made when fixing
10+ the bad habit recent bzr versions have had of leaving progress bars
11+ behind on the screen. That required calling another function before
12+ exiting the program, and it made sense to provide a full context
13+ manager at the same time. (Robert Collins)
14+
15+* The ``bzr`` front end now requires a ``bzrlib.ui.ui_factory`` which is a
16+ context manager in the Python 2.5 and above sense. The bzrlib base class
17+ is such a manager, but third party UI factories which do not derive from
18+ ``bzrlib.ui.UIFactory`` will be incompatible with the command line front
19+ end.
20+
21 * URLs like ``foo:bar/baz`` are now always parsed as a URL with scheme "foo"
22 and path "bar/baz", even if bzr does not recognize "foo" as a known URL
23 scheme. Previously these URLs would be treated as local paths.
24 (Gordon Tyler)
25
26+
27 New Features
28 ************
29
30@@ -56,6 +70,8 @@
31 test that all commands available to the test suite have help.
32 (Robert Collins, #177500)
33
34+* Progress output is cleaned up when exiting. (Aaron Bentley)
35+
36 * Raise ValueError instead of a string exception.
37 (John Arbash Meinel, #586926)
38
39@@ -181,8 +197,7 @@
40
41 * ``bzr`` does not try to guess the username as ``username@hostname``
42 and requires it to be explictly set. This can be set using ``bzr
43- whoami``.
44- (Parth Malwankar, #549310)
45+ whoami``. (Parth Malwankar, #549310)
46
47 * ``bzrlib.commands.Command`` will now raise ValueError during
48 construction if there is no __doc__ set. (Note, this will be reverted in
49
50=== modified file 'bzr'
51--- bzr 2010-06-16 12:47:51 +0000
52+++ bzr 2010-06-21 04:18:27 +0000
53@@ -135,11 +135,14 @@
54
55
56 if __name__ == '__main__':
57- bzrlib.initialize()
58- exit_val = bzrlib.commands.main()
59-
60- if profiling:
61- profile_imports.log_stack_info(sys.stderr)
62+ library_state = bzrlib.initialize()
63+ library_state.__enter__()
64+ try:
65+ exit_val = bzrlib.commands.main()
66+ if profiling:
67+ profile_imports.log_stack_info(sys.stderr)
68+ finally:
69+ library_state.__exit__(None, None, None)
70
71 # By this point we really have completed everything we want to do, and
72 # there's no point doing any additional cleanup. Abruptly exiting here
73
74=== modified file 'bzrlib/__init__.py'
75--- bzrlib/__init__.py 2010-06-16 12:47:51 +0000
76+++ bzrlib/__init__.py 2010-06-21 04:18:27 +0000
77@@ -14,7 +14,22 @@
78 # along with this program; if not, write to the Free Software
79 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
80
81-"""bzr library"""
82+"""All of bzr.
83+
84+Developer documentation is available at
85+http://doc.bazaar.canonical.com/bzr.dev/developers/
86+
87+The project website is at http://bazaar.canonical.com/
88+
89+Some particularly interesting things in bzrlib are:
90+
91+ * bzrlib.initialize -- setup the library for use
92+ * bzrlib.plugin.load_plugins -- load all installed plugins
93+ * bzrlib.branch.Branch.open -- open a branch
94+ * bzrlib.workingtree.WorkingTree.open -- open a working tree
95+
96+We hope you enjoy this library.
97+"""
98
99 import time
100
101@@ -114,15 +129,96 @@
102 __version__ = _format_version_tuple(version_info)
103 version_string = __version__
104
105-
106-def test_suite():
107- import tests
108- return tests.test_suite()
109-
110-
111-def initialize(
112- setup_ui=True,
113- stdin=None, stdout=None, stderr=None):
114+# bzr has various bits of global state that are slowly being eliminated.
115+# This variable is intended to permit any new state-like things to be attached
116+# to a BzrLibraryState object rather than getting new global variables that
117+# need to be hunted down. Accessing the current BzrLibraryState through this
118+# variable is not encouraged: it is better to pass it around as part of the
119+# context of an operation than to look it up directly, but when that is too
120+# hard, it is better to use this variable than to make a branch new global
121+# variable.
122+# If using this variable by looking it up (because it can't be easily obtained)
123+# it is important to store the reference you get, rather than looking it up
124+# repeatedly; that way your code will behave properly in the bzrlib test suite
125+# and from programs that do use multiple library contexts.
126+global_state = None
127+
128+
129+class BzrLibraryState(object):
130+ """The state about how bzrlib has been configured.
131+
132+ :ivar saved_state: The bzrlib.global_state at the time __enter__ was
133+ called.
134+ :ivar cleanups: An ObjectWithCleanups which can be used for cleanups that
135+ should occur when the use of bzrlib is completed. This is initialised
136+ in __enter__ and executed in __exit__.
137+ """
138+
139+ def __init__(self, setup_ui=True, stdin=None, stdout=None, stderr=None):
140+ """Create library start for normal use of bzrlib.
141+
142+ Most applications that embed bzrlib, including bzr itself, should just
143+ call bzrlib.initialize(), but it is possible to use the state class
144+ directly.
145+
146+ More options may be added in future so callers should use named
147+ arguments.
148+
149+ BzrLibraryState implements the Python 2.5 Context Manager protocol, and
150+ can be used with the with statement. Upon __enter__ the global
151+ variables in use by bzr are set, and they are cleared on __exit__.
152+
153+ :param setup_ui: If true (default) use a terminal UI; otherwise
154+ some other ui_factory must be assigned to `bzrlib.ui.ui_factory` by
155+ the caller.
156+ :param stdin, stdout, stderr: If provided, use these for terminal IO;
157+ otherwise use the files in `sys`.
158+ """
159+ self.setup_ui = setup_ui
160+ self.stdin = stdin
161+ self.stdout = stdout
162+ self.stderr = stderr
163+
164+ def __enter__(self):
165+ # NB: This function tweaks so much global state it's hard to test it in
166+ # isolation within the same interpreter. It's not reached on normal
167+ # in-process run_bzr calls. If it's broken, we expect that
168+ # TestRunBzrSubprocess may fail.
169+ if version_info[3] == 'final':
170+ from bzrlib.symbol_versioning import suppress_deprecation_warnings
171+ suppress_deprecation_warnings(override=True)
172+
173+ import bzrlib.cleanup
174+ import bzrlib.trace
175+ self.cleanups = bzrlib.cleanup.ObjectWithCleanups()
176+ bzrlib.trace.enable_default_logging()
177+
178+ if self.setup_ui:
179+ import bzrlib.ui
180+ stdin = self.stdin or sys.stdin
181+ stdout = self.stdout or sys.stdout
182+ stderr = self.stderr or sys.stderr
183+ bzrlib.ui.ui_factory = bzrlib.ui.make_ui_for_terminal(
184+ stdin, stdout, stderr)
185+ global global_state
186+ self.saved_state = global_state
187+ global_state = self
188+
189+ def __exit__(self, exc_type, exc_val, exc_tb):
190+ self.cleanups.cleanup_now()
191+ import bzrlib.ui
192+ bzrlib.trace._flush_stdout_stderr()
193+ bzrlib.trace._flush_trace()
194+ import bzrlib.osutils
195+ bzrlib.osutils.report_extension_load_failures()
196+ bzrlib.ui.ui_factory.__exit__(None, None, None)
197+ bzrlib.ui.ui_factory = None
198+ global global_state
199+ global_state = self.saved_state
200+ return False # propogate exceptions.
201+
202+
203+def initialize(setup_ui=True, stdin=None, stdout=None, stderr=None):
204 """Set up everything needed for normal use of bzrlib.
205
206 Most applications that embed bzrlib, including bzr itself, should call
207@@ -131,39 +227,20 @@
208 More options may be added in future so callers should use named arguments.
209
210 :param setup_ui: If true (default) use a terminal UI; otherwise
211- something else must be put into `bzrlib.ui.ui_factory`.
212+ some other ui_factory must be assigned to `bzrlib.ui.ui_factory` by
213+ the caller.
214 :param stdin, stdout, stderr: If provided, use these for terminal IO;
215 otherwise use the files in `sys`.
216+ :return: A context manager for the use of bzrlib. The __enter__ method of
217+ this context needs to be called before it takes effect, and the __exit__
218+ should be called by the caller before exiting their process or
219+ otherwise stopping use of bzrlib. Advanced callers can use
220+ BzrLibraryState directly.
221 """
222- # TODO: mention this in a guide to embedding bzrlib
223- #
224- # NB: This function tweaks so much global state it's hard to test it in
225- # isolation within the same interpreter. It's not reached on normal
226- # in-process run_bzr calls. If it's broken, we expect that
227- # TestRunBzrSubprocess may fail.
228-
229- import atexit
230- import bzrlib.trace
231-
232- bzrlib.trace.enable_default_logging()
233- atexit.register(bzrlib.trace._flush_stdout_stderr)
234- atexit.register(bzrlib.trace._flush_trace)
235-
236- import bzrlib.ui
237- if stdin is None:
238- stdin = sys.stdin
239- if stdout is None:
240- stdout = sys.stdout
241- if stderr is None:
242- stderr = sys.stderr
243-
244- if setup_ui:
245- bzrlib.ui.ui_factory = bzrlib.ui.make_ui_for_terminal(
246- stdin, stdout, stderr)
247-
248- if bzrlib.version_info[3] == 'final':
249- from bzrlib.symbol_versioning import suppress_deprecation_warnings
250- suppress_deprecation_warnings(override=True)
251-
252- import bzrlib.osutils
253- atexit.register(osutils.report_extension_load_failures)
254+ return BzrLibraryState(setup_ui=setup_ui, stdin=stdin,
255+ stdout=stdout, stderr=stderr)
256+
257+
258+def test_suite():
259+ import tests
260+ return tests.test_suite()
261
262=== modified file 'bzrlib/smart/medium.py'
263--- bzrlib/smart/medium.py 2010-06-16 07:45:53 +0000
264+++ bzrlib/smart/medium.py 2010-06-21 04:18:27 +0000
265@@ -28,9 +28,9 @@
266 import sys
267 import urllib
268
269+import bzrlib
270 from bzrlib.lazy_import import lazy_import
271 lazy_import(globals(), """
272-import atexit
273 import socket
274 import thread
275 import weakref
276@@ -494,16 +494,16 @@
277 class _DebugCounter(object):
278 """An object that counts the HPSS calls made to each client medium.
279
280- When a medium is garbage-collected, or failing that when atexit functions
281- are run, the total number of calls made on that medium are reported via
282- trace.note.
283+ When a medium is garbage-collected, or failing that when
284+ bzrlib.global_state exits, the total number of calls made on that medium
285+ are reported via trace.note.
286 """
287
288 def __init__(self):
289 self.counts = weakref.WeakKeyDictionary()
290 client._SmartClient.hooks.install_named_hook(
291 'call', self.increment_call_count, 'hpss call counter')
292- atexit.register(self.flush_all)
293+ bzrlib.global_state.cleanups.addCleanup(self.flush_all)
294
295 def track(self, medium):
296 """Start tracking calls made to a medium.
297
298=== modified file 'bzrlib/trace.py'
299--- bzrlib/trace.py 2010-05-21 06:14:50 +0000
300+++ bzrlib/trace.py 2010-06-21 04:18:27 +0000
301@@ -374,7 +374,7 @@
302 _trace_file = old_trace_file
303 bzr_logger = logging.getLogger('bzr')
304 bzr_logger.removeHandler(new_handler)
305- # must be closed, otherwise logging will try to close it atexit, and the
306+ # must be closed, otherwise logging will try to close it at exit, and the
307 # file will likely already be closed underneath.
308 new_handler.close()
309 bzr_logger.handlers = old_handlers
310@@ -540,7 +540,7 @@
311
312
313 def _flush_stdout_stderr():
314- # installed into an atexit hook by bzrlib.initialize()
315+ # called from the bzrlib library finalizer returned by bzrlib.initialize()
316 try:
317 sys.stdout.flush()
318 sys.stderr.flush()
319@@ -553,7 +553,7 @@
320
321
322 def _flush_trace():
323- # run from atexit hook
324+ # called from the bzrlib library finalizer returned by bzrlib.initialize()
325 global _trace_file
326 if _trace_file:
327 _trace_file.flush()
328
329=== modified file 'bzrlib/ui/__init__.py'
330--- bzrlib/ui/__init__.py 2010-03-25 07:34:15 +0000
331+++ bzrlib/ui/__init__.py 2010-06-21 04:18:27 +0000
332@@ -106,6 +106,9 @@
333 This tells the library how to display things to the user. Through this
334 layer different applications can choose the style of UI.
335
336+ UI Factories are also context managers, for some syntactic sugar some users
337+ need.
338+
339 :ivar suppressed_warnings: Identifiers for user warnings that should
340 no be emitted.
341 """
342@@ -123,6 +126,22 @@
343 self.suppressed_warnings = set()
344 self._quiet = False
345
346+ def __enter__(self):
347+ """Context manager entry support.
348+
349+ Override in a concrete factory class if initialisation before use is
350+ needed.
351+ """
352+
353+ def __exit__(self, exc_type, exc_val, exc_tb):
354+ """Context manager exit support.
355+
356+ Override in a concrete factory class if more cleanup than a simple
357+ self.clear_term() is needed when the UIFactory is finished with.
358+ """
359+ self.clear_term()
360+ return False # propogate exceptions.
361+
362 def be_quiet(self, state):
363 """Tell the UI to be more quiet, or not.
364
365@@ -352,7 +371,6 @@
366 "without an upgrade path.\n" % (inter.target._format,))
367
368
369-
370 class SilentUIFactory(UIFactory):
371 """A UI Factory which never prints anything.
372
373
374=== modified file 'doc/developers/overview.txt'
375--- doc/developers/overview.txt 2010-05-23 20:44:49 +0000
376+++ doc/developers/overview.txt 2010-06-21 04:18:27 +0000
377@@ -13,6 +13,38 @@
378 to the Bazaar mailing list.
379
380
381+Using bzrlib
382+############
383+
384+Within bzr
385+==========
386+
387+When using bzrlib within the ``bzr`` program (for instance as a bzr
388+plugin), bzrlib's global state is already available for use.
389+
390+From outside bzr
391+================
392+
393+To use bzrlib outside of ``bzr`` some global state needs to be setup.
394+bzrlib needs ways to handle user input, passwords, a place to emit
395+progress bars, logging setup appropriately for your program. The easiest
396+way to set all this up in the same fashion ``bzr`` does is to call
397+``bzrlib.initialize``. This returns a context manager within which bzrlib
398+functions will work correctly. See the pydoc for ``bzrlib.initialize`` for
399+more information. In Python 2.4 the ``with`` keyword is not supported and
400+so you need to use the context manager manually::
401+
402+ # This sets up your ~/.bzr.log, ui factory and so on and so forth. It is
403+ # not safe to use as a doctest.
404+ library_state = bzrlib.initialize()
405+ library_state.__enter__()
406+ try:
407+ pass
408+ # do stuff here
409+ finally:
410+ library_state.__exit__(None, None, None)
411+
412+
413 Core classes
414 ############
415