Merge lp:~gary/launchpad/pythonpath into lp:launchpad/db-devel

Proposed by Gary Poster
Status: Merged
Merged at revision: not available
Proposed branch: lp:~gary/launchpad/pythonpath
Merge into: lp:launchpad/db-devel
Diff against target: 99 lines (+51/-23)
1 file modified
buildout-templates/_pythonpath.py.in (+51/-23)
To merge this branch: bzr merge lp:~gary/launchpad/pythonpath
Reviewer Review Type Date Requested Status
Francis J. Lacoste (community) release-critical Approve
Barry Warsaw (community) code Approve
Review via email: mp+14490@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Gary Poster (gary) wrote :

Fixes bug 475550.

Diff and example usage can be found here: http://paste.ubuntu.com/310788/

I hope that the bug report, the extensive comment, and the example usage can provide the necessary context. If not, of course, please ask!

Thank you

Gary

Revision history for this message
Barry Warsaw (barry) wrote :

Thanks for working on this hairy problem! We talked about it enough in irc that this approach seems reasonable to me.

Python should really make our lives easier.

review: Approve (code)
Revision history for this message
Francis J. Lacoste (flacoste) :
review: Approve (release-critical)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'buildout-templates/_pythonpath.py.in'
--- buildout-templates/_pythonpath.py.in 2009-10-19 19:33:01 +0000
+++ buildout-templates/_pythonpath.py.in 2009-11-05 20:20:37 +0000
@@ -6,6 +6,7 @@
66
7__metaclass__ = type7__metaclass__ = type
88
9import new
9import os10import os
10import site11import site
11import sys12import sys
@@ -19,22 +20,58 @@
19site_dirs = [20site_dirs = [
20 ${indented-dir-paths}21 ${indented-dir-paths}
21 ]22 ]
22clean_modules = [
23 'os',
24 '_pythonpath',
25 'site',
26 'sitecustomize',
27 'sys',
28 ${clean-sys-modules}
29 ]
3023
31def set_path():24def set_path():
32 previously_imported = {}25 # We need to remove any modules installed using the setuptools
33 # We get a copy of the keys rather than simply iterating because we will26 # namespace package approach that generates .pth files that mutate
34 # be mutating the dictionary.27 # sys.modules. They can mask the namespace modules we actually
35 for k in sys.modules.keys():28 # want. We will try to recognize these sorts of namespace modules by
36 if k not in clean_modules:29 # looking for modules that do not have a __file__, like the ones
37 previously_imported[k] = sys.modules.pop(k)30 # generated by these .pth files. However, that's not good enough: some
31 # C modules look like this. We then look for any names in the
32 # module that are not special Python names (__*__) that are also not
33 # in sys.modules. If any one of these exist, this is not a
34 # namespace module. Otherwise, it is a namespace module. Moreover,
35 # if there were any sys.modules names from the namespace module,
36 # this _pythonpath module was imported too late: we should complain.
37 #
38 # If you have not seen the code in these .pth files, here's an
39 # example. It is all in one line in the .pth files, because of the
40 # .pth syntax for this feature.
41 #
42 # import sys,new,os;
43 # p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('lazr',));
44 # ie = os.path.exists(os.path.join(p,'__init__.py'));
45 # m = not ie and sys.modules.setdefault('lazr',new.module('lazr'));
46 # mp = (m or []) and m.__dict__.setdefault('__path__',[]);
47 # (p not in mp) and mp.append(p)
48 #
49 # Note that we will be mutating sys.modules, so we want to make sure to be
50 # getting a copy, not an iterable.
51 marker = object()
52 for k, v in sys.modules.items():
53 if k == '__main__' or not isinstance(v, new.module):
54 # These are some special cases that we'll pass by.
55 continue
56 if getattr(v, '__file__', marker) is marker:
57 # This is a .pth-generated namespace or C module.
58 for name in dir(v):
59 if not name.startswith('__') and not name.endswith('__'):
60 full_name = '.'.join((k, name))
61 if full_name in sys.modules:
62 # This is a .pth-generated namespace module that
63 # has had one of its sub-packages imported.
64 raise RuntimeError(
65 'Found unexpected module %s. '
66 'Import _pythonpath earlier!' % (full_name,))
67 else:
68 # This is a C module or something like that. Nothing
69 # to see here: move along.
70 break
71 else:
72 # It is a .pth-generated namespace module. Remove it
73 # so we can let our eggs make their own.
74 del sys.modules[k]
3875
39 # We keep the very first path because that is typically the directory76 # We keep the very first path because that is typically the directory
40 # of the file that imported us, which should continue to have precedence.77 # of the file that imported us, which should continue to have precedence.
@@ -53,15 +90,6 @@
53 for p in site_dirs:90 for p in site_dirs:
54 site.addsitedir(p)91 site.addsitedir(p)
5592
56 # We don't want to have dropped any packages that weren't already added
57 # back by what we just did. If we did, there's a good chance that the
58 # world will now be insane. Quit now, and let's fix it.
59 unexpected = set(previously_imported).difference(sys.modules)
60 if unexpected:
61 raise RuntimeError(
62 'Found unexpected module(s): %s\n\nImport _pythonpath earlier!' %
63 (', '.join(sorted(unexpected)),))
64
65 # Make subprocesses have the same environment.93 # Make subprocesses have the same environment.
66 os.environ['PYTHONPATH'] = os.pathsep.join(sys.path)94 os.environ['PYTHONPATH'] = os.pathsep.join(sys.path)
6795

Subscribers

People subscribed via source and target branches

to status/vote changes: