Merge lp:~toshio/loggerhead/mod_wsgi into lp:loggerhead

Proposed by Toshio Kuratomi
Status: Merged
Merged at revision: 465
Proposed branch: lp:~toshio/loggerhead/mod_wsgi
Merge into: lp:loggerhead
Diff against target: 205 lines (+143/-3)
7 files modified
MANIFEST.in (+1/-1)
NEWS (+2/-0)
apache-loggerhead.conf (+19/-0)
bazaar.conf (+9/-0)
docs/index.rst (+55/-0)
loggerhead.wsgi (+50/-0)
setup.py (+7/-2)
To merge this branch: bzr merge lp:~toshio/loggerhead/mod_wsgi
Reviewer Review Type Date Requested Status
Max Kanat-Alexander (community) Needs Fixing
Toshio Kuratomi (community) Needs Information
Ian Clatworthy (community) Needs Information
Review via email: mp+23630@code.launchpad.net

Description of the change

Add script, config, and documenation on running loggerhead under mod_wsgi

To post a comment you must log in.
Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

Toshio,

Thanks for this. I wonder whether we ought to put the various files into a contrib/mod_wsgi directory instead of putting them into the project root. What do you think?

Some minor doc tweaks:

* loggehead -> Loggerhead :-)
* "bazaar conf" -> bazaar.conf
* "Loggerhead's doc directory" assumes installation via a package manager doesn't it?

On the legal side, loggerhead.wsgi needs a Copyright 2010 Canonical line added above the GPL stuff. Could you please also sign the Canonical Contributor Agreement? See http://www.canonical.com/contributors.

Forgive my ignorance but how does the MANIFEST.in file come into play?

review: Needs Information
Revision history for this message
Matt Nordhoff (mnordhoff) wrote :

toshio wrote on the mailing list that the wsgi script was originally written by Stuart Colville:

http://blog.projectfondue.com/2009/11/26/loggerhead-and-mod-wsgi

So we might need him to sign the contributor agreement too.

Revision history for this message
Toshio Kuratomi (toshio) wrote :

> Thanks for this. I wonder whether we ought to put the various files into a
> contrib/mod_wsgi directory instead of putting them into the project root. What
> do you think?
>
Depends. The script is installed by setup.py so I'd lean towards putting it into the root just like serve-branches. However, if you don't want using mod_wsgi to be a supported method of running loggerhead, then I could see not installing it to /usr/bin via setup.py and moving it into a contrib directory. The example config files follow the same convention as the loggerhead.conf.example file which is in the project root currently.

> Some minor doc tweaks:
>
> * loggehead -> Loggerhead :-)
> * "bazaar conf" -> bazaar.conf

Fixed. Branch updated.

> * "Loggerhead's doc directory" assumes installation via a package manager
> doesn't it?
>
Nope. Try this::

  python setup.py install --root /var/tmp/loggerhead
  cd /var/tmp/loggerhead/usr/share/doc/loggerhead/

A package manager would likely included a version number in that final directory, though, which is why I say "Loggerhead's doc directory" instead of being more specific.

> On the legal side, loggerhead.wsgi needs a Copyright 2010 Canonical line added
> above the GPL stuff. Could you please also sign the Canonical Contributor
> Agreement? See http://www.canonical.com/contributors.
>
I'll need to send the agreement on to my manager but I don't see any difficulties with it; I know other people who have been able to sign it in order to contribute to upstart. However, I can't, in good conscience add the header by myself. As noted in the linked bug:
  https://bugs.edge.launchpad.net/loggerhead/+bug/566118

Stuart Colville (username: muffinresearch) originated the vast majority of the script. I've only done a few tweaks to make it configurable via a config file. I've emailed with him and he's fine with signing the Canonical Contributor Agreement (and also works for Canonical now.) How should we work out that aspect? Does he need to sign the agreement and then give me a copy of the file that has the header? Can I allow him to push to my branch somehow? Can you add the header after Stuart and I both sign the agreement? I just need to know what needs to be done to make the assignment clear; I think both Stuart and I are anxious to see the code committed :-)

> Forgive my ignorance but how does the MANIFEST.in file come into play?

When you use setup.py (distutils or setuptools driven), you get a command::

  python setup.py sdist

That command creates a tarball of the project. Scripts and python packages listed in the setup.py file do not have to be listed in MANIFEST.in but other files (non-standard doc files, for example) do otherwise they are not included in the tarball.

Revision history for this message
Toshio Kuratomi (toshio) wrote :

I've sent in mu Contributor Agreement. Let me know what we need to do to get Stuart's code all legal and kosher as well.

review: Needs Information
Revision history for this message
Toshio Kuratomi (toshio) wrote :

Ian, I've been added to contributor-agreement-canonical now -- let us know what we need to do so Stuart's portion is taken care of as well.

Revision history for this message
Toshio Kuratomi (toshio) wrote :

hi Max, I'm still interested in this. I don't know whether I need to rebase to trunk or anyhting but I'm willing to do that work if necessary once we get the legal situation wrt Stuart figured out.

Revision history for this message
Max Kanat-Alexander (mkanat) wrote :

Okay, I will find out what needs to happen there with Stuart.

Revision history for this message
Stuart Colville (muffinresearch) wrote :

I'm happy to place my contribution under loggerhead's license as per my current agreement as a Canonical Employee. Let me know if there's anything further you need.

Revision history for this message
Max Kanat-Alexander (mkanat) wrote :

Thanks for updating this, Toshio! Sorry that it took me so long to get to it.

I really love all of the documentation and examples, they all look great.

However, in loggerhead.wsgi, instead of having a bunch of custom code, we should re-use loggerhead.main as much as possible. See what serve-branches does, for example.

Otherwise, we're going to be disabling a lot of features when we use WSGI, and we'll also add another main entry point that we have to maintain, which I would really like to avoid.

review: Needs Fixing
Revision history for this message
Max Kanat-Alexander (mkanat) wrote :

Oh and also, yeah, you'll have to rebase to trunk since a few things have changed, mostly in setup.py (because start-loggerhead and stop-loggerhead no longer exist).

Revision history for this message
Max Kanat-Alexander (mkanat) wrote :

Also, there's already this WSGI block in loggerhead.main:

    else:
        if protocol == 'fcgi':
            from flup.server.fcgi import WSGIServer
        elif protocol == 'scgi':
            from flup.server.scgi import WSGIServer
        elif protocol == 'ajp':
            from flup.server.ajp import WSGIServer
        else:
            print 'Unknown protocol: %s.' % (protocol)
            sys.exit(1)

Could we perhaps just modify that to get direct mod_wsgi support?

Revision history for this message
Graham Binns (gmb) wrote :

I'm going to move this back to Work In Progress since it's been sat in the active reviews queue for a while.

Revision history for this message
Robert Collins (lifeless) wrote :

This can be tweaked by a reviewer and landed, its this || close.

Revision history for this message
Robert Collins (lifeless) wrote :

I've done so. I think there is room to cleanup the tree, move stuff out of root etc, but this should be sufficiently together to work. Max's suggestion about loggerhead.main would be good too, if someone wants to follow up.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'MANIFEST.in'
2--- MANIFEST.in 2008-06-22 18:04:15 +0000
3+++ MANIFEST.in 2010-04-25 04:56:25 +0000
4@@ -1,2 +1,2 @@
5-include loggerhead.conf.example README.txt NEWS
6+include loggerhead.conf.example README.txt NEWS apache-loggerhead.conf bazaar.conf
7 recursive-include loggerhead/static *
8
9=== modified file 'NEWS'
10--- NEWS 2010-04-23 00:02:29 +0000
11+++ NEWS 2010-04-25 04:56:25 +0000
12@@ -25,6 +25,8 @@
13 - Ignore readonly+ prefix when checking if Loggerhead is serving a
14 local location. (Reported by Tres Seaver.) (Matt Nordhoff)
15
16+ - Add a script and documentation for running under mod_wsgi.
17+ (Stuart Colville, Toshio Kuratomi)
18
19 1.17 [20Aug2009]
20 ----------------
21
22=== added file 'apache-loggerhead.conf'
23--- apache-loggerhead.conf 1970-01-01 00:00:00 +0000
24+++ apache-loggerhead.conf 2010-04-25 04:56:25 +0000
25@@ -0,0 +1,19 @@
26+### If you receive MemoryError tracebacks setting up loggerhead under mod_wsgi,
27+### read /usr/share/doc/loggerhead*/README for help
28+Alias /bzr/static /usr/share/loggerhead/static
29+RewriteEngine On
30+RewriteRule ^/bzr$ /bzr/ [R]
31+
32+WSGIDaemonProcess loggerhead user=apache group=apache maximum-requests=1000 display-name=loggerhead processes=4 threads=1
33+WSGISocketPrefix run/wsgi
34+WSGIRestrictStdout On
35+WSGIRestrictSignal Off
36+
37+WSGIScriptAlias /bzr /usr/bin/loggerhead.wsgi
38+
39+<Location /bzr>
40+ WSGIProcessGroup loggerhead
41+ Order deny,allow
42+ Allow from all
43+</Location>
44+
45
46=== added file 'bazaar.conf'
47--- bazaar.conf 1970-01-01 00:00:00 +0000
48+++ bazaar.conf 2010-04-25 04:56:25 +0000
49@@ -0,0 +1,9 @@
50+# directory to serve bzr branches from
51+# Non-bzr directories under this path will also be visible in loggerhead
52+#http_root_dir = '/var/www/bzr'
53+
54+# The url prefix for the bzr branches.
55+http_user_prefix = '/bzr'
56+
57+# Directory to put cache files in
58+http_sql_dir = '/var/cache/loggerhead'
59
60=== modified file 'docs/index.rst'
61--- docs/index.rst 2010-04-13 08:46:25 +0000
62+++ docs/index.rst 2010-04-25 04:56:25 +0000
63@@ -149,6 +149,61 @@
64 some path into the site, you'll need to specify it using
65 ``--prefix=/some_path``.
66
67+Serving Loggerhead with mod_wsgi
68+--------------------------------
69+
70+A second method for using Loggerhead with apache is to have apache itself
71+execute Loggerhead via mod_wsgi. You need to add configuration for apache and
72+for bazaar to make this work. Example config files are in the Loggerhead doc
73+directory as apache-loggerhead.conf and bazaar.conf. You can copy them into
74+place and use them as a starting point following these directions:
75+
76+1) Install mod_wsgi. On Ubuntu and other Debian derived distros::
77+
78+ sudo apt-get install libapache2-mod-wsgi
79+
80+ On Fedora-derived distros::
81+
82+ su -c yum install mod_wsgi
83+
84+2) Copy the bazaar.conf file where apache will find it (May be done for you if
85+ you installed Loggerhead from a distribution package)::
86+
87+ # install -d -o apache -g apache -m 0755 /etc/loggerhead
88+ # cp -p /usr/share/doc/loggerhead*/bazaar.conf /etc/loggerhead/
89+ # ln -s /etc/loggerhead /var/www/.bazaar
90+
91+3) Create the cache directory (May be done for you if you installed Loggerhead
92+ from a distribution package)::
93+
94+ # install -d -o apache -g apache -m 0700 /var/cache/loggerhead/
95+
96+4) Edit /etc/loggerhead/bazaar.conf. You need to set http_root_dir to the filesystem
97+ path that you will find your bzr branches under. Note that normal
98+ directories under that path will also be visible in Loggerhead.
99+
100+5) Install the apache conf file::
101+
102+ # cp -p /usr/share/doc/loggerhead*/apache-loggerhead.conf /etc/httpd/conf.d/loggerhead.conf
103+
104+6) Edit /etc/httpd/conf.d/loggerhead.conf to point to the url you desire to
105+ serve Loggerhead on. This should match with the setting for
106+ http_user_prefix in bazaar.conf
107+
108+7) Restart apache and you should be able to start browsing
109+
110+.. note:: If you have SELinux enabled on your system you may need to allow
111+ apache to execute files in temporary directories. You will get a
112+ MemoryError traceback from python if this is the case. This is because of
113+ the way that python ctypes interacts with libffi. To rectify this, you may
114+ have to do several things, such as mounting tmpdirs so programs can be
115+ executed on them and setting this SELinux boolean::
116+
117+ setsebool httpd_tmp_exec on
118+
119+ This bug has information about how python and/or Linux distros might solve
120+ this issue permanently and links to bugs which diagnose the root cause.
121+ https://bugzilla.redhat.com/show_bug.cgi?id=582009
122
123 Search
124 ------
125
126=== added file 'loggerhead.wsgi'
127--- loggerhead.wsgi 1970-01-01 00:00:00 +0000
128+++ loggerhead.wsgi 2010-04-25 04:56:25 +0000
129@@ -0,0 +1,50 @@
130+#!/usr/bin/python -tt
131+# This program is free software; you can redistribute it and/or modify
132+# it under the terms of the GNU General Public License as published by
133+# the Free Software Foundation; either version 2 of the License, or
134+# (at your option) any later version.
135+#
136+# This program is distributed in the hope that it will be useful,
137+# but WITHOUT ANY WARRANTY; without even the implied warranty of
138+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
139+# GNU General Public License for more details.
140+#
141+# You should have received a copy of the GNU General Public License
142+# along with this program; if not, write to the Free Software
143+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
144+
145+
146+import sys
147+import os
148+import pwd
149+sys.path.insert(0, os.path.dirname(__file__))
150+
151+from paste.httpexceptions import HTTPExceptionHandler
152+from loggerhead.apps.transport import BranchesFromTransportRoot
153+from loggerhead.apps.error import ErrorHandlerApp
154+from loggerhead.config import LoggerheadConfig
155+from bzrlib import config as bzrconfig
156+from paste.deploy.config import PrefixMiddleware
157+from bzrlib.plugin import load_plugins
158+
159+class NotConfiguredError(Exception):
160+ pass
161+
162+
163+load_plugins()
164+config = LoggerheadConfig()
165+prefix = config.get_option('user_prefix') or ''
166+# Note we could use LoggerheadConfig here if it didn't fail when a
167+# config option is not also a commandline option
168+root_dir = bzrconfig.GlobalConfig().get_user_option('http_root_dir')
169+if not root_dir:
170+ raise NotConfiguredError('You must have a ~/.bazaar/bazaar.conf file for'
171+ ' %(user)s with http_root_dir set to the base directory you want'
172+ ' to serve bazaar repositories from' %
173+ {'user': pwd.getpwuid(os.geteuid()).pw_name})
174+prefix = prefix.encode('utf-8', 'ignore')
175+root_dir = root_dir.encode('utf-8', 'ignore')
176+app = BranchesFromTransportRoot(root_dir, config)
177+app = PrefixMiddleware(app, prefix=prefix)
178+app = HTTPExceptionHandler(app)
179+application = ErrorHandlerApp(app)
180
181=== modified file 'setup.py'
182--- setup.py 2009-10-17 06:19:40 +0000
183+++ setup.py 2010-04-25 04:56:25 +0000
184@@ -30,7 +30,10 @@
185 license = "GNU GPL v2 or later",
186 maintainer = "Michael Hudson",
187 maintainer_email = "michael.hudson@canonical.com",
188- scripts = ["start-loggerhead", "stop-loggerhead", "serve-branches"],
189+ scripts = ["start-loggerhead",
190+ "stop-loggerhead",
191+ "serve-branches",
192+ "loggerhead.wsgi"],
193 packages = ["loggerhead",
194 "loggerhead/apps",
195 "loggerhead/controllers",
196@@ -67,6 +70,8 @@
197 ('share/man/man1', ['start-loggerhead.1',
198 'stop-loggerhead.1',
199 'serve-branches.1']),
200- ('share/doc/loggerhead', ['loggerhead.conf.example']),
201+ ('share/doc/loggerhead', ['loggerhead.conf.example',
202+ 'apache-loggerhead.conf',
203+ 'bazaar.conf']),
204 ],
205 )

Subscribers

People subscribed via source and target branches