`RunJob` and `CeleryRunJob` are partly "overriding general behaviour",
which makes sense to do in a subclass of `Task`, but they also implement
their own `run` methods. `celery_app.task` creates a new `Task`
instance using the decorated function as its `run` method, and so
persuading those to use `CeleryRunJob.run` instead took some effort.
However, with Python 3.6 this fails because the running task instance
doesn't seem to be an instance of `CeleryRunJob` according to `super()`.
Switching to `celery_app.register_task` avoids some unnecessary
complexity, but the core problem remains. For now, I've just switched
to the old-style way of calling the superclass using `RunJob.run(self,
job_id)`; inelegant though it is, it works on both Python 3.5 and 3.6.
I suspect that eventually we'll need to rethink how `lazr.jobrunner`'s
Celery integration works based on modern Celery best practices in order
to fix this properly.
`scripts/wsgi-archive-auth.py` runs within Apache mod_wsgi, which is a
difficult environment for Launchpad code: it doesn't activate
Launchpad's virtualenv and can't even be told to disable the automatic
import of the `site` module (similar to Python's `-S` command-line
option), so we have to take various countermeasures before manually
activating the virtualenv so that we can import other parts of
Launchpad.
In bionic, we need a new countermeasure. The system `zope.interface`
package (installed as a dependency of landscape-common) is built with
setuptools >= 31.0.0, which has new namespace package handling that
causes `sys.modules["zope"]` to exist and to point to the
system-installed package, even without importing `zope.interface`; this
then confuses Launchpad code that tries to import from `zope.interface`
later. Arrange for the namespace package to be re-imported to avoid
this problem.