Merge lp:mailman into lp:~pierfort123/mailman/site

tox-ify the test suite. First, this makes it easier to run since you don't
have to manually set up the virtualenv. Second, it allows us to later create
a python3 environment for porting purposes.

Other change:

* The `mailman conf` command no longer takes the `-t/--sort` option; the
  output is always sorted.
* The ``[database]migrations_path`` setting is removed.
* You no longer have to create a virtual environment separately when running
  the test suite. Just use `tox`.
* The RFC 2369 headers added to outgoing messages are now added in sorted
  order.
* The JSON representation `http_etag` key uses an algorithm that is
  insensitive to Python's dictionary sort order.

MANIFEST.in updates.

 * You no longer have to edit `src/mailman/testing/testing.cfg` to run the
   test suite against PostgreSQL. See `src/mailman/docs/START.rst` for
   details.

Also:

* Test fixes.
* Doc fixes.
* PostgreSQL/model fixes.

Add support for a 'coverage' environment. e.g. `tox -e coverage` will run the
test suite and provide a code coverage report.

Remove unneeded line.

s/restish/falcon/

Make sure runner subprocesses get covered during `tox -e coverage`.

Remove debugging.

Improve coverage:
* Remove some unused code
* pragma: no cover

Fix a few issues with `python2 -3` compatibility.

Repair broken test. Given by Aurélien Bompard.

Remove debugging.

Add comments and rewrite test a bit.

 * Fixed Unicode errors in the digest runner and when sending messages to the
   site owner as a fallback. Given by Aurélien Bompard. (LP: #1130957).

Also:

 * Convert some uses of the unicode() built-in to bytes.decode() in
   preparation for Python 3 and to eliminate some pyflakes errors.

 * Added LogFileMark.read() as a convenience method.

 * Fix Unicode errors when a message being added to the digest has non-ascii
   characters in its payload, but no Content-Type header defining a charset.
   Given by Aurélien Bompard. (LP: #1170347)

 * Fixed messages without a `text/plain` part crashing the `Approved` rule.
   Given by Aurélien Bompard. (LP: #1158721)

 * The address resource now has an additional '/user' sub-resource which can
   be used to GET the address's linked user if there is one. This
   sub-resource also supports POST to link an unlinked address (with an
   optional 'auto_create' flag), and PUT to link the address to a different
   user. It also supports DELETE to unlink the address. (LP: #1312884)
   Given by Aurélien Bompard based on work by nicolask.

Updated a credit.

 * Fixed getting non-ASCII filenames from RFC 2231 i18n'd messages. Given by
   Aurélien Bompard. (LP: #1060951)

/user sub-resource of the address documented by Aurélien Bompard.

 * Fixed `AttributeError` on MIME digest messages. Given by Aurélien Bompard.
   (LP: #1130696)

Prep for release

Back port from py3 branch:

 * The ``/3.0/system`` path is deprecated; use ``/3.0/system/versions`` to get
   the system version information.

Back port from the py3 branch:

 * You can access the system configuration via the resource path
   ``/3.0/system/configuration/<section>``. This returns a dictionary with
   the keys being the section's variables and the values being their value
   from ``mailman.cfg`` as verbatim strings. You can get a list of all
   section names via ``/3.0/system/configuration`` which returns a dictionary
   containing the ``http_etag`` and the section names as a sorted list under
   the ``sections`` key. The system configuration resource is read-only.

Back port from py3 branch:

 # Explicitly prevent flufl.lock from propagating its log messages
 # to its root logger, i.e. the console.

Documentation cleanup.

Update version.

Post-release housekeeping.

It's possible $HOME isn't defined.

Always pass the -C option to the subprocess.

Merge the Python 3 branch.

Bump copyright years.

We don't need the 'six' package any more.

We don't need the lazr.config workaround any more.

 * Fix calculation of default configuration file to use when the ``$var_dir``
   is created by ``mailman start``. (LP: #1411435)

Fix autonumbering in style guide.

 * Improve the documentation describing how to run Alembic to add new schema
   migrations. Given by Abhilash Raj.

Python 2 -> 3 migration.

alembic documentation change.

 * When creating a user with an email address, do not create the user record
   if the email address already exists. Given by Andrew Stuart.
   (LP: #1418280)

 * You can now DELETE an address. If the address is linked to a user, the
   user is not delete, it is just unlinked.

 * When deleting a user via REST, make sure all linked addresses are deleted.
   Found by Andrew Stuart. (LP: #1419519)

Fix some test output.

 * A new API is provided to support non-production testing infrastructures,
   allowing a client to cull all orphaned UIDs via ``DELETE`` on
   ``<api>/reserved/uids/orphans``. Note that *no guarantees* of API
   stability will ever be made for resources under ``reserved``.
   (LP: #1420083)

Also:

- Allow @dbconnection methods to be @staticmethods taking only one argument,
  the store to perform the query on.

 * Member resource JSON now include the ``member_id`` as a separate key.

Move some old bin scripts to a holding tank.

Doc fixes given by Abhilash Raj.

copybump should ignore .git directories. This isn't currently relevant for
Mailman, but I use this script for other repos.

Fix a read used during the PostgreSQL tests.

 * When trying to subscribe an address to a mailing list through the REST API
   where a case-differing version of the address is already subscribed, return
   a 409 error instead of a 500 error. Found by Ankush Sharma. (LP: #1425359)

Documentation fixes, given by Abhilash Raj.

 * ``mailman lists --domain`` was not properly handling its arguments. Given
   by Manish Gill. (LP: #1166911)

 * When deleting a user object, make sure their preferences are also deleted.
   Given by Abhishek. (LP: #1418276)

 * Be sure a mailing list's acceptable aliases are deleted when the mailing
   list itself is deleted. (LP: #1432239)

Make a test less susceptible to churn when migrations are added. Given by
abompard.

Additional fixes for importing from Mailman 2. Given by abompard.

Test importing MM2 pickles with BounceInfo. Given by abompard.

Fix some typos.

trunk merge

Update docstring.

 * The built-in example ``IArchiver`` implementations now explicitly return
   None. (LP: #1203359)

 * Domains now have a list of owners, which are ``IUser`` objects, instead of
   the single ``contact_address`` they used to have. ``IUser`` objects now
   also have a ``is_server_owner`` flag (defaulting to False) to indicate
   whether they have superuser privileges. Give by Abhliash Raj, with fixes
   and refinements by Barry Warsaw. (LP: #1423756)

 * Domains can now optionally be created with owners; domain owners can be
   added after the fact; domain owners can be deleted. Also, users now have
   an ``is_server_owner`` flag as part of their representation, which defaults
   to False, and can be PUT and PATCH'd. Given by Abhilash Raj, with fixes
   and refinements by Barry Warsaw. (LP: #1423756)

 * Mailing list subscription policy work flow has been completely rewritten.
   It now properly supports email verification and subscription confirmation
   by the user, and approval by the moderator using unique tokens.
   ``IMailingList`` objects now have a ``subscription_policy`` attribute.

Abhilash's fix for allowing singleton strings in REST interfaces that also
accept lists of strings.

Open templates for reading in utf-8; given by abompard.

Plumb the subscription policy through to the membership subscription REST API.

Update versions for 3.0rc1, but don't tag yet.

Move a doctest to the right directory.

Clean up some templates; given by raj-abhilash1

Plumb subscription request handling through the REST API.

Fix MANIFEST.in to include all the necessary test data files.

Documentation updates.

VERSION.

Remove todo from release branch.

maxking's nickname

Fix spellings.

Date in NEWS.

Set up for 3.0.0 final.

Typos; given by sumanah.

raj-abhilash1's branch to ensure we block release on Falcon 0.3. I moved and
renamed his test.

Also, backfill a relevant Launchpad bug number.

Point to the wiki release notes page.

 * The test suite now runs successfully again with PostgreSQL. Given by
   Aurélien Bompard. (LP: #1435941)

Steve's nickname.

Aurelien's nick name

 * Port the REST machinery to Falcon 0.3. (LP: #1446881)

Prep for 3.0 final.

Be sure to use Falcon 0.3.