Merge lp:~henninge/launchpad/bug-506925-oops-export into lp:launchpad

= Bug 506925 =

Each PO file has a standard header with information about the file. One of the fields in that header is "Last-Translator" which contains the name and email address of the last person that did a translation in this file. Upon export from Launchpad, this field is filled with the display_name (or title? who cares) of the last translators IPerson object (and email address). This may contain non-ascii characters as in https://launchpad.net/~danilo. The header also has a "Content-Type" field that declares the character encoding of the file. This bug used to surface when that field did not contain a charset that could represent the translators name (ideally utf-8).

This problem is not new as such clashes have happened with translations, too, which also may contain non-ascii characters. The export routine alredy knows how to deal with this and simply changes the encoding of the file to UTF-8. This behaviour needed to simply be extended to the header, i.e. the whole file.

== Pre-imp notes ==

Talked with danilo about it and we agreed that it is ok to convert the whole file to UTF-8 if necessary. Most files are UTF-8 already.

== Implmentation details ==

lib/lp/translations/utilities/gettext_po_exporter.py

 * Encode the whole file at the end instead of each chunk seperately and in different places.
 * Did away with the need of having to re-encode all chunks if the encoding changes.
 * The header is prepended at the end or the loop so that the encoding can be changed at last-minute warning.

lib/lp/translations/utilities/tests/test_gettext_po_exporter.py

 * Added test for encoding of non-ascii characters in the header.
 * Combined common code into a private method.
 * Use named format specifiers because their order changes.

== Test ==

bin/test -vvct GettextPOExporterTestCase

== Demo/QA ==

 * Import a file that has "ascii" as it's Content-type.
 * Have Danilo make a translation using ascii characters.
 * Export the file.
 * The exported file should be UTF-8 now and Данило Шеган should appear as the Last-Translator.

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/translations/utilities/gettext_po_exporter.py
  lib/lp/translations/utilities/tests/test_gettext_po_exporter.py

As discussed on IRC, I have some concerns with _try_encode_file_content method: it has very weird return semantics. Basically, they are: return encoded content according to translation_file header charset, and if it fails, return None if translation_file header wasn't UTF-8, otherwise fail with an exception. Not a very reasonable method, I'd say :)

We've looked at several solutions to that, one was to just switch the callsite to do exception handling, and not handle exceptions in this method (where the second one might fail, and would naturally propagate), though this approach loses some debugging information existing code has (file name in the exception error).

We can also just do this "exception error message improvement" in the _try_encode_file_content (i.e. catch the exception there and re-raise it unconditionally, compared to what we do now where we re-raise it only if the requested encoding was UTF-8).