Merge lp:~adiroiban/launchpad/bug-583934 into lp:launchpad

Initial API for POFile and translation statistics attributes.

= Bug 583934 =
Right now there is on API for exporting POFile attributes

== Proposed fix ==
This branch is only for POFile attributes and is part of the Translations Reporting API (https://dev.launchpad.net/Translations/Specs/ReportingAPI).

The main part of this branch is to define a translation statistics interface that is suitable for the API.

== Pre-implementation notes ==
IRC chat start from here http://irclogs.ubuntu.com/2010/06/07/%23launchpad-reviews.html#t17:21

Outstanding issues:
 * due to message sharing translation statistics are not updated in real time
   * this affects the last_updated, last_translator, untranslated ... etc
   * still, this branch is independent of this problem and we can go on and define/design the API

 * due to future work for ubuntu-upstream message sharing there may be changes in the way ubuntu changes from upstream are computed/represented.
  * we don't know how exactly this changes will look so maybe it is best to go on with this branch and make the changes only if they are required.

* Jeroen noted that (length - translated messages) is not always equal with untranslated messages, and we should count translation credits and outer automatic translations as "translated messages"

=== Launchpad Translations Reporting API Call - 2010-06-23 ===

Real-time stats update:

• There will be improvements, but they will most probably be gradual
• Not dependent on the message sharing script to complete
• For the current intended consumers the current delay is ok
• Only the series the translator is working will be updated in
real-time, other shared series will take a week to update. We assume
that the translator will mostly work on the translation focus series.

IRosettaStats:

• Not the best interface for statistics, the reason being because it is
parameterised on language
• The language parameter is only used for IPOTemplate, and in turn that
interface is only using message count
• Jeroen: it was probably done this way to a llow a script to be very
generic. It might be worth removing the IRosettaStats interface (or just
the language parameter) and see what breaks on the test suite
• Adi: replacement proposal with ITranslationStats
• An intended improvement is to also provide more descriptive names to
the interface
• AGREED that we should be working on a new translations interface

ITranslationStats:

• Adi's proposal for a new interface for statistics as a replacement for
IRosettaStats
• Right now only used for counters, which is ok if they are attributes
of POFiles
• If we want it to be generic, it might need to be extended and require
a 'target' attribute. This 'target' would be then replaced by a
reference to a language, source package, distro series, etc.
• AGREED to have a stats object to implement the ITranslationStatistics
interface and provide a method on objects to query those statistics. As
an example, a POFile would have a 'stats' attribute instead of separate
attributes for each of the counts. No conclusion on the implementation
on how to link back the object to query with the returned statistics.
Henning would like to see a merge proposal with actual code for better
understanding. Adi will submit it first with the 'target' parameter in
ITranslationStatistics approach.

Other:

• Attribute names: translated_message_*_count - if they are in their own
class, then the 'translated_message' prefix is not needed.
• Recife branch to land roughly on October. Agreed to work on the devel
branch instead for API work in order not to be a blocker. We should go
ahead with an API that's easily extensible and exposes only the
essential counts (e.g. translated, untranslated, unreviewed) and not
worry with the Recife branch changes. When the Recife branch is merged
with trunk, then we can extend the API and expose the rest.
• Jeroen and Henninge happy with a weekly call to track progress and
implementation discussion. David to follow up with Adi (dropped from the
call in the last few minutes) if he'd be happy to.

ACTIONS:

• Adi to submit a merge proposal: experiment removing the language
parameter and fixing what breaks
• Adi to resubmit the existing POFile API merge proposal:
        ∘ replace IRosettaStats with the new ITranslationStats interface
        ∘ with the 'target' parameter
        ∘ for now expose only attributes not dependent on the
        Ubuntu/upstream message sharing work (i.e. no "new" or "chaged"
        counts)
• David to set up a weekly call with Adi, Henning and Jeroen

== Implementation details ==

=== IRosettaStats problem ===
The current IRosettaStats interface is not suitable for API export since the number of various statistic are returned using a method that takes a language as a parameter.

From my point of view IRosettaStats needs a complete rework, since that language parameter is only used for IPOTemplate, and IPOTemplates only implement the message_count part of this interface. I would say that we could change the IPOTemplate to not implement IRosettaStats and remove the language parameter from all IRosettaStats methods.

=== ITranslationStatistics ===
Since this rework of IRosettaStats require some time and thinking, to move things forwards I have started a new interface ITranslationStatistics

Right now ITranslationStatistics has only those attributes required by the API.
In the future (maibe LP Epic) maybe we could decide the new shape of translation statistics interface and extend it to replace the current IRosettaStats.

There will be different TranslationStatistics object implementing the ITranslationStatistics interface in order to agregate the statistics for various translation templates sets.

=== Choosing what attribute to export for the translations statistics ===

Right now IRosettaStats contains a lot of counters, some of them having only internal API usage.

I tried to choose the one that are relevant to end users and use names that will describe their values.
Still, translation_message_new_count is challenging as that "new" can have many meanings.
Mabye we should have "translation_message_new_in_launchpad_count" and "translation_message_changed_in_launchpad_count".

Below is the list of the attribute names that will be exported in the API:

* changed_count
* new_count
* total_count
* translated_count
* unreviewed_count
* language
* template

=== Lint warnings ===

Since the lint warnings were already in the code, I will fix them after this branch is approved in order to make the review process easier.

== Tests ==

./bin/test -t xx-pofile.txt

== Demo and Q/A ==
Point your browser to https://launchpad.dev/api/devel/ubuntu/hoary/+source/evolution/+pots/evolution-2.2/es

You should see the exported attributes.

There is also an „translation_statistics_link” attribute available at :
https://launchpad.dev/api/devel/ubuntu/hoary/+source/evolution/+pots/evolution-2.2/es/+statistics

This will export the ITranslationStatistics attributes

= Launchpad lint =

Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.

Linting changed files:
  lib/lp/translations/configure.zcml
  lib/lp/translations/browser/configure.zcml
  lib/lp/translations/browser/pofile.py
  lib/lp/translations/interfaces/pofile.py
  lib/lp/translations/interfaces/statistics.py
  lib/lp/translations/interfaces/webservice.py
  lib/lp/translations/model/pofile.py
  lib/lp/translations/stories/webservice/xx-pofile.txt

== Pyflakes notices ==

lib/lp/translations/interfaces/webservice.py
    17: 'IHasTranslationImports' imported but unused
    17: 'ITranslationImportQueue' imported but unused
    17: 'ITranslationImportQueueEntry' imported but unused
    22: 'IPOTemplate' imported but unused
    24: 'IPOFile' imported but unused
    26: 'ITranslationStatistics' imported but unused

== Pylint notices ==

lib/lp/translations/browser/pofile.py
    158: [E1002, POFileBaseView.initialize] Use super on an old style class
    682: [E1002, POFileTranslateView.initialize] Use super on an old style class