Merge lp:~stub/launchpad/librarian-report into lp:launchpad

Proposed by Stuart Bishop
Status: Merged
Approved by: Jeroen T. Vermeulen
Approved revision: not available
Merged at revision: not available
Proposed branch: lp:~stub/launchpad/librarian-report
Merge into: lp:launchpad
Diff against target: 109 lines (+54/-16)
2 files modified
lib/canonical/librarian/tests/librarian-report.txt (+27/-14)
scripts/librarian-report.py (+27/-2)
To merge this branch: bzr merge lp:~stub/launchpad/librarian-report
Reviewer Review Type Date Requested Status
Jeroen T. Vermeulen (community) Approve
Review via email: mp+16522@code.launchpad.net

This proposal supersedes a proposal from 2009-12-22.

Commit message

Add date range selection to librarian-report.py

To post a comment you must log in.
Revision history for this message
Stuart Bishop (stub) wrote : Posted in a previous version of this proposal

Adds a date range selector to the existing librarian-report.py so we can generate deltas in addition to full reports.

Revision history for this message
Stuart Bishop (stub) : Posted in a previous version of this proposal
review: Needs Resubmitting
Revision history for this message
Stuart Bishop (stub) wrote :

Adds a date range selector to the existing librarian-report.py so we can generate deltas in addition to full reports.

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

Thanks for making the effort to use run_script... makes for much easier code.

I still hate you for using slashes in dates, but if you really want it this way, all that really matters is that the db accepts it.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/canonical/librarian/tests/librarian-report.txt'
--- lib/canonical/librarian/tests/librarian-report.txt 2009-12-15 05:29:47 +0000
+++ lib/canonical/librarian/tests/librarian-report.txt 2009-12-23 04:36:17 +0000
@@ -1,18 +1,31 @@
1We have a report that we run as necessary to see what is using the Librarian1We have a report that we run as necessary to see what is using the Librarian
2storage.2storage.
33
4 >>> import os.path4 >>> from canonical.launchpad.ftests.script import run_script
5 >>> from canonical.config import config5 >>> script = 'scripts/librarian-report.py'
6 >>> path = os.path.join(config.root, 'scripts', 'librarian-report.py')6
77 >>> rv, out, err = run_script(script)
8 >>> import subprocess8 >>> print rv
9 >>> proc = subprocess.Popen(9 0
10 ... path, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,10 >>> print err
11 ... stdin=subprocess.PIPE)11 >>> print '\n' + out
12 >>> (out, ignored) = proc.communicate()12 <BLANKLINE>
13 >>> print '\n' + out13 ...
14 <BLANKLINE>14 133 bytes hwsubmission in 2 files
15 ...15 ...
16 0 bytes messageapproval in 0 files16
17 ...17
18We can filter on date to produce deltas.
19
20 >>> rv, out, err = run_script(
21 ... script, ['--from=2005/01/01', '--until=2005/12/31'])
22 >>> print rv
23 0
24 >>> print err
25 >>> print '\n' + out
26 <BLANKLINE>
27 ...
28 0 bytes hwsubmission in 0 files
29 ...
30
1831
1932
=== modified file 'scripts/librarian-report.py'
--- scripts/librarian-report.py 2009-12-15 05:29:30 +0000
+++ scripts/librarian-report.py 2009-12-23 04:36:17 +0000
@@ -13,17 +13,41 @@
13from optparse import OptionParser13from optparse import OptionParser
14import sys14import sys
1515
16from canonical.database.sqlbase import connect, quoteIdentifier16from canonical.database.sqlbase import connect, quoteIdentifier, sqlvalues
17from canonical.database.postgresql import listReferences17from canonical.database.postgresql import listReferences
18from canonical.launchpad.scripts import db_options18from canonical.launchpad.scripts import db_options
1919
2020
21def main():21def main():
22 parser = OptionParser()22 parser = OptionParser()
23
23 db_options(parser)24 db_options(parser)
25 parser.add_option(
26 "-f", "--from", dest="from_date", default=None,
27 metavar="DATE", help="Only count new files since DATE (yyyy/mm/dd)")
28 parser.add_option(
29 "-u", "--until", dest="until_date", default=None,
30 metavar="DATE", help="Only count new files until DATE (yyyy/mm/dd)")
31
24 options, args = parser.parse_args()32 options, args = parser.parse_args()
25 if len(args) > 0:33 if len(args) > 0:
26 parser.error("Too many command line arguments.")34 parser.error("Too many command line arguments.")
35
36 # Handle date filters. We use LibraryFileContent.datecreated rather
37 # than LibraryFileAlias.datecreated as this report is about actual
38 # disk space usage. A new row in the database linking to a
39 # previously existing file in the Librarian takes up no new space.
40 if options.from_date is not None:
41 from_date = 'AND LFC.datecreated >= %s' % sqlvalues(
42 options.from_date)
43 else:
44 from_date = ''
45 if options.until_date is not None:
46 until_date = 'AND LFC.datecreated <= %s' % sqlvalues(
47 options.until_date)
48 else:
49 until_date = ''
50
27 con = connect(options.dbuser)51 con = connect(options.dbuser)
28 cur = con.cursor()52 cur = con.cursor()
2953
@@ -57,11 +81,12 @@
57 AND (81 AND (
58 LFA.expires IS NULL82 LFA.expires IS NULL
59 OR LFA.expires > CURRENT_TIMESTAMP AT TIME ZONE 'UTC')83 OR LFA.expires > CURRENT_TIMESTAMP AT TIME ZONE 'UTC')
84 %s %s
60 ORDER BY LFC.id85 ORDER BY LFC.id
61 ) AS Whatever86 ) AS Whatever
62 """ % (87 """ % (
63 quoted_referring_table, quoted_referring_table,88 quoted_referring_table, quoted_referring_table,
64 quoted_referring_column))89 quoted_referring_column, from_date, until_date))
65 total_bytes, formatted_size, num_files = cur.fetchone()90 total_bytes, formatted_size, num_files = cur.fetchone()
66 totals.add((total_bytes, referring_table, formatted_size, num_files))91 totals.add((total_bytes, referring_table, formatted_size, num_files))
6792