Merge lp:~jml/launchpad/flush-out-canonical into lp:launchpad
- flush-out-canonical
- Merge into devel
Proposed by
Jonathan Lange
| Status: | Merged |
|---|---|
| Merged at revision: | 12413 |
| Proposed branch: | lp:~jml/launchpad/flush-out-canonical |
| Merge into: | lp:launchpad |
| Diff against target: |
1671 lines (+213/-755) 54 files modified
configs/testrunner/launchpad-lazr.conf (+0/-1) lib/canonical/__init__.py (+5/-19) lib/canonical/autodecorate.py (+0/-29) lib/canonical/base.py (+0/-81) lib/canonical/chunkydiff.py (+0/-249) lib/canonical/config/schema-lazr.conf (+0/-3) lib/canonical/launchpad/database/message.py (+1/-1) lib/canonical/launchpad/doc/autodecorate.txt (+0/-39) lib/canonical/launchpad/doc/old-testing.txt (+0/-6) lib/canonical/launchpad/doc/private-xmlrpc.txt (+1/-1) lib/canonical/launchpad/doc/profiling.txt (+2/-2) lib/canonical/launchpad/doc/xmlrpc-authserver.txt (+1/-1) lib/canonical/launchpad/doc/xmlrpc-selftest.txt (+1/-1) lib/canonical/launchpad/ftests/test_system_documentation.py (+0/-4) lib/canonical/launchpad/helpers.py (+3/-6) lib/canonical/launchpad/pagetests/standalone/xx-opstats.txt (+1/-1) lib/canonical/launchpad/scripts/debsync.py (+1/-1) lib/canonical/launchpad/scripts/logger.py (+2/-3) lib/canonical/launchpad/testing/pages.py (+1/-2) lib/canonical/launchpad/testing/systemdocs.py (+0/-23) lib/canonical/launchpad/tests/test_chunkydiff_setting.py (+0/-26) lib/canonical/librarian/tests/test_upload.py (+0/-3) lib/canonical/tests/chunkydiff.txt (+0/-202) lib/canonical/tests/test_base.py (+0/-9) lib/canonical/tests/test_chunkydiff.py (+0/-9) lib/lp/answers/doc/person.txt (+1/-1) lib/lp/answers/doc/questionsets.txt (+1/-1) lib/lp/app/stories/launchpad-root/site-search.txt (+1/-1) lib/lp/archivepublisher/utils.py (+1/-1) lib/lp/archiveuploader/dscfile.py (+1/-1) lib/lp/archiveuploader/nascentuploadfile.py (+1/-2) lib/lp/archiveuploader/utils.py (+1/-1) lib/lp/bugs/doc/bugtracker-tokens.txt (+1/-1) lib/lp/bugs/doc/malone-xmlrpc.txt (+1/-1) lib/lp/bugs/externalbugtracker/bugzilla.py (+1/-1) lib/lp/bugs/stories/bugtracker/xx-bugtracker-handshake-tokens.txt (+1/-1) lib/lp/buildmaster/model/buildfarmjobbehavior.py (+1/-1) lib/lp/code/doc/branch-xmlrpc.txt (+1/-1) lib/lp/code/doc/xmlrpc-codeimport-scheduler.txt (+1/-1) lib/lp/registry/tests/test_doc.py (+1/-1) lib/lp/registry/tests/test_xmlrpc.py (+1/-1) lib/lp/registry/xmlrpc/mailinglist.py (+1/-1) lib/lp/services/memcache/tales.py (+2/-2) lib/lp/services/profile/__init__.py (+2/-2) lib/lp/services/profile/profile.py (+1/-1) lib/lp/services/tests/test_encoding.py (+2/-2) lib/lp/services/tests/test_utils.py (+85/-1) lib/lp/services/utils.py (+67/-1) lib/lp/soyuz/model/queue.py (+1/-1) lib/lp/soyuz/scripts/gina/packages.py (+1/-1) lib/lp/testing/factory.py (+1/-1) lib/lp/testing/xmlrpc.py (+6/-2) lib/lp_sitecustomize.py (+9/-0) utilities/migrater/file-ownership.txt (+0/-1) |
| To merge this branch: | bzr merge lp:~jml/launchpad/flush-out-canonical |
| Related bugs: |
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| Jeroen T. Vermeulen (community) | code | Approve | |
|
Review via email:
|
|||
Commit message
[r=jtv][no-qa] Move top-level modules from canonical into various places in lp. Deletes chunkydiff.
Description of the change
This branch flushes out a lot of the modules that were living in the 'canonical' package. Easiest to take them module-by-module.
autodecorate:
* moved to lp.services.utils
* doctest replaced with unit tests
base:
* moved to lp.services.utils
* doctest replaced with unit tests
* inspired new compress_hash method, changed all the call sites to use that
chunkydiff:
* Only used to display different output for page tests
* Saw the output wasn't that different, so deleted it and all its kind.
encoding:
* Moved to lp.services
functional:
* Only contained xmlrpc stuff, so moved to lp.services.xmlrpc
__init__:
* Moved the content to lp_sitecustomize (which is where it belongs)
* Added docstring
mem:
* Moved to lp.services.
To post a comment you must log in.
Revision history for this message
| Jonathan Lange (jml) wrote : | # |
I've cleared up the misspelling, made base() reject negative numbers, and use the proper 'deprecate' tag for the canonical package docstring.
Not going to going the base conversion without deeper though. Can't fix the comment because I don't know which is meant.
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
| 1 | === modified file 'configs/testrunner/launchpad-lazr.conf' |
| 2 | --- configs/testrunner/launchpad-lazr.conf 2011-02-17 14:07:09 +0000 |
| 3 | +++ configs/testrunner/launchpad-lazr.conf 2011-02-18 15:29:53 +0000 |
| 4 | @@ -6,7 +6,6 @@ |
| 5 | extends: ../development/launchpad-lazr.conf |
| 6 | |
| 7 | [canonical] |
| 8 | -chunkydiff: False |
| 9 | cron_control_url: file:lib/lp/services/scripts/tests/cronscripts.ini |
| 10 | |
| 11 | [archivepublisher] |
| 12 | |
| 13 | === modified file 'lib/canonical/__init__.py' |
| 14 | --- lib/canonical/__init__.py 2010-03-30 08:50:10 +0000 |
| 15 | +++ lib/canonical/__init__.py 2011-02-18 15:29:53 +0000 |
| 16 | @@ -1,21 +1,7 @@ |
| 17 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 18 | +# Copyright 2009-2011 Canonical Ltd. This software is licensed under the |
| 19 | # GNU Affero General Public License version 3 (see the file LICENSE). |
| 20 | |
| 21 | -# This is the python package that defines the 'canonical' package namespace. |
| 22 | - |
| 23 | -# We currently only translate launchpad specific stuff using the Zope3 i18n |
| 24 | -# routines, so this is not needed. |
| 25 | -#from zope.i18n.messageid import MessageFactory |
| 26 | -#_ = MessageFactory("canonical") |
| 27 | - |
| 28 | -# Filter all deprecation warnings for Zope 3.6, which eminate from |
| 29 | -# the zope package. |
| 30 | -import warnings |
| 31 | -filter_pattern = '.*(Zope 3.6|provide.*global site manager).*' |
| 32 | -warnings.filterwarnings( |
| 33 | - 'ignore', filter_pattern, category=DeprecationWarning) |
| 34 | - |
| 35 | -# XXX wgrant 2010-03-30 bug=551510: |
| 36 | -# Also filter apt_pkg warnings, since Lucid's python-apt has a new API. |
| 37 | -warnings.filterwarnings( |
| 38 | - 'ignore', '.*apt_pkg.*', category=DeprecationWarning) |
| 39 | +"""The canonical namespace package. |
| 40 | + |
| 41 | +:deprecated: New code should go into the `lp` package. |
| 42 | +""" |
| 43 | |
| 44 | === removed file 'lib/canonical/autodecorate.py' |
| 45 | --- lib/canonical/autodecorate.py 2009-06-25 05:30:52 +0000 |
| 46 | +++ lib/canonical/autodecorate.py 1970-01-01 00:00:00 +0000 |
| 47 | @@ -1,29 +0,0 @@ |
| 48 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 49 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 50 | - |
| 51 | -"""Metaclass to automatically decorate methods.""" |
| 52 | - |
| 53 | -from types import FunctionType |
| 54 | - |
| 55 | - |
| 56 | -__metaclass__ = type |
| 57 | -__all__ = ['AutoDecorate'] |
| 58 | - |
| 59 | - |
| 60 | -def AutoDecorate(*decorators): |
| 61 | - """Factory to generate metaclasses that automatically apply decorators.""" |
| 62 | - |
| 63 | - class AutoDecorateMetaClass(type): |
| 64 | - def __new__(cls, class_name, bases, class_dict): |
| 65 | - new_class_dict = {} |
| 66 | - for name, value in class_dict.items(): |
| 67 | - if type(value) == FunctionType: |
| 68 | - for decorator in decorators: |
| 69 | - value = decorator(value) |
| 70 | - assert callable(value), ( |
| 71 | - "Decorator %s didn't return a callable." |
| 72 | - % repr(decorator)) |
| 73 | - new_class_dict[name] = value |
| 74 | - return type.__new__(cls, class_name, bases, new_class_dict) |
| 75 | - |
| 76 | - return AutoDecorateMetaClass |
| 77 | |
| 78 | === removed file 'lib/canonical/base.py' |
| 79 | --- lib/canonical/base.py 2010-02-09 00:17:40 +0000 |
| 80 | +++ lib/canonical/base.py 1970-01-01 00:00:00 +0000 |
| 81 | @@ -1,81 +0,0 @@ |
| 82 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 83 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 84 | - |
| 85 | -"""Convert numbers to an arbitrary base numbering scheme |
| 86 | - |
| 87 | -This file is based on work from the Python Cookbook and is under the Python |
| 88 | -license. |
| 89 | - |
| 90 | -""" |
| 91 | -__all__ = ['base'] |
| 92 | - |
| 93 | -import string |
| 94 | -abc = string.digits + string.ascii_letters |
| 95 | - |
| 96 | -def base(number, radix): |
| 97 | - """Inverse function to int(str,radix) and long(str,radix) |
| 98 | - |
| 99 | - >>> base(35, 36) |
| 100 | - 'z' |
| 101 | - |
| 102 | - We can go higher than base 36, but we do this by using upper |
| 103 | - case letters. This is not a standard representation, but |
| 104 | - useful for using this method as a compression algorithm. |
| 105 | - |
| 106 | - >>> base(61, 62) |
| 107 | - 'Z' |
| 108 | - |
| 109 | - We get identical results to the hex builtin, without the 0x prefix |
| 110 | - |
| 111 | - >>> [i for i in range(0, 5000, 9) if hex(i)[2:] != base(i, 16)] |
| 112 | - [] |
| 113 | - |
| 114 | - This method is useful for shrinking sha1 and md5 hashes, but keeping |
| 115 | - them in simple ASCII suitable for URL's etc. |
| 116 | - |
| 117 | - >>> import hashlib |
| 118 | - >>> s = hashlib.sha1('foo').hexdigest() |
| 119 | - >>> s |
| 120 | - '0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33' |
| 121 | - >>> i = long(s, 16) |
| 122 | - >>> i |
| 123 | - 68123873083688143418383284816464454849230703155L |
| 124 | - >>> base(i, 62) |
| 125 | - '1HyPQr2xj1nmnkQXBCJXUdQoy5l' |
| 126 | - >>> base(int(hashlib.md5('foo').hexdigest(), 16), 62) |
| 127 | - '5fX649Stem9fET0lD46zVe' |
| 128 | - |
| 129 | - A sha1 hash can be compressed to 27 characters or less |
| 130 | - >>> len(base(long('F'*40, 16), 62)) |
| 131 | - 27 |
| 132 | - |
| 133 | - A md5 hash can be compressed to 22 characters or less |
| 134 | - >>> len(base(long('F'*32, 16), 62)) |
| 135 | - 22 |
| 136 | - |
| 137 | - """ |
| 138 | - if not 2 <= radix <= 62: |
| 139 | - raise ValueError, "radix must be in 2..62" |
| 140 | - |
| 141 | - result = [] |
| 142 | - addon = result.append |
| 143 | - if number < 0: |
| 144 | - number = -number |
| 145 | - addon('-') |
| 146 | - elif number == 0: |
| 147 | - addon('0') |
| 148 | - |
| 149 | - _divmod, _abc = divmod, abc |
| 150 | - while number: |
| 151 | - number, rdigit = _divmod(number, radix) |
| 152 | - addon(_abc[rdigit]) |
| 153 | - |
| 154 | - result.reverse() |
| 155 | - return ''.join(result) |
| 156 | - |
| 157 | -def _test(): |
| 158 | - import doctest, base |
| 159 | - doctest.testmod(base) |
| 160 | - |
| 161 | -if __name__ == '__main__': |
| 162 | - _test() |
| 163 | |
| 164 | === removed file 'lib/canonical/chunkydiff.py' |
| 165 | --- lib/canonical/chunkydiff.py 2009-06-25 05:30:52 +0000 |
| 166 | +++ lib/canonical/chunkydiff.py 1970-01-01 00:00:00 +0000 |
| 167 | @@ -1,249 +0,0 @@ |
| 168 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 169 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 170 | - |
| 171 | -"""Chunky diffs. |
| 172 | - |
| 173 | -Useful for page tests that have elisions. |
| 174 | -""" |
| 175 | - |
| 176 | -import re |
| 177 | - |
| 178 | -__metaclass__ = type |
| 179 | - |
| 180 | -def elided_source(tested, actual, debug=False, show=False, |
| 181 | - normalize_whitespace=False): |
| 182 | - if debug: |
| 183 | - import pdb; pdb.set_trace() |
| 184 | - chunks = tested.split('...') |
| 185 | - |
| 186 | - previous_chunk = None |
| 187 | - chunk = None |
| 188 | - Unknown = None |
| 189 | - currentpos = 0 |
| 190 | - results = [] |
| 191 | - for next_chunk in chunks + [None]: |
| 192 | - if chunk is None: |
| 193 | - chunk = next_chunk |
| 194 | - continue |
| 195 | - if chunk != '': |
| 196 | - if previous_chunk is None: |
| 197 | - chunk_starts_with_ellipsis = False |
| 198 | - else: |
| 199 | - chunk_starts_with_ellipsis = True |
| 200 | - if next_chunk is None: |
| 201 | - chunk_ends_with_ellipsis = False |
| 202 | - else: |
| 203 | - chunk_ends_with_ellipsis = True |
| 204 | - |
| 205 | - result = find_chunk( |
| 206 | - chunk, actual[currentpos:], |
| 207 | - anchor_start=not chunk_starts_with_ellipsis, |
| 208 | - anchor_end=not chunk_ends_with_ellipsis, |
| 209 | - debug=debug, show=show, |
| 210 | - normalize_whitespace=normalize_whitespace) |
| 211 | - if result is None: |
| 212 | - results.append(None) |
| 213 | - else: |
| 214 | - string, startofremainder = result |
| 215 | - currentpos += startofremainder |
| 216 | - # XXX: ddaa 2005-03-25: |
| 217 | - # Off by one. Should be += startofremainder + 1 |
| 218 | - results.append(ResultChunk(string, currentpos)) |
| 219 | - previous_chunk, chunk = chunk, next_chunk |
| 220 | - |
| 221 | - starts_with_ellipsis = chunks[0] == '' |
| 222 | - ends_with_ellipsis = chunks[-1] == '' |
| 223 | - |
| 224 | - resultsummary = ''.join( |
| 225 | - [mnemonic_for_result(result) for result in results] |
| 226 | - ) |
| 227 | - if re.match('^N+$', resultsummary): |
| 228 | - # If all results are None... |
| 229 | - output = actual |
| 230 | - elif re.match('^S+$', resultsummary): |
| 231 | - # If no results are None... |
| 232 | - output = '...'.join([result.text for result in results]) |
| 233 | - if starts_with_ellipsis: |
| 234 | - output = '...' + output |
| 235 | - if ends_with_ellipsis: |
| 236 | - output = output + '...' |
| 237 | - elif re.match('^S+N+$', resultsummary) and ends_with_ellipsis: |
| 238 | - # Where there are one or more None values at the end of results, |
| 239 | - # and ends_with_ellipsis, we can end without an ellipsis while |
| 240 | - # including the remainder of 'actual' from the end of the last |
| 241 | - # matched chunk. |
| 242 | - |
| 243 | - # Find last non-None result. |
| 244 | - for result in reversed(results): |
| 245 | - if result is not None: |
| 246 | - break |
| 247 | - # Get the remainder value from it. |
| 248 | - if starts_with_ellipsis: |
| 249 | - output = '...' |
| 250 | - else: |
| 251 | - # XXX: ddaa 2005-03-25: Test this code path! |
| 252 | - output = '' |
| 253 | - last_result = None |
| 254 | - for result in results: |
| 255 | - if result is not None: |
| 256 | - output += result.text |
| 257 | - last_result = result |
| 258 | - else: |
| 259 | - output += actual[last_result.remainderpos:] |
| 260 | - break |
| 261 | - |
| 262 | - else: |
| 263 | - # XXX: ddaa 2005-03-25: Test this code path! |
| 264 | - output = actual |
| 265 | - |
| 266 | - return output |
| 267 | - |
| 268 | -class ResultChunk: |
| 269 | - |
| 270 | - def __init__(self, text, remainderpos): |
| 271 | - self.text = text |
| 272 | - self.remainderpos = remainderpos |
| 273 | - |
| 274 | - |
| 275 | -def reversed(seq): |
| 276 | - L = list(seq) |
| 277 | - L.reverse() |
| 278 | - return L |
| 279 | - |
| 280 | -def mnemonic_for_result(result): |
| 281 | - """Returns 'N' if result is None, otherwise 'S'.""" |
| 282 | - if result is None: |
| 283 | - return 'N' |
| 284 | - else: |
| 285 | - return 'S' |
| 286 | - |
| 287 | -def find_chunk(chunk, actual, anchor_start=False, anchor_end=False, |
| 288 | - debug=False, show=False, normalize_whitespace=False): |
| 289 | - if debug: |
| 290 | - import pdb; pdb.set_trace() |
| 291 | - if not anchor_start: |
| 292 | - # Find the start of the chunk. |
| 293 | - beginning = '' |
| 294 | - beginning_for_regex = '' |
| 295 | - manyfound = False |
| 296 | - for char in chunk: |
| 297 | - if normalize_whitespace and char.isspace(): |
| 298 | - if beginning_for_regex[-2:] != r'\s': |
| 299 | - beginning_for_regex += r'\s' |
| 300 | - else: |
| 301 | - beginning_for_regex += re.escape(char) |
| 302 | - beginning += char |
| 303 | - numfound = len(re.findall(beginning_for_regex, actual)) |
| 304 | - #numfound = actual.count(beginning) |
| 305 | - if numfound == 0: |
| 306 | - if manyfound: |
| 307 | - beginning = manyfound_beginning |
| 308 | - beginning_for_regex = manyfound_beginning_for_regex |
| 309 | - if anchor_end: |
| 310 | - beginning_pos = list(re.finditer( |
| 311 | - beginning_for_regex, actual))[-1].start() |
| 312 | - #beginning_pos = actual.rfind(beginning) |
| 313 | - else: |
| 314 | - beginning_pos = re.search( |
| 315 | - beginning_for_regex, actual).start() |
| 316 | - # XXX ddaa 2005-03-25: This should be .span()[1]. |
| 317 | - # Needs a test. |
| 318 | - #beginning_pos = actual.find(beginning) |
| 319 | - break |
| 320 | - else: |
| 321 | - beginning = '' |
| 322 | - beginning_for_regex = '' |
| 323 | - beginning_pos = 0 |
| 324 | - break |
| 325 | - elif numfound == 1: |
| 326 | - beginning_pos = re.search( |
| 327 | - beginning_for_regex, actual).span()[1] |
| 328 | - #beginning_pos = actual.find(beginning) + len(beginning) |
| 329 | - break |
| 330 | - else: |
| 331 | - manyfound = True |
| 332 | - manyfound_beginning = beginning |
| 333 | - manyfound_beginning_for_regex = beginning_for_regex |
| 334 | - else: |
| 335 | - if manyfound: |
| 336 | - if anchor_end: |
| 337 | - beginning_pos = list(re.finditer( |
| 338 | - beginning_for_regex, actual))[-1].start() |
| 339 | - #beginning_pos = actual.rfind(beginning) |
| 340 | - else: |
| 341 | - beginning_pos = re.search( |
| 342 | - beginning_for_regex, actual).start() |
| 343 | - # XXX ddaa 2005-03-25: This should be .span()[1]. |
| 344 | - # Needs a test. |
| 345 | - #beginning_pos = actual.find(beginning) |
| 346 | - else: |
| 347 | - return None |
| 348 | - else: |
| 349 | - beginning_pos = 0 |
| 350 | - beginning = '' |
| 351 | - beginning_for_regex = '' |
| 352 | - |
| 353 | - # Find the end of the chunk. |
| 354 | - end = '' |
| 355 | - end_for_regex = '' |
| 356 | - chunk_with_no_beginning = chunk[len(beginning):] |
| 357 | - if not chunk_with_no_beginning: |
| 358 | - end_pos = beginning_pos |
| 359 | - elif not anchor_end: |
| 360 | - # Remove the beginning from the chunk. |
| 361 | - reversed_chunk = list(chunk_with_no_beginning) |
| 362 | - reversed_chunk.reverse() |
| 363 | - manyfound = False |
| 364 | - for char in reversed_chunk: |
| 365 | - end = char + end |
| 366 | - if normalize_whitespace and char.isspace(): |
| 367 | - if end_for_regex[:2] != r'\s': |
| 368 | - end_for_regex = r'\s' + end_for_regex |
| 369 | - else: |
| 370 | - end_for_regex = re.escape(char) + end_for_regex |
| 371 | - numfound = len(re.findall(end_for_regex, actual)) |
| 372 | - #numfound = actual.count(end) |
| 373 | - if numfound == 0: |
| 374 | - # None found this time around. If we previously found more |
| 375 | - # than one match, then choose the closest to the beginning. |
| 376 | - if manyfound: |
| 377 | - end = manyfound_end |
| 378 | - end_for_regex = manyfound_end_for_regex |
| 379 | - end_pos = re.search(end_for_regex, actual).start() |
| 380 | - #end_pos = actual.find(end, beginning_pos) |
| 381 | - # XXX: ddaa 2005-03-25: |
| 382 | - # This was wrong -- shouldn't be beginning_pos as |
| 383 | - # we've already chopped off the beginning! |
| 384 | - # Or is it? We chopped the beginning of the chunk, |
| 385 | - # not the actual stuff. So, using beginning_pos |
| 386 | - # still holds. Need to chop that off and add on |
| 387 | - # its length. |
| 388 | - break |
| 389 | - else: |
| 390 | - return None |
| 391 | - elif numfound == 1: |
| 392 | - end_pos = re.search(end_for_regex, actual).start() |
| 393 | - #end_pos = actual.rfind(end) |
| 394 | - # XXX: ddaa 2005-03-25: Only one found, so why not use find()? |
| 395 | - break |
| 396 | - else: |
| 397 | - manyfound = True |
| 398 | - manyfound_end = end |
| 399 | - manyfound_end_for_regex = end_for_regex |
| 400 | - else: |
| 401 | - if manyfound: |
| 402 | - end_pos = re.search(end_for_regex, actual).start() |
| 403 | - else: |
| 404 | - return None |
| 405 | - else: |
| 406 | - end_pos = len(actual) |
| 407 | - end = '' |
| 408 | - end_for_regex = '' |
| 409 | - |
| 410 | - chunk_equivalent = actual[beginning_pos:end_pos] |
| 411 | - if show: |
| 412 | - output = '[%s]%s[%s]' % (beginning, chunk_equivalent, end) |
| 413 | - else: |
| 414 | - output = '%s%s%s' % (beginning, chunk_equivalent, end) |
| 415 | - # XXX: ddaa 2005-03-25: end_pos+1 is the end of chunk_equivalent, not end. |
| 416 | - return (output, end_pos+1) |
| 417 | |
| 418 | === modified file 'lib/canonical/config/schema-lazr.conf' |
| 419 | --- lib/canonical/config/schema-lazr.conf 2011-02-17 14:07:09 +0000 |
| 420 | +++ lib/canonical/config/schema-lazr.conf 2011-02-18 15:29:53 +0000 |
| 421 | @@ -193,9 +193,6 @@ |
| 422 | |
| 423 | [canonical] |
| 424 | # datatype: boolean |
| 425 | -chunkydiff: True |
| 426 | - |
| 427 | -# datatype: boolean |
| 428 | show_tracebacks: False |
| 429 | |
| 430 | # datatype: string |
| 431 | |
| 432 | === modified file 'lib/canonical/launchpad/database/message.py' |
| 433 | --- lib/canonical/launchpad/database/message.py 2011-02-10 01:18:39 +0000 |
| 434 | +++ lib/canonical/launchpad/database/message.py 2011-02-18 15:29:53 +0000 |
| 435 | @@ -61,7 +61,6 @@ |
| 436 | from canonical.database.datetimecol import UtcDateTimeCol |
| 437 | from canonical.database.enumcol import EnumCol |
| 438 | from canonical.database.sqlbase import SQLBase |
| 439 | -from canonical.encoding import guess as ensure_unicode |
| 440 | from canonical.launchpad.helpers import get_filename_from_message_id |
| 441 | from canonical.launchpad.interfaces.librarian import ( |
| 442 | ILibraryFileAliasSet, |
| 443 | @@ -83,6 +82,7 @@ |
| 444 | PersonCreationRationale, |
| 445 | validate_public_person, |
| 446 | ) |
| 447 | +from lp.services.encoding import guess as ensure_unicode |
| 448 | from lp.services.job.model.job import Job |
| 449 | from lp.services.propertycache import cachedproperty |
| 450 | |
| 451 | |
| 452 | === removed file 'lib/canonical/launchpad/doc/autodecorate.txt' |
| 453 | --- lib/canonical/launchpad/doc/autodecorate.txt 2009-03-27 03:29:31 +0000 |
| 454 | +++ lib/canonical/launchpad/doc/autodecorate.txt 1970-01-01 00:00:00 +0000 |
| 455 | @@ -1,39 +0,0 @@ |
| 456 | - |
| 457 | -= AutoDecorate = |
| 458 | - |
| 459 | -AutoDecorate is a metaclass factory that can be used to make a class |
| 460 | -implicitely wrap all its methods with one or more decorators. |
| 461 | - |
| 462 | - >>> def decorator_1(func): |
| 463 | - ... def decorated_1(*args, **kw): |
| 464 | - ... print 'Decorated 1' |
| 465 | - ... return func(*args, **kw) |
| 466 | - ... return decorated_1 |
| 467 | - |
| 468 | - >>> def decorator_2(func): |
| 469 | - ... def decorated_2(*args, **kw): |
| 470 | - ... print 'Decorated 2' |
| 471 | - ... return func(*args, **kw) |
| 472 | - ... return decorated_2 |
| 473 | - |
| 474 | - >>> from canonical.autodecorate import AutoDecorate |
| 475 | - |
| 476 | - >>> class MyClass(object): |
| 477 | - ... __metaclass__ = AutoDecorate(decorator_1, decorator_2) |
| 478 | - ... def method_a(self): |
| 479 | - ... print 'Method A' |
| 480 | - ... def method_b(self): |
| 481 | - ... print 'Method B' |
| 482 | - |
| 483 | - >>> obj = MyClass() |
| 484 | - |
| 485 | - >>> obj.method_a() |
| 486 | - Decorated 2 |
| 487 | - Decorated 1 |
| 488 | - Method A |
| 489 | - |
| 490 | - >>> obj.method_b() |
| 491 | - Decorated 2 |
| 492 | - Decorated 1 |
| 493 | - Method B |
| 494 | - |
| 495 | |
| 496 | === modified file 'lib/canonical/launchpad/doc/old-testing.txt' |
| 497 | --- lib/canonical/launchpad/doc/old-testing.txt 2010-12-24 09:28:21 +0000 |
| 498 | +++ lib/canonical/launchpad/doc/old-testing.txt 2011-02-18 15:29:53 +0000 |
| 499 | @@ -18,12 +18,6 @@ |
| 500 | zope, we should not be testing it with the full Z3 functional test |
| 501 | harness). |
| 502 | |
| 503 | -canonical.functional.FunctionalTestCase |
| 504 | ---------------------------------------- |
| 505 | - |
| 506 | -This is a customised zope3 FunctionalTestCase and should be used when you |
| 507 | -simply need the zope3 utilities etc available. |
| 508 | - |
| 509 | PgTestSetup |
| 510 | ----------- |
| 511 | |
| 512 | |
| 513 | === modified file 'lib/canonical/launchpad/doc/private-xmlrpc.txt' |
| 514 | --- lib/canonical/launchpad/doc/private-xmlrpc.txt 2010-10-09 16:36:22 +0000 |
| 515 | +++ lib/canonical/launchpad/doc/private-xmlrpc.txt 2011-02-18 15:29:53 +0000 |
| 516 | @@ -15,7 +15,7 @@ |
| 517 | external XML-RPC port. |
| 518 | |
| 519 | >>> import xmlrpclib |
| 520 | - >>> from canonical.functional import XMLRPCTestTransport |
| 521 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 522 | >>> external_api = xmlrpclib.ServerProxy( |
| 523 | ... public_root + 'mailinglists/', |
| 524 | ... transport=XMLRPCTestTransport()) |
| 525 | |
| 526 | === modified file 'lib/canonical/launchpad/doc/profiling.txt' |
| 527 | --- lib/canonical/launchpad/doc/profiling.txt 2010-10-22 10:24:18 +0000 |
| 528 | +++ lib/canonical/launchpad/doc/profiling.txt 2011-02-18 15:29:53 +0000 |
| 529 | @@ -166,8 +166,8 @@ |
| 530 | useful to try to figure out what requests are causing the memory usage of the |
| 531 | server to increase. |
| 532 | |
| 533 | -This is not blessed for production use at this time: the implementation |
| 534 | -relies on lib/canonical/mem.py, which as of this writing warns in its |
| 535 | +This is not blessed for production use at this time: the implementation relies |
| 536 | +on lib/lp/services/profile/mem.py, which as of this writing warns in its |
| 537 | docstring that "[n]one of this should be in day-to-day use." We should |
| 538 | document the source of these concerns and evaluate them before using it in |
| 539 | production. Staging may be more acceptable. |
| 540 | |
| 541 | === modified file 'lib/canonical/launchpad/doc/xmlrpc-authserver.txt' |
| 542 | --- lib/canonical/launchpad/doc/xmlrpc-authserver.txt 2010-10-19 18:44:31 +0000 |
| 543 | +++ lib/canonical/launchpad/doc/xmlrpc-authserver.txt 2011-02-18 15:29:53 +0000 |
| 544 | @@ -28,7 +28,7 @@ |
| 545 | about users. |
| 546 | |
| 547 | >>> import xmlrpclib |
| 548 | - >>> from canonical.functional import XMLRPCTestTransport |
| 549 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 550 | >>> authserver= xmlrpclib.ServerProxy( |
| 551 | ... 'http://xmlrpc-private.launchpad.dev:8087/authserver', |
| 552 | ... transport=XMLRPCTestTransport()) |
| 553 | |
| 554 | === modified file 'lib/canonical/launchpad/doc/xmlrpc-selftest.txt' |
| 555 | --- lib/canonical/launchpad/doc/xmlrpc-selftest.txt 2010-11-08 14:16:17 +0000 |
| 556 | +++ lib/canonical/launchpad/doc/xmlrpc-selftest.txt 2011-02-18 15:29:53 +0000 |
| 557 | @@ -16,7 +16,7 @@ |
| 558 | which talks with the publisher directly. |
| 559 | |
| 560 | >>> import xmlrpclib |
| 561 | - >>> from canonical.functional import XMLRPCTestTransport |
| 562 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 563 | >>> selftest = xmlrpclib.ServerProxy( |
| 564 | ... 'http://xmlrpc.launchpad.dev/', transport=XMLRPCTestTransport()) |
| 565 | >>> selftest.concatenate('foo', 'bar') |
| 566 | |
| 567 | === modified file 'lib/canonical/launchpad/ftests/test_system_documentation.py' |
| 568 | --- lib/canonical/launchpad/ftests/test_system_documentation.py 2010-12-13 15:25:03 +0000 |
| 569 | +++ lib/canonical/launchpad/ftests/test_system_documentation.py 2011-02-18 15:29:53 +0000 |
| 570 | @@ -194,10 +194,6 @@ |
| 571 | 'old-testing.txt': LayeredDocFileSuite( |
| 572 | '../doc/old-testing.txt', layer=FunctionalLayer), |
| 573 | |
| 574 | - 'autodecorate.txt': |
| 575 | - LayeredDocFileSuite('../doc/autodecorate.txt', layer=BaseLayer), |
| 576 | - |
| 577 | - |
| 578 | # And this test want minimal environment too. |
| 579 | 'package-relationship.txt': LayeredDocFileSuite( |
| 580 | '../doc/package-relationship.txt', |
| 581 | |
| 582 | === modified file 'lib/canonical/launchpad/helpers.py' |
| 583 | --- lib/canonical/launchpad/helpers.py 2010-12-02 16:13:51 +0000 |
| 584 | +++ lib/canonical/launchpad/helpers.py 2011-02-18 15:29:53 +0000 |
| 585 | @@ -20,16 +20,15 @@ |
| 586 | import tarfile |
| 587 | import warnings |
| 588 | |
| 589 | -import gettextpo |
| 590 | from zope.component import getUtility |
| 591 | from zope.security.interfaces import ForbiddenAttribute |
| 592 | |
| 593 | -import canonical |
| 594 | from canonical.launchpad.webapp.interfaces import ILaunchBag |
| 595 | from lp.services.geoip.interfaces import ( |
| 596 | IRequestLocalLanguages, |
| 597 | IRequestPreferredLanguages, |
| 598 | ) |
| 599 | +from lp.services.utils import compress_hash |
| 600 | |
| 601 | |
| 602 | def text_replaced(text, replacements, _cache={}): |
| 603 | @@ -442,9 +441,7 @@ |
| 604 | |
| 605 | It generates a file name that's not easily guessable. |
| 606 | """ |
| 607 | - return '%s.msg' % ( |
| 608 | - canonical.base.base( |
| 609 | - long(hashlib.sha1(message_id).hexdigest(), 16), 62)) |
| 610 | + return '%s.msg' % compress_hash(hashlib.sha1(message_id)) |
| 611 | |
| 612 | |
| 613 | def intOrZero(value): |
| 614 | @@ -499,7 +496,7 @@ |
| 615 | The templates are located in 'lib/canonical/launchpad/emailtemplates'. |
| 616 | """ |
| 617 | if app is None: |
| 618 | - base = os.path.dirname(canonical.launchpad.__file__) |
| 619 | + base = os.path.dirname(__file__) |
| 620 | fullpath = os.path.join(base, 'emailtemplates', filename) |
| 621 | else: |
| 622 | import lp |
| 623 | |
| 624 | === modified file 'lib/canonical/launchpad/pagetests/standalone/xx-opstats.txt' |
| 625 | --- lib/canonical/launchpad/pagetests/standalone/xx-opstats.txt 2010-10-18 22:24:59 +0000 |
| 626 | +++ lib/canonical/launchpad/pagetests/standalone/xx-opstats.txt 2011-02-18 15:29:53 +0000 |
| 627 | @@ -4,7 +4,7 @@ |
| 628 | We can access them via XML-RPC: |
| 629 | |
| 630 | >>> import xmlrpclib |
| 631 | - >>> from canonical.functional import XMLRPCTestTransport |
| 632 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 633 | >>> lp_xmlrpc = xmlrpclib.ServerProxy( |
| 634 | ... 'http://xmlrpc.launchpad.dev/+opstats', |
| 635 | ... transport=XMLRPCTestTransport() |
| 636 | |
| 637 | === modified file 'lib/canonical/launchpad/scripts/debsync.py' |
| 638 | --- lib/canonical/launchpad/scripts/debsync.py 2010-08-20 20:31:18 +0000 |
| 639 | +++ lib/canonical/launchpad/scripts/debsync.py 2011-02-18 15:29:53 +0000 |
| 640 | @@ -17,7 +17,6 @@ |
| 641 | from zope.component import getUtility |
| 642 | |
| 643 | from canonical.database.sqlbase import flush_database_updates |
| 644 | -from canonical.encoding import guess as ensure_unicode |
| 645 | from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities |
| 646 | from canonical.launchpad.interfaces.message import ( |
| 647 | IMessageSet, |
| 648 | @@ -31,6 +30,7 @@ |
| 649 | from lp.bugs.interfaces.bugwatch import IBugWatchSet |
| 650 | from lp.bugs.interfaces.cve import ICveSet |
| 651 | from lp.bugs.scripts import debbugs |
| 652 | +from lp.services.encoding import guess as ensure_unicode |
| 653 | |
| 654 | |
| 655 | def bug_filter(bug, previous_import_set, target_bugs, target_package_set, |
| 656 | |
| 657 | === modified file 'lib/canonical/launchpad/scripts/logger.py' |
| 658 | --- lib/canonical/launchpad/scripts/logger.py 2011-02-08 15:19:24 +0000 |
| 659 | +++ lib/canonical/launchpad/scripts/logger.py 2011-02-18 15:29:53 +0000 |
| 660 | @@ -50,7 +50,6 @@ |
| 661 | from zope.component import getUtility |
| 662 | from zope.exceptions.log import Formatter |
| 663 | |
| 664 | -from canonical.base import base |
| 665 | from canonical.config import config |
| 666 | from canonical.launchpad.webapp.errorlog import ( |
| 667 | globalErrorUtility, |
| 668 | @@ -61,6 +60,7 @@ |
| 669 | UploadFailed, |
| 670 | ) |
| 671 | from lp.services.log import loglevels |
| 672 | +from lp.services.utils import compress_hash |
| 673 | |
| 674 | # Reexport our custom loglevels for old callsites. These callsites |
| 675 | # should be importing the symbols from lp.services.log.loglevels |
| 676 | @@ -149,8 +149,7 @@ |
| 677 | |
| 678 | expiry = datetime.now().replace(tzinfo=utc) + timedelta(days=90) |
| 679 | try: |
| 680 | - filename = base(long( |
| 681 | - hashlib.sha1(traceback).hexdigest(), 16), 62) + '.txt' |
| 682 | + filename = compress_hash(hashlib.sha1(traceback)) + '.txt' |
| 683 | url = librarian.remoteAddFile( |
| 684 | filename, len(traceback), StringIO(traceback), |
| 685 | 'text/plain;charset=%s' % sys.getdefaultencoding(), |
| 686 | |
| 687 | === modified file 'lib/canonical/launchpad/testing/pages.py' |
| 688 | --- lib/canonical/launchpad/testing/pages.py 2011-02-04 14:41:18 +0000 |
| 689 | +++ lib/canonical/launchpad/testing/pages.py 2011-02-18 15:29:53 +0000 |
| 690 | @@ -52,7 +52,6 @@ |
| 691 | ) |
| 692 | from canonical.launchpad.testing.systemdocs import ( |
| 693 | LayeredDocFileSuite, |
| 694 | - SpecialOutputChecker, |
| 695 | stop, |
| 696 | strip_prefix, |
| 697 | ) |
| 698 | @@ -918,7 +917,7 @@ |
| 699 | unnumberedfilenames = sorted(unnumberedfilenames) |
| 700 | |
| 701 | suite = unittest.TestSuite() |
| 702 | - checker = SpecialOutputChecker() |
| 703 | + checker = doctest.OutputChecker() |
| 704 | # Add unnumbered tests to the suite individually. |
| 705 | if unnumberedfilenames: |
| 706 | suite.addTest(LayeredDocFileSuite( |
| 707 | |
| 708 | === modified file 'lib/canonical/launchpad/testing/systemdocs.py' |
| 709 | --- lib/canonical/launchpad/testing/systemdocs.py 2010-11-08 12:52:43 +0000 |
| 710 | +++ lib/canonical/launchpad/testing/systemdocs.py 2011-02-18 15:29:53 +0000 |
| 711 | @@ -7,7 +7,6 @@ |
| 712 | __all__ = [ |
| 713 | 'default_optionflags', |
| 714 | 'LayeredDocFileSuite', |
| 715 | - 'SpecialOutputChecker', |
| 716 | 'setUp', |
| 717 | 'setGlobs', |
| 718 | 'stop', |
| 719 | @@ -26,7 +25,6 @@ |
| 720 | from zope.component import getUtility |
| 721 | from zope.testing.loggingsupport import Handler |
| 722 | |
| 723 | -from canonical.chunkydiff import elided_source |
| 724 | from canonical.config import config |
| 725 | from canonical.database.sqlbase import flush_database_updates |
| 726 | from canonical.launchpad.interfaces.launchpad import ILaunchBag |
| 727 | @@ -141,27 +139,6 @@ |
| 728 | return suite |
| 729 | |
| 730 | |
| 731 | -class SpecialOutputChecker(doctest.OutputChecker): |
| 732 | - """An OutputChecker that runs the 'chunkydiff' checker if appropriate.""" |
| 733 | - def output_difference(self, example, got, optionflags): |
| 734 | - if config.canonical.chunkydiff is False: |
| 735 | - return doctest.OutputChecker.output_difference( |
| 736 | - self, example, got, optionflags) |
| 737 | - |
| 738 | - if optionflags & doctest.ELLIPSIS: |
| 739 | - normalize_whitespace = optionflags & doctest.NORMALIZE_WHITESPACE |
| 740 | - newgot = elided_source(example.want, got, |
| 741 | - normalize_whitespace=normalize_whitespace) |
| 742 | - if newgot == example.want: |
| 743 | - # There was no difference. May be an error in |
| 744 | - # elided_source(). In any case, return the whole thing. |
| 745 | - newgot = got |
| 746 | - else: |
| 747 | - newgot = got |
| 748 | - return doctest.OutputChecker.output_difference( |
| 749 | - self, example, newgot, optionflags) |
| 750 | - |
| 751 | - |
| 752 | def ordered_dict_as_string(dict): |
| 753 | """Return the contents of a dict as an ordered string. |
| 754 | |
| 755 | |
| 756 | === removed file 'lib/canonical/launchpad/tests/test_chunkydiff_setting.py' |
| 757 | --- lib/canonical/launchpad/tests/test_chunkydiff_setting.py 2010-08-20 20:31:18 +0000 |
| 758 | +++ lib/canonical/launchpad/tests/test_chunkydiff_setting.py 1970-01-01 00:00:00 +0000 |
| 759 | @@ -1,26 +0,0 @@ |
| 760 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 761 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 762 | - |
| 763 | -"""Fail if the chunkydiff option is off. |
| 764 | - |
| 765 | -This ensures that people can't accidently commit the main config file with |
| 766 | -this option turned off to rocketfuel. |
| 767 | -""" |
| 768 | -__metaclass__ = type |
| 769 | - |
| 770 | -import unittest |
| 771 | - |
| 772 | -from canonical.config import config |
| 773 | - |
| 774 | - |
| 775 | -class TestChunkydiffSetting(unittest.TestCase): |
| 776 | - |
| 777 | - def test(self): |
| 778 | - self.failUnless( |
| 779 | - config.canonical.chunkydiff is False, |
| 780 | - 'This test is failing to ensure that the chunkydiff ' |
| 781 | - 'config setting cannot be committed in "on" mode.' |
| 782 | - ) |
| 783 | - |
| 784 | -def test_suite(): |
| 785 | - return unittest.TestLoader().loadTestsFromName(__name__) |
| 786 | |
| 787 | === modified file 'lib/canonical/librarian/tests/test_upload.py' |
| 788 | --- lib/canonical/librarian/tests/test_upload.py 2009-06-25 05:30:52 +0000 |
| 789 | +++ lib/canonical/librarian/tests/test_upload.py 2011-02-18 15:29:53 +0000 |
| 790 | @@ -54,9 +54,6 @@ |
| 791 | """Librarian upload server test helper, process a request and report what |
| 792 | happens. |
| 793 | |
| 794 | - Inspired by the canonical.functional.http function used by the Launchpad |
| 795 | - page tests. |
| 796 | - |
| 797 | Hands a request to a librarian file upload protocol, and prints the reply |
| 798 | from the server, a summary of the file uploaded, and whether the connection |
| 799 | closed, e.g.:: |
| 800 | |
| 801 | === removed file 'lib/canonical/tests/chunkydiff.txt' |
| 802 | --- lib/canonical/tests/chunkydiff.txt 2005-10-31 18:29:12 +0000 |
| 803 | +++ lib/canonical/tests/chunkydiff.txt 1970-01-01 00:00:00 +0000 |
| 804 | @@ -1,202 +0,0 @@ |
| 805 | -============ |
| 806 | -Chunky diffs |
| 807 | -============ |
| 808 | - |
| 809 | - run this using |
| 810 | - |
| 811 | - python test.py -u canonical.tests.test_chunkydiff |
| 812 | - |
| 813 | -Consider this desired output |
| 814 | - |
| 815 | - nnnnnnABCxyzDEFnnnnnn |
| 816 | - |
| 817 | -and this actual output |
| 818 | - |
| 819 | - nnnnnnABClmnopDEFnnnnnn |
| 820 | - |
| 821 | -the test says this |
| 822 | - |
| 823 | - ...ABCxyzDEF... |
| 824 | - |
| 825 | -useful output would be |
| 826 | - |
| 827 | - ...ABClmnopDEF... |
| 828 | - ^^^^^ |
| 829 | - |
| 830 | -and not |
| 831 | - |
| 832 | - nnnnnnABClmnopDEFnnnnnn |
| 833 | - ^^^^^ |
| 834 | - |
| 835 | -How do we do this? |
| 836 | - |
| 837 | -If the comparison has failed, we take the first character after the |
| 838 | -ellipsis in the test, and look for that in the actual output. If it occurs |
| 839 | -only once, then fine. If it does not occur at all, then print out the whole |
| 840 | -diff. If it occurs more than once, take the next character from the test, |
| 841 | -and look for those two characters in the actual output. Repeat until |
| 842 | -there is no occurence, or there is just one occurence. |
| 843 | - |
| 844 | -The same goes for the characters before the trailing ellipsis. |
| 845 | - |
| 846 | -The search can be narrowed because the characters at the end must follow |
| 847 | -those at the start. |
| 848 | - |
| 849 | - >>> from canonical.chunkydiff import elided_source |
| 850 | - |
| 851 | - >>> actual = "nnnnnnABClmnopDEFnnnnnn" |
| 852 | - >>> tested = "...ABCxyzDEF..." |
| 853 | - >>> elided_source(tested, actual) |
| 854 | - '...ABClmnopDEF...' |
| 855 | - |
| 856 | -Trivial modification, so that the code needs to take the input into account |
| 857 | -and not just parrot out a hard-coded return value. |
| 858 | - |
| 859 | - >>> actual = "nnnnnnABClmnopzDEFnnnnnn" |
| 860 | - >>> tested = "...ABCxyzDEF..." |
| 861 | - >>> elided_source(tested, actual) |
| 862 | - '...ABClmnopzDEF...' |
| 863 | - |
| 864 | -No interesting output between the arbitrary markers. This is really no |
| 865 | -different from the above, as far as the system is concerned. |
| 866 | - |
| 867 | - >>> actual = "nnnnnnABCDEFnnnnnn" |
| 868 | - >>> tested = "...ABCxyzDEF..." |
| 869 | - >>> elided_source(tested, actual) |
| 870 | - '...ABCDEF...' |
| 871 | - |
| 872 | -If there are two chunks that differ by the second or third characters in, |
| 873 | -choose the one that matches best. |
| 874 | - |
| 875 | - >>> actual = "nnnnnnABClmnopzDEFnnnnnABXuuuuXEFnnnn" |
| 876 | - >>> tested = "...ABCxyzDEF..." |
| 877 | - >>> elided_source(tested, actual) |
| 878 | - '...ABClmnopzDEF...' |
| 879 | - |
| 880 | -What happens when there is no ellipsis at the start? |
| 881 | - |
| 882 | - >>> actual = "ABClmnopzDEFnnnn" |
| 883 | - >>> tested = "ABCxyzDEF..." |
| 884 | - >>> elided_source(tested, actual) |
| 885 | - 'ABClmnopzDEF...' |
| 886 | - |
| 887 | -What happens when there is no ellipsis at the end, but extra data at the end? |
| 888 | - |
| 889 | - >>> actual = "ABClmnopzDEFnnnn" |
| 890 | - >>> tested = "...ABCxyzDEF" |
| 891 | - >>> elided_source(tested, actual) |
| 892 | - '...ABClmnopzDEFnnnn' |
| 893 | - |
| 894 | -What happens when there is no ellipsis at the end? |
| 895 | - |
| 896 | - >>> actual = "nnnnABClmnopzDEF" |
| 897 | - >>> tested = "...ABCxyzDEF" |
| 898 | - >>> elided_source(tested, actual) |
| 899 | - '...ABClmnopzDEF' |
| 900 | - |
| 901 | -What happens when there is no ellipsis at all? |
| 902 | - |
| 903 | - >>> actual = "ABClmnopzDEF" |
| 904 | - >>> tested = "ABCxyzDEF" |
| 905 | - >>> elided_source(tested, actual) |
| 906 | - 'ABClmnopzDEF' |
| 907 | - |
| 908 | -What happens when there is more than one chunk? |
| 909 | - |
| 910 | - >>> actual = "nnnnnnABClmnopzDEFnnnnnGHIzponmJKLnnnn" |
| 911 | - >>> tested = "...ABCxyzDEF...GHIxyzJKL..." |
| 912 | - >>> elided_source(tested, actual) |
| 913 | - '...ABClmnopzDEF...GHIzponmJKL...' |
| 914 | - |
| 915 | -What about when the chunks are presented in the wrong order? |
| 916 | - |
| 917 | -The first chunk from "tested" will have been found, but the second chunk |
| 918 | -will be absent. We want to keep the "nnnn" at the end of 'actual' because |
| 919 | -it is not matched by anything. We want to elide the start as it matches the |
| 920 | -elision, but not the end, as there is unmatched stuff in tested that we may |
| 921 | -want to compare. |
| 922 | - |
| 923 | -We may want to choose from among the following possible output in this case: |
| 924 | - |
| 925 | -'...GHIzponmJKLnnnn' |
| 926 | -'...GHIzponmJKL...' |
| 927 | -'...ABCmnopzDEF...GHIzponmJKL...' |
| 928 | - |
| 929 | -We'll use the first case for now, and see how it works in practice. |
| 930 | - |
| 931 | -Implementing this involves recognising how much of the actual string has |
| 932 | -been consumed by matching each chunk, and using only that remainder for the |
| 933 | -next chunks. |
| 934 | - |
| 935 | - >>> actual = "nnnnnnABClmnopzDEFnnnnnGHIzponmJKLnnnn" |
| 936 | - >>> tested = "...GHIzponmJKL...ABClmnopzDEF..." |
| 937 | - >>> elided_source(tested, actual) |
| 938 | - '...GHIzponmJKLnnnn' |
| 939 | - |
| 940 | -Where there is more than one option for the end match, choose the closest |
| 941 | -one. |
| 942 | - |
| 943 | - >>> actual = "nnnnnnABClmnopzDEFnnnnnGHIzponmDEFnnnn" |
| 944 | - >>> tested = "...ABCxxxxDEF..." |
| 945 | - >>> elided_source(tested, actual) |
| 946 | - '...ABClmnopzDEF...' |
| 947 | - |
| 948 | -Check anchoring to the start with elided junk after the first matched chunk. |
| 949 | - |
| 950 | - >>> actual = "ABClmnopznnnDEFzponmGHInnnn" |
| 951 | - >>> tested = "ABC...DEFxxxGHI..." |
| 952 | - >>> elided_source(tested, actual) |
| 953 | - 'ABC...DEFzponmGHI...' |
| 954 | - |
| 955 | -Check anchoring to the end with elided junk immediately before. |
| 956 | - |
| 957 | - >>> actual = "ABCDEzxcvbX" |
| 958 | - >>> tested = "ABCDE...X" |
| 959 | - >>> elided_source(tested, actual) |
| 960 | - 'ABCDE...X' |
| 961 | - |
| 962 | -Test single character within ellipses. |
| 963 | - |
| 964 | - >>> actual = "abcdeXfghij" |
| 965 | - >>> tested = "...X..." |
| 966 | - >>> elided_source(tested, actual) |
| 967 | - '...X...' |
| 968 | - |
| 969 | -Multiple single characters. |
| 970 | - |
| 971 | - >>> actual = "ABCDEnnXnnXnnX" |
| 972 | - >>> tested = "ABCDE...X" |
| 973 | - >>> elided_source(tested, actual) |
| 974 | - 'ABCDE...X' |
| 975 | - |
| 976 | - |
| 977 | - >>> actual = "ABCDEnnXnn" |
| 978 | - >>> tested = "ABCDE...X..." |
| 979 | - >>> elided_source(tested, actual) |
| 980 | - 'ABCDE...X...' |
| 981 | - |
| 982 | -Test with differences in whitespace. |
| 983 | - |
| 984 | - >>> actual = "ABC\nxxx DEF" |
| 985 | - >>> tested = "ABC ... DEF" |
| 986 | - >>> elided_source(tested, actual)#xx |
| 987 | - 'ABC\nxxx ...DEF' |
| 988 | - |
| 989 | - >>> actual = "ABC xxx DEF" |
| 990 | - >>> tested = "ABC\n... DEF" |
| 991 | - >>> elided_source(tested, actual, normalize_whitespace=True) |
| 992 | - 'ABC\n... DEF' |
| 993 | - |
| 994 | -Test with multiple whitespace characters. |
| 995 | - |
| 996 | - >>> actual = "ABC xxx DEF" |
| 997 | - >>> tested = "ABC\n\n... DEF" |
| 998 | - >>> elided_source(tested, actual, normalize_whitespace=True) |
| 999 | - 'ABC\n\n... DEF' |
| 1000 | - |
| 1001 | -Test brad's case: |
| 1002 | - |
| 1003 | - >>> actual = '\'Bug #7: "firefox crashes all the time" added\'\n' |
| 1004 | - >>> tested = "'...task added'\n" |
| 1005 | - >>> elided_source(tested, actual) |
| 1006 | - '\'...ttime" added\'\n' |
| 1007 | |
| 1008 | === removed file 'lib/canonical/tests/test_base.py' |
| 1009 | --- lib/canonical/tests/test_base.py 2010-10-12 01:11:41 +0000 |
| 1010 | +++ lib/canonical/tests/test_base.py 1970-01-01 00:00:00 +0000 |
| 1011 | @@ -1,9 +0,0 @@ |
| 1012 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 1013 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 1014 | - |
| 1015 | -from doctest import DocTestSuite |
| 1016 | -import canonical.base |
| 1017 | - |
| 1018 | -def test_suite(): |
| 1019 | - suite = DocTestSuite(canonical.base) |
| 1020 | - return suite |
| 1021 | |
| 1022 | === removed file 'lib/canonical/tests/test_chunkydiff.py' |
| 1023 | --- lib/canonical/tests/test_chunkydiff.py 2009-06-25 05:30:52 +0000 |
| 1024 | +++ lib/canonical/tests/test_chunkydiff.py 1970-01-01 00:00:00 +0000 |
| 1025 | @@ -1,9 +0,0 @@ |
| 1026 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 1027 | -# GNU Affero General Public License version 3 (see the file LICENSE). |
| 1028 | - |
| 1029 | -from canonical.launchpad.testing.systemdocs import LayeredDocFileSuite |
| 1030 | - |
| 1031 | - |
| 1032 | -def test_suite(): |
| 1033 | - return LayeredDocFileSuite('chunkydiff.txt', stdout_logging=False) |
| 1034 | - |
| 1035 | |
| 1036 | === modified file 'lib/lp/answers/doc/person.txt' |
| 1037 | --- lib/lp/answers/doc/person.txt 2010-10-18 22:24:59 +0000 |
| 1038 | +++ lib/lp/answers/doc/person.txt 2011-02-18 15:29:53 +0000 |
| 1039 | @@ -169,7 +169,7 @@ |
| 1040 | But Carlos has one. |
| 1041 | |
| 1042 | # Because not everyone uses a real editor <wink> |
| 1043 | - >>> from canonical.encoding import ascii_smash |
| 1044 | + >>> from lp.services.encoding import ascii_smash |
| 1045 | >>> carlos_raw = personset.getByName('carlos') |
| 1046 | >>> carlos = IQuestionsPerson(carlos_raw) |
| 1047 | >>> for question in carlos.searchQuestions( |
| 1048 | |
| 1049 | === modified file 'lib/lp/answers/doc/questionsets.txt' |
| 1050 | --- lib/lp/answers/doc/questionsets.txt 2010-11-15 21:56:43 +0000 |
| 1051 | +++ lib/lp/answers/doc/questionsets.txt 2011-02-18 15:29:53 +0000 |
| 1052 | @@ -48,7 +48,7 @@ |
| 1053 | regular full text algorithm. |
| 1054 | |
| 1055 | # Because not everyone uses a real editor <wink> |
| 1056 | - >>> from canonical.encoding import ascii_smash |
| 1057 | + >>> from lp.services.encoding import ascii_smash |
| 1058 | >>> for question in question_set.searchQuestions(search_text='firefox'): |
| 1059 | ... print ascii_smash(question.title), question.target.displayname |
| 1060 | Problemas de Impressao no Firefox Mozilla Firefox |
| 1061 | |
| 1062 | === modified file 'lib/lp/app/stories/launchpad-root/site-search.txt' |
| 1063 | --- lib/lp/app/stories/launchpad-root/site-search.txt 2010-09-27 19:39:21 +0000 |
| 1064 | +++ lib/lp/app/stories/launchpad-root/site-search.txt 2011-02-18 15:29:53 +0000 |
| 1065 | @@ -5,7 +5,7 @@ |
| 1066 | specific search with Launchpad's prominent objects (projects, bugs, |
| 1067 | teams, etc.). |
| 1068 | |
| 1069 | - >>> from canonical.encoding import ascii_smash |
| 1070 | + >>> from lp.services.encoding import ascii_smash |
| 1071 | |
| 1072 | # Our very helpful function for printing all the page results. |
| 1073 | |
| 1074 | |
| 1075 | === modified file 'lib/lp/archivepublisher/utils.py' |
| 1076 | --- lib/lp/archivepublisher/utils.py 2011-02-04 09:07:36 +0000 |
| 1077 | +++ lib/lp/archivepublisher/utils.py 2011-02-18 15:29:53 +0000 |
| 1078 | @@ -31,7 +31,7 @@ |
| 1079 | IStoreSelector, |
| 1080 | MAIN_STORE, |
| 1081 | ) |
| 1082 | -from canonical.mem import resident |
| 1083 | +from lp.services.profile.mem import resident |
| 1084 | from lp.soyuz.enums import ArchivePurpose |
| 1085 | from lp.soyuz.interfaces.archive import ( |
| 1086 | default_name_by_purpose, |
| 1087 | |
| 1088 | === modified file 'lib/lp/archiveuploader/dscfile.py' |
| 1089 | --- lib/lp/archiveuploader/dscfile.py 2010-12-02 16:15:46 +0000 |
| 1090 | +++ lib/lp/archiveuploader/dscfile.py 2011-02-18 15:29:53 +0000 |
| 1091 | @@ -28,7 +28,7 @@ |
| 1092 | from debian.deb822 import Deb822Dict |
| 1093 | from zope.component import getUtility |
| 1094 | |
| 1095 | -from canonical.encoding import guess as guess_encoding |
| 1096 | +from lp.services.encoding import guess as guess_encoding |
| 1097 | from canonical.launchpad.interfaces.gpghandler import ( |
| 1098 | GPGVerificationError, |
| 1099 | IGPGHandler, |
| 1100 | |
| 1101 | === modified file 'lib/lp/archiveuploader/nascentuploadfile.py' |
| 1102 | --- lib/lp/archiveuploader/nascentuploadfile.py 2010-10-19 09:00:29 +0000 |
| 1103 | +++ lib/lp/archiveuploader/nascentuploadfile.py 2011-02-18 15:29:53 +0000 |
| 1104 | @@ -30,7 +30,6 @@ |
| 1105 | |
| 1106 | from zope.component import getUtility |
| 1107 | |
| 1108 | -from canonical.encoding import guess as guess_encoding |
| 1109 | from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet |
| 1110 | from canonical.librarian.utils import filechunks |
| 1111 | from lp.app.errors import NotFoundError |
| 1112 | @@ -47,10 +46,10 @@ |
| 1113 | re_valid_version, |
| 1114 | ) |
| 1115 | from lp.buildmaster.enums import BuildStatus |
| 1116 | +from lp.services.encoding import guess as guess_encoding |
| 1117 | from lp.soyuz.enums import ( |
| 1118 | BinaryPackageFormat, |
| 1119 | PackagePublishingPriority, |
| 1120 | - PackagePublishingStatus, |
| 1121 | PackageUploadCustomFormat, |
| 1122 | ) |
| 1123 | from lp.soyuz.interfaces.binarypackagename import IBinaryPackageNameSet |
| 1124 | |
| 1125 | === modified file 'lib/lp/archiveuploader/utils.py' |
| 1126 | --- lib/lp/archiveuploader/utils.py 2010-12-09 16:20:20 +0000 |
| 1127 | +++ lib/lp/archiveuploader/utils.py 2011-02-18 15:29:53 +0000 |
| 1128 | @@ -32,7 +32,7 @@ |
| 1129 | import signal |
| 1130 | import subprocess |
| 1131 | |
| 1132 | -from canonical.encoding import ( |
| 1133 | +from lp.services.encoding import ( |
| 1134 | ascii_smash, |
| 1135 | guess as guess_encoding, |
| 1136 | ) |
| 1137 | |
| 1138 | === modified file 'lib/lp/bugs/doc/bugtracker-tokens.txt' |
| 1139 | --- lib/lp/bugs/doc/bugtracker-tokens.txt 2010-10-09 16:36:22 +0000 |
| 1140 | +++ lib/lp/bugs/doc/bugtracker-tokens.txt 2011-02-18 15:29:53 +0000 |
| 1141 | @@ -4,7 +4,7 @@ |
| 1142 | |
| 1143 | >>> import xmlrpclib |
| 1144 | >>> from zope.component import getUtility |
| 1145 | - >>> from canonical.functional import XMLRPCTestTransport |
| 1146 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 1147 | >>> from canonical.launchpad.interfaces.logintoken import ILoginTokenSet |
| 1148 | >>> bugtracker_api = xmlrpclib.ServerProxy( |
| 1149 | ... 'http://xmlrpc-private.launchpad.dev:8087/bugs', |
| 1150 | |
| 1151 | === modified file 'lib/lp/bugs/doc/malone-xmlrpc.txt' |
| 1152 | --- lib/lp/bugs/doc/malone-xmlrpc.txt 2010-10-19 18:44:31 +0000 |
| 1153 | +++ lib/lp/bugs/doc/malone-xmlrpc.txt 2011-02-18 15:29:53 +0000 |
| 1154 | @@ -3,7 +3,7 @@ |
| 1155 | Malone provides an XML-RPC interface for filing bugs. |
| 1156 | |
| 1157 | >>> import xmlrpclib |
| 1158 | - >>> from canonical.functional import XMLRPCTestTransport |
| 1159 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 1160 | >>> filebug_api = xmlrpclib.ServerProxy( |
| 1161 | ... 'http://test@canonical.com:test@xmlrpc.launchpad.dev/bugs/', |
| 1162 | ... transport=XMLRPCTestTransport()) |
| 1163 | |
| 1164 | === modified file 'lib/lp/bugs/externalbugtracker/bugzilla.py' |
| 1165 | --- lib/lp/bugs/externalbugtracker/bugzilla.py 2011-02-08 20:52:21 +0000 |
| 1166 | +++ lib/lp/bugs/externalbugtracker/bugzilla.py 2011-02-18 15:29:53 +0000 |
| 1167 | @@ -21,7 +21,6 @@ |
| 1168 | from zope.component import getUtility |
| 1169 | from zope.interface import implements |
| 1170 | |
| 1171 | -from canonical import encoding |
| 1172 | from canonical.config import config |
| 1173 | from canonical.launchpad.interfaces.message import IMessageSet |
| 1174 | from canonical.launchpad.webapp.url import ( |
| 1175 | @@ -52,6 +51,7 @@ |
| 1176 | ISupportsCommentPushing, |
| 1177 | UNKNOWN_REMOTE_IMPORTANCE, |
| 1178 | ) |
| 1179 | +from lp.services import encoding |
| 1180 | |
| 1181 | |
| 1182 | class Bugzilla(ExternalBugTracker): |
| 1183 | |
| 1184 | === modified file 'lib/lp/bugs/stories/bugtracker/xx-bugtracker-handshake-tokens.txt' |
| 1185 | --- lib/lp/bugs/stories/bugtracker/xx-bugtracker-handshake-tokens.txt 2009-10-22 13:02:12 +0000 |
| 1186 | +++ lib/lp/bugs/stories/bugtracker/xx-bugtracker-handshake-tokens.txt 2011-02-18 15:29:53 +0000 |
| 1187 | @@ -5,7 +5,7 @@ |
| 1188 | done using the internal XML-RPC service. |
| 1189 | |
| 1190 | >>> import xmlrpclib |
| 1191 | - >>> from canonical.functional import XMLRPCTestTransport |
| 1192 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 1193 | >>> bugtracker_api = xmlrpclib.ServerProxy( |
| 1194 | ... 'http://xmlrpc-private.launchpad.dev:8087/bugs', |
| 1195 | ... transport=XMLRPCTestTransport()) |
| 1196 | |
| 1197 | === modified file 'lib/lp/buildmaster/model/buildfarmjobbehavior.py' |
| 1198 | --- lib/lp/buildmaster/model/buildfarmjobbehavior.py 2010-11-11 13:06:58 +0000 |
| 1199 | +++ lib/lp/buildmaster/model/buildfarmjobbehavior.py 2011-02-18 15:29:53 +0000 |
| 1200 | @@ -22,7 +22,6 @@ |
| 1201 | from zope.interface import implements |
| 1202 | from zope.security.proxy import removeSecurityProxy |
| 1203 | |
| 1204 | -from canonical import encoding |
| 1205 | from canonical.librarian.interfaces import ILibrarianClient |
| 1206 | from lp.buildmaster.interfaces.builder import ( |
| 1207 | BuildSlaveFailure, |
| 1208 | @@ -32,6 +31,7 @@ |
| 1209 | BuildBehaviorMismatch, |
| 1210 | IBuildFarmJobBehavior, |
| 1211 | ) |
| 1212 | +from lp.services import encoding |
| 1213 | from lp.services.job.interfaces.job import JobStatus |
| 1214 | |
| 1215 | |
| 1216 | |
| 1217 | === modified file 'lib/lp/code/doc/branch-xmlrpc.txt' |
| 1218 | --- lib/lp/code/doc/branch-xmlrpc.txt 2010-10-03 15:30:06 +0000 |
| 1219 | +++ lib/lp/code/doc/branch-xmlrpc.txt 2011-02-18 15:29:53 +0000 |
| 1220 | @@ -5,7 +5,7 @@ |
| 1221 | >>> from datetime import datetime |
| 1222 | >>> import pytz |
| 1223 | >>> import xmlrpclib |
| 1224 | - >>> from canonical.functional import XMLRPCTestTransport |
| 1225 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 1226 | >>> branchset_api = xmlrpclib.ServerProxy( |
| 1227 | ... 'http://foo.bar@canonical.com:test@xmlrpc.launchpad.dev/bazaar/', |
| 1228 | ... transport=XMLRPCTestTransport()) |
| 1229 | |
| 1230 | === modified file 'lib/lp/code/doc/xmlrpc-codeimport-scheduler.txt' |
| 1231 | --- lib/lp/code/doc/xmlrpc-codeimport-scheduler.txt 2010-10-03 15:30:06 +0000 |
| 1232 | +++ lib/lp/code/doc/xmlrpc-codeimport-scheduler.txt 2011-02-18 15:29:53 +0000 |
| 1233 | @@ -44,7 +44,7 @@ |
| 1234 | The point of all this is for it to be accessed over XMLRPC. |
| 1235 | |
| 1236 | >>> import xmlrpclib |
| 1237 | - >>> from canonical.functional import XMLRPCTestTransport |
| 1238 | + >>> from lp.testing.xmlrpc import XMLRPCTestTransport |
| 1239 | >>> codeimportscheduler = xmlrpclib.ServerProxy( |
| 1240 | ... 'http://xmlrpc-private.launchpad.dev:8087/codeimportscheduler', |
| 1241 | ... transport=XMLRPCTestTransport()) |
| 1242 | |
| 1243 | === modified file 'lib/lp/registry/tests/test_doc.py' |
| 1244 | --- lib/lp/registry/tests/test_doc.py 2010-10-04 19:50:45 +0000 |
| 1245 | +++ lib/lp/registry/tests/test_doc.py 2011-02-18 15:29:53 +0000 |
| 1246 | @@ -51,7 +51,7 @@ |
| 1247 | # Use a real XMLRPC server proxy so that the same test is run through the |
| 1248 | # full security machinery. This is more representative of the real-world, |
| 1249 | # but more difficult to debug. |
| 1250 | - from canonical.functional import XMLRPCTestTransport |
| 1251 | + from lp.testing.xmlrpc import XMLRPCTestTransport |
| 1252 | from xmlrpclib import ServerProxy |
| 1253 | mailinglist_api = ServerProxy( |
| 1254 | 'http://xmlrpc-private.launchpad.dev:8087/mailinglists/', |
| 1255 | |
| 1256 | === modified file 'lib/lp/registry/tests/test_xmlrpc.py' |
| 1257 | --- lib/lp/registry/tests/test_xmlrpc.py 2010-10-20 20:51:26 +0000 |
| 1258 | +++ lib/lp/registry/tests/test_xmlrpc.py 2011-02-18 15:29:53 +0000 |
| 1259 | @@ -11,7 +11,6 @@ |
| 1260 | from zope.component import getUtility |
| 1261 | from zope.security.proxy import removeSecurityProxy |
| 1262 | |
| 1263 | -from canonical.functional import XMLRPCTestTransport |
| 1264 | from canonical.launchpad.interfaces.account import AccountStatus |
| 1265 | from canonical.launchpad.interfaces.launchpad import IPrivateApplication |
| 1266 | from canonical.launchpad.webapp.servers import LaunchpadTestRequest |
| 1267 | @@ -24,6 +23,7 @@ |
| 1268 | ) |
| 1269 | from lp.registry.xmlrpc.softwarecenteragent import SoftwareCenterAgentAPI |
| 1270 | from lp.testing import TestCaseWithFactory |
| 1271 | +from lp.testing.xmlrpc import XMLRPCTestTransport |
| 1272 | |
| 1273 | |
| 1274 | class TestSoftwareCenterAgentAPI(TestCaseWithFactory): |
| 1275 | |
| 1276 | === modified file 'lib/lp/registry/xmlrpc/mailinglist.py' |
| 1277 | --- lib/lp/registry/xmlrpc/mailinglist.py 2010-10-26 03:51:12 +0000 |
| 1278 | +++ lib/lp/registry/xmlrpc/mailinglist.py 2011-02-18 15:29:53 +0000 |
| 1279 | @@ -16,7 +16,7 @@ |
| 1280 | from zope.security.proxy import removeSecurityProxy |
| 1281 | |
| 1282 | from canonical.config import config |
| 1283 | -from canonical.encoding import escape_nonascii_uniquely |
| 1284 | +from lp.services.encoding import escape_nonascii_uniquely |
| 1285 | from canonical.launchpad.interfaces.emailaddress import ( |
| 1286 | EmailAddressStatus, |
| 1287 | IEmailAddressSet, |
| 1288 | |
| 1289 | === renamed file 'lib/canonical/encoding.py' => 'lib/lp/services/encoding.py' |
| 1290 | === modified file 'lib/lp/services/memcache/tales.py' |
| 1291 | --- lib/lp/services/memcache/tales.py 2010-12-13 18:04:24 +0000 |
| 1292 | +++ lib/lp/services/memcache/tales.py 2011-02-18 15:29:53 +0000 |
| 1293 | @@ -27,11 +27,11 @@ |
| 1294 | ) |
| 1295 | from zope.tales.interfaces import ITALESExpression |
| 1296 | |
| 1297 | -from canonical.base import base |
| 1298 | from canonical.config import config |
| 1299 | from lp.app import versioninfo |
| 1300 | from canonical.launchpad.webapp.interfaces import ILaunchBag |
| 1301 | from lp.services.memcache.interfaces import IMemcacheClient |
| 1302 | +from lp.services.utils import compress_hash |
| 1303 | |
| 1304 | # Request annotation key. |
| 1305 | COUNTER_KEY = 'lp.services.memcache.tales.counter' |
| 1306 | @@ -238,7 +238,7 @@ |
| 1307 | # with a hash. A short hash is good, provided it is still unique, |
| 1308 | # to preserve readability as much as possible. We include the |
| 1309 | # unsanitized URL in the hash to ensure uniqueness. |
| 1310 | - key_hash = base(int(md5(key + url).hexdigest(), 16), 62) |
| 1311 | + key_hash = compress_hash(md5(key + url)) |
| 1312 | key = key[:250-len(key_hash)] + key_hash |
| 1313 | |
| 1314 | return key |
| 1315 | |
| 1316 | === modified file 'lib/lp/services/profile/__init__.py' |
| 1317 | --- lib/lp/services/profile/__init__.py 2010-07-01 01:39:46 +0000 |
| 1318 | +++ lib/lp/services/profile/__init__.py 2011-02-18 15:29:53 +0000 |
| 1319 | @@ -1,7 +1,7 @@ |
| 1320 | -# Copyright 2010 Canonical Ltd. This software is licensed under the |
| 1321 | +# Copyright 2010-2011 Canonical Ltd. This software is licensed under the |
| 1322 | # GNU Affero General Public License version 3 (see the file LICENSE). |
| 1323 | |
| 1324 | -"""lp.services.profile - profiling for zope applications. |
| 1325 | +"""Profiling for Python and Zope applications. |
| 1326 | |
| 1327 | Tests for this package are currently services stories. |
| 1328 | """ |
| 1329 | |
| 1330 | === renamed file 'lib/canonical/mem.py' => 'lib/lp/services/profile/mem.py' |
| 1331 | === modified file 'lib/lp/services/profile/profile.py' |
| 1332 | --- lib/lp/services/profile/profile.py 2010-10-22 10:24:18 +0000 |
| 1333 | +++ lib/lp/services/profile/profile.py 2011-02-18 15:29:53 +0000 |
| 1334 | @@ -26,7 +26,7 @@ |
| 1335 | from canonical.config import config |
| 1336 | import canonical.launchpad.webapp.adapter as da |
| 1337 | from canonical.launchpad.webapp.interfaces import IStartRequestEvent |
| 1338 | -from canonical.mem import ( |
| 1339 | +from lp.services.profile.mem import ( |
| 1340 | memory, |
| 1341 | resident, |
| 1342 | ) |
| 1343 | |
| 1344 | === renamed file 'lib/canonical/tests/test_encoding.py' => 'lib/lp/services/tests/test_encoding.py' |
| 1345 | --- lib/canonical/tests/test_encoding.py 2010-10-12 01:11:41 +0000 |
| 1346 | +++ lib/lp/services/tests/test_encoding.py 2011-02-18 15:29:53 +0000 |
| 1347 | @@ -3,8 +3,8 @@ |
| 1348 | |
| 1349 | from doctest import DocTestSuite, ELLIPSIS |
| 1350 | |
| 1351 | -import canonical.encoding |
| 1352 | +import lp.services.encoding |
| 1353 | |
| 1354 | def test_suite(): |
| 1355 | - suite = DocTestSuite(canonical.encoding, optionflags=ELLIPSIS) |
| 1356 | + suite = DocTestSuite(lp.services.encoding, optionflags=ELLIPSIS) |
| 1357 | return suite |
| 1358 | |
| 1359 | === modified file 'lib/lp/services/tests/test_utils.py' |
| 1360 | --- lib/lp/services/tests/test_utils.py 2011-02-09 10:59:00 +0000 |
| 1361 | +++ lib/lp/services/tests/test_utils.py 2011-02-18 15:29:53 +0000 |
| 1362 | @@ -1,15 +1,19 @@ |
| 1363 | # Copyright 2009 Canonical Ltd. This software is licensed under the |
| 1364 | # GNU Affero General Public License version 3 (see the file LICENSE). |
| 1365 | |
| 1366 | -"""Module docstring goes here.""" |
| 1367 | +"""Tests for lp.services.utils.""" |
| 1368 | |
| 1369 | __metaclass__ = type |
| 1370 | |
| 1371 | from contextlib import contextmanager |
| 1372 | +import hashlib |
| 1373 | import itertools |
| 1374 | import unittest |
| 1375 | |
| 1376 | from lp.services.utils import ( |
| 1377 | + AutoDecorate, |
| 1378 | + base, |
| 1379 | + compress_hash, |
| 1380 | CachingIterator, |
| 1381 | decorate_with, |
| 1382 | docstring_dedent, |
| 1383 | @@ -19,6 +23,86 @@ |
| 1384 | from lp.testing import TestCase |
| 1385 | |
| 1386 | |
| 1387 | + |
| 1388 | +class TestAutoDecorate(TestCase): |
| 1389 | + """Tests for AutoDecorate.""" |
| 1390 | + |
| 1391 | + def setUp(self): |
| 1392 | + super(TestAutoDecorate, self).setUp() |
| 1393 | + self.log = None |
| 1394 | + |
| 1395 | + def decorator_1(self, f): |
| 1396 | + def decorated(*args, **kwargs): |
| 1397 | + self.log.append(1) |
| 1398 | + return f(*args, **kwargs) |
| 1399 | + return decorated |
| 1400 | + |
| 1401 | + def decorator_2(self, f): |
| 1402 | + def decorated(*args, **kwargs): |
| 1403 | + self.log.append(2) |
| 1404 | + return f(*args, **kwargs) |
| 1405 | + return decorated |
| 1406 | + |
| 1407 | + def test_auto_decorate(self): |
| 1408 | + # All of the decorators passed to AutoDecorate are applied as |
| 1409 | + # decorators in reverse order. |
| 1410 | + |
| 1411 | + class AutoDecoratedClass: |
| 1412 | + __metaclass__ = AutoDecorate(self.decorator_1, self.decorator_2) |
| 1413 | + def method_a(s): |
| 1414 | + self.log.append('a') |
| 1415 | + def method_b(s): |
| 1416 | + self.log.append('b') |
| 1417 | + |
| 1418 | + obj = AutoDecoratedClass() |
| 1419 | + self.log = [] |
| 1420 | + obj.method_a() |
| 1421 | + self.assertEqual([2, 1, 'a'], self.log) |
| 1422 | + self.log = [] |
| 1423 | + obj.method_b() |
| 1424 | + self.assertEqual([2, 1, 'b'], self.log) |
| 1425 | + |
| 1426 | + |
| 1427 | +class TestBase(TestCase): |
| 1428 | + |
| 1429 | + def test_simple_base(self): |
| 1430 | + # 35 in base 36 is lowercase 'z' |
| 1431 | + self.assertEqual('z', base(35, 36)) |
| 1432 | + |
| 1433 | + def test_extended_base(self): |
| 1434 | + # There is no standard representation for numbers in bases above 36 |
| 1435 | + # (all the digits, all the letters of the English alphabet). However, |
| 1436 | + # we can represent bases up to 62 by using upper case letters on top |
| 1437 | + # of lower case letters. This is useful as a cheap compression |
| 1438 | + # algorithm. |
| 1439 | + self.assertEqual('A', base(36, 62)) |
| 1440 | + self.assertEqual('B', base(37, 62)) |
| 1441 | + self.assertEqual('Z', base(61, 62)) |
| 1442 | + |
| 1443 | + def test_negative_numbers(self): |
| 1444 | + # We don't convert negative numbers at all. |
| 1445 | + self.assertRaises(ValueError, base, -43, 62) |
| 1446 | + |
| 1447 | + def test_base_matches_builtin_hex(self): |
| 1448 | + # We get identical results to the hex builtin, without the 0x prefix |
| 1449 | + numbers = list(range(5000)) |
| 1450 | + using_hex = [hex(i)[2:] for i in numbers] |
| 1451 | + using_base = [base(i, 16) for i in numbers] |
| 1452 | + self.assertEqual(using_hex, using_base) |
| 1453 | + |
| 1454 | + def test_compress_md5_hash(self): |
| 1455 | + # compress_hash compresses MD5 hashes down to 22 URL-safe characters. |
| 1456 | + compressed = compress_hash(hashlib.md5('foo')) |
| 1457 | + self.assertEqual('5fX649Stem9fET0lD46zVe', compressed) |
| 1458 | + self.assertEqual(22, len(compressed)) |
| 1459 | + |
| 1460 | + def test_compress_sha1_hash(self): |
| 1461 | + # compress_hash compresses SHA1 hashes down to 27 URL-safe characters. |
| 1462 | + compressed = compress_hash(hashlib.sha1('foo')) |
| 1463 | + self.assertEqual('1HyPQr2xj1nmnkQXBCJXUdQoy5l', compressed) |
| 1464 | + self.assertEqual(27, len(compressed)) |
| 1465 | + |
| 1466 | + |
| 1467 | class TestIterateSplit(TestCase): |
| 1468 | """Tests for iter_split.""" |
| 1469 | |
| 1470 | |
| 1471 | === modified file 'lib/lp/services/utils.py' |
| 1472 | --- lib/lp/services/utils.py 2011-02-08 21:17:56 +0000 |
| 1473 | +++ lib/lp/services/utils.py 2011-02-18 15:29:53 +0000 |
| 1474 | @@ -1,4 +1,4 @@ |
| 1475 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 1476 | +# Copyright 2009-2011 Canonical Ltd. This software is licensed under the |
| 1477 | # GNU Affero General Public License version 3 (see the file LICENSE). |
| 1478 | |
| 1479 | """Generic Python utilities. |
| 1480 | @@ -9,7 +9,10 @@ |
| 1481 | |
| 1482 | __metaclass__ = type |
| 1483 | __all__ = [ |
| 1484 | + 'AutoDecorate', |
| 1485 | + 'base', |
| 1486 | 'CachingIterator', |
| 1487 | + 'compress_hash', |
| 1488 | 'decorate_with', |
| 1489 | 'docstring_dedent', |
| 1490 | 'iter_split', |
| 1491 | @@ -20,14 +23,77 @@ |
| 1492 | ] |
| 1493 | |
| 1494 | from itertools import tee |
| 1495 | +import string |
| 1496 | import sys |
| 1497 | from textwrap import dedent |
| 1498 | +from types import FunctionType |
| 1499 | |
| 1500 | from lazr.enum import BaseItem |
| 1501 | from twisted.python.util import mergeFunctionMetadata |
| 1502 | from zope.security.proxy import isinstance as zope_isinstance |
| 1503 | |
| 1504 | |
| 1505 | +def AutoDecorate(*decorators): |
| 1506 | + """Factory to generate metaclasses that automatically apply decorators. |
| 1507 | + |
| 1508 | + AutoDecorate is a metaclass factory that can be used to make a class |
| 1509 | + implicitly wrap all of its methods with one or more decorators. |
| 1510 | + """ |
| 1511 | + |
| 1512 | + class AutoDecorateMetaClass(type): |
| 1513 | + def __new__(cls, class_name, bases, class_dict): |
| 1514 | + new_class_dict = {} |
| 1515 | + for name, value in class_dict.items(): |
| 1516 | + if type(value) == FunctionType: |
| 1517 | + for decorator in decorators: |
| 1518 | + value = decorator(value) |
| 1519 | + assert callable(value), ( |
| 1520 | + "Decorator %s didn't return a callable." |
| 1521 | + % repr(decorator)) |
| 1522 | + new_class_dict[name] = value |
| 1523 | + return type.__new__(cls, class_name, bases, new_class_dict) |
| 1524 | + |
| 1525 | + return AutoDecorateMetaClass |
| 1526 | + |
| 1527 | + |
| 1528 | +def base(number, radix): |
| 1529 | + """Convert 'number' to an arbitrary base numbering scheme, 'radix'. |
| 1530 | + |
| 1531 | + This function is based on work from the Python Cookbook and is under the |
| 1532 | + Python license. |
| 1533 | + |
| 1534 | + Inverse function to int(str, radix) and long(str, radix) |
| 1535 | + """ |
| 1536 | + if not 2 <= radix <= 62: |
| 1537 | + raise ValueError("radix must be between 2 and 62: %s" % (radix,)) |
| 1538 | + |
| 1539 | + if number < 0: |
| 1540 | + raise ValueError("number must be non-negative: %s" % (number,)) |
| 1541 | + |
| 1542 | + result = [] |
| 1543 | + addon = result.append |
| 1544 | + if number == 0: |
| 1545 | + addon('0') |
| 1546 | + |
| 1547 | + ABC = string.digits + string.ascii_letters |
| 1548 | + while number: |
| 1549 | + number, rdigit = divmod(number, radix) |
| 1550 | + addon(ABC[rdigit]) |
| 1551 | + |
| 1552 | + result.reverse() |
| 1553 | + return ''.join(result) |
| 1554 | + |
| 1555 | + |
| 1556 | +def compress_hash(hash_obj): |
| 1557 | + """Compress a hash_obj using `base`. |
| 1558 | + |
| 1559 | + Given an ``md5`` or ``sha1`` hash object, compress it down to either 22 or |
| 1560 | + 27 characters in a way that's safe to be used in URLs. Takes the hex of |
| 1561 | + the hash and converts it to base 62. |
| 1562 | + """ |
| 1563 | + return base(int(hash_obj.hexdigest(), 16), 62) |
| 1564 | + |
| 1565 | + |
| 1566 | def iter_split(string, splitter): |
| 1567 | """Iterate over ways to split 'string' in two with 'splitter'. |
| 1568 | |
| 1569 | |
| 1570 | === modified file 'lib/lp/soyuz/model/queue.py' |
| 1571 | --- lib/lp/soyuz/model/queue.py 2011-01-27 15:05:34 +0000 |
| 1572 | +++ lib/lp/soyuz/model/queue.py 2011-02-18 15:29:53 +0000 |
| 1573 | @@ -40,7 +40,7 @@ |
| 1574 | SQLBase, |
| 1575 | sqlvalues, |
| 1576 | ) |
| 1577 | -from canonical.encoding import ( |
| 1578 | +from lp.services.encoding import ( |
| 1579 | ascii_smash, |
| 1580 | guess as guess_encoding, |
| 1581 | ) |
| 1582 | |
| 1583 | === modified file 'lib/lp/soyuz/scripts/gina/packages.py' |
| 1584 | --- lib/lp/soyuz/scripts/gina/packages.py 2011-02-04 02:26:34 +0000 |
| 1585 | +++ lib/lp/soyuz/scripts/gina/packages.py 2011-02-18 15:29:53 +0000 |
| 1586 | @@ -28,7 +28,6 @@ |
| 1587 | import shutil |
| 1588 | import tempfile |
| 1589 | |
| 1590 | -from canonical import encoding |
| 1591 | from canonical.database.constants import UTC_NOW |
| 1592 | from canonical.launchpad.scripts import log |
| 1593 | from canonical.launchpad.validators.version import valid_debian_version |
| 1594 | @@ -39,6 +38,7 @@ |
| 1595 | extract_dpkg_source, |
| 1596 | ) |
| 1597 | from lp.registry.interfaces.gpg import GPGKeyAlgorithm |
| 1598 | +from lp.services import encoding |
| 1599 | from lp.soyuz.enums import PackagePublishingPriority |
| 1600 | from lp.soyuz.scripts.gina import ( |
| 1601 | call, |
| 1602 | |
| 1603 | === modified file 'lib/lp/testing/factory.py' |
| 1604 | --- lib/lp/testing/factory.py 2011-02-16 11:18:48 +0000 |
| 1605 | +++ lib/lp/testing/factory.py 2011-02-18 15:29:53 +0000 |
| 1606 | @@ -61,7 +61,6 @@ |
| 1607 | removeSecurityProxy, |
| 1608 | ) |
| 1609 | |
| 1610 | -from canonical.autodecorate import AutoDecorate |
| 1611 | from canonical.config import config |
| 1612 | from canonical.database.constants import ( |
| 1613 | DEFAULT, |
| 1614 | @@ -233,6 +232,7 @@ |
| 1615 | from lp.services.mail.signedmessage import SignedMessage |
| 1616 | from lp.services.openid.model.openididentifier import OpenIdIdentifier |
| 1617 | from lp.services.propertycache import clear_property_cache |
| 1618 | +from lp.services.utils import AutoDecorate |
| 1619 | from lp.services.worlddata.interfaces.country import ICountrySet |
| 1620 | from lp.services.worlddata.interfaces.language import ILanguageSet |
| 1621 | from lp.soyuz.adapters.packagelocation import PackageLocation |
| 1622 | |
| 1623 | === renamed file 'lib/canonical/functional.py' => 'lib/lp/testing/xmlrpc.py' |
| 1624 | --- lib/canonical/functional.py 2009-06-25 05:30:52 +0000 |
| 1625 | +++ lib/lp/testing/xmlrpc.py 2011-02-18 15:29:53 +0000 |
| 1626 | @@ -1,7 +1,11 @@ |
| 1627 | -# Copyright 2009 Canonical Ltd. This software is licensed under the |
| 1628 | +# Copyright 2009-2011 Canonical Ltd. This software is licensed under the |
| 1629 | # GNU Affero General Public License version 3 (see the file LICENSE). |
| 1630 | |
| 1631 | -"""Alter the standard functional testing environment for Launchpad.""" |
| 1632 | +"""Tools for testing XML-RPC services.""" |
| 1633 | + |
| 1634 | +__all__ = [ |
| 1635 | + 'XMLRPCTestTransport', |
| 1636 | + ] |
| 1637 | |
| 1638 | from cStringIO import StringIO |
| 1639 | import httplib |
| 1640 | |
| 1641 | === modified file 'lib/lp_sitecustomize.py' |
| 1642 | --- lib/lp_sitecustomize.py 2010-12-24 13:03:02 +0000 |
| 1643 | +++ lib/lp_sitecustomize.py 2011-02-18 15:29:53 +0000 |
| 1644 | @@ -92,6 +92,15 @@ |
| 1645 | "ignore", |
| 1646 | category=DeprecationWarning, |
| 1647 | module="Crypto") |
| 1648 | + # Filter all deprecation warnings for Zope 3.6, which emanate from |
| 1649 | + # the zope package. |
| 1650 | + filter_pattern = '.*(Zope 3.6|provide.*global site manager).*' |
| 1651 | + warnings.filterwarnings( |
| 1652 | + 'ignore', filter_pattern, category=DeprecationWarning) |
| 1653 | + # XXX wgrant 2010-03-30 bug=551510: |
| 1654 | + # Also filter apt_pkg warnings, since Lucid's python-apt has a new API. |
| 1655 | + warnings.filterwarnings( |
| 1656 | + 'ignore', '.*apt_pkg.*', category=DeprecationWarning) |
| 1657 | |
| 1658 | |
| 1659 | def customize_logger(): |
| 1660 | |
| 1661 | === modified file 'utilities/migrater/file-ownership.txt' |
| 1662 | --- utilities/migrater/file-ownership.txt 2010-11-25 04:42:51 +0000 |
| 1663 | +++ utilities/migrater/file-ownership.txt 2011-02-18 15:29:53 +0000 |
| 1664 | @@ -3846,7 +3846,6 @@ |
| 1665 | ./tests/test_branchtarget.py |
| 1666 | ./tests/test_branchurifield.py |
| 1667 | ./tests/test_bugnotification.py |
| 1668 | - ./tests/test_chunkydiff_setting.py |
| 1669 | ./tests/test_datetimeutils.py |
| 1670 | ./tests/test_helpers.py |
| 1671 | ./tests/test_imports.py |
Splendid. Thanks for cleaning this up, and for converting the doctest in particular. Only a few notes discussed on IRC:
* It'd be nice if we could use an off-the-shelf base64, but that's something for another day… benji mentioned base64.
* We're not sure how base() will react to the lowest possible twos-complement number, so might as well just reject negative numbers.
* s/eminate/emanate/ in a comment you moved (but obviously did not write).
* That same comment is unclear in that it may be about "all deprecation warnings (which emanate from zope)," or "all deprecation warnings that emanate from zope."
Keep 'em coming.