Storm

Merge lp:~cjwatson/storm/py3-strings into lp:storm

  1. py3-strings
  2. Merge into trunk
Proposed by Colin Watson
Status: Merged
Merged at revision: 526
Proposed branch: lp:~cjwatson/storm/py3-strings
Merge into: lp:storm
Prerequisite: lp:~cjwatson/storm/py3-tests-prepare-strings
Diff against target: 848 lines (+140/-103)
20 files modified
storm/cextensions.c (+9/-7)
storm/database.py (+3/-1)
storm/databases/postgres.py (+2/-5)
storm/databases/sqlite.py (+13/-9)
storm/expr.py (+26/-24)
storm/info.py (+2/-2)
storm/properties.py (+1/-0)
storm/references.py (+1/-1)
storm/sqlobject.py (+15/-7)
storm/tests/databases/base.py (+2/-1)
storm/tests/databases/postgres.py (+6/-2)
storm/tests/expr.py (+2/-2)
storm/tests/mocker.py (+1/-1)
storm/tests/sqlobject.py (+2/-2)
storm/tests/store/base.py (+4/-4)
storm/tests/tracer.py (+2/-1)
storm/tests/variables.py (+15/-11)
storm/tracer.py (+7/-2)
storm/tz.py (+2/-2)
storm/variables.py (+25/-19)
To merge this branch: bzr merge lp:~cjwatson/storm/py3-strings
Reviewer Review Type Date Requested Status
Simon Poirier (community) Approve
Review via email: mp+371173@code.launchpad.net

Commit message

Update string handling for Python 3.

Description of the change

This is inspired by work done by Thiago Bellini. I've taken a different approach in a few areas which are useful to discuss briefly here (and some of this should end up in release notes):

 * I generally wrote "bytes" rather than "six.binary_type", since it's shorter and works in all supported versions of Python.

 * In the SQLite backend, there's no need to use memoryview, because the sqlite3 module in Python 3 automatically converts between the SQLite BLOB type and bytes.

 * Some exception messages have changed slightly for clarity.

 * On Python 3, raw=True and token=True in storm.expr.Compile.__call__ only treats str specially, not bytes and str, because ultimately the compiler is assembling a text string to send to the database.

 * On Python 3, storm.tracer.BaseStatementTracer.connection_raw_execute renders text parameters using ascii() rather than by encoding to bytes and then calling repr(). While this does result in slightly different output from Python 2, it's normally more useful since the encoding is in terms of Unicode codepoints rather than UTF-8.

 * storm.sqlobject.AutoUnicodeVariable (and hence StringCol) explicitly documents that it only accepts text on Python 3, since native strings are already Unicode there so there's much less need for the porting affordance.

I also removed a special case for unicode query parameters to PostgreSQL, since psycopg2 has supported these natively since 1.99.3 or thereabouts, or at any rate as far back as we support.

To post a comment you must log in.
Revision history for this message
Simon Poirier (simpoir) wrote :

+1 looks good

minor nitpicks inline.

review: Approve
lp:~cjwatson/storm/py3-strings updated
514. By Colin Watson

Simplify type-checking conditions in Compile.__call__.

Revision history for this message
Colin Watson (cjwatson) wrote :

Thanks for spotting my any() foolishness.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'storm/cextensions.c'
2--- storm/cextensions.c 2019-08-11 09:48:05 +0000
3+++ storm/cextensions.c 2019-09-16 13:26:27 +0000
4@@ -1560,10 +1560,13 @@
5
6 /*
7 expr_type = type(expr)
8- if (expr_type is SQLRaw or
9- raw and (expr_type is str or expr_type is unicode)):
10+ string_types = (str,) if six.PY3 else (str, unicode)
11+ if expr_type is SQLRaw or (raw and expr_type in string_types):
12 return expr
13 */
14+ /* Note that PyString_CheckExact(o) is defined at the top of this file
15+ to 0 on Python 3, so we can safely translate the string_types checks
16+ here to PyString_CheckExact || PyUnicode_CheckExact. */
17 if ((PyObject *)expr->ob_type == SQLRaw ||
18 (raw && (PyString_CheckExact(expr) || PyUnicode_CheckExact(expr)))) {
19 /* Pass our reference on. */
20@@ -1571,7 +1574,7 @@
21 }
22
23 /*
24- if token and (expr_type is str or expr_type is unicode):
25+ if token and expr_type in string_types:
26 expr = SQLToken(expr)
27 */
28 if (token && (PyString_CheckExact(expr) || PyUnicode_CheckExact(expr))) {
29@@ -1601,8 +1604,8 @@
30 PyObject *subexpr = PySequence_Fast_GET_ITEM(sequence, i);
31 /*
32 subexpr_type = type(subexpr)
33- if subexpr_type is SQLRaw or raw and (subexpr_type is str or
34- subexpr_type is unicode):
35+ if (subexpr_type is SQLRaw or
36+ (raw and subexpr_type in string_types)):
37 */
38 if ((PyObject *)subexpr->ob_type == (PyObject *)SQLRaw ||
39 (raw && (PyString_CheckExact(subexpr) ||
40@@ -1623,8 +1626,7 @@
41 /* else: */
42 } else {
43 /*
44- if token and (subexpr_type is unicode or
45- subexpr_type is str):
46+ if token and subexpr_type in string_types:
47 */
48 if (token && (PyUnicode_CheckExact(subexpr) ||
49 PyString_CheckExact(subexpr))) {
50
51=== modified file 'storm/database.py'
52--- storm/database.py 2019-07-02 15:55:44 +0000
53+++ storm/database.py 2019-09-16 13:26:27 +0000
54@@ -33,6 +33,8 @@
55 from collections import Callable
56 from functools import wraps
57
58+import six
59+
60 from storm.expr import Expr, State, compile
61 # Circular import: imported at the end of the module.
62 # from storm.tracer import trace
63@@ -677,7 +679,7 @@
64 - "anything:..." Where 'anything' has previously been registered
65 with L{register_scheme}.
66 """
67- if isinstance(uri, basestring):
68+ if isinstance(uri, six.string_types):
69 uri = URI(uri)
70 if uri.scheme in _database_schemes:
71 factory = _database_schemes[uri.scheme]
72
73=== modified file 'storm/databases/postgres.py'
74--- storm/databases/postgres.py 2019-08-11 14:37:04 +0000
75+++ storm/databases/postgres.py 2019-09-16 13:26:27 +0000
76@@ -308,17 +308,14 @@
77 def to_database(self, params):
78 """
79 Like L{Connection.to_database}, but this converts datetime
80- types to strings, unicode to UTF-8 encoded strings, and
81- strings to L{psycopg2.Binary} instances.
82+ types to strings, and bytes to L{psycopg2.Binary} instances.
83 """
84 for param in params:
85 if isinstance(param, Variable):
86 param = param.get(to_db=True)
87 if isinstance(param, (datetime, date, time, timedelta)):
88 yield str(param)
89- elif isinstance(param, unicode):
90- yield param.encode("UTF-8")
91- elif isinstance(param, str):
92+ elif isinstance(param, bytes):
93 yield psycopg2.Binary(param)
94 else:
95 yield param
96
97=== modified file 'storm/databases/sqlite.py'
98--- storm/databases/sqlite.py 2019-06-25 21:33:26 +0000
99+++ storm/databases/sqlite.py 2019-09-16 13:26:27 +0000
100@@ -24,6 +24,8 @@
101 from time import sleep, time as now
102 import sys
103
104+import six
105+
106 from storm.databases import dummy
107
108 try:
109@@ -83,21 +85,23 @@
110
111 @staticmethod
112 def set_variable(variable, value):
113- if isinstance(variable, RawStrVariable):
114+ if (isinstance(variable, RawStrVariable) and
115+ isinstance(value, six.text_type)):
116 # pysqlite2 may return unicode.
117- value = str(value)
118+ value = value.encode("UTF-8")
119 variable.set(value, from_db=True)
120
121 @staticmethod
122 def from_database(row):
123- """Convert MySQL-specific datatypes to "normal" Python types.
124+ """Convert SQLite-specific datatypes to "normal" Python types.
125
126- If there are anny C{buffer} instances in the row, convert them
127- to strings.
128+ On Python 2, if there are any C{buffer} instances in the row,
129+ convert them to bytes. On Python 3, BLOB types are converted to
130+ bytes, which is already what we want.
131 """
132 for value in row:
133- if isinstance(value, buffer):
134- yield str(value)
135+ if six.PY2 and isinstance(value, buffer):
136+ yield bytes(value)
137 else:
138 yield value
139
140@@ -112,7 +116,7 @@
141 def to_database(params):
142 """
143 Like L{Connection.to_database}, but this also converts
144- instances of L{datetime} types to strings, and strings
145+ instances of L{datetime} types to strings, and (on Python 2) bytes
146 instances to C{buffer} instances.
147 """
148 for param in params:
149@@ -120,7 +124,7 @@
150 param = param.get(to_db=True)
151 if isinstance(param, (datetime, date, time, timedelta)):
152 yield str(param)
153- elif isinstance(param, str):
154+ elif six.PY2 and isinstance(param, bytes):
155 yield buffer(param)
156 else:
157 yield param
158
159=== modified file 'storm/expr.py'
160--- storm/expr.py 2019-08-12 14:57:51 +0000
161+++ storm/expr.py 2019-09-16 13:26:27 +0000
162@@ -157,21 +157,23 @@
163 created internally (and thus can't be accessed).
164 @param join: The string token to use to put between
165 subexpressions. Defaults to ", ".
166- @param raw: If true, any string or unicode expression or
167- subexpression will not be further compiled.
168- @param token: If true, any string or unicode expression will
169- be considered as a SQLToken, and quoted properly.
170+ @param raw: If true, any string (str or unicode on Python 2, str on
171+ Python 3) expression or subexpression will not be further
172+ compiled.
173+ @param token: If true, any string (str or unicode on Python 2, str
174+ on Python 3) expression will be considered as a SQLToken, and
175+ quoted properly.
176 """
177 # FASTPATH This method is part of the fast path. Be careful when
178 # changing it (try to profile any changes).
179
180 expr_type = type(expr)
181+ string_types = (str,) if six.PY3 else (str, unicode)
182
183- if (expr_type is SQLRaw or
184- raw and (expr_type is str or expr_type is unicode)):
185+ if expr_type is SQLRaw or (raw and expr_type in string_types):
186 return expr
187
188- if token and (expr_type is str or expr_type is unicode):
189+ if token and expr_type in string_types:
190 expr = SQLToken(expr)
191
192 if state is None:
193@@ -182,15 +184,14 @@
194 compiled = []
195 for subexpr in expr:
196 subexpr_type = type(subexpr)
197- if subexpr_type is SQLRaw or raw and (subexpr_type is str or
198- subexpr_type is unicode):
199+ if (subexpr_type is SQLRaw or
200+ (raw and subexpr_type in string_types)):
201 statement = subexpr
202 elif subexpr_type is tuple or subexpr_type is list:
203 state.precedence = outer_precedence
204 statement = self(subexpr, state, join, raw, token)
205 else:
206- if token and (subexpr_type is unicode or
207- subexpr_type is str):
208+ if token and subexpr_type in string_types:
209 subexpr = SQLToken(subexpr)
210 statement = self._compile_single(subexpr, state,
211 outer_precedence)
212@@ -307,13 +308,13 @@
213 # --------------------------------------------------------------------
214 # Builtin type support
215
216-@compile.when(str)
217-def compile_str(compile, expr, state):
218+@compile.when(bytes)
219+def compile_bytes(compile, expr, state):
220 state.parameters.append(RawStrVariable(expr))
221 return "?"
222
223-@compile.when(unicode)
224-def compile_unicode(compile, expr, state):
225+@compile.when(six.text_type)
226+def compile_text(compile, expr, state):
227 state.parameters.append(UnicodeVariable(expr))
228 return "?"
229
230@@ -362,7 +363,8 @@
231 return "NULL"
232
233
234-@compile_python.when(str, unicode, float, type(None), *six.integer_types)
235+@compile_python.when(
236+ bytes, six.text_type, float, type(None), *six.integer_types)
237 def compile_python_builtin(compile, expr, state):
238 return repr(expr)
239
240@@ -516,20 +518,20 @@
241 return Upper(self)
242
243 def startswith(self, prefix):
244- if not isinstance(prefix, unicode):
245- raise ExprError("Expected unicode argument, got %r" % type(prefix))
246+ if not isinstance(prefix, six.text_type):
247+ raise ExprError("Expected text argument, got %r" % type(prefix))
248 pattern = prefix.translate(like_escape) + u"%"
249 return Like(self, pattern, u"!")
250
251 def endswith(self, suffix):
252- if not isinstance(suffix, unicode):
253- raise ExprError("Expected unicode argument, got %r" % type(suffix))
254+ if not isinstance(suffix, six.text_type):
255+ raise ExprError("Expected text argument, got %r" % type(suffix))
256 pattern = u"%" + suffix.translate(like_escape)
257 return Like(self, pattern, u"!")
258
259 def contains_string(self, substring):
260- if not isinstance(substring, unicode):
261- raise ExprError("Expected unicode argument, got %r" % type(substring))
262+ if not isinstance(substring, six.text_type):
263+ raise ExprError("Expected text argument, got %r" % type(substring))
264 pattern = u"%" + substring.translate(like_escape) + u"%"
265 return Like(self, pattern, u"!")
266
267@@ -1422,7 +1424,7 @@
268 # --------------------------------------------------------------------
269 # Plain SQL expressions.
270
271-class SQLRaw(str):
272+class SQLRaw(six.text_type):
273 """Subtype to mark a string as something that shouldn't be compiled.
274
275 This is handled internally by the compiler.
276@@ -1430,7 +1432,7 @@
277 __slots__ = ()
278
279
280-class SQLToken(str):
281+class SQLToken(six.text_type):
282 """Marker for strings that should be considered as a single SQL token.
283
284 These strings will be quoted, when needed.
285
286=== modified file 'storm/info.py'
287--- storm/info.py 2019-08-12 14:57:51 +0000
288+++ storm/info.py 2019-09-16 13:26:27 +0000
289@@ -77,7 +77,7 @@
290
291 self.cls = cls
292
293- if isinstance(self.table, basestring):
294+ if isinstance(self.table, six.string_types):
295 self.table = Table(self.table)
296
297 pairs = []
298@@ -135,7 +135,7 @@
299 __order__ = (__order__,)
300 self.default_order = []
301 for item in __order__:
302- if isinstance(item, basestring):
303+ if isinstance(item, six.string_types):
304 if item.startswith("-"):
305 prop = Desc(getattr(cls, item[1:]))
306 else:
307
308=== modified file 'storm/properties.py'
309--- storm/properties.py 2019-06-07 12:17:35 +0000
310+++ storm/properties.py 2019-09-16 13:26:27 +0000
311@@ -147,6 +147,7 @@
312 class Decimal(SimpleProperty):
313 variable_class = DecimalVariable
314
315+# Bytes might be a clearer name nowadays, but we keep this for compatibility.
316 class RawStr(SimpleProperty):
317 variable_class = RawStrVariable
318
319
320=== modified file 'storm/references.py'
321--- storm/references.py 2019-08-12 14:57:51 +0000
322+++ storm/references.py 2019-09-16 13:26:27 +0000
323@@ -916,7 +916,7 @@
324 def resolve_one(self, property):
325 if type(property) is tuple:
326 return self.resolve(property)
327- elif isinstance(property, basestring):
328+ elif isinstance(property, six.string_types):
329 return self._resolve_string(property)
330 elif isinstance(property, SuffixExpr):
331 # XXX This covers cases like order_by=Desc("Bar.id"), see #620369.
332
333=== modified file 'storm/sqlobject.py'
334--- storm/sqlobject.py 2019-08-21 09:40:30 +0000
335+++ storm/sqlobject.py 2019-09-16 13:26:27 +0000
336@@ -209,7 +209,7 @@
337
338
339 id_type = dict.setdefault("_idType", int)
340- id_cls = {int: Int, str: RawStr, unicode: AutoUnicode}[id_type]
341+ id_cls = {int: Int, bytes: RawStr, six.text_type: AutoUnicode}[id_type]
342 dict["id"] = id_cls(id_name, primary=True, default=AutoReload)
343 attr_to_prop[id_name] = "id"
344
345@@ -332,7 +332,7 @@
346 if not isinstance(orderBy, (tuple, list)):
347 orderBy = (orderBy,)
348 for item in orderBy:
349- if isinstance(item, basestring):
350+ if isinstance(item, six.string_types):
351 desc = item.startswith("-")
352 if desc:
353 item = item[1:]
354@@ -609,7 +609,7 @@
355 return self._copy(prejoinClauseTables=prejoinClauseTables)
356
357 def sum(self, attribute):
358- if isinstance(attribute, basestring):
359+ if isinstance(attribute, six.string_types):
360 attribute = SQL(attribute)
361 result_set = self._without_prejoins()._result_set
362 return result_set.sum(attribute)
363@@ -681,13 +681,21 @@
364
365
366 class AutoUnicodeVariable(Variable):
367- """Unlike UnicodeVariable, this will try to convert str to unicode."""
368+ """A more relaxed version of UnicodeVariable that accepts native strings.
369+
370+ On Python 2, this will try to convert bytes to text, to make it easier
371+ to port code from SQLObject that expects to be able to set this variable
372+ to a native string.
373+
374+ On Python 3, this behaves the same way as UnicodeVariable and only
375+ accepts text, since native strings are already Unicode.
376+ """
377 __slots__ = ()
378
379 def parse_set(self, value, from_db):
380- if not isinstance(value, basestring):
381- raise TypeError("Expected basestring, found %s" % repr(type(value)))
382- return unicode(value)
383+ if not isinstance(value, six.string_types):
384+ raise TypeError("Expected a string type, found %r" % type(value))
385+ return six.text_type(value)
386
387 class AutoUnicode(SimpleProperty):
388 variable_class = AutoUnicodeVariable
389
390=== modified file 'storm/tests/databases/base.py'
391--- storm/tests/databases/base.py 2019-08-12 17:07:08 +0000
392+++ storm/tests/databases/base.py 2019-09-16 13:26:27 +0000
393@@ -27,6 +27,7 @@
394 import sys
395 import os
396
397+import six
398 from six.moves import cPickle as pickle
399
400 from storm.uri import URI
401@@ -151,7 +152,7 @@
402 self.assertTrue(isinstance(result, Result))
403 row = result.get_one()
404 self.assertEquals(row, ("Title 10",))
405- self.assertTrue(isinstance(row[0], unicode))
406+ self.assertTrue(isinstance(row[0], six.text_type))
407
408 def test_execute_params(self):
409 result = self.connection.execute("SELECT one FROM number "
410
411=== modified file 'storm/tests/databases/postgres.py'
412--- storm/tests/databases/postgres.py 2019-08-12 17:07:08 +0000
413+++ storm/tests/databases/postgres.py 2019-09-16 13:26:27 +0000
414@@ -24,6 +24,8 @@
415 import os
416 import json
417
418+import six
419+
420 from storm.databases.postgres import (
421 Postgres, compile, currval, Returning, Case, PostgresTimeoutTracer,
422 make_dsn, JSONElement, JSONTextElement, JSON)
423@@ -157,7 +159,7 @@
424 result = connection.execute("SELECT title FROM test WHERE id=1")
425 title = result.get_one()[0]
426
427- self.assertTrue(isinstance(title, unicode))
428+ self.assertTrue(isinstance(title, six.text_type))
429 self.assertEquals(title, uni_str)
430
431 def test_unicode_array(self):
432@@ -681,7 +683,9 @@
433
434 connection = self.database.connect()
435 value = {"a": 3, "b": "foo", "c": None}
436- db_value = json.dumps(value).decode("utf-8")
437+ db_value = json.dumps(value)
438+ if six.PY2:
439+ db_value = db_value.decode("utf-8")
440 connection.execute(
441 "INSERT INTO json_test (json) VALUES (?)", (db_value,))
442 connection.commit()
443
444=== modified file 'storm/tests/expr.py'
445--- storm/tests/expr.py 2019-08-14 13:36:13 +0000
446+++ storm/tests/expr.py 2019-09-16 13:26:27 +0000
447@@ -2165,11 +2165,11 @@
448
449 def test_bytes(self):
450 py_expr = compile_python(b"str")
451- self.assertEquals(py_expr, "'str'")
452+ self.assertEquals(py_expr, "b'str'" if six.PY3 else "'str'")
453
454 def test_unicode(self):
455 py_expr = compile_python(u"str")
456- self.assertEquals(py_expr, "u'str'")
457+ self.assertEquals(py_expr, "'str'" if six.PY3 else "u'str'")
458
459 def test_int(self):
460 py_expr = compile_python(1)
461
462=== modified file 'storm/tests/mocker.py'
463--- storm/tests/mocker.py 2019-08-21 11:14:38 +0000
464+++ storm/tests/mocker.py 2019-09-16 13:26:27 +0000
465@@ -559,7 +559,7 @@
466 explicitly requested via the L{passthrough()}
467 method.
468 """
469- if isinstance(object, basestring):
470+ if isinstance(object, six.string_types):
471 if name is None:
472 name = object
473 import_stack = object.split(".")
474
475=== modified file 'storm/tests/sqlobject.py'
476--- storm/tests/sqlobject.py 2019-08-14 13:36:13 +0000
477+++ storm/tests/sqlobject.py 2019-09-16 13:26:27 +0000
478@@ -124,7 +124,7 @@
479 _defaultOrder = "-Person.name"
480 _table = "person"
481 _idName = "name"
482- _idType = unicode
483+ _idType = six.text_type
484 age = IntCol()
485 ts = UtcDateTimeCol()
486
487@@ -1190,7 +1190,7 @@
488 # properties:
489 class Person(self.SQLObject):
490 _idName = "name"
491- _idType = unicode
492+ _idType = six.text_type
493 address = ForeignKey(foreignKey="Phone", dbName="address_id",
494 notNull=True)
495
496
497=== modified file 'storm/tests/store/base.py'
498--- storm/tests/store/base.py 2019-08-21 09:40:30 +0000
499+++ storm/tests/store/base.py 2019-09-16 13:26:27 +0000
500@@ -1051,7 +1051,7 @@
501 def test_find_max_unicode(self):
502 title = self.store.find(Foo).max(Foo.title)
503 self.assertEquals(title, "Title 30")
504- self.assertTrue(isinstance(title, unicode))
505+ self.assertTrue(isinstance(title, six.text_type))
506
507 def test_find_max_with_empty_result_and_disallow_none(self):
508 class Bar(object):
509@@ -1072,7 +1072,7 @@
510 def test_find_min_unicode(self):
511 title = self.store.find(Foo).min(Foo.title)
512 self.assertEquals(title, "Title 10")
513- self.assertTrue(isinstance(title, unicode))
514+ self.assertTrue(isinstance(title, six.text_type))
515
516 def test_find_min_with_empty_result_and_disallow_none(self):
517 class Bar(object):
518@@ -1156,7 +1156,7 @@
519 values = list(values)
520 self.assertEquals(values, ["Title 30", "Title 20", "Title 10"])
521 self.assertEquals([type(value) for value in values],
522- [unicode, unicode, unicode])
523+ [six.text_type, six.text_type, six.text_type])
524
525 def test_find_multiple_values(self):
526 result = self.store.find(Foo).order_by(Foo.id)
527@@ -5933,7 +5933,7 @@
528 try:
529 self.assertEquals(myfoo.title, title)
530 except AssertionError as e:
531- raise AssertionError(unicode(e, 'replace') +
532+ raise AssertionError(six.text_type(e, 'replace') +
533 " (ensure your database was created with CREATE DATABASE"
534 " ... CHARACTER SET utf8)")
535
536
537=== modified file 'storm/tests/tracer.py'
538--- storm/tests/tracer.py 2019-08-12 17:07:08 +0000
539+++ storm/tests/tracer.py 2019-09-16 13:26:27 +0000
540@@ -534,7 +534,8 @@
541 [var1])
542 self.assertEqual(
543 [(conn, 'cursor',
544- "Unformattable query: '%s %s' with params [u'substring'].")],
545+ "Unformattable query: '%%s %%s' with params [%r]." %
546+ u'substring')],
547 tracer.calls)
548
549
550
551=== modified file 'storm/tests/variables.py'
552--- storm/tests/variables.py 2019-08-14 13:36:13 +0000
553+++ storm/tests/variables.py 2019-09-16 13:26:27 +0000
554@@ -446,7 +446,8 @@
555 variable = RawStrVariable()
556 variable.set(b"str")
557 self.assertEquals(variable.get(), b"str")
558- variable.set(buffer(b"buffer"))
559+ buffer_type = memoryview if six.PY3 else buffer
560+ variable.set(buffer_type(b"buffer"))
561 self.assertEquals(variable.get(), b"buffer")
562 self.assertRaises(TypeError, variable.set, u"unicode")
563
564@@ -480,7 +481,7 @@
565
566 def test_get_set_from_database(self):
567 datetime_str = "1977-05-04 12:34:56.78"
568- datetime_uni = unicode(datetime_str)
569+ datetime_uni = six.text_type(datetime_str)
570 datetime_obj = datetime(1977, 5, 4, 12, 34, 56, 780000)
571
572 variable = DateTimeVariable()
573@@ -493,7 +494,7 @@
574 self.assertEquals(variable.get(), datetime_obj)
575
576 datetime_str = "1977-05-04 12:34:56"
577- datetime_uni = unicode(datetime_str)
578+ datetime_uni = six.text_type(datetime_str)
579 datetime_obj = datetime(1977, 5, 4, 12, 34, 56)
580
581 variable.set(datetime_str, from_db=True)
582@@ -558,7 +559,7 @@
583
584 def test_get_set_from_database(self):
585 date_str = "1977-05-04"
586- date_uni = unicode(date_str)
587+ date_uni = six.text_type(date_str)
588 date_obj = date(1977, 5, 4)
589 datetime_obj = datetime(1977, 5, 4, 0, 0, 0)
590
591@@ -602,7 +603,7 @@
592
593 def test_get_set_from_database(self):
594 time_str = "12:34:56.78"
595- time_uni = unicode(time_str)
596+ time_uni = six.text_type(time_str)
597 time_obj = time(12, 34, 56, 780000)
598
599 variable = TimeVariable()
600@@ -615,7 +616,7 @@
601 self.assertEquals(variable.get(), time_obj)
602
603 time_str = "12:34:56"
604- time_uni = unicode(time_str)
605+ time_uni = six.text_type(time_str)
606 time_obj = time(12, 34, 56)
607
608 variable.set(time_str, from_db=True)
609@@ -672,7 +673,7 @@
610
611 def test_get_set_from_database(self):
612 delta_str = "42 days 12:34:56.78"
613- delta_uni = unicode(delta_str)
614+ delta_uni = six.text_type(delta_str)
615 delta_obj = timedelta(days=42, hours=12, minutes=34,
616 seconds=56, microseconds=780000)
617
618@@ -686,7 +687,7 @@
619 self.assertEquals(variable.get(), delta_obj)
620
621 delta_str = "1 day, 12:34:56"
622- delta_uni = unicode(delta_str)
623+ delta_uni = six.text_type(delta_str)
624 delta_obj = timedelta(days=1, hours=12, minutes=34, seconds=56)
625
626 variable.set(delta_str, from_db=True)
627@@ -901,7 +902,10 @@
628
629 class JSONVariableTest(EncodedValueVariableTestMixin, TestHelper):
630
631- encode = staticmethod(lambda data: json.dumps(data).decode("utf-8"))
632+ if six.PY3:
633+ encode = staticmethod(lambda data: json.dumps(data))
634+ else:
635+ encode = staticmethod(lambda data: json.dumps(data).decode("utf-8"))
636 variable_type = JSONVariable
637
638 def is_supported(self):
639@@ -914,11 +918,11 @@
640 self.assertRaises(TypeError, variable.set, b'"abc"', from_db=True)
641
642 def test_unicode_to_db(self):
643- # JSONVariable._dumps() works around unicode/str handling issues in
644+ # JSONVariable._dumps() works around text/bytes handling issues in
645 # json.
646 variable = self.variable_type()
647 variable.set({u"a": 1})
648- self.assertTrue(isinstance(variable.get(to_db=True), unicode))
649+ self.assertTrue(isinstance(variable.get(to_db=True), six.text_type))
650
651
652 class ListVariableTest(TestHelper):
653
654=== modified file 'storm/tracer.py'
655--- storm/tracer.py 2019-06-05 11:41:07 +0000
656+++ storm/tracer.py 2019-09-16 13:26:27 +0000
657@@ -5,6 +5,8 @@
658 import sys
659 import threading
660
661+import six
662+
663 # Circular import: imported at the end of the module.
664 # from storm.database import convert_param_marks
665 from storm.exceptions import TimeoutError
666@@ -171,8 +173,11 @@
667 # string parameters which represent encoded binary data.
668 render_params = []
669 for param in query_params:
670- if isinstance(param, unicode):
671- render_params.append(repr(param.encode('utf8')))
672+ if isinstance(param, six.text_type):
673+ if six.PY3:
674+ render_params.append(ascii(param))
675+ else:
676+ render_params.append(repr(param.encode('utf8')))
677 else:
678 render_params.append(repr(param))
679 try:
680
681=== modified file 'storm/tz.py'
682--- storm/tz.py 2019-08-12 14:57:51 +0000
683+++ storm/tz.py 2019-09-16 13:26:27 +0000
684@@ -198,7 +198,7 @@
685 # ftp://elsie.nci.nih.gov/pub/tz*.tar.gz
686
687 def __init__(self, fileobj):
688- if isinstance(fileobj, basestring):
689+ if isinstance(fileobj, six.string_types):
690 self._filename = fileobj
691 fileobj = open(fileobj)
692 elif hasattr(fileobj, "name"):
693@@ -699,7 +699,7 @@
694 if not rrule:
695 from dateutil import rrule
696
697- if isinstance(fileobj, basestring):
698+ if isinstance(fileobj, six.string_types):
699 self._s = fileobj
700 fileobj = open(fileobj)
701 elif hasattr(fileobj, "name"):
702
703=== modified file 'storm/variables.py'
704--- storm/variables.py 2019-06-07 17:14:33 +0000
705+++ storm/variables.py 2019-09-16 13:26:27 +0000
706@@ -57,6 +57,12 @@
707 ]
708
709
710+if six.PY3:
711+ _buffer_type = memoryview
712+else:
713+ _buffer_type = buffer
714+
715+
716 class LazyValue(object):
717 """Marker to be used as a base class on lazily evaluated values."""
718 __slots__ = ()
719@@ -348,7 +354,7 @@
720
721 @staticmethod
722 def parse_set(value, from_db):
723- if (from_db and isinstance(value, basestring) or
724+ if (from_db and isinstance(value, six.string_types) or
725 isinstance(value, six.integer_types)):
726 value = Decimal(value)
727 elif not isinstance(value, Decimal):
728@@ -359,7 +365,7 @@
729 @staticmethod
730 def parse_get(value, to_db):
731 if to_db:
732- return unicode(value)
733+ return six.text_type(value)
734 return value
735
736
737@@ -367,10 +373,10 @@
738 __slots__ = ()
739
740 def parse_set(self, value, from_db):
741- if isinstance(value, buffer):
742- value = str(value)
743- elif not isinstance(value, str):
744- raise TypeError("Expected str, found %r: %r"
745+ if isinstance(value, _buffer_type):
746+ value = bytes(value)
747+ elif not isinstance(value, bytes):
748+ raise TypeError("Expected bytes, found %r: %r"
749 % (type(value), value))
750 return value
751
752@@ -379,8 +385,8 @@
753 __slots__ = ()
754
755 def parse_set(self, value, from_db):
756- if not isinstance(value, unicode):
757- raise TypeError("Expected unicode, found %r: %r"
758+ if not isinstance(value, six.text_type):
759+ raise TypeError("Expected text, found %r: %r"
760 % (type(value), value))
761 return value
762
763@@ -396,7 +402,7 @@
764 if from_db:
765 if isinstance(value, datetime):
766 pass
767- elif isinstance(value, (str, unicode)):
768+ elif isinstance(value, six.string_types):
769 if " " not in value:
770 raise ValueError("Unknown date/time format: %r" % value)
771 date_str, time_str = value.split(" ")
772@@ -430,7 +436,7 @@
773 return value.date()
774 if isinstance(value, date):
775 return value
776- if not isinstance(value, (str, unicode)):
777+ if not isinstance(value, six.string_types):
778 raise TypeError("Expected date, found %s" % repr(value))
779 if " " in value:
780 value, time_str = value.split(" ")
781@@ -453,7 +459,7 @@
782 return None
783 if isinstance(value, time):
784 return value
785- if not isinstance(value, (str, unicode)):
786+ if not isinstance(value, six.string_types):
787 raise TypeError("Expected time, found %s" % repr(value))
788 if " " in value:
789 date_str, value = value.split(" ")
790@@ -476,7 +482,7 @@
791 return None
792 if isinstance(value, timedelta):
793 return value
794- if not isinstance(value, (str, unicode)):
795+ if not isinstance(value, six.string_types):
796 raise TypeError("Expected timedelta, found %s" % repr(value))
797 return _parse_interval(value)
798 else:
799@@ -489,7 +495,7 @@
800 __slots__ = ()
801
802 def parse_set(self, value, from_db):
803- if from_db and isinstance(value, basestring):
804+ if from_db and isinstance(value, six.string_types):
805 value = uuid.UUID(value)
806 elif not isinstance(value, uuid.UUID):
807 raise TypeError("Expected UUID, found %r: %r"
808@@ -498,7 +504,7 @@
809
810 def parse_get(self, value, to_db):
811 if to_db:
812- return unicode(value)
813+ return six.text_type(value)
814 return value
815
816
817@@ -581,8 +587,8 @@
818
819 def parse_set(self, value, from_db):
820 if from_db:
821- if isinstance(value, buffer):
822- value = str(value)
823+ if isinstance(value, _buffer_type):
824+ value = bytes(value)
825 return self._loads(value)
826 else:
827 return value
828@@ -615,7 +621,7 @@
829 __slots__ = ()
830
831 def _loads(self, value):
832- if not isinstance(value, unicode):
833+ if not isinstance(value, six.text_type):
834 raise TypeError(
835 "Cannot safely assume encoding of byte string %r." % value)
836 return json.loads(value)
837@@ -623,9 +629,9 @@
838 def _dumps(self, value):
839 # http://www.ietf.org/rfc/rfc4627.txt states that JSON is text-based
840 # and so we treat it as such here. In other words, this method returns
841- # unicode and never str.
842+ # Unicode text and never bytes.
843 dump = json.dumps(value, ensure_ascii=False)
844- if not isinstance(dump, unicode):
845+ if not isinstance(dump, six.text_type):
846 # json.dumps() does not always return unicode. See
847 # http://code.google.com/p/simplejson/issues/detail?id=40 for one
848 # of many discussions of str/unicode handling in simplejson.

Subscribers

People subscribed via source and target branches

to status/vote changes: