Storm

Merge lp:~jbaker/storm/oracle_support into lp:storm

  1. oracle_support
  2. Merge into trunk
Proposed by Jason Baker
Status: Needs review
Proposed branch: lp:~jbaker/storm/oracle_support
Merge into: lp:storm
Diff against target: 1573 lines (+1287/-39)
9 files modified
README (+38/-2)
oracle_test_setup.sql (+13/-0)
storm/databases/oracle.py (+590/-0)
storm/expr.py (+1/-1)
storm/variables.py (+2/-0)
tests/databases/base.py (+32/-31)
tests/databases/oracle.py (+372/-0)
tests/store/base.py (+5/-5)
tests/store/oracle.py (+234/-0)
To merge this branch: bzr merge lp:~jbaker/storm/oracle_support
Reviewer Review Type Date Requested Status
Jamu Kakar (community) Abstain
Storm Developers Pending
Review via email: mp+15442@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Jason Baker (jbaker) wrote :

This should be ready for merge. I've commented out the disconnection tests for now. Some parts of this are still somewhat ugly, but it's a good starting point.

Reply
Revision history for this message
Jamu Kakar (jkakar) wrote :

[1]

I've managed to get Oracle XE installed, but I'm having trouble
getting cx_Oracle in place, so I can't run the test suite yet. I'm
going to keep at it, but if you have any hints on how to get
cx_Oracle in place (on a 64-bit machine) I'd appreciate some help.
Ideally, packages for Ubuntu would be nice.

[2]

I've pushed a branch that adds some details about getting Oracle XE
installed on a 64-bit machine to the README file and makes some
cosmetic changes to make the code match the Storm coding style more
closely. Please merge:

lp:~jkakar/storm/oracle-support-tweaks

[3]

Why isn't the builtin reserved words functionality being used?

I've made a broad pass over the changeset and it looks really good.
I'd like to get the test suite running before I do a more detailed
pass.

Reply
Revision history for this message
Jason Baker (jbaker) wrote :

> [1]
>
> I've managed to get Oracle XE installed, but I'm having trouble
> getting cx_Oracle in place, so I can't run the test suite yet. I'm
> going to keep at it, but if you have any hints on how to get
> cx_Oracle in place (on a 64-bit machine) I'd appreciate some help.
> Ideally, packages for Ubuntu would be nice.

I found this blog post to be a good help: http://catherinedevlin.blogspot.com/2007/10/cxoracle-and-oracle-xe-on-ubuntu.html ... Although note that they're a little bit outdated. I don't believe it's necessary to install the XE patch.

If you can give me a specific error message, I might be able to help out.

> [2]
>
> I've pushed a branch that adds some details about getting Oracle XE
> installed on a 64-bit machine to the README file and makes some
> cosmetic changes to make the code match the Storm coding style more
> closely. Please merge:
>
> lp:~jkakar/storm/oracle-support-tweaks

I'll look at them. Thanks!

> [3]
>
> Why isn't the builtin reserved words functionality being used?

I don't recall the *exact* reason why I did that, but there was some issue with the reserved word keylist considering words to be reserved that weren't. Perhaps a better solution would be to update the reserved word list. It's a bit dated (SQL 1992).

Reply
Revision history for this message
Jason Baker (jbaker) wrote :

> lp:~jkakar/storm/oracle-support-tweaks

Erm.... This branch doesn't seem to exist. Unless of course I'm doing something wrong (which is possible).

Reply
Revision history for this message
Jamu Kakar (jkakar) wrote :

Oops, the branch was private by mistake. It should be visible now.

Reply
lp:~jbaker/storm/oracle_support updated
357. By Jason Baker

Merging in changes from trunk.

358. By Jason Baker

Merging some changes in from lp:~jkakar/storm/oracle-support-tweaks

359. By Jason Baker

Updating from trunk.

360. By Jason Baker

Making the oracle backend properly handle disconnection errors

Revision history for this message
Jason Baker (jbaker) wrote :

Where do we stand on this? I've just merged in the latest revisions from trunk and also added a bugfix that makes the disconnection tests now pass.

Reply
Revision history for this message
Jamu Kakar (jkakar) wrote :

I had spent some time looking at this a while ago. I got as far as
getting Oracle installed (which was hard on 64-bit) and then stopped
when I had to figure out how to compile cxOracle DB-API drivers. I
looked over the code, and it looks good, but want to actually be
able to run the test suite.

I've since installed a new disk in my machine, so I'll need to setup
Oracle et al. again to continue. I've been thinking that we really
need a buildbot, since getting Oracle in place is rather awkward and
most Storm developers probably won't want to do it.

Reply
Revision history for this message
Jamu Kakar (jkakar) wrote :

I haven't managed to get Oracle installed. I'm going to abstain
from reviewing this for now.

review: Abstain
Reply

Unmerged revisions

360. By Jason Baker

Making the oracle backend properly handle disconnection errors

359. By Jason Baker

Updating from trunk.

358. By Jason Baker

Merging some changes in from lp:~jkakar/storm/oracle-support-tweaks

357. By Jason Baker

Merging in changes from trunk.

356. By Jason Baker <jason@ubuntu>

Added instructions for getting started with Oracle.

355. By Jason Baker <jason@ubuntu>

Adding in a script to set up a test user in an oracle database.

354. By Jason Baker <jason@ubuntu>

Reverting the error handling back to the way it was for now. For whatever reason, this led to spurious error reports when running tests.

353. By Jason Baker <jason@ubuntu>

Temporarily commenting out the Disconnection tests until they can be fixed.

352. By Jason Baker <jason@ubuntu>

Adding an error description for cx_Oracle not being found

351. By Jason Baker <jason@ubuntu>

Merge from trunk.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'README'
--- README 2009-10-08 07:23:43 +0000
+++ README 2010-03-15 15:48:27 +0000
@@ -95,8 +95,8 @@
95same procedure will probably work without changes on a Debian system95same procedure will probably work without changes on a Debian system
96and with minimal changes on a non-Debian-based linux distribution.96and with minimal changes on a non-Debian-based linux distribution.
97In order to run the test suite, and exercise all supported backends,97In order to run the test suite, and exercise all supported backends,
98you will need to install MySQL and PostgreSQL, along with the98you will need to install MySQL, PostgreSQL and Oracle XE, along with
99related Python database drivers:99the related Python database drivers:
100100
101 $ sudo apt-get install python-mysqldb python-psycopg2 mysql-server \101 $ sudo apt-get install python-mysqldb python-psycopg2 mysql-server \
102 postgresql build-essential102 postgresql build-essential
@@ -110,6 +110,34 @@
110asked to enter a password multiple times. Leave it blank in each110asked to enter a password multiple times. Leave it blank in each
111case.111case.
112112
113To install Oracle on a 32-bit system you must first add the
114following line to your /etc/apt/sources.list:
115
116 deb http://oss.oracle.com/debian unstable main non-free
117
118You can then install Oracle XE by running the following commands:
119
120 $ sudo wget http://oss.oracle.com/el4/RPM-GPG-KEY-oracle -O- | sudo apt-key add -
121 $ sudo apt-get update
122 $ sudo apt-get install oracle-xe
123
124Binaries for Oracle XE on 64-bit systems are not available, so
125you'll need to install the 32-bit versions. Start by downloading
126the packages:
127
128 $ wget -c http://oss.oracle.com/debian/dists/unstable/main/binary-i386/libaio_0.3.104-1_i386.deb http://oss.oracle.com/debian/dists/unstable/non-free/binary-i386/oracle-xe-universal_10.2.0.1-1.1_i386.deb
129
130Once you have the packages you need to install them manually:
131
132 $ sudo apt-get install libc6-i386 bc
133 $ sudo dpkg -i --force-architecture libaio_0.3.104-1_i386.deb
134 $ sudo dpkg -i --force-architecture oracle-xe-universal_10.2.0.1-1.1_i386.deb
135
136Finally, when the packages are installed, you can then run the
137following line to configure the database:
138
139 $ sudo /etc/init.d/oracle-xe configure
140
113Setting up database users and access security141Setting up database users and access security
114---------------------------------------------142---------------------------------------------
115143
@@ -145,6 +173,14 @@
145mysql> GRANT ALL PRIVILEGES ON *.* TO 'YOUR_USERNAME'@'localhost'173mysql> GRANT ALL PRIVILEGES ON *.* TO 'YOUR_USERNAME'@'localhost'
146IDENTIFIED BY '' WITH GRANT OPTION;174IDENTIFIED BY '' WITH GRANT OPTION;
147175
176For Oracle, we've included a script to set up a test user. To run it,
177do the following:
178
179 $ sqlplus sys/<sys_password>@localhost:1521
180 SQL> @oracle_test_setup
181
182And follow the prompts to create a user.
183
148Creating test databases184Creating test databases
149-----------------------185-----------------------
150186
151187
=== added file 'oracle_test_setup.sql'
--- oracle_test_setup.sql 1970-01-01 00:00:00 +0000
+++ oracle_test_setup.sql 2010-03-15 15:48:27 +0000
@@ -0,0 +1,13 @@
1-- Note that this is a script that is intended to be run to set up a
2-- test database. You probably don't want to run this on a production
3-- database!
4
5CREATE USER &&user IDENTIFIED BY &&password;
6GRANT CONNECT TO &&user;
7GRANT CREATE ANY TABLE TO &&user;
8GRANT CREATE ANY TRIGGER TO &&user;
9GRANT CREATE ANY SEQUENCE TO &&user;
10GRANT DROP ANY SEQUENCE TO &&user;
11GRANT DROP ANY TABLE TO &&user;
12GRANT DROP ANY TRIGGER TO &&user;
13GRANT UNLIMITED TABLESPACE TO &&user;
0\ No newline at end of file14\ No newline at end of file
115
=== added file 'storm/databases/oracle.py'
--- storm/databases/oracle.py 1970-01-01 00:00:00 +0000
+++ storm/databases/oracle.py 2010-03-15 15:48:27 +0000
@@ -0,0 +1,590 @@
1# Copyright (c) 2008 Alfaiati
2#
3# Written by Gustavo Noronha <kov@alfaiati.net>
4# Willi Langenberger <wlang@wu-wien.ac.at>
5#
6# This file is part of Storm Object Relational Mapper.
7#
8# Storm is free software; you can redistribute it and/or modify
9# it under the terms of the GNU Lesser General Public License as
10# published by the Free Software Foundation; either version 2.1 of
11# the License, or (at your option) any later version.
12#
13# Storm is distributed in the hope that it will be useful,
14# but WITHOUT ANY WARRANTY; without even the implied warranty of
15# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16# GNU Lesser General Public License for more details.
17#
18# You should have received a copy of the GNU Lesser General Public License
19# along with this program. If not, see <http://www.gnu.org/licenses/>.
20
21import os
22import sys
23from decimal import Decimal
24
25from storm.databases import dummy
26
27# If no NLS_LANG is set, we'll use english with UTF-8. Note that this
28# variable *must* be set before importing cx_Oracle.
29if not os.environ.get("NLS_LANG", None):
30 os.environ["NLS_LANG"] = 'american_america.utf8'
31try:
32 import cx_Oracle as oracle
33except Exception, e:
34 oracle = dummy
35
36from storm.tracer import trace
37from storm.variables import Variable, RawStrVariable, PickleVariable
38from storm.database import Database, Connection, Result
39from storm.exceptions import (
40 install_exceptions, ClosedError, DatabaseModuleError, DatabaseError,
41 OperationalError)
42from storm.info import get_cls_info, ClassAlias
43from storm.expr import (
44 Undef, SetExpr, Select, Insert, Alias, And, Eq, FuncExpr, SQLRaw,
45 Le, Gt, Column, is_safe_token, Except, Expr, Sequence, SQLToken,
46 COLUMN, COLUMN_NAME, COLUMN_PREFIX, TABLE, compile, compile_eq,
47 compile_select, compile_insert, compile_set_expr, compile_sql_token,
48 State)
49
50
51install_exceptions(oracle)
52compile = compile.create_child()
53
54
55RESERVED = set(
56"""ACCESS ADD ALL ALTER AND ANY AS ASC AUDIT BETWEEN BY CHAR
57 CHECK CLUSTER COLUMN COMMENT COMPRESS CONNECT CREATE CURRENT
58 DATE DECIMAL DEFAULT DELETE DESC DISTINCT DROP ELSE EXCLUSIVE
59 EXISTS FILE FLOAT FOR FROM GRANT GROUP HAVING IDENTIFIED
60 IMMEDIATE IN INCREMENT INDEX INITIAL INSERT INTEGER INTERSECT
61 INTO IS LEVEL LIKE LOCK LONG MAXEXTENTS MINUS MLSLABEL MODE
62 MODIFY NOAUDIT NOCOMPRESS NOT NOWAIT NULL NUMBER OF OFFLINE
63 ON ONLINE OPTION OR ORDER PCTFREE PRIOR PRIVILEGES PUBLIC
64 RAW RENAME RESOURCE REVOKE ROW ROWS SELECT SESSION
65 SET SHARE SIZE SMALLINT START SUCCESSFUL SYNONYM SYSDATE
66 TABLE THEN TO TRIGGER UID UNION UNIQUE UPDATE USER VALIDATE
67 VALUES VARCHAR VARCHAR2 VIEW WHENEVER WHERE WITH""".split())
68
69
70def alias_names():
71 ct = 0
72 while 1:
73 yield "_%x" % ct
74 ct += 1
75
76
77@compile.when(type)
78def compile_type(compile, expr, state):
79 cls_info = get_cls_info(expr)
80 table = compile(cls_info.table, state)
81 if state.context is TABLE and issubclass(expr, ClassAlias):
82 return "%s %s" % (compile(cls_info.cls, state), table)
83 return table
84
85
86@compile.when(Alias)
87def compile_alias(compile, alias, state):
88 name = compile(alias.name, state, token=True)
89 if state.context is COLUMN or state.context is TABLE:
90 return "%s %s" % (compile(alias.expr, state), name)
91 return name
92
93
94@compile.when(Sequence)
95def compile_sequence_oracle(compile, sequence, state):
96 return "%s.NEXTVAL" % sequence.name
97
98
99class Minus(SetExpr):
100 oper = " MINUS "
101
102
103@compile.when(Except)
104def compile_except_oracle(compile, expr, state):
105 new_expr = Minus()
106 new_expr.exprs = expr.exprs
107 new_expr.all = expr.all
108 new_expr.order_by = expr.order_by
109 new_expr.limit = expr.limit
110 new_expr.offset = expr.offset
111 return compile_set_expr_oracle(compile, new_expr, state)
112
113
114@compile.when(SetExpr)
115def compile_set_expr_oracle(compile, expr, state):
116 names = alias_names()
117 if isinstance(expr, Minus):
118 # Build new set expression without arguments (order_by, etc).
119 new_expr = expr.__class__()
120 new_expr.exprs = expr.exprs
121 new_expr.all = expr.all
122
123 if expr.order_by is not Undef:
124 # Make sure that state.aliases isn't None, since we want them to
125 # compile our order_by statement below.
126 no_aliases = state.aliases is None
127 if no_aliases:
128 state.push("aliases", {})
129
130 aliases = {}
131 for subexpr in expr.exprs:
132 if isinstance(subexpr, Select):
133 columns = subexpr.columns
134 if not isinstance(columns, (tuple, list)):
135 columns = [columns]
136 else:
137 columns = list(columns)
138 for i, column in enumerate(columns):
139 if column not in aliases:
140 if isinstance(column, Column):
141 aliases[column] = columns[i] = Alias(
142 column, name=names.next())
143 elif isinstance(column, Alias):
144 aliases[column.expr] = column
145 subexpr.columns = columns
146 aliases.update(state.aliases)
147 state.aliases = aliases
148 aliases = None
149
150 set_statement = SQLRaw("(%s)" % compile(expr.exprs, state,
151 join=expr.oper))
152
153 if expr.order_by is not Undef:
154 # Build order_by statement, using aliases.
155 state.push("context", COLUMN_NAME)
156 order_by_statement = SQLRaw(compile(expr.order_by, state))
157 state.pop()
158 else:
159 order_by_statement = Undef
160
161 # Build wrapping select statement.
162 select = Select(
163 SQLRaw("*"), tables=Alias(set_statement, name=names.next()),
164 limit=expr.limit, offset=expr.offset, order_by=order_by_statement)
165 return compile_select(compile, select, state)
166 return compile_set_expr(compile, expr, state)
167
168
169@compile.when(Select)
170def compile_select_oracle(compile, select, state):
171 limit = select.limit
172 offset = select.offset
173 # Make sure limit is Undef'ed.
174 select.offset = select.limit = Undef
175
176 names = alias_names()
177
178 if select.default_tables is Undef:
179 select.default_tables = ["DUAL"]
180
181 if select.order_by is not Undef:
182 # Copied from expr.py's compile_set_expr.
183 aliases = {}
184 columns = select.columns
185 if not isinstance(columns, (tuple, list)):
186 columns = [columns]
187 else:
188 columns = list(columns)
189 for i, column in enumerate(columns):
190 if column not in aliases:
191 if isinstance(column, Column):
192 aliases[column] = columns[i] = Alias(
193 column, name=names.next())
194 elif isinstance(column, Alias):
195 aliases[column.expr] = column
196 select.columns = columns
197 # Copied from expr.py's compile_set_expr.
198 statement = SQLRaw("(%s)" % compile_select(compile, select, state))
199 select = Select(SQLRaw("*"), tables=Alias(statement,
200 name=names.next()))
201
202 if (limit is not Undef) and (offset is not Undef):
203 rownum_alias = Alias(SQLRaw("ROWNUM"), name=names.next())
204
205 # If we have an SQLRaw here that is because we are dealing with a
206 # subquery.
207 if isinstance(select.columns, SQLRaw):
208 select.columns = [SQLRaw('"' + select.tables.name + '".*'),
209 rownum_alias]
210 else:
211 select.columns.append(rownum_alias)
212
213 where_expr = Le(SQLRaw("ROWNUM"), limit + offset)
214 if select.where is Undef:
215 select.where = where_expr
216 else:
217 select.where = And(select.where, where_expr)
218
219 statement = SQLRaw("(%s)" % compile_select(compile, select, state))
220 select = Select(SQLRaw("*"), tables=Alias(statement, names.next()),
221 where=Gt(rownum_alias, offset))
222 elif limit is not Undef:
223 expr = Le(SQLRaw("ROWNUM"), limit)
224 if select.where is Undef:
225 select.where = expr
226 else:
227 select.where = And(select.where, expr)
228 elif offset is not Undef:
229 rownum_alias = Alias(SQLRaw("ROWNUM"), name=names.next())
230
231 # If we have an SQLRaw here that is because we are dealing with a
232 # subquery.
233 if isinstance(select.columns, SQLRaw):
234 select.columns = [SQLRaw('"' + select.tables.name + '".*'),
235 rownum_alias]
236 else:
237 select.columns.append(rownum_alias)
238
239 statement = SQLRaw("(%s)" % compile_select(compile, select, state))
240 select = Select(SQLRaw("*"), tables=Alias(statement,
241 name=names.next()),
242 where=Gt(rownum_alias, offset))
243
244 return compile_select(compile, select, state)
245
246
247@compile.when(Insert)
248def compile_insert_oracle(compile, insert, state):
249 # Shamelessly copied from PostgreSQL.
250 if not insert.map and insert.primary_columns is not Undef:
251 insert.map.update(dict.fromkeys(insert.primary_columns,
252 SQLRaw("DEFAULT")))
253 return compile_insert(compile, insert, state)
254
255
256@compile.when(Sequence)
257def compile_sequence_oracle(compile, sequence, state):
258 return "%s.nextval" % sequence.name
259
260
261@compile.when(bool)
262def compile_bool(compile, expr, state):
263 return compile_eq(compile, Eq(1, int(expr)), state)
264
265
266class currval(FuncExpr):
267
268 name = "currval"
269
270 def __init__(self, column):
271 self.column = column
272
273@compile.when(currval)
274def compile_currval(compile, expr, state):
275 """Compile a L{currval}."""
276 state.push("context", COLUMN_PREFIX)
277 table = compile(expr.column.table, state, token=True)
278 state.pop()
279 return "%s_seq.currval" % (table)
280
281
282class Rowid(Expr):
283
284 def __init__(self, rowid):
285 self.rowid = rowid
286
287@compile.when(Rowid)
288def compile_rowid(compile, expr, state):
289 state.parameters.append(expr.rowid)
290 return "?"
291
292
293@compile.when(SQLToken)
294def compile_oracle_sql_token(compile, expr, state):
295 if "." in expr and state.context in (TABLE, COLUMN_PREFIX):
296 return ".".join(compile_sql_token(compile, subexpr, state)
297 for subexpr in expr.split("."))
298 # A bit of a hack: we're skipping the built-in reserved word list and
299 # using our own set.
300 if is_safe_token(expr) and not str(expr) in RESERVED:
301 return expr
302 elif (state.context in (COLUMN, COLUMN_NAME)
303 and str(expr).lower() in ("rowid", "rownum")):
304 return expr
305 return '"%s"' % expr.replace('"', "|")
306
307
308class OracleResult(Result):
309
310 def __init__(self, connection, raw_cursor, rowid = None):
311 super(OracleResult, self).__init__(connection, raw_cursor)
312 self.lastrowid = rowid
313
314 def get_insert_identity(self, primary_key, primary_variables):
315 return Eq(Column("rowid"), Rowid(self.lastrowid))
316
317 @staticmethod
318 def set_variable(variable, value):
319 if isinstance(value, float):
320 value = Decimal(str(value))
321
322 variable.set(value, from_db=True)
323
324 @staticmethod
325 def from_database(row):
326 """Convert Oracle-specific datatypes to 'normal' Python types.
327
328 If there are anny C{buffer} instances in the row, convert them
329 to strings.
330 """
331 for value in row:
332 if isinstance(value, oracle.LOB):
333 yield value.read()
334 else:
335 yield value
336
337
338class OracleConnection(Connection):
339
340 result_factory = OracleResult
341 compile = compile
342 param_mark = "?"
343
344 def as_read_committed(self):
345 return _isolation_context(self)
346
347 def is_disconnection_error(self, exc):
348 if isinstance(exc, (oracle.OperationalError, oracle.DatabaseError)):
349 error, = exc.args
350 # Sometimes exceptions get caught here without a code attribute.
351 if hasattr(error, "code") and error.code in (3135, 3113):
352 return True
353 return False
354
355 def execute(self, statement, params=None, noresult=False):
356 """Execute a statement with the given parameters.
357
358 This method is completely overidden because the original from the base
359 class expects to receive only a C{raw_cursor} from C{raw_execute}, and
360 we need to receive also the C{rowid}, as we cannot set it in the
361 cursor object.
362
363 @type statement: L{Expr} or C{str}.
364 @param statement: The statement to execute. It will be compiled if
365 necessary.
366 @param noresult: If True, no result will be returned.
367 @raise DisconnectionError: Raised when the connection is lost.
368 Reconnection happens automatically on rollback.
369 @return: The result of C{self.result_factory}, or None if C{noresult}
370 is True.
371 """
372 if self._closed:
373 raise ClosedError("Connection is closed")
374 self._ensure_connected()
375 if self._event:
376 self._event.emit("register-transaction")
377 if isinstance(statement, Expr):
378 if params is not None:
379 raise ValueError("Can't pass parameters with expressions")
380 state = State()
381 statement = self.compile(statement, state)
382 params = state.parameters
383 statement = convert_to_sequential(statement)
384 raw_cursor, rowid = self.raw_execute(statement, params)
385 if noresult:
386 self._check_disconnect(raw_cursor.close)
387 return None
388 return self.result_factory(self, raw_cursor, rowid)
389
390 def raw_execute(self, statement, params):
391 """Execute a raw statement with the given parameters.
392
393 This method is completely overidden because the original from the base
394 class converts params to a tuple, and we need a dictionary! It's
395 acceptable to override this method in subclasses, but it is not
396 intended to be called externally. If the global C{DEBUG} flag is
397 True, the statement will be printed to standard out.
398
399 @return: The DBAPI cursor object, as fetched from L{build_raw_cursor}.
400 """
401 rowid = None
402 raw_cursor = self.build_raw_cursor()
403
404 statement = str(statement)
405
406 if statement.startswith("INSERT INTO"):
407 statement = statement + " RETURNING ROWID INTO :out_rowid"
408
409 # Make sure params is a list as we need to add to it.
410 if params is None:
411 params = []
412 elif not isinstance(params, list):
413 params = list(params)
414
415 rowid = raw_cursor.var(oracle.ROWID)
416 params.append(rowid)
417
418 if not params:
419 params = ()
420 else:
421 params = tuple(self.to_database(params))
422
423 trace("connection_raw_execute", self, raw_cursor, statement,
424 params or ())
425
426 try:
427 self._check_disconnect(raw_cursor.execute, statement, params)
428 if rowid:
429 rowid = rowid.getvalue()
430 except DatabaseError, de:
431 error, = de.args
432 if error == 8177:
433 raise OperationalError("database is locked")
434 else:
435 raise
436 except Exception, error:
437 trace("connection_raw_execute_error", self, raw_cursor,
438 statement, params or (), error)
439 raise
440 else:
441 trace("connection_raw_execute_success", self, raw_cursor,
442 statement, params or ())
443 return raw_cursor, rowid
444
445 @staticmethod
446 def to_database(params):
447 for bind_var in params:
448 if isinstance(bind_var, (RawStrVariable, PickleVariable)):
449 yield oracle.Binary(bind_var.get(to_db=True))
450 elif isinstance(bind_var, Variable):
451 yield bind_var.get(to_db=True)
452 else:
453 yield bind_var
454
455
456def convert_to_sequential(statement):
457 """Convert a query using ? bind variables to a query using
458 sequential bind variables. For example, SELECT ? FROM DUAL
459 will be converted to SELECT :1 FROM DUAL"""
460 param_no = 1
461 tokens = statement.split("'")
462 for i in range(0, len(tokens), 2):
463 while True:
464 old_tokens = tokens[i]
465 new_tokens = old_tokens.replace('?', ':%s' % param_no, 1)
466 if old_tokens == new_tokens:
467 break
468 else:
469 tokens[i] = new_tokens
470 param_no += 1
471 return "'".join(tokens)
472
473
474class _isolation_context(object):
475
476 def __init__ (self, connection):
477 self.connection = connection
478
479 def __enter__(self):
480 self.connection.commit()
481 self.connection.execute(
482 "ALTER SESSION SET isolation_level = read committed")
483
484 def __exit__(self, exc_type, exc_value, traceback):
485 if exc_value:
486 self.connection.rollback()
487 else:
488 self.connection.commit()
489 self.connection.execute(
490 "ALTER SESSION SET isolation_level = serializable")
491 self.connection.commit()
492
493
494class _type_converter(object):
495 """This class wraps an input type handler and an output type handler.
496 The reason being to allow for saving the encoding for the session."""
497 def __init__(self, encoding):
498 self.encoding = encoding
499
500 def convert_unicode(self, value):
501 if isinstance(value, Variable):
502 value = value.get(to_db=True)
503 if not isinstance(value, unicode):
504 return unicode(value, "utf8")
505 else:
506 return value
507
508 def OutputTypeHandler(self, cursor, name, defaultType, size, precision,
509 scale):
510 """Prepare cx_Oracle to turn an Oracle datatype into a Python datatype.
511
512 This function's purpose is to tell cx_Oracle how to convert an Oracle
513 datatype into a Python datatype.
514 """
515 if defaultType in (oracle.UNICODE, oracle.FIXED_UNICODE):
516 return cursor.var(unicode, size, cursor.arraysize,
517 outconverter=self.convert_unicode)
518 elif defaultType in (oracle.STRING, oracle.FIXED_CHAR):
519 return cursor.var(str, size, cursor.arraysize)
520
521 def InputTypeHandler(self, cursor, value, numElements):
522 """Prepare cx_Oracle to turn a Python datatype into an Oracle datatype.
523
524 This function does the opposite of OutputTypeHandler: it tells
525 cx_Oracle how to convert a Python datatype into an Oracle datatype.
526 """
527 if isinstance(value, unicode):
528 return cursor.var(unicode, arraysize=numElements,
529 inconverter=self.convert_unicode)
530
531class Oracle(Database):
532
533 connection_factory = OracleConnection
534 converter_factory = _type_converter
535 raw_connection_factory = oracle.connect
536
537 def __init__(self, uri):
538 if oracle is dummy:
539 raise DatabaseModuleError("'cx_Oracle' failed to load")
540
541 if not uri.port:
542 uri.port = 1521
543
544 isolation = uri.options.get("isolation", "serializable")
545 isolation_mapping = {
546 "serializable": "SERIALIZABLE",
547 "read-committed": "READ COMMITTED",
548 }
549 try:
550 self._isolation = isolation_mapping[isolation]
551 except KeyError:
552 raise ValueError(
553 "Unknown serialization level %r: expected one of "
554 "'serializable' or 'read-committed'" %
555 (isolation,))
556
557 # Optionally set ORACLE_HOME and TNS_ADMIN environment
558 # variables for controlling tnsnames.ora lookup
559 oracle_home = uri.options.get('oracle_home')
560 if oracle_home:
561 os.environ["ORACLE_HOME"] = oracle_home
562
563 # If tns is specified in the options, treat the host part of the URI as
564 # the TNS name, and therefore the DSN.
565 if uri.options.get("tns", False):
566 self._dsn = uri.host
567 else:
568 self._dsn = oracle.makedsn(uri.host, uri.port, uri.database)
569
570 self._username = uri.username
571 self._password = uri.password
572
573 def raw_connect(self):
574 if oracle is dummy:
575 raise ImportError, "Could not import cx_Oracle"
576
577 raw_connection = self.raw_connection_factory(self._username,
578 self._password, self._dsn)
579
580 c = raw_connection.cursor()
581 c.execute("alter session set isolation_level=%s" % self._isolation)
582 c.close()
583
584 type_handler = self.converter_factory(raw_connection.nencoding)
585 raw_connection.inputtypehandler = type_handler.InputTypeHandler
586 raw_connection.outputtypehandler = type_handler.OutputTypeHandler
587
588 return raw_connection
589
590create_from_uri = Oracle
0591
=== modified file 'storm/expr.py'
--- storm/expr.py 2009-11-02 11:11:20 +0000
+++ storm/expr.py 2010-03-15 15:48:27 +0000
@@ -1000,7 +1000,7 @@
1000 return "%s = %s" % (compile(eq.expr1, state), compile(eq.expr2, state))1000 return "%s = %s" % (compile(eq.expr1, state), compile(eq.expr2, state))
10011001
1002@compile_python.when(Eq)1002@compile_python.when(Eq)
1003def compile_eq(compile, eq, state):1003def compile_python_eq(compile, eq, state):
1004 return "%s == %s" % (compile(eq.expr1, state), compile(eq.expr2, state))1004 return "%s == %s" % (compile(eq.expr1, state), compile(eq.expr2, state))
10051005
10061006
10071007
=== modified file 'storm/variables.py'
--- storm/variables.py 2009-10-03 16:08:04 +0000
+++ storm/variables.py 2010-03-15 15:48:27 +0000
@@ -436,6 +436,8 @@
436 if from_db:436 if from_db:
437 if value is None:437 if value is None:
438 return None438 return None
439 if isinstance(value, datetime):
440 return value.date()
439 if isinstance(value, date):441 if isinstance(value, date):
440 return value442 return value
441 if not isinstance(value, (str, unicode)):443 if not isinstance(value, (str, unicode)):
442444
=== modified file 'tests/databases/base.py'
--- tests/databases/base.py 2009-09-19 15:56:13 +0000
+++ tests/databases/base.py 2010-03-15 15:48:27 +0000
@@ -75,7 +75,7 @@
75 raise NotImplementedError75 raise NotImplementedError
7676
77 def create_sample_data(self):77 def create_sample_data(self):
78 self.connection.execute("INSERT INTO number VALUES (1, 2, 3)")78 self.connection.execute("INSERT INTO number_table VALUES (1, 2, 3)")
79 self.connection.execute("INSERT INTO test VALUES (10, 'Title 10')")79 self.connection.execute("INSERT INTO test VALUES (10, 'Title 10')")
80 self.connection.execute("INSERT INTO test VALUES (20, 'Title 20')")80 self.connection.execute("INSERT INTO test VALUES (20, 'Title 20')")
81 self.connection.commit()81 self.connection.commit()
@@ -84,7 +84,7 @@
84 pass84 pass
8585
86 def drop_tables(self):86 def drop_tables(self):
87 for table in ["number", "test", "datetime_test", "bin_test"]:87 for table in ["number_table", "test", "datetime_test", "bin_test"]:
88 try:88 try:
89 self.connection.execute("DROP TABLE " + table)89 self.connection.execute("DROP TABLE " + table)
90 self.connection.commit()90 self.connection.commit()
@@ -131,7 +131,7 @@
131 self.assertTrue(result.get_one())131 self.assertTrue(result.get_one())
132132
133 def test_execute_result(self):133 def test_execute_result(self):
134 result = self.connection.execute("SELECT 1")134 result = self.connection.execute(Select(1))
135 self.assertTrue(isinstance(result, Result))135 self.assertTrue(isinstance(result, Result))
136 self.assertTrue(result.get_one())136 self.assertTrue(result.get_one())
137137
@@ -143,15 +143,15 @@
143 self.assertTrue(isinstance(row[0], unicode))143 self.assertTrue(isinstance(row[0], unicode))
144144
145 def test_execute_params(self):145 def test_execute_params(self):
146 result = self.connection.execute("SELECT one FROM number "146 result = self.connection.execute("SELECT one FROM number_table "
147 "WHERE 1=?", (1,))147 "WHERE 1=?", (1,))
148 self.assertTrue(result.get_one())148 self.assertTrue(result.get_one())
149 result = self.connection.execute("SELECT one FROM number "149 result = self.connection.execute("SELECT one FROM number_table "
150 "WHERE 1=?", (2,))150 "WHERE 1=?", (2,))
151 self.assertFalse(result.get_one())151 self.assertFalse(result.get_one())
152152
153 def test_execute_empty_params(self):153 def test_execute_empty_params(self):
154 result = self.connection.execute("SELECT one FROM number", ())154 result = self.connection.execute("SELECT one FROM number_table", ())
155 self.assertTrue(result.get_one())155 self.assertTrue(result.get_one())
156156
157 def test_execute_expression(self):157 def test_execute_expression(self):
@@ -356,7 +356,7 @@
356 event.hook("register-transaction", register_transaction)356 event.hook("register-transaction", register_transaction)
357357
358 connection = self.database.connect(event)358 connection = self.database.connect(event)
359 connection.execute("SELECT 1")359 connection.execute(Select(1))
360 self.assertEqual(len(calls), 1)360 self.assertEqual(len(calls), 1)
361 self.assertEqual(calls[0], marker)361 self.assertEqual(calls[0], marker)
362362
@@ -364,17 +364,17 @@
364 return [int(item)+1 for item in row]364 return [int(item)+1 for item in row]
365365
366 def test_wb_result_get_one_goes_through_from_database(self):366 def test_wb_result_get_one_goes_through_from_database(self):
367 result = self.connection.execute("SELECT one, two FROM number")367 result = self.connection.execute("SELECT one, two FROM number_table")
368 result.from_database = self.from_database368 result.from_database = self.from_database
369 self.assertEquals(result.get_one(), (2, 3))369 self.assertEquals(result.get_one(), (2, 3))
370370
371 def test_wb_result_get_all_goes_through_from_database(self):371 def test_wb_result_get_all_goes_through_from_database(self):
372 result = self.connection.execute("SELECT one, two FROM number")372 result = self.connection.execute("SELECT one, two FROM number_table")
373 result.from_database = self.from_database373 result.from_database = self.from_database
374 self.assertEquals(result.get_all(), [(2, 3)])374 self.assertEquals(result.get_all(), [(2, 3)])
375375
376 def test_wb_result_iter_goes_through_from_database(self):376 def test_wb_result_iter_goes_through_from_database(self):
377 result = self.connection.execute("SELECT one, two FROM number")377 result = self.connection.execute("SELECT one, two FROM number_table")
378 result.from_database = self.from_database378 result.from_database = self.from_database
379 self.assertEquals(iter(result).next(), (2, 3))379 self.assertEquals(iter(result).next(), (2, 3))
380380
@@ -505,6 +505,7 @@
505 "%s.default_port" % (type(self).__name__,505 "%s.default_port" % (type(self).__name__,
506 type(self).__name__))506 type(self).__name__))
507 uri_str = os.environ.get(self.host_environment_variable)507 uri_str = os.environ.get(self.host_environment_variable)
508
508 if uri_str:509 if uri_str:
509 uri = URI(uri_str)510 uri = URI(uri_str)
510 if not uri.host:511 if not uri.host:
@@ -542,24 +543,24 @@
542543
543 def test_proxy_works(self):544 def test_proxy_works(self):
544 """Ensure that we can talk to the database through the proxy."""545 """Ensure that we can talk to the database through the proxy."""
545 result = self.connection.execute("SELECT 1")546 result = self.connection.execute(Select(1))
546 self.assertEqual(result.get_one(), (1,))547 self.assertEqual(result.get_one(), (1,))
547548
548 def test_catch_disconnect_on_execute(self):549 def test_catch_disconnect_on_execute(self):
549 """Test that database disconnections get caught on execute()."""550 """Test that database disconnections get caught on execute()."""
550 result = self.connection.execute("SELECT 1")551 result = self.connection.execute(Select(1))
551 self.assertTrue(result.get_one())552 self.assertTrue(result.get_one())
552 self.proxy.restart()553 self.proxy.restart()
553 self.assertRaises(DisconnectionError,554 self.assertRaises(DisconnectionError,
554 self.connection.execute, "SELECT 1")555 self.connection.execute, Select(1))
555556
556 def test_catch_disconnect_on_commit(self):557 def test_catch_disconnect_on_commit(self):
557 """Test that database disconnections get caught on commit()."""558 """Test that database disconnections get caught on commit()."""
558 result = self.connection.execute("SELECT 1")559 result = self.connection.execute(Select(1))
559 self.assertTrue(result.get_one())560 self.assertTrue(result.get_one())
560 self.proxy.restart()561 self.proxy.restart()
561 self.assertRaises(DisconnectionError, self.connection.commit)562 self.assertRaises(DisconnectionError, self.connection.commit)
562563
563 def test_wb_catch_already_disconnected_on_rollback(self):564 def test_wb_catch_already_disconnected_on_rollback(self):
564 """Connection.rollback() swallows disconnection errors.565 """Connection.rollback() swallows disconnection errors.
565566
@@ -567,13 +568,13 @@
567 then it is possible that Storm won't see the disconnection.568 then it is possible that Storm won't see the disconnection.
568 It should be able to recover from this situation though.569 It should be able to recover from this situation though.
569 """570 """
570 result = self.connection.execute("SELECT 1")571 result = self.connection.execute(Select(1))
571 self.assertTrue(result.get_one())572 self.assertTrue(result.get_one())
572 self.proxy.restart()573 self.proxy.restart()
573 # Perform an action that should result in a disconnection.574 # Perform an action that should result in a disconnection.
574 try:575 try:
575 cursor = self.connection._raw_connection.cursor()576 cursor = self.connection._raw_connection.cursor()
576 cursor.execute("SELECT 1")577 cursor.execute(Select(1))
577 cursor.fetchone()578 cursor.fetchone()
578 except Error, exc:579 except Error, exc:
579 self.assertTrue(self.connection.is_disconnection_error(exc))580 self.assertTrue(self.connection.is_disconnection_error(exc))
@@ -600,59 +601,59 @@
600 then it is possible that Storm won't see the disconnection.601 then it is possible that Storm won't see the disconnection.
601 It should be able to recover from this situation though.602 It should be able to recover from this situation though.
602 """603 """
603 result = self.connection.execute("SELECT 1")604 result = self.connection.execute(Select(1))
604 self.assertTrue(result.get_one())605 self.assertTrue(result.get_one())
605 self.proxy.restart()606 self.proxy.restart()
606 # Perform an action that should result in a disconnection.607 # Perform an action that should result in a disconnection.
607 try:608 try:
608 cursor = self.connection._raw_connection.cursor()609 cursor = self.connection._raw_connection.cursor()
609 cursor.execute("SELECT 1")610 cursor.execute(Select(1))
610 cursor.fetchone()611 cursor.fetchone()
611 except DatabaseError, exc:612 except DatabaseError, exc:
612 self.assertTrue(self.connection.is_disconnection_error(exc))613 self.assertTrue(self.connection.is_disconnection_error(exc))
613 else:614 else:
614 self.fail("Disconnection was not caught.")615 self.fail("Disconnection was not caught.")
615 self.assertRaises(DisconnectionError,616 self.assertRaises(DisconnectionError,
616 self.connection.execute, "SELECT 1")617 self.connection.execute, Select(1))
617618
618 def test_connection_stays_disconnected_in_transaction(self):619 def test_connection_stays_disconnected_in_transaction(self):
619 """Test that connection does not immediately reconnect."""620 """Test that connection does not immediately reconnect."""
620 result = self.connection.execute("SELECT 1")621 result = self.connection.execute(Select(1))
621 self.assertTrue(result.get_one())622 self.assertTrue(result.get_one())
622 self.proxy.restart()623 self.proxy.restart()
623 self.assertRaises(DisconnectionError,624 self.assertRaises(DisconnectionError,
624 self.connection.execute, "SELECT 1")625 self.connection.execute, Select(1))
625 self.assertRaises(DisconnectionError,626 self.assertRaises(DisconnectionError,
626 self.connection.execute, "SELECT 1")627 self.connection.execute, Select(1))
627628
628 def test_reconnect_after_rollback(self):629 def test_reconnect_after_rollback(self):
629 """Test that we reconnect after rolling back the connection."""630 """Test that we reconnect after rolling back the connection."""
630 result = self.connection.execute("SELECT 1")631 result = self.connection.execute(Select(1))
631 self.assertTrue(result.get_one())632 self.assertTrue(result.get_one())
632 self.proxy.restart()633 self.proxy.restart()
633 self.assertRaises(DisconnectionError,634 self.assertRaises(DisconnectionError,
634 self.connection.execute, "SELECT 1")635 self.connection.execute, Select(1))
635 self.connection.rollback()636 self.connection.rollback()
636 result = self.connection.execute("SELECT 1")637 result = self.connection.execute(Select(1))
637 self.assertTrue(result.get_one())638 self.assertTrue(result.get_one())
638639
639 def test_catch_disconnect_on_reconnect(self):640 def test_catch_disconnect_on_reconnect(self):
640 """Test that reconnection failures result in DisconnectionError."""641 """Test that reconnection failures result in DisconnectionError."""
641 result = self.connection.execute("SELECT 1")642 result = self.connection.execute(Select(1))
642 self.assertTrue(result.get_one())643 self.assertTrue(result.get_one())
643 self.proxy.stop()644 self.proxy.stop()
644 self.assertRaises(DisconnectionError,645 self.assertRaises(DisconnectionError,
645 self.connection.execute, "SELECT 1")646 self.connection.execute, Select(1))
646 # Rollback the connection, but because the proxy is still647 # Rollback the connection, but because the proxy is still
647 # down, we get a DisconnectionError again.648 # down, we get a DisconnectionError again.
648 self.connection.rollback()649 self.connection.rollback()
649 self.assertRaises(DisconnectionError,650 self.assertRaises(DisconnectionError,
650 self.connection.execute, "SELECT 1")651 self.connection.execute, Select(1))
651652
652 def test_close_connection_after_disconnect(self):653 def test_close_connection_after_disconnect(self):
653 result = self.connection.execute("SELECT 1")654 result = self.connection.execute(Select(1))
654 self.assertTrue(result.get_one())655 self.assertTrue(result.get_one())
655 self.proxy.stop()656 self.proxy.stop()
656 self.assertRaises(DisconnectionError,657 self.assertRaises(DisconnectionError,
657 self.connection.execute, "SELECT 1")658 self.connection.execute, Select(1))
658 self.connection.close()659 self.connection.close()
659660
=== added file 'tests/databases/oracle.py'
--- tests/databases/oracle.py 1970-01-01 00:00:00 +0000
+++ tests/databases/oracle.py 2010-03-15 15:48:27 +0000
@@ -0,0 +1,372 @@
1from __future__ import with_statement
2
3import os
4import pickle
5from binascii import hexlify
6from datetime import time, timedelta
7from unittest import TestSuite, defaultTestLoader as loader, TestCase
8
9from cx_Oracle import makedsn, DatabaseError, LOB
10
11from storm.database import *
12from storm.databases.oracle import (
13 _type_converter, convert_to_sequential, Oracle, _isolation_context,
14 OracleConnection, OracleResult, compile)
15from storm.event import EventSystem
16from storm.exceptions import NotSupportedError
17from storm.expr import Select, SQLToken, Alias
18from storm.tracer import debug
19from storm.uri import URI
20from storm.variables import PickleVariable, RawStrVariable, TimeDeltaVariable
21
22from tests.databases.base import (
23 DatabaseTest, DatabaseDisconnectionTest, UnsupportedDatabaseTest)
24from tests.helper import TestHelper
25from tests.mocker import ANY, ARGS
26from tests.store.oracle import create_table_helpers, drop_tables
27
28
29DEBUG = True
30debug(DEBUG)
31
32class Marker(object):
33 pass
34
35marker = Marker()
36
37class OracleDatabaseTest(DatabaseTest, TestHelper):
38 _table_names = ['number_table', 'test', 'datetime_test', 'bin_test',
39 'like_case_insensitive_test', 'insert_returning_test']
40 supports_microseconds = False
41 def is_supported(self):
42 return bool(os.environ.get('STORM_ORACLE_URI'))
43
44 def create_database(self):
45 self.database = create_database(os.environ['STORM_ORACLE_URI'])
46
47 def create_tables(self):
48 try:
49 with self.connection.as_read_committed():
50 self.connection.execute("CREATE TABLE number_table "
51 "(id INTEGER, one INTEGER, two INTEGER,"
52 " three INTEGER)")
53 self.connection.execute("CREATE TABLE test "
54 "(id INTEGER PRIMARY KEY, title NVARCHAR2(2000))")
55 self.connection.execute("CREATE TABLE datetime_test "
56 "(id INTEGER PRIMARY KEY,"
57 " dt TIMESTAMP(9), d DATE, t DATE, "
58 " td INTERVAL DAY TO SECOND)")
59 self.connection.execute("CREATE TABLE bin_test "
60 "(id INTEGER PRIMARY KEY, b BLOB)")
61 self.connection.execute("CREATE TABLE like_case_insensitive_test "
62 "(id INTEGER PRIMARY KEY, description CLOB)")
63 self.connection.execute("CREATE TABLE insert_returning_test "
64 "(id INTEGER PRIMARY KEY, "
65 "id1 INTEGER DEFAULT 123, "
66 " id2 INTEGER DEFAULT 456)")
67 create_table_helpers(self.connection, self._table_names)
68
69 except Exception, e:
70 self.connection.rollback()
71 raise
72 finally:
73 debug(DEBUG)
74
75 def create_sample_data(self):
76 self.connection.execute("INSERT INTO number_table (one, two, three) VALUES (1, 2, 3)")
77 self.connection.execute("INSERT INTO test VALUES (10, 'Title 10')")
78 self.connection.execute("INSERT INTO test VALUES (20, 'Title 20')")
79 self.connection.commit()
80
81 def drop_tables(self):
82 try:
83 drop_tables(self.connection, self._table_names)
84 finally:
85 debug(DEBUG)
86
87 def test_binary(self):
88 """Ensure database works with high bits and embedded zeros.
89 Overriding this because we need to send the data as a hexlified
90 string."""
91
92 value = "\xff\x00\xff\x00"
93 self.connection.execute("INSERT INTO bin_test (b) VALUES (?)",
94 (hexlify(value),))
95 self.connection.commit()
96 result = self.connection.execute("SELECT b FROM bin_test")
97 variable = RawStrVariable()
98 result.set_variable(variable, result.get_one()[0])
99 self.assertEquals(variable.get(), value)
100
101 def test_binary_ascii(self):
102 """Some databases like pysqlite2 may return unicode for strings.
103 Overriding this because we need to send the data as a hexlified
104 string."""
105 bin_text = 'Value'
106 self.connection.execute("INSERT INTO bin_test VALUES (10, ?)",
107 (hexlify(bin_text),))
108 result = self.connection.execute("SELECT b FROM bin_test")
109 variable = RawStrVariable()
110 # If the following doesn't raise a TypeError we're good.
111 result.set_variable(variable, result.get_one()[0])
112 self.assertEquals(variable.get(), "Value")
113
114 def test_execute_result(self):
115 result = self.connection.execute("SELECT 1 FROM DUAL")
116 self.assertTrue(isinstance(result, Result))
117 self.assertTrue(result.get_one())
118
119 def test_execute_sends_event(self):
120 event = EventSystem(marker)
121 calls = []
122 def register_transaction(owner):
123 calls.append(owner)
124 event.hook("register-transaction", register_transaction)
125
126 connection = self.database.connect(event)
127 connection.execute("SELECT 1 FROM DUAL")
128 self.assertEqual(len(calls), 1)
129 self.assertEqual(calls[0], marker)
130
131 def test_timedelta(self):
132 value = timedelta(12, 34)
133 self.connection.execute("INSERT INTO datetime_test (td) VALUES (?)",
134 (value,))
135 result = self.connection.execute("SELECT td FROM datetime_test")
136 variable = TimeDeltaVariable()
137 result.set_variable(variable, result.get_one()[0])
138 self.assertEquals(variable.get(), value)
139
140 def test_pickle(self):
141 value = {"a": 1, "b": 2}
142 value_dump = pickle.dumps(value, -1)
143 self.connection.execute("INSERT INTO bin_test (b) VALUES (?)",
144 (hexlify(value_dump),))
145 result = self.connection.execute("SELECT b FROM bin_test")
146 variable = PickleVariable()
147 result.set_variable(variable, result.get_one()[0])
148 self.assertEquals(variable.get(), value)
149
150 def test_time(self):
151 """Oracle doesn't have a time-only datatype, so we need to test
152 that using one fails gracefully."""
153 self.assertRaises(NotSupportedError, self.connection.execute,
154 'SELECT * FROM ', (time(1, 2, 3),))
155
156
157class OracleUnsupportedDatabaseTest(UnsupportedDatabaseTest, TestHelper):
158 dbapi_module_names = ['cx_Oracle']
159 db_module_name = 'oracle'
160
161# FIXME - Oracle doesn't seem to work with these disconnection tests
162# class OracleDatabaseDisconnectionTest(DatabaseDisconnectionTest, TestHelper):
163# environment_variable = "STORM_ORACLE_URI"
164# host_environment_variable = "STORM_ORACLE_HOST_URI"
165# default_port=1521
166
167class MockObject(object):
168 def __init__(self, allowed_sets, allowed_gets, *args, **argd):
169 self.allowed_sets = allowed_sets
170 self.allowed_gets = allowed_gets
171 self.args = args
172 self.argd = argd
173 self.was_set = set()
174 self.was_gotten = set()
175
176 def __getattr__(self, name):
177 if self.allowed_gets.get(name, False):
178 self.was_gotten.add(name)
179 return self.allowed_gets[name]
180
181 else:
182 raise NotImplementedError()
183
184 def __setattr__(self, name, val):
185
186 if self.allowed_sets.get(name, False):
187 self.was_set.add(name)
188 self.__dict__[name] = val
189 else:
190 raise NotImplementedError()
191
192 def __delattr__(self, name):
193 raise NotImplementedError()
194
195 def __call__(self, *args, **argd):
196 return self
197
198def dummy_func(*args, **argd):
199 pass
200
201
202class UnicodeUnitTests(TestCase):
203 unit = True
204 def setUp(self):
205 self.converter = _type_converter('utf8')
206
207 def test_identity(self):
208 expected = u"Some string"
209 actual = self.converter.convert_unicode(expected)
210 self.assertEqual(actual, expected)
211
212 def test_cross_conversion(self):
213 initial = unicode('Some string', 'latin-1')
214 expected = unicode('Some string', 'utf8')
215 actual = self.converter.convert_unicode(initial)
216 self.assertEqual(actual, expected)
217
218class OracleUnitTests(TestHelper):
219 unit = True
220 def test_tns_uri(self):
221 uri = URI('oracle://foo:bar@test?tns=true')
222 ora = Oracle(uri)
223 self.assertEqual(ora._dsn, 'test')
224 self.assertEqual(ora._username, 'foo')
225 self.assertEqual(ora._password, 'bar')
226
227 def test_host_str(self):
228 uri = URI('oracle://foo:bar@localhost:1521/test')
229 ora = Oracle(uri)
230 self.assertEqual(ora._username, 'foo')
231 self.assertEqual(ora._password, 'bar')
232 expected_dsn = makedsn('localhost', 1521, 'test')
233 self.assertEqual(ora._dsn, expected_dsn)
234
235 def test_type_handlers_set(self):
236 mocker = self.mocker
237 mock = mocker.mock()
238 converter = mocker.mock()
239 mock(ARGS)
240 mocker.result(mock)
241 mock.cursor()
242 mocker.result(mock)
243 mock.execute(ARGS)
244 mocker.result(None)
245 mock.nencoding
246 mocker.result('utf8')
247 converter('utf8')
248 mocker.result(converter)
249
250 converter.InputTypeHandler
251 mocker.result(converter)
252
253 mock.inputtypehandler = converter
254 mocker.result(None)
255
256 converter.OutputTypeHandler
257 mocker.result(converter)
258 mock.outputtypehandler = converter
259 mock.close()
260 mocker.result(None)
261 mocker.replay()
262
263 class DummyOracle(Oracle):
264 raw_connection_factory = mock
265 converter_factory = converter
266 def __init__(self):
267 self._username = ''
268 self._password = ''
269 self._dsn = ''
270 self._isolation = ''
271
272 DummyOracle().raw_connect()
273
274
275class OracleConnectionTests(TestHelper):
276 unit = True
277 def test_isolation_context_pass(self):
278 mocker = self.mocker
279 mock = mocker.mock()
280 mock.commit()
281 mocker.count(0, None)
282 mock.execute('ALTER SESSION SET isolation_level = read committed')
283 mock.execute('ALTER SESSION SET isolation_level = serializable')
284 mocker.replay()
285
286 with _isolation_context(mock):
287 pass
288
289 def test_isolation_context_fail(self):
290 mocker = self.mocker
291 mock = mocker.mock()
292 mock.commit()
293 mocker.result(None)
294 mock.execute('ALTER SESSION SET isolation_level = read committed')
295 mock.rollback()
296 mocker.result(None)
297 mock.execute('ALTER SESSION SET isolation_level = serializable')
298 mock.commit()
299 mocker.result(None)
300 mocker.replay()
301
302 try:
303 with _isolation_context(mock):
304 raise TypeError()
305 except TypeError:
306 pass
307
308 def test_is_disconnection_error(self):
309 codes = [3135, 3113]
310 non_codes = [0, 1000]
311
312 database = self.mocker.mock()
313 database.raw_connect()
314 self.mocker.result(database)
315 self.mocker.replay()
316
317 conn = OracleConnection(database)
318
319 class DummyError(object):
320 def __init__(self, code):
321 self.code = code
322
323 for code in codes:
324 exc = DatabaseError(DummyError(code))
325 assert conn.is_disconnection_error(exc)
326
327 for code in non_codes:
328 exc = DatabaseError(DummyError(code))
329 assert not conn.is_disconnection_error(exc)
330
331 def test_from_database_LOB(self):
332 mocker = self.mocker
333 mock = mocker.mock(LOB)
334
335 iter(mock)
336 mocker.generate([mock])
337 expected = 'result string'
338 mock.read()
339 mocker.result(expected)
340 mocker.replay()
341
342 row_iter = OracleResult.from_database(mock)
343 self.assertEqual(row_iter.next(), expected)
344 self.assertRaises(StopIteration, row_iter.next)
345
346
347class QueryUnitTests(TestHelper):
348 unit = True
349 def test_convert_to_sequential(self):
350 initial = 'SELECT ? FROM DUAL'
351 expected = 'SELECT :1 FROM DUAL'
352 actual = convert_to_sequential(initial)
353 self.assertEqual(expected, actual)
354
355 def test_convert_to_sequential_in_str(self):
356 expected = "SELECT '?' FROM DUAL"
357 actual = convert_to_sequential(expected)
358 self.assertEqual(expected, actual)
359
360 def test_conver_to_sequential_not_in_str(self):
361 initial = "SELECT 'Foo', ? FROM DUAL"
362 expected = "SELECT 'Foo', :1 FROM DUAL"
363 actual = convert_to_sequential(initial)
364 self.assertEqual(expected, actual)
365
366 def test_double_quotes(self):
367 initial = 'word"word'
368 expected = '"word|word"'
369 actual = compile(SQLToken(initial))
370 self.assertEqual(expected, actual)
371
372
0373
=== modified file 'tests/store/base.py'
--- tests/store/base.py 2010-02-08 11:30:44 +0000
+++ tests/store/base.py 2010-03-15 15:48:27 +0000
@@ -280,11 +280,11 @@
280 return store._cache280 return store._cache
281281
282 def test_execute(self):282 def test_execute(self):
283 result = self.store.execute("SELECT 1")283 result = self.store.execute(Select(1))
284 self.assertTrue(isinstance(result, Result))284 self.assertTrue(isinstance(result, Result))
285 self.assertEquals(result.get_one(), (1,))285 self.assertEquals(result.get_one(), (1,))
286286
287 result = self.store.execute("SELECT 1", noresult=True)287 result = self.store.execute(Select(1), noresult=True)
288 self.assertEquals(result, None)288 self.assertEquals(result, None)
289289
290 def test_execute_params(self):290 def test_execute_params(self):
@@ -302,7 +302,7 @@
302 def test_close(self):302 def test_close(self):
303 store = Store(self.database)303 store = Store(self.database)
304 store.close()304 store.close()
305 self.assertRaises(ClosedError, store.execute, "SELECT 1")305 self.assertRaises(ClosedError, store.execute, Select(1))
306306
307 def test_get(self):307 def test_get(self):
308 foo = self.store.get(Foo, 10)308 foo = self.store.get(Foo, 10)
@@ -1423,7 +1423,7 @@
1423 result.group_by(Foo)1423 result.group_by(Foo)
1424 foo1 = self.store.get(Foo, 10)1424 foo1 = self.store.get(Foo, 10)
1425 foo2 = self.store.get(Foo, 20)1425 foo2 = self.store.get(Foo, 20)
1426 self.assertEquals(list(result), [(5, foo1), (16, foo2)])1426 self.assertEquals(sorted(list(result)), [(5, foo1), (16, foo2)])
14271427
1428 def test_find_group_by_table_contains(self):1428 def test_find_group_by_table_contains(self):
1429 result = self.store.find(1429 result = self.store.find(
@@ -5675,7 +5675,7 @@
5675 def register_transaction(owner):5675 def register_transaction(owner):
5676 calls.append(owner)5676 calls.append(owner)
5677 self.store._event.hook("register-transaction", register_transaction)5677 self.store._event.hook("register-transaction", register_transaction)
5678 self.store.execute("SELECT 1")5678 self.store.execute(Select(1))
5679 self.assertEqual(len(calls), 1)5679 self.assertEqual(len(calls), 1)
5680 self.assertEqual(calls[0], self.store)5680 self.assertEqual(calls[0], self.store)
56815681
56825682
=== added file 'tests/store/oracle.py'
--- tests/store/oracle.py 1970-01-01 00:00:00 +0000
+++ tests/store/oracle.py 2010-03-15 15:48:27 +0000
@@ -0,0 +1,234 @@
1#
2# Copyright (c) 2006, 2007 Canonical
3#
4# Written by Gustavo Niemeyer <gustavo@niemeyer.net>
5#
6# This file is part of Storm Object Relational Mapper.
7#
8# Storm is free software; you can redistribute it and/or modify
9# it under the terms of the GNU Lesser General Public License as
10# published by the Free Software Foundation; either version 2.1 of
11# the License, or (at your option) any later version.
12#
13# Storm is distributed in the hope that it will be useful,
14# but WITHOUT ANY WARRANTY; without even the implied warranty of
15# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16# GNU Lesser General Public License for more details.
17#
18# You should have received a copy of the GNU Lesser General Public License
19# along with this program. If not, see <http://www.gnu.org/licenses/>.
20#
21from __future__ import with_statement
22import os, sys
23
24from storm.database import create_database
25from storm.databases.oracle import Oracle
26from storm.uri import URI
27from cx_Oracle import DatabaseError
28
29from storm.database import Result
30from tests.store.base import StoreTest, EmptyResultSetTest
31from tests.store.base import Foo
32from storm.properties import Int, Float, RawStr, Unicode, Property, Pickle
33from tests.helper import TestHelper, MakePath
34
35from storm.info import ClassAlias
36
37from storm.tracer import debug
38
39DEBUG = False
40debug(DEBUG)
41
42def _safe_drop(connection, query):
43 """
44 This function will execute a given DROP query
45 and ignore any error codes that indicate that
46 the object doesn't exist.
47 """
48 with connection.as_read_committed():
49 try:
50 connection.execute(query)
51 except DatabaseError, exc:
52 error, = exc.args
53 if error.code not in [942, 4080, 2289, 903]:
54 raise
55
56def create_table_helpers(connection, table_names):
57 with connection.as_read_committed():
58 for table_name in table_names:
59 connection.execute("CREATE SEQUENCE %s_s "
60 "START WITH 1 "
61 "INCREMENT BY 1 "
62 "NOMAXVALUE "
63 "ORDER NOCYCLE" % (table_name),
64 )
65 connection.execute("CREATE OR REPLACE TRIGGER %s_t "
66 "BEFORE INSERT ON %s "
67 "FOR EACH ROW "
68 "WHEN (new.id is null) "
69 "BEGIN "
70 "SELECT %s_s.nextval INTO :new.id FROM DUAL; "
71 "END;" % (table_name, table_name, table_name),
72 )
73
74def drop_tables(connection, tables):
75 for table in tables:
76 _safe_drop(connection, "DROP TRIGGER %s_t" % table)
77 _safe_drop(connection, "DROP SEQUENCE %s_s" % table)
78
79 _safe_drop(connection, "DROP TABLE %s" % table)
80
81class OracleStoreTest(TestHelper, StoreTest):
82 DUMMY_SELECT = 'SELECT 1 FROM DUAL'
83 helpers = [MakePath]
84 _tables = ['foo', 'bar', 'bin', 'link', 'money', 'selfref', 'foovalue']
85
86 def create_tables(self):
87 connection = self.connection
88 with connection.as_read_committed():
89 connection.execute("CREATE TABLE foo (id NUMBER, "
90 "title NVARCHAR2(2000) DEFAULT 'Default Title')")
91 connection.execute("CREATE TABLE bar "
92 "(id NUMBER, foo_id NUMBER, "
93 "title NVARCHAR2(2000))")
94 connection.execute("CREATE TABLE bin "
95 "(id NUMBER, bin BLOB, foo_id INTEGER)")
96 connection.execute("CREATE TABLE link "
97 "(id NUMBER, foo_id NUMBER, bar_id NUMBER)")
98 connection.execute("CREATE TABLE money "
99 "(id NUMBER, value NUMBER(6,4))")
100 connection.execute("CREATE TABLE selfref "
101 "(id NUMBER, title NVARCHAR2(2000),"
102 " selfref_id NUMBER)")
103 connection.execute("CREATE TABLE foovalue "
104 "(id NUMBER, foo_id INTEGER,"
105 " value1 INTEGER, value2 INTEGER)")
106 create_table_helpers(connection, self._tables)
107
108 def drop_tables(self):
109 drop_tables(self.connection, self._tables)
110
111 def create_sample_data(self):
112 connection = self.connection
113 connection.execute("INSERT INTO foo (id, title)"
114 " VALUES (10, 'Title 30')")
115 connection.execute("INSERT INTO foo (id, title)"
116 " VALUES (20, 'Title 20')")
117 connection.execute("INSERT INTO foo (id, title)"
118 " VALUES (30, 'Title 10')")
119 connection.execute("INSERT INTO bar (id, foo_id, title)"
120 " VALUES (100, 10, 'Title 300')")
121 connection.execute("INSERT INTO bar (id, foo_id, title)"
122 " VALUES (200, 20, 'Title 200')")
123 connection.execute("INSERT INTO bar (id, foo_id, title)"
124 " VALUES (300, 30, 'Title 100')")
125 connection.execute("INSERT INTO bin (id, bin) VALUES (10, rawtohex('Blob 30'))")
126 connection.execute("INSERT INTO bin (id, bin) VALUES (20, rawtohex('Blob 20'))")
127 connection.execute("INSERT INTO bin (id, bin) VALUES (30, rawtohex('Blob 10'))")
128 connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (10, 100)")
129 connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (10, 200)")
130 connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (10, 300)")
131 connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (20, 100)")
132 connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (20, 200)")
133 connection.execute("INSERT INTO link (foo_id, bar_id) VALUES (30, 300)")
134 connection.execute("INSERT INTO money (id, value)"
135 " VALUES (10, '12.3455')")
136 connection.execute("INSERT INTO selfref (id, title, selfref_id)"
137 " VALUES (15, 'SelfRef 15', NULL)")
138 connection.execute("INSERT INTO selfref (id, title, selfref_id)"
139 " VALUES (25, 'SelfRef 25', NULL)")
140 connection.execute("INSERT INTO selfref (id, title, selfref_id)"
141 " VALUES (35, 'SelfRef 35', 15)")
142 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
143 " VALUES (1, 10, 2, 1)")
144 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
145 " VALUES (2, 10, 2, 1)")
146 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
147 " VALUES (3, 10, 2, 1)")
148 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
149 " VALUES (4, 10, 2, 2)")
150 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
151 " VALUES (5, 20, 1, 3)")
152 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
153 " VALUES (6, 20, 1, 3)")
154 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
155 " VALUES (7, 20, 1, 4)")
156 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
157 " VALUES (8, 20, 1, 4)")
158 connection.execute("INSERT INTO foovalue (id, foo_id, value1, value2)"
159 " VALUES (9, 20, 1, 2)")
160 connection.commit()
161
162 #Overriding this because SELECT ? should be SELECT ? FROM DUAL
163 def test_execute_params(self):
164 result = self.store.execute("SELECT ? FROM DUAL", (1,))
165 self.assertTrue(isinstance(result, Result))
166 self.assertEquals(result.get_one(), (1,))
167
168 def test_is_in_empty_list(self):
169 result2 = self.store.find(Foo, Foo.id.is_in([]))
170 self.assertEquals(result2.count(), 0)
171
172 def setUp(self):
173 TestHelper.setUp(self)
174 StoreTest.setUp(self)
175 self.connection = self.database.connect()
176
177 def tearDown(self):
178 TestHelper.tearDown(self)
179 StoreTest.tearDown(self)
180 self.connection.close()
181
182 def is_supported(self):
183 return bool(os.environ.get("STORM_ORACLE_URI"))
184
185 def create_database(self):
186 self.database = create_database(os.environ["STORM_ORACLE_URI"])
187
188
189class testOracleEmptyResultSet(TestHelper, EmptyResultSetTest):
190
191 helpers = [MakePath]
192
193 def setUp(self):
194 TestHelper.setUp(self)
195 EmptyResultSetTest.setUp(self)
196 self.connection = self.database.connect()
197
198 def tearDown(self):
199 TestHelper.tearDown(self)
200 EmptyResultSetTest.tearDown(self)
201 self.connection.close()
202
203 def is_supported(self):
204 return bool(os.environ.get("STORM_ORACLE_URI"))
205
206 def create_database(self):
207 self.database = create_database(os.environ["STORM_ORACLE_URI"])
208
209 def create_tables(self):
210 connection = self.connection
211 with connection.as_read_committed():
212 connection.execute("CREATE TABLE foo (id NUMBER, "
213 "title VARCHAR2(4000) DEFAULT 'Default Title')")
214 #connection.execute("CREATE OR REPLACE SEQUENCE foo_seq "
215 #"START WITH 1 "
216 #"INCREMENT BY 1 "
217 #"NOMAXVALUE")
218 #connection.execute("CREATE OR REPLACE TRIGGER foo_trigger "
219 #"BEFORE INSERT ON foo "
220 #"FOR EACH ROW "
221 #"BEGIN "
222 #"SELECT foo_seq.nextval INTO :new.id FROM DUAL; "
223 #"END")
224
225 def drop_tables(self):
226 for table in ["foo"]:
227 connection = self.connection
228 try:
229 #connection.execute("DROP TRIGGER %s_trigger" % table)
230 #connection.execute("DROP SEQUENCE %s_seq" % table)
231 connection.execute("DROP TABLE %s" % table)
232 connection.commit()
233 except:
234 connection.rollback()

Subscribers

People subscribed via source and target branches

to status/vote changes: