Burrow

Merge lp:~eday/burrow/backend-unittests into lp:burrow

  1. backend-unittests
  2. Merge into trunk
Proposed by Eric Day
Status: Merged
Approved by: Eric Day
Approved revision: 32
Merged at revision: 30
Proposed branch: lp:~eday/burrow/backend-unittests
Merge into: lp:burrow
Diff against target: 840 lines (+325/-305)
3 files modified
burrow/backend/memory.py (+6/-6)
burrow/backend/sqlite.py (+309/-292)
burrow/frontend/wsgi.py (+10/-7)
To merge this branch: bzr merge lp:~eday/burrow/backend-unittests
Reviewer Review Type Date Requested Status
Burrow Core Team Pending
Review via email: mp+70820@code.launchpad.net

Description of the change

Cleaned up SQLite backend, added accounts table.

To post a comment you must log in.
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :
Download full text (88.7 KiB)

The attempt to merge lp:~eday/burrow/backend-unittests into lp:burrow failed. Below is the output from the failed tests.

running test
Checking .pth file support in .
/usr/bin/python -E -c pass
Searching for webob
Reading http://pypi.python.org/simple/webob/
Reading http://pythonpaste.org/webob/
Reading http://webob.org/
Best match: WebOb 1.1beta1
Downloading http://pypi.python.org/packages/source/W/WebOb/WebOb-1.1beta1.zip#md5=7c3d1e3812159838e7a392a9738998e0
Processing WebOb-1.1beta1.zip
Running WebOb-1.1beta1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-mPBWGS/WebOb-1.1beta1/egg-dist-tmp-c8lTC8

Installed /tmp/tmp_V2pdO/WebOb-1.1beta1-py2.7.egg
Searching for routes
Reading http://pypi.python.org/simple/routes/
Reading http://routes.groovie.org/
Best match: Routes 1.12.3
Downloading http://pypi.python.org/packages/source/R/Routes/Routes-1.12.3.tar.gz#md5=9740ff424ff6b841632c784a38fb2be3
Processing Routes-1.12.3.tar.gz
Running Routes-1.12.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-s6qyPV/Routes-1.12.3/egg-dist-tmp-3sQIKT

Installed /tmp/tmp_V2pdO/Routes-1.12.3-py2.7.egg
running egg_info
creating burrow.egg-info
writing requirements to burrow.egg-info/requires.txt
writing burrow.egg-info/PKG-INFO
writing top-level names to burrow.egg-info/top_level.txt
writing dependency_links to burrow.egg-info/dependency_links.txt
writing manifest file 'burrow.egg-info/SOURCES.txt'
reading manifest file 'burrow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'burrow.egg-info/SOURCES.txt'
running build_ext

no previously-included directories found matching '*.pyc'
no previously-included directories found matching '*.pyo'
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
warning: no previously-included files matching '*.hgignore' found anywhere in distribution
warning: no previously-included files matching '*.hgtags' found anywhere in distribution
warning: no files found matching 'ChangeLog'
test_accounts (test.backend.test_memory.TestMemory) ... ok
test_accounts_delete_detail_all (test.backend.test_memory.TestMemory) ... ok
test_accounts_delete_detail_bad (test.backend.test_memory.TestMemory) ... ok
test_accounts_delete_detail_id (test.backend.test_memory.TestMemory) ... ok
test_accounts_delete_detail_none (test.backend.test_memory.TestMemory) ... ok
test_accounts_delete_limit (test.backend.test_memory.TestMemory) ... ok
test_accounts_delete_marker (test.backend.test_memory.TestMemory) ... ok
test_accounts_delete_marker_limit (test.backend.test_memory.TestMemory) ... ok
test_accounts_get_detail_all (test.backend.test_memory.TestMemory) ... ok
test_accounts_get_detail_bad (test.backend.test_memory.TestMemory) ... ok
test_accounts_get_detail_id (test.backend.test_memory.TestMemory) ... ok
test_accounts_get_detail_none (test.backend.test_memory.TestMemory) ... ok
test_accounts_get_limit (test.backend.test_memory.TestMemory) ... ok
test_accounts_get_marker (test.backend.test_memory.TestMemory) ... ok
test_accounts_get_marker_limit (test.backend.test_memory.TestMemory) ... ok
test_messages (test.backend.test_memory.TestMemory) ... ok
test_messages_delete_detail_all (test.b...

lp:~eday/burrow/backend-unittests updated
32. By Eric Day

Cleaned up more of the SQLite backend, fixed a few bugs introduced with single message operations.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'burrow/backend/memory.py'
2--- burrow/backend/memory.py 2011-08-08 20:04:40 +0000
3+++ burrow/backend/memory.py 2011-08-09 08:07:03 +0000
4@@ -130,10 +130,10 @@
5 def delete_message(self, account, queue, message, filters={}):
6 account, queue = self.accounts.get_queue(account, queue)
7 if queue is None:
8- return None
9+ raise burrow.backend.NotFound()
10 message = queue.messages.get(message)
11 if message is None:
12- return None
13+ raise burrow.backend.NotFound()
14 queue.messages.delete(message.id)
15 if queue.messages.count() == 0:
16 self.accounts.delete_queue(account.id, queue.id)
17@@ -142,21 +142,21 @@
18 def get_message(self, account, queue, message, filters={}):
19 account, queue = self.accounts.get_queue(account, queue)
20 if queue is None:
21- return None
22+ raise burrow.backend.NotFound()
23 message = queue.messages.get(message)
24 if message is None:
25- return None
26+ raise burrow.backend.NotFound()
27 return message.detail()
28
29 def update_message(self, account, queue, message, attributes={},
30 filters={}):
31 account, queue = self.accounts.get_queue(account, queue)
32 if queue is None:
33- return None
34+ raise burrow.backend.NotFound()
35 ttl, hide = self._get_attributes(attributes)
36 message = queue.messages.get(message)
37 if message is None:
38- return None
39+ raise burrow.backend.NotFound()
40 if ttl is not None:
41 message.ttl = ttl
42 if hide is not None:
43
44=== modified file 'burrow/backend/sqlite.py'
45--- burrow/backend/sqlite.py 2011-08-08 20:04:40 +0000
46+++ burrow/backend/sqlite.py 2011-08-09 08:07:03 +0000
47@@ -23,6 +23,10 @@
48 # Default configuration values for this module.
49 DEFAULT_DATABASE = ':memory:'
50
51+# Maximum number of parameters to pass to execute. Testing shows a max of
52+# 999, so leave a few extra for parameters not added by a list of IDs.
53+MAXIMUM_PARAMETERS = 990
54+
55
56 class Backend(burrow.backend.Backend):
57
58@@ -36,316 +40,386 @@
59 self.db = sqlite3.connect(database)
60 self.db.isolation_level = None
61 queries = [
62+ 'CREATE TABLE IF NOT EXISTS accounts ('
63+ 'account VARCHAR(255) NOT NULL,'
64+ 'PRIMARY KEY (account))',
65 'CREATE TABLE IF NOT EXISTS queues ('
66- 'account VARCHAR(255) NOT NULL,'
67+ 'account INT UNSIGNED NOT NULL,'
68 'queue VARCHAR(255) NOT NULL,'
69 'PRIMARY KEY (account, queue))',
70 'CREATE TABLE IF NOT EXISTS messages ('
71 'queue INT UNSIGNED NOT NULL,'
72- 'name VARCHAR(255) NOT NULL,'
73+ 'message VARCHAR(255) NOT NULL,'
74 'ttl INT UNSIGNED NOT NULL,'
75 'hide INT UNSIGNED NOT NULL,'
76 'body BLOB NOT NULL,'
77- 'PRIMARY KEY (queue, name))']
78+ 'PRIMARY KEY (queue, message))']
79 for query in queries:
80 self.db.execute(query)
81
82 def delete_accounts(self, filters={}):
83 if len(filters) == 0:
84- count = self.db.execute('SELECT COUNT(*) FROM queues').fetchall()
85- if count[0][0] == 0:
86+ query = 'SELECT rowid FROM accounts LIMIT 1'
87+ if len(self.db.execute(query).fetchall()) == 0:
88 raise burrow.backend.NotFound()
89+ self.db.execute('DELETE FROM accounts')
90 self.db.execute('DELETE FROM queues')
91 self.db.execute('DELETE FROM messages')
92 return
93- query = 'SELECT rowid,account FROM queues'
94- values = tuple()
95- marker = filters.get('marker', None)
96- if marker is not None:
97- query += ' WHERE account >= ?'
98- values += (marker,)
99- limit = filters.get('limit', None)
100- detail = self._get_detail(filters, 'id')
101- current_account = None
102+ count = 0
103+ detail = self._get_detail(filters)
104 ids = []
105- marker_found = False
106- count = 0
107- for row in self.db.execute(query, values):
108- if marker == row[1]:
109- marker_found = True
110- continue
111- elif marker is not None and not marker_found:
112- break
113- if current_account != row[1]:
114- if limit is not None:
115- if limit == 0:
116- break
117- limit -= 1
118- current_account = row[1]
119- count += 1
120- if detail == 'id':
121- yield row[1]
122- elif detail == 'all':
123- yield dict(id=row[1])
124+ query = 'SELECT rowid,account FROM accounts'
125+ for row in self._get_accounts(query, filters):
126+ count += 1
127+ if detail is not None:
128+ yield self._detail(row[1:], detail)
129 ids.append(row[0])
130- if len(ids) == 999:
131- self._delete_queues(ids)
132+ if len(ids) == MAXIMUM_PARAMETERS:
133+ self._delete_accounts(ids)
134 ids = []
135- if marker is not None and not marker_found:
136- filters = dict(filters)
137- filters.pop('marker')
138- for account in self.delete_accounts(filters):
139- count += 1
140- yield account
141 if count == 0:
142 raise burrow.backend.NotFound()
143 if len(ids) > 0:
144- self._delete_queues(ids)
145+ self._delete_accounts(ids)
146+
147+ def _delete_accounts(self, ids):
148+ ids = tuple(ids)
149+ query_values = '(?' + (',?' * (len(ids) - 1)) + ')'
150+ queue_ids = []
151+ queue_query = 'DELETE FROM messages WHERE queue IN '
152+ queue_query_values = '(?' + (',?' * (MAXIMUM_PARAMETERS - 1)) + ')'
153+ query = 'SELECT rowid FROM queues WHERE account IN '
154+ for row in self.db.execute(query + query_values, ids):
155+ queue_ids.append(row[0])
156+ if len(queue_ids) == MAXIMUM_PARAMETERS:
157+ self.db.execute(queue_query + queue_query_values, queue_ids)
158+ queue_ids = []
159+ if len(queue_ids) > 0:
160+ queue_query_values = '(?' + (',?' * (len(queue_ids) - 1)) + ')'
161+ self.db.execute(queue_query + queue_query_values, queue_ids)
162+ query = 'DELETE FROM queues WHERE account IN '
163+ self.db.execute(query + query_values, ids)
164+ query = 'DELETE FROM accounts WHERE rowid IN '
165+ self.db.execute(query + query_values, ids)
166+
167+ def _get_detail(self, filters, default=None):
168+ detail = filters.get('detail', default)
169+ if detail == 'none':
170+ detail = None
171+ elif detail is not None and detail not in ['id', 'all']:
172+ raise burrow.backend.BadDetail(detail)
173+ return detail
174+
175+ def _detail(self, row, detail):
176+ if detail == 'id':
177+ return row[0]
178+ return dict(id=row[0])
179
180 def get_accounts(self, filters={}):
181- query = 'SELECT DISTINCT account FROM queues'
182+ count = 0
183+ detail = self._get_detail(filters, 'id')
184+ query = 'SELECT account FROM accounts'
185+ for row in self._get_accounts(query, filters):
186+ count += 1
187+ if detail is not None:
188+ yield self._detail(row, detail)
189+ if count == 0:
190+ raise burrow.backend.NotFound()
191+
192+ def _get_accounts(self, query, filters):
193 values = tuple()
194- limit = filters.get('limit', None)
195 marker = filters.get('marker', None)
196 if marker is not None:
197- query += ' WHERE account >= ?'
198- values += (marker,)
199- if limit is not None:
200- limit += 1
201+ try:
202+ marker = self._get_account(marker)
203+ query += ' WHERE rowid > ?'
204+ values += (marker,)
205+ except burrow.backend.NotFound:
206+ marker = None
207+ limit = filters.get('limit', None)
208 if limit is not None:
209 query += ' LIMIT ?'
210 values += (limit,)
211- detail = self._get_detail(filters, 'id')
212- marker_found = False
213- count = 0
214- for row in self.db.execute(query, values):
215- if marker == row[0]:
216- marker_found = True
217- continue
218- elif marker is not None and not marker_found:
219- break
220- count += 1
221- if detail == 'id':
222- yield row[0]
223- elif detail == 'all':
224- yield dict(id=row[0])
225- if marker is not None and not marker_found:
226- filters = dict(filters)
227- filters.pop('marker')
228- for account in self.get_accounts(filters):
229- count += 1
230- yield account
231- if count == 0:
232+ return self.db.execute(query, values)
233+
234+ def _get_account(self, account):
235+ query = 'SELECT rowid FROM accounts WHERE account=?'
236+ account_rowid = self.db.execute(query, (account,)).fetchall()
237+ if len(account_rowid) == 0:
238 raise burrow.backend.NotFound()
239+ return account_rowid[0][0]
240
241 def delete_queues(self, account, filters={}):
242- query = 'SELECT rowid,queue FROM queues WHERE account=?'
243- values = (account,)
244- query, values, marker = self._add_queue_filters(query, values, filters)
245- detail = self._get_detail(filters, None)
246+ account_rowid = self._get_account(account)
247+ count = 0
248+ detail = self._get_detail(filters)
249 ids = []
250- marker_found = False
251- count = 0
252- for row in self.db.execute(query, values):
253- if marker == row[1]:
254- marker_found = True
255- continue
256- elif marker is not None and not marker_found:
257- break
258+ query = 'SELECT rowid,queue FROM queues'
259+ for row in self._get_queues(query, account_rowid, filters):
260 count += 1
261- if detail == 'id':
262- yield row[1]
263- elif detail == 'all':
264- yield dict(id=row[1])
265+ if detail is not None:
266+ yield self._detail(row[1:], detail)
267 ids.append(row[0])
268- if len(ids) == 999:
269+ if len(ids) == MAXIMUM_PARAMETERS:
270 self._delete_queues(ids)
271 ids = []
272- if marker is not None and not marker_found:
273- filters = dict(filters)
274- filters.pop('marker')
275- for queue in self.delete_queues(account, filters):
276- count += 1
277- yield queue
278 if count == 0:
279 raise burrow.backend.NotFound()
280 if len(ids) > 0:
281 self._delete_queues(ids)
282+ self._check_empty_account(account_rowid)
283+
284+ def _delete_queues(self, ids):
285+ ids = tuple(ids)
286+ query_values = '(?' + (',?' * (len(ids) - 1)) + ')'
287+ query = 'DELETE FROM messages WHERE queue IN '
288+ self.db.execute(query + query_values, ids)
289+ query = 'DELETE FROM queues WHERE rowid IN '
290+ self.db.execute(query + query_values, ids)
291+
292+ def _check_empty_account(self, account_rowid):
293+ query = 'SELECT rowid FROM queues WHERE account=? LIMIT 1'
294+ if len(self.db.execute(query, (account_rowid,)).fetchall()) == 0:
295+ query = 'DELETE FROM accounts WHERE rowid=?'
296+ self.db.execute(query, (account_rowid,))
297
298 def get_queues(self, account, filters={}):
299- query = 'SELECT queue FROM queues WHERE account=?'
300- values = (account,)
301- query, values, marker = self._add_queue_filters(query, values, filters)
302+ account_rowid = self._get_account(account)
303+ count = 0
304 detail = self._get_detail(filters, 'id')
305- marker_found = False
306- count = 0
307- for row in self.db.execute(query, values):
308- if marker == row[0]:
309- marker_found = True
310- continue
311- elif marker is not None and not marker_found:
312- break
313+ query = 'SELECT queue FROM queues'
314+ for row in self._get_queues(query, account_rowid, filters):
315 count += 1
316- if detail == 'id':
317- yield row[0]
318- elif detail == 'all':
319- yield dict(id=row[0])
320- if marker is not None and not marker_found:
321- filters = dict(filters)
322- filters.pop('marker')
323- for queue in self.get_queues(account, filters):
324- count += 1
325- yield queue
326+ if detail is not None:
327+ yield self._detail(row, detail)
328 if count == 0:
329 raise burrow.backend.NotFound()
330
331- def _add_queue_filters(self, query, values, filters):
332- limit = filters.get('limit', None)
333+ def _get_queues(self, query, account_rowid, filters):
334+ query += ' WHERE account=?'
335+ values = (account_rowid,)
336 marker = filters.get('marker', None)
337 if marker is not None:
338- query += ' AND queue >= ?'
339- values += (marker,)
340- if limit is not None:
341- limit += 1
342+ try:
343+ marker = self._get_queue(account_rowid, marker)
344+ query += ' AND rowid > ?'
345+ values += (marker,)
346+ except burrow.backend.NotFound:
347+ marker = None
348+ limit = filters.get('limit', None)
349 if limit is not None:
350 query += ' LIMIT ?'
351 values += (limit,)
352- return query, values, marker
353-
354- def _delete_queues(self, ids):
355- query = 'DELETE FROM messages WHERE queue IN (?' + \
356- (',?' * (len(ids) - 1)) + ')'
357- self.db.execute(query, tuple(ids))
358- query = 'DELETE FROM queues WHERE rowid IN (?' + \
359- (',?' * (len(ids) - 1)) + ')'
360- self.db.execute(query, tuple(ids))
361-
362- def _get_detail(self, filters, default=None):
363- detail = filters.get('detail', default)
364- if detail == 'none':
365- detail = None
366- elif detail is not None and detail not in ['id', 'all']:
367- raise burrow.backend.BadDetail(detail)
368- return detail
369+ return self.db.execute(query, values)
370+
371+ def _get_queue(self, account_rowid, queue):
372+ query = 'SELECT rowid FROM queues WHERE account=? AND queue=?'
373+ queue_rowid = self.db.execute(query, (account_rowid, queue)).fetchall()
374+ if len(queue_rowid) == 0:
375+ raise burrow.backend.NotFound()
376+ return queue_rowid[0][0]
377
378 def delete_messages(self, account, queue, filters={}):
379- result = self._get_messages(account, queue, filters)
380- rowid = result.next()
381+ account_rowid = self._get_account(account)
382+ queue_rowid = self._get_queue(account_rowid, queue)
383+ count = 0
384+ detail = self._get_message_detail(filters)
385 ids = []
386- detail = self._get_message_detail(filters)
387- for message in result:
388- ids.append(message[0])
389+ query = 'SELECT rowid,message,ttl,hide,body FROM messages'
390+ for row in self._get_messages(query, queue_rowid, filters):
391+ count += 1
392 if detail is not None:
393- yield self._message_detail(message, detail)
394- values = (rowid,) + tuple(ids)
395- query = 'DELETE FROM messages WHERE queue=? AND name IN (%s)'
396- self.db.execute(query % ','.join('?' * len(ids)), values)
397+ yield self._message_detail(row[1:], detail)
398+ ids.append(row[0])
399+ if len(ids) == MAXIMUM_PARAMETERS:
400+ self._delete_messages(ids)
401+ ids = []
402+ if count == 0:
403+ raise burrow.backend.NotFound()
404+ if len(ids) > 0:
405+ self._delete_messages(ids)
406+ self._check_empty_queue(account_rowid, queue_rowid)
407+
408+ def _delete_messages(self, ids):
409+ ids = tuple(ids)
410+ query_values = '(?' + (',?' * (len(ids) - 1)) + ')'
411+ query = 'DELETE FROM messages WHERE rowid IN '
412+ self.db.execute(query + query_values, ids)
413+
414+ def _check_empty_queue(self, account_rowid, queue_rowid):
415 query = 'SELECT rowid FROM messages WHERE queue=? LIMIT 1'
416- if len(self.db.execute(query, (rowid,)).fetchall()) == 0:
417- query = 'DELETE FROM queues WHERE rowid=?'
418- self.db.execute(query, (rowid,))
419+ if len(self.db.execute(query, (queue_rowid,)).fetchall()) == 0:
420+ self.db.execute('DELETE FROM queues WHERE rowid=?', (queue_rowid,))
421+ self._check_empty_account(account_rowid)
422+
423+ def _get_message_detail(self, filters, default=None):
424+ detail = filters.get('detail', default)
425+ options = ['id', 'attributes', 'body', 'all']
426+ if detail == 'none':
427+ detail = None
428+ elif detail is not None and detail not in options:
429+ raise burrow.backend.BadDetail(detail)
430+ return detail
431+
432+ def _message_detail(self, row, detail):
433+ if detail == 'id':
434+ return row[0]
435+ elif detail == 'body':
436+ return str(row[3])
437+ ttl = row[1]
438+ if ttl > 0:
439+ ttl -= int(time.time())
440+ hide = row[2]
441+ if hide > 0:
442+ hide -= int(time.time())
443+ if detail == 'attributes':
444+ return dict(id=row[0], ttl=ttl, hide=hide)
445+ return dict(id=row[0], ttl=ttl, hide=hide, body=str(row[3]))
446
447 def get_messages(self, account, queue, filters={}):
448- result = self._get_messages(account, queue, filters)
449- result.next()
450+ account_rowid = self._get_account(account)
451+ queue_rowid = self._get_queue(account_rowid, queue)
452+ count = 0
453 detail = self._get_message_detail(filters, 'all')
454- for message in result:
455+ query = 'SELECT message,ttl,hide,body FROM messages'
456+ for row in self._get_messages(query, queue_rowid, filters):
457+ count += 1
458 if detail is not None:
459- yield self._message_detail(message, detail)
460+ yield self._message_detail(row, detail)
461+ if count == 0:
462+ raise burrow.backend.NotFound()
463+
464+ def _get_messages(self, query, queue_rowid, filters):
465+ query += ' WHERE queue=?'
466+ values = (queue_rowid,)
467+ marker = filters.get('marker', None)
468+ if marker is not None:
469+ try:
470+ marker = self._get_message(queue_rowid, marker)
471+ query += ' AND rowid > ?'
472+ values += (marker,)
473+ except burrow.backend.NotFound:
474+ marker = None
475+ match_hidden = filters.get('match_hidden', False)
476+ if match_hidden is False:
477+ query += ' AND hide=0'
478+ limit = filters.get('limit', None)
479+ if limit is not None:
480+ query += ' LIMIT ?'
481+ values += (limit,)
482+ return self.db.execute(query, values)
483+
484+ def _get_message(self, queue_rowid, message):
485+ query = 'SELECT rowid FROM messages WHERE queue=? AND message=?'
486+ values = (queue_rowid, message)
487+ message_rowid = self.db.execute(query, values).fetchall()
488+ if len(message_rowid) == 0:
489+ raise burrow.backend.NotFound()
490+ return message_rowid[0][0]
491
492 def update_messages(self, account, queue, attributes={}, filters={}):
493- result = self._get_messages(account, queue, filters)
494- rowid = result.next()
495+ account_rowid = self._get_account(account)
496+ queue_rowid = self._get_queue(account_rowid, queue)
497+ count = 0
498+ detail = self._get_message_detail(filters)
499 ids = []
500+ notify = False
501 ttl = attributes.get('ttl', None)
502 if ttl is not None and ttl > 0:
503 ttl += int(time.time())
504 hide = attributes.get('hide', None)
505 if hide is not None and hide > 0:
506 hide += int(time.time())
507- detail = self._get_message_detail(filters)
508- for message in result:
509- message = list(message)
510- ids.append(message[0])
511- if ttl is not None:
512- message[1] = ttl
513- if hide is not None:
514- message[2] = hide
515+ query = 'SELECT rowid,message,ttl,hide,body FROM messages'
516+ for row in self._get_messages(query, queue_rowid, filters):
517+ count += 1
518 if detail is not None:
519- yield self._message_detail(message, detail)
520- query = 'UPDATE messages SET'
521+ row = list(row)
522+ if ttl is not None:
523+ row[2] = ttl
524+ if hide is not None:
525+ row[3] = hide
526+ yield self._message_detail(row[1:], detail)
527+ ids.append(row[0])
528+ if len(ids) == MAXIMUM_PARAMETERS:
529+ if self._update_messages(ttl, hide, ids):
530+ notify = True
531+ ids = []
532+ if count == 0:
533+ raise burrow.backend.NotFound()
534+ if len(ids) > 0:
535+ if self._update_messages(ttl, hide, ids):
536+ notify = True
537+ if notify:
538+ self.notify(account, queue)
539+
540+ def _update_messages(self, ttl, hide, ids):
541+ query = 'UPDATE messages SET '
542+ query_values = ' WHERE rowid IN (?' + (',?' * (len(ids) - 1)) + ')'
543+ values = []
544 comma = ''
545- values = tuple()
546 if ttl is not None:
547- query += '%s ttl=?' % comma
548- values += (ttl,)
549+ query += comma + 'ttl=?'
550+ values.append(ttl)
551 comma = ','
552 if hide is not None:
553- query += '%s hide=?' % comma
554- values += (hide,)
555+ query += comma + 'hide=?'
556+ values.append(hide)
557 comma = ','
558 if comma == '':
559- return
560- values += (rowid,)
561- values += tuple(ids)
562- query += ' WHERE queue=? AND name IN (%s)'
563- self.db.execute(query % ','.join('?' * len(ids)), values)
564- self.notify(account, queue)
565+ return False
566+ self.db.execute(query + query_values, tuple(values + ids))
567+ return True
568
569 def create_message(self, account, queue, message, body, attributes={}):
570- query = "SELECT rowid FROM queues " \
571- "WHERE account='%s' AND queue='%s'" % (account, queue)
572- result = self.db.execute(query).fetchall()
573- if len(result) == 0:
574- query = "INSERT INTO queues VALUES ('%s', '%s')" % (account, queue)
575- rowid = self.db.execute(query).lastrowid
576- else:
577- rowid = result[0][0]
578- query = "SELECT rowid FROM messages WHERE queue=%d AND name='%s'" % \
579- (rowid, message)
580- result = self.db.execute(query).fetchall()
581+ try:
582+ account_rowid = self._get_account(account)
583+ except burrow.backend.NotFound:
584+ query = 'INSERT INTO accounts VALUES (?)'
585+ account_rowid = self.db.execute(query, (account,)).lastrowid
586+ try:
587+ queue_rowid = self._get_queue(account_rowid, queue)
588+ except burrow.backend.NotFound:
589+ query = 'INSERT INTO queues VALUES (?, ?)'
590+ values = (account_rowid, queue)
591+ queue_rowid = self.db.execute(query, values).lastrowid
592 ttl = attributes.get('ttl', 0)
593 if ttl > 0:
594 ttl += int(time.time())
595 hide = attributes.get('hide', 0)
596 if hide > 0:
597 hide += int(time.time())
598- if len(result) == 0:
599- query = "INSERT INTO messages VALUES (?, ?, ?, ?, ?)"
600- self.db.execute(query, (rowid, message, ttl, hide, body))
601+ query = 'SELECT rowid FROM messages WHERE queue=? AND message=?'
602+ values = (queue_rowid, message)
603+ message_rowid = self.db.execute(query, values).fetchall()
604+ if len(message_rowid) == 0:
605+ query = 'INSERT INTO messages VALUES (?,?,?,?,?)'
606+ self.db.execute(query, (queue_rowid, message, ttl, hide, body))
607 self.notify(account, queue)
608 return True
609- query = "UPDATE messages SET ttl=?, hide=?, body=? WHERE rowid=?"
610- self.db.execute(query, (ttl, hide, body, result[0][0]))
611+ query = 'UPDATE messages SET ttl=?,hide=?,body=? WHERE rowid=?'
612+ self.db.execute(query, (ttl, hide, body, message_rowid[0][0]))
613 if hide == 0:
614 self.notify(account, queue)
615 return False
616
617 def delete_message(self, account, queue, message):
618- rowid = self._get_queue(account, queue)
619- if rowid is None:
620- return None
621+ account_rowid = self._get_account(account)
622+ queue_rowid = self._get_queue(account_rowid, queue)
623+ message_rowid = self._get_message(queue_rowid, message)
624 message = self.get_message(account, queue, message)
625- if message is None:
626- return None
627- query = "DELETE FROM messages WHERE queue=%d AND name='%s'" % \
628- (rowid, message['id'])
629- self.db.execute(query)
630- query = "SELECT rowid FROM messages WHERE queue=%d LIMIT 1" % rowid
631- if len(self.db.execute(query).fetchall()) == 0:
632- query = "DELETE FROM queues WHERE rowid=%d" % rowid
633- self.db.execute(query)
634+ self.db.execute('DELETE FROM messages WHERE rowid=?', (message_rowid,))
635+ self._check_empty_queue(account_rowid, queue_rowid)
636 return message
637
638 def get_message(self, account, queue, message):
639- rowid = self._get_queue(account, queue)
640- if rowid is None:
641- return None
642- query = "SELECT name,ttl,hide,body FROM messages " \
643- "WHERE queue=%d AND name='%s'" % (rowid, message)
644- result = self.db.execute(query).fetchall()
645+ queue_rowid = self._get_queue(self._get_account(account), queue)
646+ query = 'SELECT message,ttl,hide,body FROM messages ' \
647+ 'WHERE queue=? AND message=?'
648+ result = self.db.execute(query, (queue_rowid, message)).fetchall()
649 if len(result) == 0:
650- return None
651+ raise burrow.backend.NotFound()
652 row = result[0]
653 ttl = row[1]
654 if ttl > 0:
655@@ -356,41 +430,43 @@
656 return dict(id=row[0], ttl=ttl, hide=hide, body=str(row[3]))
657
658 def update_message(self, account, queue, message, attributes):
659- rowid = self._get_queue(account, queue)
660- if rowid is None:
661- return None
662+ queue_rowid = self._get_queue(self._get_account(account), queue)
663 message = self.get_message(account, queue, message)
664 if message is None:
665- return None
666- query = "UPDATE messages SET"
667+ raise burrow.backend.NotFound()
668+ query = 'UPDATE messages SET'
669+ values = tuple()
670 comma = ''
671 ttl = attributes.get('ttl', None)
672- hide = attributes.get('hide', None)
673 if ttl is not None:
674 message['ttl'] = ttl
675 if ttl > 0:
676 ttl += int(time.time())
677- query += "%s ttl=%d" % (comma, ttl)
678+ query += comma + ' ttl=?'
679+ values += (ttl,)
680 comma = ','
681+ hide = attributes.get('hide', None)
682 if hide is not None:
683 message['hide'] = hide
684 if hide > 0:
685 hide += int(time.time())
686- query += "%s hide=%d" % (comma, hide)
687+ query += comma + ' hide=?'
688+ values += (hide,)
689 comma = ','
690 if comma == '':
691 return message
692- query += " WHERE queue=%d AND name='%s'" % (rowid, message['id'])
693- self.db.execute(query)
694+ query += ' WHERE queue=? AND message=?'
695+ values += (queue_rowid, message['id'])
696+ self.db.execute(query, values)
697 if hide == 0:
698 self.notify(account, queue)
699 return message
700
701 def clean(self):
702 now = int(time.time())
703- query = "SELECT rowid,queue FROM messages " \
704- "WHERE ttl > 0 AND ttl <= %d" % now
705- result = self.db.execute(query).fetchall()
706+ query = 'SELECT rowid,queue FROM messages ' \
707+ 'WHERE ttl > 0 AND ttl <= ?'
708+ result = self.db.execute(query, (now,)).fetchall()
709 if len(result) > 0:
710 messages = []
711 queues = []
712@@ -400,12 +476,11 @@
713 query = 'DELETE FROM messages WHERE rowid in (%s)' % \
714 ','.join(messages)
715 self.db.execute(query)
716- for queue in queues:
717- query = "SELECT rowid FROM messages WHERE queue=%d LIMIT 1" % \
718- queue
719- if len(self.db.execute(query).fetchall()) == 0:
720- query = "DELETE FROM queues WHERE rowid=%d" % queue
721- self.db.execute(query)
722+ for queue_rowid in queues:
723+ query = 'SELECT account FROM queues WHERE rowid=?'
724+ account_rowid = self.db.execute(query, (queue_rowid,))
725+ account_rowid = account_rowid.fetchall()[0][0]
726+ self._check_empty_queue(account_rowid, queue_rowid)
727 query = "SELECT rowid,queue FROM messages WHERE " \
728 "hide > 0 AND hide <= %d" % now
729 result = self.db.execute(query).fetchall()
730@@ -419,67 +494,9 @@
731 ','.join(messages)
732 self.db.execute(query)
733 for queue in queues:
734- query = "SELECT account,queue FROM queues WHERE rowid=%d" % \
735- queue
736- result = self.db.execute(query).fetchall()[0]
737+ query = 'SELECT accounts.account,queues.queue ' \
738+ 'FROM queues JOIN accounts ' \
739+ 'ON queues.account=accounts.rowid ' \
740+ 'WHERE queues.rowid=?'
741+ result = self.db.execute(query, (queue,)).fetchall()[0]
742 self.notify(result[0], result[1])
743-
744- def _get_queue(self, account, queue):
745- query = "SELECT rowid FROM queues " \
746- "WHERE account='%s' AND queue='%s'" % \
747- (account, queue)
748- result = self.db.execute(query).fetchall()
749- if len(result) == 0:
750- return None
751- return result[0][0]
752-
753- def _get_messages(self, account, queue, filters):
754- rowid = self._get_queue(account, queue)
755- yield rowid
756- if rowid is None:
757- raise burrow.backend.NotFound()
758- marker = None
759- if 'marker' in filters and filters['marker'] is not None:
760- query = "SELECT rowid FROM messages " \
761- "WHERE queue=%d AND name='%s'" % (rowid, filters['marker'])
762- result = self.db.execute(query).fetchall()
763- if len(result) > 0:
764- marker = result[0][0]
765- query = "SELECT name,ttl,hide,body FROM messages WHERE queue=%d" % \
766- rowid
767- if marker is not None:
768- query += " AND rowid > %d" % marker
769- if 'match_hidden' not in filters or filters['match_hidden'] is False:
770- query += " AND hide == 0"
771- if 'limit' in filters and filters['limit'] is not None:
772- query += " LIMIT %d" % filters['limit']
773- count = 0
774- for row in self.db.execute(query):
775- yield row
776- count += 1
777- if count == 0:
778- raise burrow.backend.NotFound()
779-
780- def _get_message_detail(self, filters, default=None):
781- detail = filters.get('detail', default)
782- options = ['id', 'attributes', 'body', 'all']
783- if detail == 'none':
784- detail = None
785- elif detail is not None and detail not in options:
786- raise burrow.backend.BadDetail(detail)
787- return detail
788-
789- def _message_detail(self, row, detail):
790- if detail == 'id':
791- return row[0]
792- elif detail == 'body':
793- return str(row[3])
794- ttl = row[1]
795- if ttl > 0:
796- ttl -= int(time.time())
797- hide = row[2]
798- if hide > 0:
799- hide -= int(time.time())
800- if detail == 'attributes':
801- return dict(id=row[0], ttl=ttl, hide=hide)
802- return dict(id=row[0], ttl=ttl, hide=hide, body=str(row[3]))
803
804=== modified file 'burrow/frontend/wsgi.py'
805--- burrow/frontend/wsgi.py 2011-08-04 23:43:09 +0000
806+++ burrow/frontend/wsgi.py 2011-08-09 08:07:03 +0000
807@@ -172,24 +172,27 @@
808
809 @webob.dec.wsgify
810 def _delete_message(self, req, account, queue, message):
811- message = self.backend.delete_message(account, queue, message)
812- if message is None:
813+ try:
814+ message = self.backend.delete_message(account, queue, message)
815+ except burrow.backend.NotFound:
816 return self._response(status=404)
817 return self._return_message(req, account, queue, message, 'none')
818
819 @webob.dec.wsgify
820 def _get_message(self, req, account, queue, message):
821- message = self.backend.get_message(account, queue, message)
822- if message is None:
823+ try:
824+ message = self.backend.get_message(account, queue, message)
825+ except burrow.backend.NotFound:
826 return self._response(status=404)
827 return self._return_message(req, account, queue, message, 'all')
828
829 @webob.dec.wsgify
830 def _post_message(self, req, account, queue, message):
831 attributes = self._parse_attributes(req)
832- message = self.backend.update_message(account, queue, message,
833- attributes)
834- if message is None:
835+ try:
836+ message = self.backend.update_message(account, queue, message,
837+ attributes)
838+ except burrow.backend.NotFound:
839 return self._response(status=404)
840 return self._return_message(req, account, queue, message, 'id')
841

Subscribers

People subscribed via source and target branches