Gwibber

Merge lp:~barry/gwibber/testlogger into lp:~barry/gwibber/py3

  1. testlogger
  2. Merge into py3
Proposed by Robert Bruce Park
Status: Merged
Merged at revision: 1437
Proposed branch: lp:~barry/gwibber/testlogger
Merge into: lp:~barry/gwibber/py3
Diff against target: 863 lines (+208/-443)
10 files modified
gwibber/gwibber/errors.py (+3/-0)
gwibber/gwibber/protocols/flickr.py (+4/-4)
gwibber/gwibber/testing/helpers.py (+0/-49)
gwibber/gwibber/testing/mocks.py (+84/-0)
gwibber/gwibber/tests/test_account.py (+3/-2)
gwibber/gwibber/tests/test_flickr.py (+27/-17)
gwibber/gwibber/tests/test_foursquare.py (+19/-11)
gwibber/gwibber/tests/test_twitter.py (+13/-11)
gwibber/gwibber/utils/base.py (+55/-15)
gwibber/microblog/plugins/foursquare/__init__.py (+0/-334)
To merge this branch: bzr merge lp:~barry/gwibber/testlogger
Reviewer Review Type Date Requested Status
Barry Warsaw Pending
Review via email: mp+125623@code.launchpad.net

Description of the change

Hmmm, looks like the only way for me to comment is with an mp ;-)

So I like the looks of the threading.Thread subclass, that's a clever idea by itself. I do think that mocking out this _SYNCHRONIZE constant is a bit ugly. Is there no better way for _OperationThread to discover that it's running inside a testsuite, without relying on a mocked value? Surely there must be some non-ugly way to inspect the runtime environment and discover that there are unittest instances in memory. Or maybe this could somehow be linked to args.test from main.py, or something ;-)

Or maybe, why don't you just define the subclass in testing.helpers, and then let the live code use threading.Thread directly, only mocking it out for the synchronous one in the unittests directly? I think that would be a bit less ugly, fewer layers of indirection that way. Mocking base.__dict__ is pretty ugly ;-)

To post a comment you must log in.
Revision history for this message
Barry Warsaw (barry) wrote :

On Sep 21, 2012, at 03:04 AM, Robert Bruce Park wrote:

>Hmmm, looks like the only way for me to comment is with an mp ;-)

Well, of course it's also possible to just bzr diff and send an email, but
your right that a merge proposal is the best way to comment on a branch for
the permanent public record.

>So I like the looks of the threading.Thread subclass, that's a clever idea by
>itself. I do think that mocking out this _SYNCHRONIZE constant is a bit
>ugly. Is there no better way for _OperationThread to discover that it's
>running inside a testsuite, without relying on a mocked value? Surely there
>must be some non-ugly way to inspect the runtime environment and discover
>that there are unittest instances in memory. Or maybe this could somehow be
>linked to args.test from main.py, or something ;-)
>
>Or maybe, why don't you just define the subclass in testing.helpers, and then
>let the live code use threading.Thread directly, only mocking it out for the
>synchronous one in the unittests directly? I think that would be a bit less
>ugly, fewer layers of indirection that way. Mocking base.__dict__ is pretty
>ugly ;-)

Don't worry, I knew it (and frankly, mocking out various module 'log'
attributes) was ugly in the branch, which is why I did the work in a branch
before merging to what is effectively our trunk. I also knew I could clean it
up, and I did! :)

Now, in order to mock the various 'logs' you create an instance of LogMock()
in your setUp(), passing in the list of modules you want to mock. The last
component can be the wildcard '*' so if you want to mock base.py and
everything in protocols/ you would do the following:

    self.log_mock = LogMock('gwibber.utils.base', 'gwibber.protcools.*')

Instantiating this object starts the mocking. Your tearDown() needs to stop
the mocking, so it must call

    self.log_mock.stop()

To read all the accumulated log entries, including in subthreads, just do:

    text = self.log_mock.empty()

and this will return a big string of the log contents up to that point. By
default, empty() will trim some of the variable details out of any tracebacks,
but of course you can disable that if you want (you probably wouldn't want to
except in doctests, which we're not using).

As for enabling synchronization, now all you do is set the class attribute
Base._SYNCHRONIZE = True. Do that in your setUp() and be sure to remember to
Base._SYNCHRONIZE = False in your tearDown().

Take a look at the merge to lp:~barry/gwibber/py3. So much prettier, and
nothing magical going on at all!

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'gwibber/gwibber/errors.py'
2--- gwibber/gwibber/errors.py 2012-09-06 20:48:07 +0000
3+++ gwibber/gwibber/errors.py 2012-09-21 03:03:20 +0000
4@@ -32,6 +32,9 @@
5 self.account = account
6 self.message = message
7
8+ def __str__(self):
9+ return '{} (account: {})'.format(self.message, self.account)
10+
11
12 class UnsupportedProtocolError(GwibberError):
13 def __init__(self, protocol):
14
15=== modified file 'gwibber/gwibber/protocols/flickr.py'
16--- gwibber/gwibber/protocols/flickr.py 2012-09-19 22:21:39 +0000
17+++ gwibber/gwibber/protocols/flickr.py 2012-09-21 03:03:20 +0000
18@@ -19,7 +19,6 @@
19 ]
20
21
22-import gettext
23 import logging
24
25 from gwibber.errors import AuthorizationError
26@@ -31,7 +30,6 @@
27
28
29 log = logging.getLogger('gwibber.service')
30-_ = gettext.lgettext
31
32
33 # This is contact information for the Flickr REST service.
34@@ -142,9 +140,9 @@
35 """Download all of a user's public photos."""
36 user_id = self._get_nsid()
37 if user_id is None:
38- log.error('Flickr: {}'.format(_('No NSID available')))
39+ log.error('Flickr: No NSID available')
40 raise AuthorizationError(
41- self.account, _('No Flickr user id available'))
42+ self.account['id'], 'No Flickr user id available')
43 # The user is logged into Flickr.
44 GET_arguments = dict(
45 api_key = API_KEY,
46@@ -157,3 +155,5 @@
47 response = get_json(REST_SERVER, GET_arguments)
48 return [self._reformat(data)
49 for data in response.get('photos', {}).get('photo', [])]
50+
51+ receive = images
52
53=== modified file 'gwibber/gwibber/testing/helpers.py'
54--- gwibber/gwibber/testing/helpers.py 2012-09-07 21:51:08 +0000
55+++ gwibber/gwibber/testing/helpers.py 2012-09-21 03:03:20 +0000
56@@ -16,43 +16,11 @@
57
58 __all__ = [
59 'FakeAccount',
60- 'LogPreserver',
61- 'SettingsIterMock',
62 ]
63
64
65-import logging
66 import threading
67
68-from gwibber.utils.logging import initialize
69-
70-
71-class LogPreserver:
72- """Preserve and restore the state of the gwibber.service logger."""
73-
74- def __init__(self):
75- # Some of side-effects of these tests will be to output messages to
76- # the logger, but we don't want that output cluttering up the test
77- # results. Grab the logger object and shut it up, but preserve its
78- # state so that it can be restored after these tests run.
79- #
80- # We can't just blast away the logging object because the modules
81- # under test will already have a handle on it via their module globals
82- # (extracted at initial import time). So we have to perform our
83- # tricks on the actual shared logging object.
84- initialize()
85- log = logging.getLogger('gwibber.service')
86- self._propagate = log.propagate
87- self._level = log.getEffectiveLevel()
88- log.propagate = False
89- log.setLevel(logging.CRITICAL)
90-
91- def restore(self):
92- """Restore the logger."""
93- log = logging.getLogger('gwibber.service')
94- log.propagate = self._propagate
95- log.setLevel(self._level)
96-
97
98 class FakeAccount(dict):
99 """A fake account object for testing purposes."""
100@@ -61,20 +29,3 @@
101 self.login_lock = threading.Lock()
102 self.global_id = 'fake global id'
103 self['id'] = 'faker/than fake'
104-
105-
106-class SettingsIterMock:
107- """Mimic the weird libaccounts AgAccountSettingIter semantics.
108-
109- The default Python mapping of this object does not follow standard Python
110- iterator semantics.
111- """
112-
113- def __init__(self):
114- self.items = [(True, 'key', 'value')]
115-
116- def next(self):
117- if self.items:
118- return self.items.pop()
119- else:
120- return (False, None, None)
121
122=== modified file 'gwibber/gwibber/testing/mocks.py'
123--- gwibber/gwibber/testing/mocks.py 2012-09-14 20:29:11 +0000
124+++ gwibber/gwibber/testing/mocks.py 2012-09-21 03:03:20 +0000
125@@ -17,12 +17,20 @@
126 __all__ = [
127 'FakeData',
128 'FakeOpen',
129+ 'SettingsIterMock',
130+ 'TestLogger',
131 ]
132
133
134 import hashlib
135+import logging
136
137+from io import StringIO
138+from logging.handlers import QueueHandler
139 from pkg_resources import resource_string
140+from queue import Empty, Queue
141+
142+NEWLINE = '\n'
143
144
145 # A test double which shortens the given URL by returning the hex hash of the
146@@ -81,3 +89,79 @@
147 return FakeInfo()
148 self.call_count += 1
149 return FakeOpen(self._data, self._charset)
150+
151+
152+class SettingsIterMock:
153+ """Mimic the weird libaccounts AgAccountSettingIter semantics.
154+
155+ The default Python mapping of this object does not follow standard Python
156+ iterator semantics.
157+ """
158+
159+ def __init__(self):
160+ self.items = [(True, 'key', 'value')]
161+
162+ def next(self):
163+ if self.items:
164+ return self.items.pop()
165+ else:
166+ return (False, None, None)
167+
168+
169+class TestLogger:
170+ """A wrapper for capturing logging output in protocol classes.
171+
172+ This ensures that the standard gwibber.service log file isn't polluted by
173+ the tests, and that the logging output in a sub-thread can be tested in
174+ the main thread.
175+
176+ To use, instantiate one of these, then mock your class with a decorator
177+ such as:
178+
179+ @mock.patch.dict('gwibber.URLs.base.__dict__', {'log': my_logger.log})
180+
181+ Then in your test, you can retrieve all the logged output (and clear the
182+ logger instance for the next test) by doing:
183+
184+ my_logger.empty()
185+ """
186+ def __init__(self):
187+ self._queue = Queue()
188+ self.log = logging.getLogger('gwibber.test')
189+ self.log.addHandler(QueueHandler(self._queue))
190+ # Capture effectively everything.
191+ self.log.setLevel(logging.NOTSET)
192+ self.log.propagate = False
193+
194+ def empty(self, trim=True):
195+ """Return all the log messages written to this log.
196+
197+ :param trim: Trim exception text to just the first and last line, with
198+ ellipses in between. You will usually want to do this since the
199+ exception details will contain file tracebacks with paths specific
200+ to your testing environment.
201+ :type trim: bool
202+ """
203+ output = StringIO()
204+ while True:
205+ try:
206+ record = self._queue.get_nowait()
207+ except Empty:
208+ # The queue is exhausted.
209+ break
210+ # We have to print both the message, and explicitly the exc_text,
211+ # otherwise we won't see the exception traceback in the output.
212+ args = [record.getMessage()]
213+ if record.exc_text is None:
214+ # Nothing to include.
215+ pass
216+ elif trim:
217+ exc_lines = record.exc_text.splitlines()
218+ # Leave just the first and last lines, but put ellipses in
219+ # between.
220+ exc_lines[1:-1] = [' ...']
221+ args.append(NEWLINE.join(exc_lines))
222+ else:
223+ args.append(record.exc_text)
224+ print(*args, file=output)
225+ return output.getvalue()
226
227=== modified file 'gwibber/gwibber/tests/test_account.py'
228--- gwibber/gwibber/tests/test_account.py 2012-09-18 21:27:54 +0000
229+++ gwibber/gwibber/tests/test_account.py 2012-09-21 03:03:20 +0000
230@@ -24,9 +24,10 @@
231 import unittest
232
233 from gwibber.errors import UnsupportedProtocolError
234-from gwibber.testing.helpers import FakeAccount, SettingsIterMock
235+from gwibber.protocols.flickr import Flickr
236+from gwibber.testing.helpers import FakeAccount
237+from gwibber.testing.mocks import SettingsIterMock
238 from gwibber.utils.account import Account, AccountManager
239-from gwibber.protocols.flickr import Flickr
240
241 try:
242 # Python 3.3
243
244=== modified file 'gwibber/gwibber/tests/test_flickr.py'
245--- gwibber/gwibber/tests/test_flickr.py 2012-09-19 22:21:39 +0000
246+++ gwibber/gwibber/tests/test_flickr.py 2012-09-21 03:03:20 +0000
247@@ -22,8 +22,8 @@
248 import unittest
249
250 from gwibber.errors import AuthorizationError
251-from gwibber.testing.helpers import FakeAccount, LogPreserver
252-from gwibber.testing.mocks import FakeData
253+from gwibber.testing.helpers import FakeAccount
254+from gwibber.testing.mocks import FakeData, TestLogger
255 from gwibber.protocols.flickr import Flickr
256
257 try:
258@@ -33,31 +33,41 @@
259 import mock
260
261
262-# Disable i18n translations so we only need to test English responses.
263-@mock.patch.dict('gwibber.protocols.flickr.__dict__', {'_': lambda s: s})
264+test_logger = TestLogger()
265+
266+
267+# Ensure synchronicity between the main thread and the sub-thread. Also, set
268+# up the loggers for the modules-under-test so that we can assert their error
269+# messages.
270+@mock.patch.dict('gwibber.utils.base.__dict__',
271+ {'_SYNCHRONIZE': True,
272+ 'log': test_logger.log,
273+ })
274+@mock.patch.dict('gwibber.protocols.flickr.__dict__', {'log': test_logger.log})
275 class TestFlickr(unittest.TestCase):
276 """Test the Flickr API."""
277
278- @classmethod
279- def setUpClass(cls):
280- cls._log_state = LogPreserver()
281-
282- @classmethod
283- def tearDownClass(cls):
284- cls._log_state.restore()
285-
286 def setUp(self):
287 self.account = FakeAccount()
288 self.protocol = Flickr(self.account)
289
290+ def tearDown(self):
291+ # Ensure that any log entries we haven't tested just get consumed so
292+ # as to isolate out test logger from other tests.
293+ test_logger.empty()
294+
295 def test_failed_login(self):
296 # Force the Flickr login to fail.
297 with mock.patch.object(self.protocol, '_get_nsid', return_value=None):
298- with self.assertRaises(AuthorizationError) as cm:
299- self.protocol.images()
300- self.assertEqual(cm.exception.account, self.protocol.account)
301- self.assertEqual(cm.exception.message,
302- 'No Flickr user id available')
303+ self.protocol('receive')
304+ self.assertEqual(test_logger.empty(), """\
305+Flickr: No NSID available
306+Gwibber operation exception:
307+ Traceback (most recent call last):
308+ ...
309+gwibber.errors.AuthorizationError:\
310+ No Flickr user id available (account: faker/than fake)
311+""")
312
313 @mock.patch('gwibber.utils.download.urlopen',
314 FakeData('gwibber.tests.data', 'flickr-nophotos.dat'))
315
316=== modified file 'gwibber/gwibber/tests/test_foursquare.py'
317--- gwibber/gwibber/tests/test_foursquare.py 2012-09-20 16:26:53 +0000
318+++ gwibber/gwibber/tests/test_foursquare.py 2012-09-21 03:03:20 +0000
319@@ -23,8 +23,8 @@
320
321 from gi.repository import Dee
322
323-from gwibber.testing.helpers import FakeAccount, LogPreserver
324-from gwibber.testing.mocks import FakeData
325+from gwibber.testing.helpers import FakeAccount
326+from gwibber.testing.mocks import FakeData, TestLogger
327 from gwibber.protocols.foursquare import FourSquare
328 from gwibber.utils.model import COLUMN_TYPES
329
330@@ -40,22 +40,30 @@
331 TestModel = Dee.SharedModel.new('com.Gwibber.TestSharedModel')
332 TestModel.set_schema_full(COLUMN_TYPES)
333
334-
335+test_logger = TestLogger()
336+
337+
338+# Ensure synchronicity between the main thread and the sub-thread. Also, set
339+# up the loggers for the modules-under-test so that we can assert their error
340+# messages.
341+@mock.patch.dict('gwibber.utils.base.__dict__',
342+ {'_SYNCHRONIZE': True,
343+ 'log': test_logger.log,
344+ })
345+@mock.patch.dict('gwibber.protocols.foursquare.__dict__',
346+ {'log': test_logger.log})
347 class TestFourSquare(unittest.TestCase):
348 """Test the FourSquare API."""
349
350- @classmethod
351- def setUpClass(cls):
352- cls._log_state = LogPreserver()
353-
354- @classmethod
355- def tearDownClass(cls):
356- cls._log_state.restore()
357-
358 def setUp(self):
359 self.account = FakeAccount()
360 self.protocol = FourSquare(self.account)
361
362+ def tearDown(self):
363+ # Ensure that any log entries we haven't tested just get consumed so
364+ # as to isolate out test logger from other tests.
365+ test_logger.empty()
366+
367 def test_protocol_info(self):
368 self.assertEqual(self.protocol.__class__.__name__, 'FourSquare')
369
370
371=== modified file 'gwibber/gwibber/tests/test_twitter.py'
372--- gwibber/gwibber/tests/test_twitter.py 2012-09-12 22:15:43 +0000
373+++ gwibber/gwibber/tests/test_twitter.py 2012-09-21 03:03:20 +0000
374@@ -22,7 +22,8 @@
375 import unittest
376
377 from gwibber.protocols.twitter import Twitter
378-from gwibber.testing.helpers import FakeAccount, LogPreserver
379+from gwibber.testing.helpers import FakeAccount
380+from gwibber.testing.mocks import TestLogger
381
382
383 try:
384@@ -32,19 +33,20 @@
385 import mock
386
387
388-# Disable i18n translations so we only need to test English responses.
389-@mock.patch.dict('gwibber.protocols.twitter.__dict__', {'_': lambda s: s})
390+test_logger = TestLogger()
391+
392+
393+# Ensure synchronicity between the main thread and the sub-thread. Also, set
394+# up the loggers for the modules-under-test so that we can assert their error
395+# messages.
396+@mock.patch.dict('gwibber.utils.base.__dict__',
397+ {'_SYNCHRONIZE': True,
398+ 'log': test_logger.log,
399+ })
400+@mock.patch.dict('gwibber.protocols.flickr.__dict__', {'log': test_logger.log})
401 class TestTwitter(unittest.TestCase):
402 """Test the Twitter API."""
403
404- @classmethod
405- def setUpClass(cls):
406- cls._log_state = LogPreserver()
407-
408- @classmethod
409- def tearDownClass(cls):
410- cls._log_state.restore()
411-
412 def setUp(self):
413 self.account = FakeAccount()
414 self.protocol = Twitter(self.account)
415
416=== modified file 'gwibber/gwibber/utils/base.py'
417--- gwibber/gwibber/utils/base.py 2012-09-20 16:26:53 +0000
418+++ gwibber/gwibber/utils/base.py 2012-09-21 03:03:20 +0000
419@@ -22,12 +22,19 @@
420
421 import re
422 import string
423-
424-from threading import Lock, Thread
425+import logging
426+import threading
427
428 from gwibber.utils.model import COLUMN_INDICES, SCHEMA, DEFAULTS, Model
429
430
431+# When the system is under test, mock this value to true to enable
432+# synchronizing the operations threads with the main thread. In this way, you
433+# can ensure that the results of calling an operation on a protocol will
434+# complete before the assertions testing the results of that operation.
435+_SYNCHRONIZE = False
436+
437+
438 IGNORED = string.punctuation + string.whitespace
439 SCHEME_RE = re.compile('http[s]?://|gwibber:/', re.IGNORECASE)
440 EMPTY_STRING = ''
441@@ -43,7 +50,9 @@
442
443 # Protocol __call__() methods run in threads, so we need to serialize
444 # publishing new data into the SharedModel.
445-_publish_lock = Lock()
446+_publish_lock = threading.Lock()
447+
448+log = logging.getLogger('gwibber.service')
449
450
451 def _make_key(row):
452@@ -75,28 +84,59 @@
453 return EMPTY_STRING.join(char for char in key if char not in IGNORED)
454
455
456+class _OperationThread(threading.Thread):
457+ """Catch, log, and swallow all exceptions in the sub-thread."""
458+
459+ def __init__(self, barrier, *args, **kws):
460+ # The barrier will only be provided when the system is under test.
461+ self._barrier = barrier
462+ super().__init__(*args, **kws)
463+
464+ # Always run these as daemon threads, so they don't block the main thread,
465+ # i.e. gwibber-service, from exiting.
466+ daemon = True
467+
468+ def run(self):
469+ try:
470+ super().run()
471+ except Exception:
472+ log.exception('Gwibber operation exception:\n')
473+ # If the system is under test, indicate that we've reached the
474+ # barrier, so that the main thread, i.e. the test thread waiting for
475+ # the results, can then proceed.
476+ if self._barrier is not None:
477+ self._barrier.wait()
478+
479+
480 class Base:
481 def __init__(self, account):
482 self.account = account
483
484 def __call__(self, operation, *args, **kwargs):
485- """Call an operation, i.e. a method, with arguments.
486+ """Call an operation, i.e. a method, with arguments in a sub-thread.
487
488- The operation is called in a thread so as not to block the main
489- gwibber-service thread. Note that there is no expectation, and no
490- race-condition safe way, of returning a result from the operation.
491+ Sub-threads do not currently communicate any state to the main thread,
492+ and any exception that occurs in the sub-thread is simply logged and
493+ discarded.
494 """
495 if operation.startswith('_') or not hasattr(self, operation):
496 raise NotImplementedError(operation)
497-
498 method = getattr(self, operation)
499-
500- # Run the operation in a thread, but make sure it's a daemon thread so
501- # that the main thread (i.e. gwibber-service) can exit even if this
502- # thread has not yet completed.
503- thread = Thread(target=method, args=args, kwargs=kwargs)
504- thread.daemon = True
505- thread.start()
506+ # When the system is under test, or at least tests which assert the
507+ # results of operations in a sub-thread, then this flag will be set to
508+ # true, in which case we want to pass a barrier to the sub-thread.
509+ # The sub-thread will complete, either successfully or unsuccessfully,
510+ # but in either case, it will always indicate that it's reached the
511+ # barrier before exiting. The main thread, i.e. this one, will not
512+ # proceed until that barrier has been reached, thus allowing the main
513+ # thread to assert the results of the sub-thread.
514+ barrier = (threading.Barrier(parties=2) if _SYNCHRONIZE else None)
515+ _OperationThread(barrier,
516+ target=method, args=args, kwargs=kwargs).start()
517+ # When under synchronous testing, wait until the sub-thread completes
518+ # before returning.
519+ if barrier is not None:
520+ barrier.wait()
521
522 def _publish(self, account_id, message_id, **kwargs):
523 """Publish fresh data into the model, ignoring duplicates.
524
525=== removed directory 'gwibber/microblog/plugins/foursquare'
526=== removed file 'gwibber/microblog/plugins/foursquare/__init__.py'
527--- gwibber/microblog/plugins/foursquare/__init__.py 2012-06-26 19:24:15 +0000
528+++ gwibber/microblog/plugins/foursquare/__init__.py 1970-01-01 00:00:00 +0000
529@@ -1,334 +0,0 @@
530-from gwibber.microblog import network, util
531-from gwibber.microblog.util import resources
532-from gwibber.microblog.util.auth import Authentication
533-
534-import json, re
535-from gettext import lgettext as _
536-
537-import logging
538-logger = logging.getLogger("FourSquare")
539-logger.debug("Initializing.")
540-
541-import json, urllib2
542-
543-PROTOCOL_INFO = {
544- "name": "Foursquare",
545- "version": "5.0",
546-
547- "config": [
548- "private:secret_token",
549- "access_token",
550- "receive_enabled",
551- "username",
552- "color",
553- ],
554-
555- "authtype": "oauth2",
556- "color": "#990099",
557-
558- "features": [
559- "receive",
560- ],
561-
562- "default_streams": [
563- "receive",
564- ],
565-}
566-
567-URL_PREFIX = "https://api.foursquare.com/v2"
568-
569-class Client:
570- """Query Foursquare and converts data.
571-
572- The Client class is responsible for querying Foursquare and turning the data obtained
573- into data that Gwibber can understand. Foursquare uses a version of OAuth for security.
574-
575- Tokens have already been obtained when the account was set up in Gwibber and are used to
576- authenticate when getting data.
577-
578- """
579- def __init__(self, acct):
580- self.service = util.getbus("Service")
581- self.account = acct
582- self._loop = None
583-
584- def _retrieve_user_id(self):
585- #Make a request with our new token for the user's own data
586- url = "https://api.foursquare.com/v2/users/self?oauth_token=" + self.account["access_token"]
587- data = json.load(urllib2.urlopen(url))
588- fullname = ""
589- if isinstance(data, dict):
590- if data["response"]["user"].has_key("firstName"):
591- fullname += data["response"]["user"]["firstName"] + " "
592- if data["response"]["user"].has_key("lastName"):
593- fullname += data["response"]["user"]["lastName"]
594- self.account["username"] = fullname.encode("utf-8")
595- self.account["user_id"] = data["response"]["user"]["id"]
596- self.account["uid"] = self.account["user_id"]
597- else:
598- logger.error("Couldn't get user ID")
599-
600- def _login(self):
601- old_token = self.account.get("access_token", None)
602- with self.account.login_lock:
603- # Perform the login only if it wasn't already performed by another thread
604- # while we were waiting to the lock
605- if self.account.get("access_token", None) == old_token:
606- self._locked_login(old_token)
607-
608- return "access_token" in self.account and \
609- self.account["access_token"] != old_token
610-
611- def _locked_login(self, old_token):
612- logger.debug("Re-authenticating" if old_token else "Logging in")
613-
614- auth = Authentication(self.account, logger)
615- reply = auth.login()
616- if reply and reply.has_key("AccessToken"):
617- self.account["access_token"] = reply["AccessToken"]
618- self._retrieve_user_id()
619- logger.debug("User id is: %s" % self.account["uid"])
620- else:
621- logger.error("Didn't find token in session: %s", (reply,))
622-
623- def _message(self, data):
624- """Parses messages into Gwibber compatible forms.
625-
626- Arguments:
627- data -- A data object obtained from Foursquare containing a complete checkin
628-
629- Returns:
630- m -- A data object compatible with inserting into the Gwibber database for that checkin
631-
632- """
633- m = {};
634- m["mid"] = str(data["id"])
635- m["service"] = "foursquare"
636- m["account"] = self.account["id"]
637- m["time"] = data["createdAt"]
638-
639- shouttext = ""
640- text = ""
641-
642- if data.has_key("shout"):
643- shout = data["shout"]
644- shout = shout.replace('& ', '& ')
645-
646- if data.has_key("venue"):
647- venuename = data["venue"]["name"]
648- venuename = venuename.replace('& ', '& ')
649-
650- if data.has_key("shout"):
651- shouttext += shout + "\n\n"
652- text += shout + "\n"
653- if data["venue"].has_key("id"):
654- m["url"] = "https://foursquare.com/venue/%s" % data["venue"]["id"]
655- else:
656- m["url"] = "https://foursquare.com"
657- shouttext += "Checked in at <a href='" + m["url"] + "'>" + venuename + "</a>"
658- text += "Checked in at " + venuename
659- if data["venue"]["location"].has_key("address"):
660- shouttext += ", " + data["venue"]["location"]["address"]
661- text += ", " + data["venue"]["location"]["address"]
662- if data["venue"]["location"].has_key("crossstreet"):
663- shouttext += " and " + data["venue"]["location"]["crossstreet"]
664- text += " and " + data["venue"]["location"]["crossstreet"]
665- if data["venue"]["location"].has_key("city"):
666- shouttext += ", " + data["venue"]["location"]["city"]
667- text += ", " + data["venue"]["location"]["city"]
668- if data["venue"]["location"].has_key("state"):
669- shouttext += ", " + data["venue"]["location"]["state"]
670- text += ", " + data["venue"]["location"]["state"]
671- if data.has_key("event"):
672- if data["event"].has_key("name"):
673- shouttext += " for " + data["event"]["name"]
674- else:
675- if data.has_key("shout"):
676- shouttext += shout + "\n\n"
677- text += shout + "\n"
678- else:
679- text= "Checked in off the grid"
680- shouttext= "Checked in off the grid"
681-
682- m["text"] = text
683- m["content"] = shouttext
684- m["html"] = shouttext
685-
686- m["sender"] = {}
687- m["sender"]["id"] = data["user"]["id"]
688- m["sender"]["image"] = data["user"]["photo"]
689- m["sender"]["url"] = "https://www.foursquare.com/user/" + data["user"]["id"]
690- if data["user"]["relationship"] == "self":
691- m["sender"]["is_me"] = True
692- else:
693- m["sender"]["is_me"] = False
694- fullname = ""
695- if data["user"].has_key("firstName"):
696- fullname += data["user"]["firstName"] + " "
697- if data["user"].has_key("lastName"):
698- fullname += data["user"]["lastName"]
699-
700- if data.has_key("photos"):
701- if data["photos"]["count"] > 0:
702- m["photo"] = {}
703- m["photo"]["url"] = ""
704- m["photo"]["picture"] = data["photos"]["items"][0]["url"]
705- m["photo"]["name"] = ""
706- m["type"] = "photo"
707-
708- if data.has_key("likes"):
709- if data["likes"]["count"] > 0:
710- m["likes"]["count"] = data["likes"]["count"]
711-
712- m["sender"]["name"] = fullname
713- m["sender"]["nick"] = fullname
714-
715- if data.has_key("source"):
716- m["source"] = "<a href='" + data["source"]["url"] + "'>" + data["source"]["name"] + "</a>"
717- else:
718- m["source"] = "<a href='https://foursquare.com/'>Foursquare</a>"
719-
720- if data.has_key("comments"):
721- if data["comments"]["count"] > 0:
722-
723- m["comments"] = []
724- comments = self._get_comments(data["id"])
725- for comment in comments:
726- # Get the commenter's name
727- fullname = ""
728- if comment["user"].has_key("firstName"):
729- fullname += comment["user"]["firstName"] + " "
730- if comment["user"].has_key("lastName"):
731- fullname += comment["user"]["lastName"]
732-
733- # Create a sender
734- sender = {
735- "name" : fullname,
736- "id" : comment["user"]["id"],
737- "is_me": False,
738- "image": comment["user"]["photo"],
739- "url" : "https://www.foursquare.com/user/" + comment["user"]["id"]
740- }
741- # Create a comment
742- m["comments"].append({
743- "text" : comment["text"],
744- "time" : comment["createdAt"],
745- "sender": sender,
746- })
747-
748- return m
749-
750- def _check_error(self, data):
751- """Checks to ensure the data obtained by Foursquare is in the correct form.
752-
753- If it's not in the correct form, an error is logged in gwibber.log
754-
755- Arguments:
756- data -- A data structure obtained from Foursquare
757-
758- Returns:
759- True if data is valid (is a dictionary and contains a 'recent' parameter).
760- If the data is not valid, then return False.
761-
762- """
763- if isinstance(data, dict) and "recent" in data:
764- return True
765- else:
766- logger.error("Foursquare error %s", data)
767- return False
768-
769- def _get_comments(self, checkin_id):
770- """Gets comments on a particular check in ID.
771-
772- Arguments:
773- checkin_id -- The checkin id of the checkin
774-
775- Returns:
776- A comment object
777-
778- """
779- url = "/".join((URL_PREFIX, "checkins", checkin_id))
780- url = url + "?oauth_token=" + self.token
781- data = network.Download(url, None, False).get_json()["response"]
782- return data["checkin"]["comments"]["items"]
783-
784- def _get(self, path, parse="message", post=False, single=False, **args):
785- """Establishes a connection with Foursquare and gets the data requested.
786-
787- Arguments:
788- path -- The end of the URL to look up on Foursquare
789- parse -- The function to use to parse the data returned (message by default)
790- post -- True if using POST, for example the send operation. False if using GET, most operations other than send. (False by default)
791- single -- True if a single checkin is requested, False if multiple (False by default)
792- **args -- Arguments to be added to the URL when accessed
793-
794- Returns:
795- A list of Gwibber compatible objects which have been parsed by the parse function.
796-
797- """
798- if "access_token" not in self.account and not self._login():
799- logstr = """%s: %s - %s""" % (PROTOCOL_INFO["name"], _("Authentication failed"), "Auth needs updating")
800- logger.error("%s", logstr)
801- return [{"error": {"type": "auth", "account": self.account, "message": _("Authentication failed, please re-authorize")}}]
802-
803- url = "/".join((URL_PREFIX, path))
804-
805- url = url + "?oauth_token=" + self.account["access_token"]
806-
807- data = network.Download(url, None, post).get_json()["response"]
808-
809- resources.dump(self.account["service"], self.account["id"], data)
810-
811- if isinstance(data, dict) and data.get("errors", 0):
812- if "authenticate" in data["errors"][0]["message"]:
813- # Try again, if we get a new token
814- if self._login():
815- logger.debug("Authentication error, logging in again")
816- return self._get(path, parse, post, single, args)
817- else:
818- logstr = """%s: %s - %s""" % (PROTOCOL_INFO["name"], _("Authentication failed"), data["errors"][0]["message"])
819- logger.error("%s", logstr)
820- return [{"error": {"type": "auth", "account": self.account, "message": data["errors"][0]["message"]}}]
821- else:
822- for error in data["errors"]:
823- logstr = """%s: %s - %s""" % (PROTOCOL_INFO["name"], _("Unknown failure"), error["message"])
824- return [{"error": {"type": "unknown", "account": self.account, "message": error["message"]}}]
825- elif isinstance(data, dict) and data.get("error", 0):
826- if "Incorrect signature" in data["error"]:
827- # Try again, if we get a new token
828- if self._login():
829- logger.debug("Authentication error, logging in again")
830- return self._get(path, parse, post, single, args)
831- else:
832- logstr = """%s: %s - %s""" % (PROTOCOL_INFO["name"], _("Request failed"), data["error"])
833- logger.error("%s", logstr)
834- return [{"error": {"type": "auth", "account": self.account, "message": data["error"]}}]
835- elif isinstance(data, str):
836- logstr = """%s: %s - %s""" % (PROTOCOL_INFO["name"], _("Request failed"), data)
837- logger.error("%s", logstr)
838- return [{"error": {"type": "request", "account": self.account, "message": data}}]
839- if not self._check_error(data):
840- return []
841-
842- checkins = data["recent"]
843- if single: return [getattr(self, "_%s" % parse)(checkins)]
844- if parse: return [getattr(self, "_%s" % parse)(m) for m in checkins]
845- else: return []
846-
847- def __call__(self, opname, **args):
848- return getattr(self, opname)(**args)
849-
850- def receive(self):
851- """Gets a list of each friend's most recent check-ins.
852-
853- The list of each friend's recent check-ins is then
854- saved to the database.
855-
856- Arguments:
857- None
858-
859- Returns:
860- A list of Gwibber compatible objects which have been parsed by the parse function.
861-
862- """
863- return self._get("checkins/recent", v=20120612)

Subscribers

People subscribed via source and target branches

to all changes: