Merge lp:~marco-gallotta/ibid/contest into lp:~ibid-core/ibid/old-trunk-1.6
- contest
- Merge into old-trunk-1.6
Status: | Superseded | ||||
---|---|---|---|---|---|
Proposed branch: | lp:~marco-gallotta/ibid/contest | ||||
Merge into: | lp:~ibid-core/ibid/old-trunk-1.6 | ||||
Diff against target: |
276 lines (+262/-0) 2 files modified
ibid/plugins/contest.py (+259/-0) scripts/ibid-plugin (+3/-0) |
||||
To merge this branch: | bzr merge lp:~marco-gallotta/ibid/contest | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Stefano Rivera | Needs Fixing | ||
Review via email: mp+17469@code.launchpad.net |
This proposal has been superseded by a proposal from 2010-01-19.
Commit message
Description of the change
marcog (marco-gallotta) wrote : | # |
Stefano Rivera (stefanor) wrote : | # |
Run pyflakes. Otherwise you are simply asking me to do it for you:
ibid/plugins/
ibid/plugins/
ibid/plugins/
ibid/plugins/
ibid/plugins/
The name "contest" is too board. It should be called programming_
Also, it's really esoteric, so it should have autoload = False on all the Processors.
> priority = -20
why? Put a comment in explaining please.
> urlencode({u'NAME': user, u'PASSWORD': password})
URLencode doesn't encode to utf-8, so utf-8 encode the parameters yourself, and give it bytestrings.
Applies to all your urlencodes()
> if font.text and font.text.
More pythonic:
if font.text and u'Please try again' in font.text:
This crops up again in: division = [b.text for b in etree.getiterat
> class UsacoException(
> pass
I believe that if you are creating your own exception to carry an exception message it should do so and not rely on Exception, as Python 3's Exception doesn't have a message. We don't support Python 3 yet, but it's good practice.
> .filter(and_(
> Identity.identity == user,
> Identity.source == event.source)) \
You can save a line by doing:
.filter(
.filter(
> usaco_account = [attr.value for attr in account.attributes if attr.name == 'usaco_account']
If you are going to do that, you do want to eager load attributes.
> __redacted__
I'd prefer [redacted]. It looks nicer :)
> def setup(self):
> pass
Be aware that if lp:~stefanor/ibid/ibid-plugin-507489 gets in first, it'll clash.
- 851. By marcog
-
Remove unused imports
- 852. By marcog
-
Handle usaco user not found
- 853. By marcog
-
Collection of minor changes: autoload=False, utf-8 encode params and other less important changes
- 854. By marcog
-
Make UsacoException have a msg so that it works in Python3
- 855. By marcog
-
Eagerload attributes and shave a line with filters
- 856. By marcog
-
__redacted__ -> [redacted], to make stefano happy :)
marcog (marco-gallotta) wrote : | # |
> The name "contest" is too board. It should be called programming_
> anything.
We still need to agree on a name. I agree that contest is too general. I think programming_contest is too long (20 characters and the longest current name is 12); if we do go with this, I think we should drop the _ to follow the style of other plugins (gameservers, buildbot). Vhata suggested progcomp, but as he said that's ugly. There's also the possibility that maths contests are added: what then?
> Run pyflakes. Otherwise you are simply asking me to do it for you:
>
> ibid/plugins/
> ibid/plugins/
> ibid/plugins/
> ibid/plugins/
> ibid/plugins/
> used
>
> Also, it's really esoteric, so it should have autoload = False on all the
> Processors.
>
> > priority = -20
> why? Put a comment in explaining please.
>
> > urlencode({u'NAME': user, u'PASSWORD': password})
> URLencode doesn't encode to utf-8, so utf-8 encode the parameters yourself,
> and give it bytestrings.
>
> Applies to all your urlencodes()
>
> > if font.text and font.text.
> More pythonic:
> if font.text and u'Please try again' in font.text:
>
> This crops up again in: division = [b.text for b in etree.getiterat
> b.text and b.text.
>
> > class UsacoException(
> > pass
>
> I believe that if you are creating your own exception to carry an exception
> message it should do so and not rely on Exception, as Python 3's Exception
> doesn't have a message. We don't support Python 3 yet, but it's good practice.
> > .filter(and_(
> > Identity.identity == user,
> > Identity.source == event.source)) \
>
> You can save a line by doing:
> .filter(
> .filter(
>
> > usaco_account = [attr.value for attr in account.attributes if attr.name ==
> 'usaco_account']
>
> If you are going to do that, you do want to eager load attributes.
>
> > __redacted__
> I'd prefer [redacted]. It looks nicer :)
Done
> > def setup(self):
> > pass
>
> Be aware that if lp:~stefanor/ibid/ibid-plugin-507489 gets in first, it'll
> clash.
Hopefully someone notices :)
Stefano Rivera (stefanor) wrote : | # |
ibid/plugins/
And the name needs to be sorted. But otherwise I approve
Stefano Rivera (stefanor) wrote : | # |
Can you add a copyright header to the top (see any other file in current trunk)
marcog (marco-gallotta) wrote : | # |
> Can you add a copyright header to the top (see any other file in current
> trunk)
Done
- 862. By marcog
-
Copyright header
- 863. By marcog
-
Search for usaco results also by name and usaco account when not linked
- 864. By marcog
-
Add timeout for fetching USACO results and suggest that the site might be down
- 865. By marcog
-
Forbid giving passwords in public
- 866. By marcog
-
Allow admins to link usaco accounts without a usaco password
- 867. By marcog
-
Tell the user when usaco site appears dead
- 868. By marcog
-
Change from looking up Identities to Accounts table to find users not on the current source
- 869. By marcog
-
Improve the 'you need an account' msg
- 870. By marcog
-
Put division inline if only one person is listed in usaco results
- 871. By marcog
-
Correct the test for 'not a teacher account'
- 872. By marcog
-
Rename contest-
>codecontest - 873. By marcog
-
Small tidy-up
Unmerged revisions
Preview Diff
1 | === added file 'ibid/plugins/contest.py' |
2 | --- ibid/plugins/contest.py 1970-01-01 00:00:00 +0000 |
3 | +++ ibid/plugins/contest.py 2010-01-19 13:45:24 +0000 |
4 | @@ -0,0 +1,259 @@ |
5 | +# Copyright (c) 2010, Marco Gallotta |
6 | +# Released under terms of the MIT/X/Expat Licence. See COPYING for details. |
7 | + |
8 | +import re |
9 | +from urllib2 import HTTPError |
10 | + |
11 | +from urllib import urlencode |
12 | + |
13 | +from ibid.config import Option |
14 | +from ibid.db import eagerload |
15 | +from ibid.db.models import Account, Attribute, Identity |
16 | +from ibid.plugins import Processor, match |
17 | +from ibid.utils import cacheable_download |
18 | +from ibid.utils.html import get_html_parse_tree |
19 | + |
20 | +help = {u'usaco': u'Query USACO sections, divisions and more. Since this info is private, users are required to provide their USACO password when linking their USACO account to their ibid account and only linked accounts can be queried. Your password is used only to confirm that the account is yours and is discarded immediately.'} |
21 | + |
22 | +class UsacoException(Exception): |
23 | + def __init__(self, msg): |
24 | + self.msg = msg |
25 | + |
26 | + def __unicode__(self): |
27 | + return unicode(self.msg) |
28 | + |
29 | +class Usaco(Processor): |
30 | + """usaco <section|division> for <user> |
31 | + usaco <contest> results [for <user>] |
32 | + i am <usaco_username> on usaco password <usaco_password>""" |
33 | + |
34 | + admin_user = Option('admin_user', 'Admin user on USACO', None) |
35 | + admin_password = Option('admin_password', 'Admin password on USACO', None) |
36 | + |
37 | + feature = 'usaco' |
38 | + # Clashes with identity, so lower our priority since if we match, then |
39 | + # this is the better match |
40 | + priority = -20 |
41 | + autoload = False |
42 | + |
43 | + def _login(self, user, password): |
44 | + params = urlencode({'NAME': user.encode('utf-8'), 'PASSWORD': password.encode('utf-8')}) |
45 | + etree = get_html_parse_tree(u'http://ace.delos.com/usacogate', data=params, treetype=u'etree') |
46 | + for font in etree.getiterator(u'font'): |
47 | + if font.text and u'Please try again' in font.text: |
48 | + return None |
49 | + return etree |
50 | + |
51 | + def _check_login(self, user, password): |
52 | + return self._login(user, password) is not None |
53 | + |
54 | + def _get_section(self, monitor_url, usaco_user, user): |
55 | + etree = get_html_parse_tree(monitor_url, treetype=u'etree') |
56 | + usaco_user = usaco_user.lower() |
57 | + header = True |
58 | + for tr in etree.getiterator(u'tr'): |
59 | + if header: |
60 | + header = False |
61 | + continue |
62 | + tds = [t.text for t in tr.getiterator(u'td')] |
63 | + section = u'is on section %s' % tds[5] |
64 | + if tds[5] == u'DONE': |
65 | + section = u'has completed USACO training' |
66 | + if tds[0] and tds[0].lower() == usaco_user: |
67 | + return u'%(user)s (%(usaco_user)s on USACO) %(section)s and last logged in %(days)s ago' % { |
68 | + 'user': user, |
69 | + 'usaco_user': usaco_user, |
70 | + 'days': tds[3], |
71 | + 'section': section, |
72 | + } |
73 | + |
74 | + return None |
75 | + |
76 | + def _add_user(self, monitor_url, user): |
77 | + matches = re.search(r'a=(.+)&', monitor_url) |
78 | + auth = matches.group(1) |
79 | + params = urlencode({'STUDENTID': user.encode('utf-8'), 'ADD': 'ADD STUDENT', |
80 | + 'a': auth.encode('utf-8'), 'monitor': '1'}) |
81 | + etree = get_html_parse_tree(monitor_url, treetype=u'etree', data=params) |
82 | + for font in etree.getiterator(u'font'): |
83 | + if font.text and u'No STATUS file for' in font.text: |
84 | + raise UsacoException(u'Sorry, user %s not found' % user) |
85 | + |
86 | + def _get_monitor_url(self): |
87 | + if self.admin_user is None or self.admin_password is None: |
88 | + raise UsacoException(u'Sorry, you need to configure a USACO admin account') |
89 | + return |
90 | + etree = self._login(self.admin_user, self.admin_password) |
91 | + if etree is None: |
92 | + raise UsacoException(u'Sorry, the configured USACO admin account is invalid') |
93 | + |
94 | + urls = [a.get(u'href') for a in etree.getiterator(u'a')] |
95 | + monitor_url = [url for url in urls if u'monitor' in url][0] |
96 | + if len(monitor_url) == 0: |
97 | + raise UsacoException(u'USACO admin account does not have teacher status') |
98 | + |
99 | + return monitor_url |
100 | + |
101 | + def _get_usaco_user(self, event, user): |
102 | + account = event.session.query(Account) \ |
103 | + .options(eagerload('attributes')) \ |
104 | + .filter(Account.username == user) \ |
105 | + .first() |
106 | + if account is None: |
107 | + account = event.session.query(Account) \ |
108 | + .options(eagerload('attributes')) \ |
109 | + .join('identities') \ |
110 | + .filter(Identity.identity == user) \ |
111 | + .filter(Identity.source == event.source) \ |
112 | + .first() |
113 | + if account is None: |
114 | + raise UsacoException(u'Sorry, %s has not been linked to a USACO account yet' % user) |
115 | + |
116 | + usaco_account = [attr.value for attr in account.attributes if attr.name == 'usaco_account'] |
117 | + if len(usaco_account) == 0: |
118 | + raise UsacoException(u'Sorry, %s has not been linked to a USACO account yet' % user) |
119 | + return usaco_account[0] |
120 | + |
121 | + def _get_usaco_users(self, event): |
122 | + accounts = event.session.query(Identity) \ |
123 | + .join(['account', 'attributes']) \ |
124 | + .add_entity(Attribute) \ |
125 | + .filter(Attribute.name == u'usaco_account') \ |
126 | + .filter(Identity.source == event.source) \ |
127 | + .all() |
128 | + |
129 | + users = {} |
130 | + for a in accounts: |
131 | + users[a[1].value] = a[0].identity |
132 | + return users |
133 | + |
134 | + @match(r'^usaco\s+section\s+(?:for\s+)?(.+)$') |
135 | + def get_section(self, event, user): |
136 | + try: |
137 | + usaco_user = self._get_usaco_user(event, user) |
138 | + monitor_url = self._get_monitor_url() |
139 | + except UsacoException, e: |
140 | + event.addresponse(e) |
141 | + return |
142 | + |
143 | + section = self._get_section(monitor_url, usaco_user, user) |
144 | + if section: |
145 | + event.addresponse(section) |
146 | + return |
147 | + |
148 | + try: |
149 | + self._add_user(monitor_url, user) |
150 | + except UsacoException, e: |
151 | + event.addresponse(e) |
152 | + return |
153 | + |
154 | + event.addresponse(self._get_section(monitor_url, usaco_user, user)) |
155 | + |
156 | + @match(r'^usaco\s+division\s+(?:for\s+)?(.+)$') |
157 | + def get_division(self, event, user): |
158 | + try: |
159 | + usaco_user = self._get_usaco_user(event, user) |
160 | + except UsacoException, e: |
161 | + event.addresponse(e) |
162 | + return |
163 | + |
164 | + params = urlencode({'id': usaco_user.encode('utf-8'), 'search': 'SEARCH'}) |
165 | + etree = get_html_parse_tree(u'http://ace.delos.com/showdiv', data=params, treetype=u'etree') |
166 | + division = [b.text for b in etree.getiterator(u'b') if b.text and usaco_user in b.text][0] |
167 | + if division.find(u'would compete') != -1: |
168 | + event.addresponse(u'%(user)s (%(usaco_user)s on USACO) has not competed in a USACO before', |
169 | + {u'user': user, u'usaco_user': usaco_user}) |
170 | + matches = re.search(r'(\w+) Division', division) |
171 | + division = matches.group(1).lower() |
172 | + event.addresponse(u'%(user)s (%(usaco_user)s on USACO) is in the %(division)s division', |
173 | + {u'user': user, u'usaco_user': usaco_user, u'division': division}) |
174 | + |
175 | + def _redact(self, event, term): |
176 | + for type in ['raw', 'deaddressed', 'clean', 'stripped']: |
177 | + event['message'][type] = re.sub(r'(.*)(%s)' % re.escape(term), r'\1[redacted]', event['message'][type]) |
178 | + |
179 | + @match(r'^i\s+am\s+(\S+)\s+on\s+usaco\s+password\s+(\S+)$') |
180 | + def usaco_account(self, event, user, password): |
181 | + self._redact(event, password) |
182 | + |
183 | + if not self._check_login(user, password): |
184 | + event.addresponse(u'Sorry, that account is invalid') |
185 | + return |
186 | + if not event.account: |
187 | + event.addresponse(u'Sorry, you need to create an account first') |
188 | + return |
189 | + |
190 | + try: |
191 | + monitor_url = self._get_monitor_url() |
192 | + except UsacoException, e: |
193 | + event.addresponse(e) |
194 | + return |
195 | + |
196 | + self._add_user(monitor_url, user) |
197 | + |
198 | + account = event.session.query(Account).get(event.account) |
199 | + usaco_account = [attr for attr in account.attributes if attr.name == u'usaco_account'] |
200 | + if usaco_account: |
201 | + usaco_account[0].value = user |
202 | + else: |
203 | + account.attributes.append(Attribute('usaco_account', user)) |
204 | + event.session.save_or_update(account) |
205 | + event.session.commit() |
206 | + |
207 | + event.addresponse(u'Done') |
208 | + |
209 | + @match(r'^usaco\s+(\S+)\s+results(?:\s+for\s+(\S+))?$') |
210 | + def usaco_results(self, event, contest, user): |
211 | + if user is not None: |
212 | + try: |
213 | + usaco_user = self._get_usaco_user(event, user) |
214 | + except UsacoException, e: |
215 | + event.addresponse(e) |
216 | + return |
217 | + |
218 | + url = u'http://ace.delos.com/%sresults' % contest.upper() |
219 | + try: |
220 | + filename = cacheable_download(url, u'usaco/results_%s' % contest.upper()) |
221 | + except HTTPError: |
222 | + event.addresponse(u"Sorry, the results for %s aren't released yet", contest) |
223 | + |
224 | + if user is not None: |
225 | + users = {usaco_user: user} |
226 | + else: |
227 | + users = self._get_usaco_users(event) |
228 | + |
229 | + text = open(filename, 'r').read().decode('ISO-8859-2') |
230 | + divisions = [u'gold', u'silver', u'bronze'] |
231 | + results = [[], [], []] |
232 | + division = None |
233 | + for line in text.splitlines(): |
234 | + for index, d in enumerate(divisions): |
235 | + if d in line.lower(): |
236 | + division = index |
237 | + # Example results line: |
238 | + # 2010 POL Jakub Pachocki meret1 ***** ***** 270 ***** ***** * 396 ***** ***** ** 324 1000 |
239 | + matches = re.match(r'^\s*(\d{4})\s+([A-Z]{3})\s+(.+?)\s+(\S+\d)\s+([\*xts\.e0-9 ]+?)\s+(\d+)\s*$', line) |
240 | + if matches: |
241 | + year = matches.group(1) |
242 | + country = matches.group(2) |
243 | + name = matches.group(3) |
244 | + usaco_user = matches.group(4) |
245 | + scores = matches.group(5) |
246 | + total = matches.group(6) |
247 | + if usaco_user in users.keys(): |
248 | + results[division].append((year, country, name, usaco_user, scores, total)) |
249 | + |
250 | + response = [] |
251 | + for i, division in enumerate(divisions): |
252 | + if results[i]: |
253 | + response.append(u'%s division results:' % division.title()) |
254 | + for result in results[i]: |
255 | + response.append(u'%(user)s (%(usaco_user)s on USACO) scored %(total)s (%(scores)s)' % { |
256 | + u'user': users[result[3]], |
257 | + u'usaco_user': result[3], |
258 | + u'total': result[5], |
259 | + u'scores': result[4], |
260 | + }) |
261 | + event.addresponse(u'\n'.join(response), conflate=False) |
262 | + |
263 | +# vi: set et sta sw=4 ts=4: |
264 | |
265 | === modified file 'scripts/ibid-plugin' |
266 | --- scripts/ibid-plugin 2010-01-18 22:13:23 +0000 |
267 | +++ scripts/ibid-plugin 2010-01-19 13:45:25 +0000 |
268 | @@ -65,6 +65,9 @@ |
269 | permissions = [] |
270 | supports = ('action', 'multiline', 'notice') |
271 | |
272 | + def setup(self): |
273 | + pass |
274 | + |
275 | def logging_name(self, name): |
276 | return name |
277 |
First iteration of the contest plugin, with USACO scriping pretty much done.