GNU Mailman

Merge lp:~danci-emanuel/mailman/dlist_runner into lp:~danci-emanuel/mailman/mm_3.0b2

  1. dlist_runner
  2. Merge into mm_3.0b2
Proposed by Emanuel Danci
Status: Needs review
Proposed branch: lp:~danci-emanuel/mailman/dlist_runner
Merge into: lp:~danci-emanuel/mailman/mm_3.0b2
Diff against target: 382 lines (+157/-9)
16 files modified
src/mailman/config/config.py (+1/-0)
src/mailman/config/mailman.cfg (+8/-0)
src/mailman/config/schema.cfg (+3/-0)
src/mailman/core/pipelines.py (+2/-0)
src/mailman/database/schema/postgres.sql (+5/-0)
src/mailman/database/schema/sqlite.sql (+1/-0)
src/mailman/interfaces/mailinglist.py (+5/-2)
src/mailman/interfaces/user.py (+5/-0)
src/mailman/model/mailinglist.py (+1/-0)
src/mailman/model/user.py (+9/-0)
src/mailman/rest/configuration.py (+3/-3)
src/mailman/rules/max_size.py (+0/-1)
src/mailman/runners/dlist.py (+34/-0)
src/mailman/styles/default.py (+1/-3)
src/mailman/testing/helpers.py (+75/-0)
src/mailman/testing/testing.cfg (+4/-0)
To merge this branch: bzr merge lp:~danci-emanuel/mailman/dlist_runner
Reviewer Review Type Date Requested Status
Robin J Pending
Review via email: mp+123832@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Robin J (robin-jeffries) wrote :

What are you doing with this stuff about "test_field"? It's clearly something not finished yet (which is OK), but what is it a proxy for?

in sqlite.sql, lets make this more clear. Instead of
   Is the Dynamic sublists feature enabled for this list?

How about:
   Allow people to (un)subscribe to individual conversations?

What does the change to maxsize.py mean? (looks like you took out part of the copyright notice)

I don't see that you have the actual Runner functionality. Did I miss that or does it go somewhere else?

 Could you for the next round (since it is part of the runner), look at what Sneha Priscilla did to split the dlistRunner into an early and a late part (the reason was to deal with emails with a to: address of the form <email address hidden> that also have 'unsubscribe' in the subject line. The actual dlist work had to go late, but shunting these off to keep them from being picked up by the administrivia processor had to happen early. It could be that things are done in a different order here, and you might be able to show that this isn't needed, but please look into it as part of the Runner work.

Reply
Revision history for this message
Emanuel Danci (danci-emanuel) wrote :

> What are you doing with this stuff about "test_field"? It's clearly something
> not finished yet (which is OK), but what is it a proxy for?

 I added it for some testing, but I will remove it as soon as the DlistUtils will be completed.

> in sqlite.sql, lets make this more clear. Instead of
> Is the Dynamic sublists feature enabled for this list?
>
> How about:
> Allow people to (un)subscribe to individual conversations?

Done, I have changed it.

> What does the change to maxsize.py mean? (looks like you took out part of the
> copyright notice)

Due to some quick editing in vim, I deleted it by mistake. I placed the missing row back.

> I don't see that you have the actual Runner functionality. Did I miss that or
> does it go somewhere else?
>
> Could you for the next round (since it is part of the runner), look at what
> Sneha Priscilla did to split the dlistRunner into an early and a late part
> (the reason was to deal with emails with a to: address of the form
> <email address hidden> that also have 'unsubscribe' in the subject
> line. The actual dlist work had to go late, but shunting these off to keep
> them from being picked up by the administrivia processor had to happen early.
> It could be that things are done in a different order here, and you might be
> able to show that this isn't needed, but please look into it as part of the
> Runner work.

Ok, I will look at what Sneha Priscilla did and see how I can adapt that here.

Reply

Unmerged revisions

7164. By Emanuel Danci

The DlistRunner was added.

7163. By Emanuel Danci

DlistRunner added

7162. By Emanuel Danci

Merge from the main branch

7161. By Emanuel Danci

Add the dlist runner

7160. By Emanuel Danci

Contains the changes needed for supporting Dynamic sublists

7159. By Emanuel Danci

Contains the changes needed for supporting Dynamic sublists

7158. By Emanuel Danci

Dynamic sublists support added

7157. By Emanuel Danci

Contains the changes needed for supporting Dynamic sublists

7156. By Emanuel Danci

contains UI changes needed for supporing Dynamic sublists

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/mailman/config/config.py'
2--- src/mailman/config/config.py 2012-07-02 20:56:35 +0000
3+++ src/mailman/config/config.py 2012-09-11 20:41:22 +0000
4@@ -154,6 +154,7 @@
5 messages_dir = category.messages_dir,
6 archive_dir = category.archive_dir,
7 queue_dir = category.queue_dir,
8+ dlist_queue_dir = category.dlist_queue_dir,
9 var_dir = var_dir,
10 template_dir = (
11 os.path.dirname(mailman.templates.__file__)
12
13=== modified file 'src/mailman/config/mailman.cfg'
14--- src/mailman/config/mailman.cfg 2012-03-31 10:51:42 +0000
15+++ src/mailman/config/mailman.cfg 2012-09-11 20:41:22 +0000
16@@ -88,4 +88,12 @@
17 [runner.digest]
18 class: mailman.runners.digest.DigestRunner
19
20+# add the dlist runner
21+[runner.dlist]
22+class: mailman.runners.dlist.DlistRunner
23+
24 [style.default]
25+
26+[database]
27+class: mailman.database.postgresql.PostgreSQLDatabase
28+url: postgres://mailman:mailman@localhost/mailman
29
30=== modified file 'src/mailman/config/schema.cfg'
31--- src/mailman/config/schema.cfg 2012-08-16 01:08:39 +0000
32+++ src/mailman/config/schema.cfg 2012-09-11 20:41:22 +0000
33@@ -98,6 +98,9 @@
34 var_dir: /var/tmp/mailman
35 # This is where the Mailman queue files directories will be created.
36 queue_dir: $var_dir/queue
37+
38+dlist_queue_dir: $queue_dir/dlist
39+
40 # This is the directory containing the Mailman 'runner' and 'master' commands
41 # if set to the string '$argv', it will be taken as the directory containing
42 # the 'bin/mailman' command.
43
44=== modified file 'src/mailman/core/pipelines.py'
45--- src/mailman/core/pipelines.py 2012-04-26 02:08:22 +0000
46+++ src/mailman/core/pipelines.py 2012-09-11 20:41:22 +0000
47@@ -111,6 +111,8 @@
48 name = 'default-posting-pipeline'
49 description = _('The built-in posting pipeline.')
50
51+ ## dlist_custom ##
52+ ## TODO: add the dlist handler to the default-posting-pipeline ##
53 _default_handlers = (
54 'mime-delete',
55 'tagger',
56
57=== modified file 'src/mailman/database/schema/postgres.sql'
58--- src/mailman/database/schema/postgres.sql 2012-07-23 14:40:53 +0000
59+++ src/mailman/database/schema/postgres.sql 2012-09-11 20:41:22 +0000
60@@ -19,6 +19,7 @@
61 admin_notify_mchanges BOOLEAN,
62 administrivia BOOLEAN,
63 advertised BOOLEAN,
64+ dlists_enabled BOOLEAN,
65 anonymous_list BOOLEAN,
66 archive BOOLEAN,
67 archive_private BOOLEAN,
68@@ -174,6 +175,10 @@
69 id SERIAL NOT NULL,
70 display_name TEXT,
71 password BYTEA,
72+
73+ --- custom ----
74+ test_field TEXT,
75+ ---------------
76 _user_id UUID,
77 _created_on TIMESTAMP,
78 _preferred_address_id INTEGER,
79
80=== modified file 'src/mailman/database/schema/sqlite.sql'
81--- src/mailman/database/schema/sqlite.sql 2012-04-08 16:15:29 +0000
82+++ src/mailman/database/schema/sqlite.sql 2012-09-11 20:41:22 +0000
83@@ -118,6 +118,7 @@
84 admin_notify_mchanges BOOLEAN,
85 administrivia BOOLEAN,
86 advertised BOOLEAN,
87+ dlists_enabled BOOLEAN,
88 anonymous_list BOOLEAN,
89 archive BOOLEAN,
90 archive_private BOOLEAN,
91
92=== modified file 'src/mailman/interfaces/mailinglist.py'
93--- src/mailman/interfaces/mailinglist.py 2012-09-05 01:31:50 +0000
94+++ src/mailman/interfaces/mailinglist.py 2012-09-11 20:41:22 +0000
95@@ -132,6 +132,9 @@
96 """Advertise this mailing list when people ask for an overview of the
97 available mailing lists.""")
98
99+ dlists_enabled = Attribute(
100+ """Is the Dynamic sublists feature enabled for this list?""")
101+
102 # Contact addresses
103
104 posting_address = Attribute(
105@@ -205,8 +208,8 @@
106 administrators = Attribute(
107 """The IUser administrators of this mailing list.
108
109- This includes the IUsers who are both owners and moderators of the
110- mailing list.""")
111+ This includes the IUsers who are both owners and moderators of the
112+ mailing list.""")
113
114 members = Attribute(
115 """An iterator over all the members of the mailing list, regardless of
116
117=== modified file 'src/mailman/interfaces/user.py'
118--- src/mailman/interfaces/user.py 2012-04-07 21:59:30 +0000
119+++ src/mailman/interfaces/user.py 2012-09-11 20:41:22 +0000
120@@ -53,6 +53,11 @@
121 class IUser(Interface):
122 """A basic user."""
123
124+ ## dlist custom ##
125+ test_field = Attribute(
126+ """Test field""")
127+ #######################
128+
129 display_name = Attribute(
130 """This user's display name.""")
131
132
133=== modified file 'src/mailman/model/mailinglist.py'
134--- src/mailman/model/mailinglist.py 2012-09-05 01:31:50 +0000
135+++ src/mailman/model/mailinglist.py 2012-09-11 20:41:22 +0000
136@@ -84,6 +84,7 @@
137 allow_list_posts = Bool()
138 include_rfc2369_headers = Bool()
139 advertised = Bool()
140+ dlists_enabled = Bool()
141 anonymous_list = Bool()
142 # Attributes not directly modifiable via the web u/i
143 created_at = DateTime()
144
145=== modified file 'src/mailman/model/user.py'
146--- src/mailman/model/user.py 2012-04-26 02:08:22 +0000
147+++ src/mailman/model/user.py 2012-09-11 20:41:22 +0000
148@@ -52,6 +52,10 @@
149 """Mailman users."""
150
151 id = Int(primary=True)
152+
153+ #####dlist custom#######
154+ test_field = Unicode()
155+ ################
156 display_name = Unicode()
157 _password = RawStr(name='password')
158 _user_id = UUID()
159@@ -73,6 +77,11 @@
160 self._user_id = user_id
161 self.display_name = ('' if display_name is None else display_name)
162 self.preferences = preferences
163+
164+
165+ ### dlist custom ###
166+ #init test_field
167+ ###############################
168 store.add(self)
169
170 def __repr__(self):
171
172=== modified file 'src/mailman/rest/configuration.py'
173--- src/mailman/rest/configuration.py 2012-08-21 00:55:24 +0000
174+++ src/mailman/rest/configuration.py 2012-09-11 20:41:22 +0000
175@@ -163,6 +163,7 @@
176 admin_notify_mchanges=GetterSetter(as_boolean),
177 administrivia=GetterSetter(as_boolean),
178 advertised=GetterSetter(as_boolean),
179+ dlists_enabled = GetterSetter(as_boolean),
180 anonymous_list=GetterSetter(as_boolean),
181 autorespond_owner=GetterSetter(enum_validator(ResponseAction)),
182 autorespond_postings=GetterSetter(enum_validator(ResponseAction)),
183@@ -206,11 +207,10 @@
184 welcome_message_uri=GetterSetter(unicode),
185 )
186
187-
188 VALIDATORS = ATTRIBUTES.copy()
189 for attribute, gettersetter in VALIDATORS.items():
190 if gettersetter.decoder is None:
191- del VALIDATORS[attribute]
192+ del VALIDATORS[attribute]
193
194
195
196
197@@ -234,7 +234,7 @@
198 return http.bad_request(
199 [], b'Unknown attribute: {0}'.format(self._attribute))
200 else:
201- attribute = self._attribute
202+ attribute = self._attribute
203 value = ATTRIBUTES[attribute].get(self._mlist, attribute)
204 resource[attribute] = value
205 return http.ok([], etag(resource))
206
207=== modified file 'src/mailman/rules/max_size.py'
208--- src/mailman/rules/max_size.py 2012-04-26 02:08:22 +0000
209+++ src/mailman/rules/max_size.py 2012-09-11 20:41:22 +0000
210@@ -3,7 +3,6 @@
211 # This file is part of GNU Mailman.
212 #
213 # GNU Mailman is free software: you can redistribute it and/or modify it under
214-# the terms of the GNU General Public License as published by the Free
215 # Software Foundation, either version 3 of the License, or (at your option)
216 # any later version.
217 #
218
219=== added file 'src/mailman/runners/dlist.py'
220--- src/mailman/runners/dlist.py 1970-01-01 00:00:00 +0000
221+++ src/mailman/runners/dlist.py 2012-09-11 20:41:22 +0000
222@@ -0,0 +1,34 @@
223+# Copyright (C) 1998,1999,2000,2001 by the Free Software Foundation, Inc.
224+# This program is free software; you can redistribute it and/or
225+# modify it under the terms of the GNU General Public License
226+ # as published by the Free Software Foundation; either version 2
227+# of the License, or (at your option) any later version.
228+#
229+# This program is distributed in the hope that it will be useful,
230+# but WITHOUT ANY WARRANTY; without even the implied warranty of
231+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
232+# GNU General Public License for more details.
233+#
234+# You should have received a copy of the GNU General Public License
235+# along with this program; if not, write to the Free Software
236+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-130 7, USA.
237+
238+"""Dynamic sublist processor."""
239+
240+from mailman.config import config
241+from mailman.core.runner import Runner
242+
243+__metaclass__ = type
244+__all__ = [
245+ 'DlistRunner'
246+ ]
247+
248+class DlistRunner(Runner):
249+ """The dlist runner."""
250+
251+ def _dispose(self, mlist, msg, msgdata):
252+ """See 'IRunner'."""
253+
254+ config.switchboards['out'].enqueue(msg, msgdata)
255+ return False
256+
257
258=== modified file 'src/mailman/styles/default.py'
259--- src/mailman/styles/default.py 2012-09-05 01:31:50 +0000
260+++ src/mailman/styles/default.py 2012-09-11 20:41:22 +0000
261@@ -24,12 +24,9 @@
262 'DefaultStyle',
263 ]
264
265-
266 # XXX Styles need to be reconciled with lazr.config.
267-
268 from datetime import timedelta
269 from zope.interface import implementer
270-
271 from mailman.core.i18n import _
272 from mailman.interfaces.action import Action, FilterAction
273 from mailman.interfaces.archiver import ArchivePolicy
274@@ -63,6 +60,7 @@
275 mlist.new_member_options = 256
276 mlist.respond_to_post_requests = True
277 mlist.advertised = True
278+ mlist.dlists_enabled = False
279 mlist.max_num_recipients = 10
280 mlist.max_message_size = 40 # KB
281 mlist.reply_goes_to_list = ReplyToMunging.no_munging
282
283=== modified file 'src/mailman/testing/helpers.py'
284--- src/mailman/testing/helpers.py 2012-07-21 22:06:43 +0000
285+++ src/mailman/testing/helpers.py 2012-09-11 20:41:22 +0000
286@@ -38,6 +38,14 @@
287 'subscribe',
288 'temporary_db',
289 'wait_for_webservice',
290+ 'print_clog',
291+ 'enq_messages',
292+ 'CLS',
293+ 'print_lists',
294+ 'check_qfiles',
295+ 'check_dlist_qfiles',
296+ 'check_out_qfiles',
297+ 'check_shunt_qfiles',
298 ]
299
300
301@@ -531,3 +539,70 @@
302 bytes_payload = payload.encode('utf-8')
303 for line in bytes_payload.splitlines():
304 yield line
305+
306+#######################################
307+# dlist_custom classes/functions
308+# XXX: remove the classes/functions
309+# that are no longer needed after
310+# development is complete
311+#######################################
312+
313+# print a custom message into the testing log file
314+def print_clog(string, filename=None):
315+
316+ if filename is None:
317+ filename = "/home/zalman/Desktop/testing_log.txt"
318+
319+ with open(filename, 'w+') as f:
320+ f.write(string)
321+
322+# enqueue test messages into a particular queue
323+def enq_messages(queue, messages):
324+
325+ switchboard = config.switchboards[queue]
326+
327+ for i in range(len(messages)):
328+ switchboard.enqueue(messages[i].msg, messages[i].msgdata)
329+
330+# used to clear the python console while testing code
331+class CLS(object):
332+ def __repr__(self):
333+ os.system("clear")
334+ return ''
335+
336+# print existing lists
337+def print_lists():
338+
339+ from mailman.interfaces.listmanager import IListManager
340+ from zope.component import getUtility
341+
342+ list_manager = getUtility(IListManager)
343+ for name in sorted(list_manager.names):
344+ print (name)
345+
346+# check the number of files from a given queue
347+def check_qfiles(qdir):
348+ files = {}
349+ for qfile in os.listdir(qdir):
350+ root, ext = os.path.splitext(qfile)
351+ files[ext] = files.get(ext, 0) + 1
352+ if len(files) == 0:
353+ print ('empty')
354+ for ext in sorted(files):
355+ print ('{0}: {1}'.format(ext, files[ext]))
356+
357+# check the number of files from the dlist qdir
358+def check_dlist_qfiles():
359+ check_qfiles(config.DLIST_QUEUE_DIR)
360+
361+# check the number of files from the out qdir
362+def check_out_qfiles():
363+ qdir = os.path.join(config.QUEUE_DIR, 'out')
364+ check_qfiles(qdir)
365+
366+# check the number of files from the shunt qdir
367+def check_shunt_qfiles():
368+ qdir = os.path.join(config.QUEUE_DIR, 'shunt')
369+ check_qfiles(qdir)
370+
371+
372
373=== modified file 'src/mailman/testing/testing.cfg'
374--- src/mailman/testing/testing.cfg 2012-08-16 01:08:39 +0000
375+++ src/mailman/testing/testing.cfg 2012-09-11 20:41:22 +0000
376@@ -88,3 +88,7 @@
377 [language.fr]
378 description: French
379 charset: iso-8859-1
380+
381+[database]
382+class: mailman.database.postgresql.PostgreSQLDatabase
383+url: postgres://mailman:mailman@local:/mailman

Subscribers

People subscribed via source and target branches

to all changes: