Merge lp:~stub/launchpad/replication into lp:launchpad/db-devel

Proposed by Stuart Bishop
Status: Merged
Merged at revision: not available
Proposed branch: lp:~stub/launchpad/replication
Merge into: lp:launchpad/db-devel
Diff against target: 68 lines (+36/-4)
3 files modified
database/replication/Makefile (+1/-0)
database/replication/authdb_sequences.sql (+22/-0)
database/replication/new-slave.py (+13/-4)
To merge this branch: bzr merge lp:~stub/launchpad/replication
Reviewer Review Type Date Requested Status
Henning Eggers (community) code Approve
Review via email: mp+17278@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Stuart Bishop (stub) wrote :

Bug #504807

Changes made late last year to the staging rebuild broke things - sequence values in the authdb replication set stopped being restored. This broke parts of the system, such as account creation, adding email addresses etc. and required manual repair by the losas.

This branch automates the repair work as part of the rebuild process, as restoring the sequence from the actual database dumps would involve much more invasive refactoring and gain us nothing.

Revision history for this message
Stuart Bishop (stub) wrote :

Also fix Bug #504751, where new-slave.py stopped subscribing the new slave to the authdb replication set after we move its master to a new database.

Revision history for this message
Henning Eggers (henninge) wrote :

Looks good, thank you!

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'database/replication/Makefile'
--- database/replication/Makefile 2010-01-06 15:53:35 +0000
+++ database/replication/Makefile 2010-01-13 08:43:16 +0000
@@ -110,6 +110,7 @@
110 --no-acl --no-owner --disable-triggers --data-only \110 --no-acl --no-owner --disable-triggers --data-only \
111 --table=$$table ${STAGING_DUMP}; \111 --table=$$table ${STAGING_DUMP}; \
112 done112 done
113 psql -q -d lpmain_staging_new -f authdb_sequences.sql
113 # Uninstall Slony-I if it is installed - a pg_dump of a DB with114 # Uninstall Slony-I if it is installed - a pg_dump of a DB with
114 # Slony-I installed isn't usable without this step.115 # Slony-I installed isn't usable without this step.
115 LPCONFIG=${NEW_STAGING_CONFIG} ./repair-restored-db.py116 LPCONFIG=${NEW_STAGING_CONFIG} ./repair-restored-db.py
116117
=== added file 'database/replication/authdb_sequences.sql'
--- database/replication/authdb_sequences.sql 1970-01-01 00:00:00 +0000
+++ database/replication/authdb_sequences.sql 2010-01-13 08:43:16 +0000
@@ -0,0 +1,22 @@
1-- Repair sequences in the authdb replication set. We need to do this because
2-- we cannot restore the sequence values from the dump when restoring the
3-- data using pg_restore --data-only.
4
5SELECT setval('account_id_seq', max(id)) AS Account
6FROM Account;
7
8SELECT setval('accountpassword_id_seq', max(id)) AS AccountPassword
9FROM AccountPassword;
10
11SELECT setval('authtoken_id_seq', max(id)) AS AuthToken
12FROM AuthToken;
13
14SELECT setval('emailaddress_id_seq', max(id)) AS EmailAddress
15FROM EmailAddress;
16
17SELECT setval('openidauthorization_id_seq', max(id)) AS OpenIDAuthorization
18FROM OpenIDAuthorization;
19
20SELECT setval('openidrpsummary_id_seq', max(id)) AS OpenIDRPSummary
21FROM OpenIDRPSummary;
22
023
=== modified file 'database/replication/new-slave.py'
--- database/replication/new-slave.py 2009-12-03 09:47:24 +0000
+++ database/replication/new-slave.py 2010-01-13 08:43:16 +0000
@@ -162,13 +162,22 @@
162 target_con.commit()162 target_con.commit()
163 del target_con163 del target_con
164164
165 # Get a list of existing set ids.165 # Get a list of existing set ids that can be subscribed too. This
166 # is all sets where the origin is the master_node, and set 2 if
167 # the master happens to be configured as a forwarding slave. We
168 # don't allow other sets where the master is configured as a
169 # forwarding slave as we have to special case rebuilding the database
170 # schema (such as we do for the authdb replication set 2).
166 source_connection.rollback()171 source_connection.rollback()
167 master_node = replication.helpers.get_master_node(source_connection)172 master_node = replication.helpers.get_master_node(source_connection)
168 cur = source_connection.cursor()173 cur = source_connection.cursor()
169 cur.execute(174 cur.execute("""
170 "SELECT set_id FROM _sl.sl_set WHERE set_origin=%d"175 SELECT set_id FROM _sl.sl_set WHERE set_origin=%d
171 % master_node.node_id)176 UNION
177 SELECT sub_set AS set_id FROM _sl.sl_subscribe
178 WHERE sub_receiver=%d AND sub_forward IS TRUE AND sub_active IS TRUE
179 AND sub_set=2
180 """ % (master_node.node_id, master_node.node_id))
172 set_ids = [set_id for set_id, in cur.fetchall()]181 set_ids = [set_id for set_id, in cur.fetchall()]
173 log.debug("Discovered set ids %s" % repr(list(set_ids)))182 log.debug("Discovered set ids %s" % repr(list(set_ids)))
174183

Subscribers

People subscribed via source and target branches

to status/vote changes: