Launchpad itself

Merge lp:~stub/launchpad/replication into lp:launchpad/db-devel

  1. replication
  2. Merge into db-devel
Proposed by Stuart Bishop
Status: Merged
Merged at revision: not available
Proposed branch: lp:~stub/launchpad/replication
Merge into: lp:launchpad/db-devel
Diff against target: 68 lines (+36/-4)
3 files modified
database/replication/Makefile (+1/-0)
database/replication/authdb_sequences.sql (+22/-0)
database/replication/new-slave.py (+13/-4)
To merge this branch: bzr merge lp:~stub/launchpad/replication
Reviewer Review Type Date Requested Status
Henning Eggers (community) code Approve
Review via email: mp+17278@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Stuart Bishop (stub) wrote :

Bug #504807

Changes made late last year to the staging rebuild broke things - sequence values in the authdb replication set stopped being restored. This broke parts of the system, such as account creation, adding email addresses etc. and required manual repair by the losas.

This branch automates the repair work as part of the rebuild process, as restoring the sequence from the actual database dumps would involve much more invasive refactoring and gain us nothing.

Revision history for this message
Stuart Bishop (stub) wrote :

Also fix Bug #504751, where new-slave.py stopped subscribing the new slave to the authdb replication set after we move its master to a new database.

Revision history for this message
Henning Eggers (henninge) wrote :

Looks good, thank you!

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'database/replication/Makefile'
2--- database/replication/Makefile 2010-01-06 15:53:35 +0000
3+++ database/replication/Makefile 2010-01-13 08:43:16 +0000
4@@ -110,6 +110,7 @@
5 --no-acl --no-owner --disable-triggers --data-only \
6 --table=$$table ${STAGING_DUMP}; \
7 done
8+ psql -q -d lpmain_staging_new -f authdb_sequences.sql
9 # Uninstall Slony-I if it is installed - a pg_dump of a DB with
10 # Slony-I installed isn't usable without this step.
11 LPCONFIG=${NEW_STAGING_CONFIG} ./repair-restored-db.py
12
13=== added file 'database/replication/authdb_sequences.sql'
14--- database/replication/authdb_sequences.sql 1970-01-01 00:00:00 +0000
15+++ database/replication/authdb_sequences.sql 2010-01-13 08:43:16 +0000
16@@ -0,0 +1,22 @@
17+-- Repair sequences in the authdb replication set. We need to do this because
18+-- we cannot restore the sequence values from the dump when restoring the
19+-- data using pg_restore --data-only.
20+
21+SELECT setval('account_id_seq', max(id)) AS Account
22+FROM Account;
23+
24+SELECT setval('accountpassword_id_seq', max(id)) AS AccountPassword
25+FROM AccountPassword;
26+
27+SELECT setval('authtoken_id_seq', max(id)) AS AuthToken
28+FROM AuthToken;
29+
30+SELECT setval('emailaddress_id_seq', max(id)) AS EmailAddress
31+FROM EmailAddress;
32+
33+SELECT setval('openidauthorization_id_seq', max(id)) AS OpenIDAuthorization
34+FROM OpenIDAuthorization;
35+
36+SELECT setval('openidrpsummary_id_seq', max(id)) AS OpenIDRPSummary
37+FROM OpenIDRPSummary;
38+
39
40=== modified file 'database/replication/new-slave.py'
41--- database/replication/new-slave.py 2009-12-03 09:47:24 +0000
42+++ database/replication/new-slave.py 2010-01-13 08:43:16 +0000
43@@ -162,13 +162,22 @@
44 target_con.commit()
45 del target_con
46
47- # Get a list of existing set ids.
48+ # Get a list of existing set ids that can be subscribed too. This
49+ # is all sets where the origin is the master_node, and set 2 if
50+ # the master happens to be configured as a forwarding slave. We
51+ # don't allow other sets where the master is configured as a
52+ # forwarding slave as we have to special case rebuilding the database
53+ # schema (such as we do for the authdb replication set 2).
54 source_connection.rollback()
55 master_node = replication.helpers.get_master_node(source_connection)
56 cur = source_connection.cursor()
57- cur.execute(
58- "SELECT set_id FROM _sl.sl_set WHERE set_origin=%d"
59- % master_node.node_id)
60+ cur.execute("""
61+ SELECT set_id FROM _sl.sl_set WHERE set_origin=%d
62+ UNION
63+ SELECT sub_set AS set_id FROM _sl.sl_subscribe
64+ WHERE sub_receiver=%d AND sub_forward IS TRUE AND sub_active IS TRUE
65+ AND sub_set=2
66+ """ % (master_node.node_id, master_node.node_id))
67 set_ids = [set_id for set_id, in cur.fetchall()]
68 log.debug("Discovered set ids %s" % repr(list(set_ids)))
69

Subscribers

People subscribed via source and target branches

to status/vote changes: