Merge lp:~spiv/bzr/push-relock into lp:bzr

Proposed by Andrew Bennetts
Status: Work in progress
Proposed branch: lp:~spiv/bzr/push-relock
Merge into: lp:bzr
Diff against target: 102 lines (+28/-3)
4 files modified
NEWS (+3/-0)
bzrlib/builtins.py (+2/-0)
bzrlib/bzrdir.py (+3/-2)
bzrlib/push.py (+20/-1)
To merge this branch: bzr merge lp:~spiv/bzr/push-relock
Reviewer Review Type Date Requested Status
bzr-core Pending
Review via email: mp+23767@code.launchpad.net

Commit message

Avoid relocking in cmd_push.

Description of the change

This fixes more relock warnings, this time in cmd_push.

This one is a bit uglier than the previous patches: most of the time cmd_push just needs a read-lock on br_from (the source branch), but occasionally it wants to set the remembered push location. And sometimes it won't know that it will want to until after reading the branch.conf -- i.e. after the read-lock has been acquired.

Unfortunately, we don't have an API to upgrade a read lock to write lock, so instead this patch opens a separate instance of the (already read-locked) branch so it can write-lock that and set the push location. That second instance is then discarded. The downside, as explained in the comment, is that the read-locked version will not see the update, but in this case I don't think that matters.

To post a comment you must log in.
Revision history for this message
Robert Collins (lifeless) wrote :

This won't work with older bzr formats that use a mutex across all processes
on write locks. I don't know if that matters, but it should be at least
documented, I think. Other than that it seems that perhaps its worth doing
an api to upgrade to a write lock?

Revision history for this message
Andrew Bennetts (spiv) wrote :

Hmm. I see what you mean (about older formats). I suppose the thing to
do is to catch LockContention, and if it occurs emit a warning to the
user that the push location could not be set due lock contention, and
then let the rest of the code proceed as normal. I think even for older
formats that's probably an improvement in behaviour: I'd rather the
overall push worked immediately even if bzr can't remember the location
because the branch is locked.

I'd love an API to upgrade to a write lock — and I think we'll need one
if we ever decide to pursue a zero relocks as a goal. But it looks like
a moderately large chunk of work. Maybe I'm wrong?

Revision history for this message
John A Meinel (jameinel) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Bennetts wrote:
> Hmm. I see what you mean (about older formats). I suppose the thing to
> do is to catch LockContention, and if it occurs emit a warning to the
> user that the push location could not be set due lock contention, and
> then let the rest of the code proceed as normal. I think even for older
> formats that's probably an improvement in behaviour: I'd rather the
> overall push worked immediately even if bzr can't remember the location
> because the branch is locked.
>
> I'd love an API to upgrade to a write lock — and I think we'll need one
> if we ever decide to pursue a zero relocks as a goal. But it looks like
> a moderately large chunk of work. Maybe I'm wrong?

Well, we already have *some* of that in the code today. Because Dirstate
can be updated within a logical 'read' lock. (status updating the hash
cache). So it has some "try to switch to a write lock" code.

I don't know what that would imply for something like Branch, though.
For Repository, it is trivial, because a write-lock is *still* a no-op
until you 'commit_write_group'. Branch and WT need to actually do
something to write-lock, though.

And we'd also need to decide what to do when the write-lock is already
taken.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvRveYACgkQJdeBCYSNAAMvcQCgyiKttmHFv3LjucqKkXbf2IE6
MRUAnR+fgSwPNzlgx+zb3Mt/Ev7ysFpN
=wyUS
-----END PGP SIGNATURE-----

Unmerged revisions

5148. By Andrew Bennetts

Tweak NEWS entry wording for consistency.

5147. By Andrew Bennetts

Merge lp:bzr.

5146. By Andrew Bennetts

Add NEWS entry.

5145. By Andrew Bennetts

Try to avoid relocking in cmd_push using an ugly hack for the --remember case.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'NEWS'
2--- NEWS 2010-04-20 10:30:30 +0000
3+++ NEWS 2010-04-20 14:02:15 +0000
4@@ -93,6 +93,9 @@
5 * ``bzr pull`` locks the branches and tree only once.
6 (Andrew Bennetts)
7
8+* ``bzr push`` read-locks the source branch only once.
9+ (Andrew Bennetts)
10+
11 * Index lookups in pack repositories search recently hit pack files first.
12 In repositories with many pack files this can greatly reduce the
13 number of files accessed, the number of bytes read, and the number of
14
15=== modified file 'bzrlib/builtins.py'
16--- bzrlib/builtins.py 2010-04-19 13:04:30 +0000
17+++ bzrlib/builtins.py 2010-04-20 14:02:15 +0000
18@@ -1127,6 +1127,8 @@
19 # Get the source branch
20 (tree, br_from,
21 _unused) = bzrdir.BzrDir.open_containing_tree_or_branch(directory)
22+ br_from.lock_read()
23+ self.add_cleanup(br_from.unlock)
24 # Get the tip's revision_id
25 revision = _get_one_revision('push', revision)
26 if revision is not None:
27
28=== modified file 'bzrlib/bzrdir.py'
29--- bzrlib/bzrdir.py 2010-04-15 15:03:15 +0000
30+++ bzrlib/bzrdir.py 2010-04-20 14:02:15 +0000
31@@ -61,6 +61,7 @@
32 sha_string,
33 )
34 from bzrlib.push import (
35+ _set_push_location,
36 PushResult,
37 )
38 from bzrlib.repofmt import pack_repo
39@@ -1283,7 +1284,7 @@
40 repository_to.fetch(source.repository, revision_id=revision_id)
41 br_to = source.clone(self, revision_id=revision_id)
42 if source.get_push_location() is None or remember:
43- source.set_push_location(br_to.base)
44+ _set_push_location(source, br_to.base)
45 push_result.stacked_on = None
46 push_result.branch_push_result = None
47 push_result.old_revno = None
48@@ -1294,7 +1295,7 @@
49 else:
50 # We have successfully opened the branch, remember if necessary:
51 if source.get_push_location() is None or remember:
52- source.set_push_location(br_to.base)
53+ _set_push_location(source, br_to.base)
54 try:
55 tree_to = self.open_workingtree()
56 except errors.NotLocalUrl:
57
58=== modified file 'bzrlib/push.py'
59--- bzrlib/push.py 2010-02-16 16:08:40 +0000
60+++ bzrlib/push.py 2010-04-20 14:02:15 +0000
61@@ -20,6 +20,7 @@
62 builtins,
63 branch,
64 bzrdir,
65+ cleanup,
66 errors,
67 revision as _mod_revision,
68 transport,
69@@ -57,6 +58,24 @@
70 self.branch_push_result.report(to_file)
71
72
73+def _set_push_location(br, location):
74+ """Helper for br.set_push_location(location) that tries to workaround br
75+ being read-locked.
76+ """
77+ try:
78+ br.set_push_location(location)
79+ except errors.ReadOnlyError:
80+ # XXX: Hack to work-around lack of a way to upgrade a read-lock to a
81+ # write-lock. This has the down-side that the config cached in 'br'
82+ # won't be aware of the changed push location, but the only code that
83+ # could notice would be a plugin that overrides cmd_push.
84+ br_unlocked = br.bzrdir.open_branch()
85+ op = cleanup.OperationWithCleanups(br_unlocked.set_push_location)
86+ br_unlocked.lock_write()
87+ op.add_cleanup(br_unlocked.unlock)
88+ op.run_simple(location)
89+
90+
91 def _show_push_branch(br_from, revision_id, location, to_file, verbose=False,
92 overwrite=False, remember=False, stacked_on=None, create_prefix=False,
93 use_existing_dir=False):
94@@ -131,7 +150,7 @@
95 push_result.old_revid = _mod_revision.NULL_REVISION
96 push_result.old_revno = 0
97 if br_from.get_push_location() is None or remember:
98- br_from.set_push_location(br_to.base)
99+ _set_push_location(br_from, br_to.base)
100 else:
101 if stacked_on is not None:
102 warning("Ignoring request for a stacked branch as repository "