Bazaar

Merge lp:~vila/bzr/697815-repair-repo into lp:bzr

  1. 697815-repair-repo
  2. Merge into bzr.dev
Proposed by Vincent Ladeuil
Status: Work in progress
Proposed branch: lp:~vila/bzr/697815-repair-repo
Merge into: lp:bzr
Diff against target: 177 lines (+101/-36)
3 files modified
bzrlib/builtins.py (+27/-2)
bzrlib/check.py (+1/-34)
tools/repair-repo.py (+73/-0)
To merge this branch: bzr merge lp:~vila/bzr/697815-repair-repo
Reviewer Review Type Date Requested Status
bzr-core Pending
Review via email: mp+45945@code.launchpad.net

Description of the change

This add a script that repair repos containing an empty pack file.

As seen in bug #697815 this repair a broken repository but is an incomplete solution as the branch is left in a broken state.

Still, I don't feel like addressing the later but having this script easily available sounds like a useful thing anyway.

I wanted to add some tests but... this script is not a command and I don't feel like starting a new test infrastructure for that...

So I thought I may ask for feedback about whether it's acceptable to land this first and what people think about:
- turning it into its own command,
- somehow make it part of check/reconcile
- something else

To post a comment you must log in.
Revision history for this message
Martin Pool (mbp) wrote :

That's useful.

I think putting it into a hidden command is better than putting into
tools/, because the latter will not be easily accessible to people
running bzr from a package or a binary installer.

Ultimately I think this should move into probably reconcile or check.
It might be enough to just give them an option to run this particular
fix, like

 check --fixer empty_packs

should run this code.

--
Martin

Reply
lp:~vila/bzr/697815-repair-repo updated
5598. By Vincent Ladeuil

Remove code deprecated in 1.6.0 and 1.16.0.

5599. By Vincent Ladeuil

Sort the lazy registered commands in ascii order.

5600. By Vincent Ladeuil

Unfinished stuff

Unmerged revisions

5600. By Vincent Ladeuil

Unfinished stuff

5599. By Vincent Ladeuil

Sort the lazy registered commands in ascii order.

5598. By Vincent Ladeuil

Remove code deprecated in 1.6.0 and 1.16.0.

5597. By Vincent Ladeuil

Give some context and mention the missing features/tests.

5596. By Vincent Ladeuil

Add the copyright notice.

5595. By Vincent Ladeuil

Add the working repair-repo script as-is.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'bzrlib/builtins.py'
2--- bzrlib/builtins.py 2011-01-21 23:21:18 +0000
3+++ bzrlib/builtins.py 2011-01-25 13:41:43 +0000
4@@ -1638,6 +1638,31 @@
5 reconcile(dir, canonicalize_chks=canonicalize_chks)
6
7
8+class cmd_repair_repo(Command):
9+ __doc__ = """Repair a repo with an empty pack file.
10+
11+ Under some (yet) unclear circumstances (but generally involving ext4),
12+ system crashes left some file systems in an inconsistent state: pack file
13+ content get lost even if later modifications are taken into account by such
14+ file systems. The most commonly observed symptom is that an empty pack file
15+ is referenced in a repository pack-names file.
16+
17+ The content of the file can't be recovered, but deleting it from the
18+ pack-names file makes the repository usable again.
19+
20+ Unless you've been instructed to use this command, please file a bug at
21+ <https://bugs.launchpad.net/bzr/+filebug> explaining as best as you can how
22+ you end up in such a situation (OS and file system used at least and
23+ whether you had a crash).
24+ """
25+
26+ takes_args = ['pack_name']
27+ hidden = True
28+
29+ def run(self, pack_name):
30+ pass
31+
32+
33 class cmd_revision_history(Command):
34 __doc__ = """Display the list of revision ids on a branch."""
35
36@@ -6118,11 +6143,11 @@
37 for (name, aliases, module_name) in [
38 ('cmd_bundle_info', [], 'bzrlib.bundle.commands'),
39 ('cmd_config', [], 'bzrlib.config'),
40+ ('cmd_conflicts', [], 'bzrlib.conflicts'),
41 ('cmd_dpush', [], 'bzrlib.foreign'),
42- ('cmd_version_info', [], 'bzrlib.cmd_version_info'),
43 ('cmd_resolve', ['resolved'], 'bzrlib.conflicts'),
44- ('cmd_conflicts', [], 'bzrlib.conflicts'),
45 ('cmd_sign_my_commits', [], 'bzrlib.sign_my_commits'),
46 ('cmd_test_script', [], 'bzrlib.cmd_test_script'),
47+ ('cmd_version_info', [], 'bzrlib.cmd_version_info'),
48 ]:
49 builtin_command_registry.register_lazy(name, aliases, module_name)
50
51=== modified file 'bzrlib/check.py'
52--- bzrlib/check.py 2011-01-13 00:14:31 +0000
53+++ bzrlib/check.py 2011-01-25 13:41:43 +0000
54@@ -1,4 +1,4 @@
55-# Copyright (C) 2005, 2006 Canonical Ltd
56+# Copyright (C) 2005-2011 Canonical Ltd
57 #
58 # This program is free software; you can redistribute it and/or modify
59 # it under the terms of the GNU General Public License as published by
60@@ -330,39 +330,6 @@
61 self.text_key_references[key] = True
62
63
64-@deprecated_function(deprecated_in((1,6,0)))
65-def check(branch, verbose):
66- """Run consistency checks on a branch.
67-
68- Results are reported through logging.
69-
70- Deprecated in 1.6. Please use check_dwim instead.
71-
72- :raise BzrCheckError: if there's a consistency error.
73- """
74- check_branch(branch, verbose)
75-
76-
77-@deprecated_function(deprecated_in((1,16,0)))
78-def check_branch(branch, verbose):
79- """Run consistency checks on a branch.
80-
81- Results are reported through logging.
82-
83- :raise BzrCheckError: if there's a consistency error.
84- """
85- branch.lock_read()
86- try:
87- needed_refs = {}
88- for ref in branch._get_check_refs():
89- needed_refs.setdefault(ref, []).append(branch)
90- result = branch.repository.check([branch.last_revision()], needed_refs)
91- branch_result = result.other_results[0]
92- finally:
93- branch.unlock()
94- branch_result.report_results(verbose)
95-
96-
97 def scan_branch(branch, needed_refs, to_unlock):
98 """Scan a branch for refs.
99
100
101=== added file 'tools/repair-repo.py'
102--- tools/repair-repo.py 1970-01-01 00:00:00 +0000
103+++ tools/repair-repo.py 2011-01-25 13:41:43 +0000
104@@ -0,0 +1,73 @@
105+#!/usr/bin/env python
106+# Copyright (C) 2010 Canonical Ltd
107+#
108+# This program is free software; you can redistribute it and/or modify
109+# it under the terms of the GNU General Public License as published by
110+# the Free Software Foundation; either version 2 of the License, or
111+# (at your option) any later version.
112+#
113+# This program is distributed in the hope that it will be useful,
114+# but WITHOUT ANY WARRANTY; without even the implied warranty of
115+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
116+# GNU General Public License for more details.
117+#
118+# You should have received a copy of the GNU General Public License
119+# along with this program; if not, write to the Free Software
120+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
121+
122+"""Repair a repo with an empty pack file.
123+
124+Under some (yet) unclear circumstances, crashes left some file systems in an
125+inconsistent state: file content get lost even if later modifications are taken
126+into account by such file systems. The most commonly observed symptom is that
127+an empty pack file is referenced in a repository pack-names file.
128+
129+The content of the file can't be recovered, but deleting it from the pack-names
130+file makes the repository usable again.
131+
132+Unless you've been instructed to use this file, please file a bug at
133+<https://bugs.launchpad.net/bzr/+filebug> explaining as best as you can how you
134+end up in such a situation (OS and file system used at least and whether you
135+had a crash).
136+
137+Usage:
138+
139+ repair-repo.py <pack file base name>
140+
141+"""
142+
143+# FIXME: This should be turned into a proper command, possibly hidden
144+
145+# Missing features/tests:
146+# - apply only to pack-based repositories
147+# - should detect all empty pack files and report them
148+# - should have a --dry-run option
149+
150+# - should provide help to repair the associated branch too (generally the
151+# empty pack file happen for the last commit and the branch where this commit
152+# occurred mentions the (now lost) revid in its last-revision file).
153+
154+import sys
155+
156+from bzrlib import branch
157+
158+pack_name = sys.argv[1]
159+
160+from bzrlib import repository, btree_index, transport
161+
162+location = u'.'
163+b = branch.Branch.open_containing(location)[0]
164+
165+t = b.repository._transport
166+pn_index = btree_index.BTreeGraphIndex(t, 'pack-names', None)
167+nodes = pn_index.iter_all_entries()
168+
169+new_index = btree_index.BTreeBuilder(0, 1)
170+
171+for _, key, value in nodes:
172+ if key == (pack_name,):
173+ continue
174+ new_index.add_node(key, value)
175+
176+new_index_content = new_index.finish()
177+t.put_file('pack-names', new_index_content)