Merge lp:~abentley/bzr/appenddir into lp:bzr

Proposed by Aaron Bentley
Status: Work in progress
Proposed branch: lp:~abentley/bzr/appenddir
Merge into: lp:bzr
Diff against target: 158 lines (+43/-14)
5 files modified
bzrlib/config.py (+12/-9)
bzrlib/help_topics/en/configuration.txt (+17/-1)
bzrlib/tests/test_config.py (+7/-1)
doc/developers/contribution-quickstart.txt (+2/-2)
doc/en/release-notes/bzr-2.4.txt (+5/-1)
To merge this branch: bzr merge lp:~abentley/bzr/appenddir
Reviewer Review Type Date Requested Status
Vincent Ladeuil Abstain
Review via email: mp+50060@code.launchpad.net

Commit message

Support appenddir configuration policy. (Aaron Bentley)

Description of the change

Hi all,

This branch introduces a new configuration policy, "appenddir". This new
policy is similar to appendpath, except that it appends only the last path
element to the value. Here's an example configuration:

[/home/abentley/bzr]
public_branch = lp:~abentley/bzr
public_branch:policy = appenddir

This means that the public branch for "/home/abentley/bzr/foo" will be
"lp:~abentley/bzr/foo", as with appendpath. But the public branch of
"/home/abentley/bzr/bar/foo" will *also* be "lp:~abentley/bzr/foo", whereas
appendpath would make it "lp:~abentley/bzr/bar/foo". While the appendpath
behaviour may be useful for some, it produces too-long branch names in many
cases.

My motivating case is "bzr-pipeline". When a pipeline is created with
"reconfigure-pipeline", the branch is converted into a lightweight checkout,
with branches at ".bzr/pipes/". If the push location had been set with
appendpath, a push location of "lp:~abentley/bzr/foo" would become
"lp:~abentley/bzr/.bzr/pipes/foo", which is undesirable. With appenddir, it
would retain the value "lp:~abentley/bzr/foo". I believe bzr-colo has the same
issue.

While the new policy is syntactically compatible with existing config files,
current versions of bzr treat unknown configuration policies as internal
errors:
*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.

I would be happy to create a bugfix for earlier bzrs so that they emit a
user-friendly error when they encounter unknown configuration policies.

To post a comment you must log in.
Revision history for this message
Vincent Ladeuil (vila) wrote :

Thanks for working on this, this is clearly a sore point in out config handling.

I've got a couple of questions about the implementation:

- are you aware of the work in progress documented in configuration.txt in the lp:~bzr/bzr/devnotes ? The interpolation part especially may provide a more scalable means to achieve the same result.

- why did you chose 'appendir' instead of say 'nick' ? I'm well aware that they may not match but the later would also work for looms,

27 STORE_LOCATION = POLICY_NONE
28 STORE_LOCATION_NORECURSE = POLICY_NORECURSE
29 STORE_LOCATION_APPENDPATH = POLICY_APPENDPATH
30 STORE_BRANCH = 3
31 STORE_GLOBAL = 4
32 +STORE_LOCATION_APPENDDIR = POLICY_APPENDDIR

The overlapping between STORE_* and POLICY_* is already a concern in the current implementation, could you think of a better way to articulate their relationship ?

This shows even more in:

51 - if store not in [STORE_LOCATION,
52 - STORE_LOCATION_NORECURSE,
53 - STORE_LOCATION_APPENDPATH]:
54 + if store not in _policy_name:

Err, wait, checking 'store' in '_policy_name' ? This at least requires a comment...

review: Needs Information
Revision history for this message
Aaron Bentley (abentley) wrote :
Download full text (3.2 KiB)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11-02-17 04:47 AM, Vincent Ladeuil wrote:
> Review: Needs Information
> Thanks for working on this, this is clearly a sore point in out config handling.
>
> I've got a couple of questions about the implementation:
>
> - are you aware of the work in progress documented in configuration.txt in the lp:~bzr/bzr/devnotes ?

No. I'm glad that serious re-thinking is going on. I would rather not
have to wait for a ground-up rewrite, unless it is immanent.

> The interpolation part especially may provide a more scalable means to achieve the same result.
>
> - why did you chose 'appendir' instead of say 'nick' ?

Initially, I did choose 'nick', but locations.conf is not branch-aware,
and in fact is sometimes used without a branch. So making it
branch-aware didn't seem like a good fit.

> I'm well aware that they may not match but the later would also work for looms,

Supporting looms would be nice, but holds little value for me. I
haven't used looms since I wrote bzr-pipeline.

> 27 STORE_LOCATION = POLICY_NONE
> 28 STORE_LOCATION_NORECURSE = POLICY_NORECURSE
> 29 STORE_LOCATION_APPENDPATH = POLICY_APPENDPATH
> 30 STORE_BRANCH = 3
> 31 STORE_GLOBAL = 4
> 32 +STORE_LOCATION_APPENDDIR = POLICY_APPENDDIR
>
> The overlapping between STORE_* and POLICY_* is already a concern in the current implementation, could you think of a better way to articulate their relationship ?

It looks like inheritance to me-- All STORE values indicate which file
to store the value in, while the STORE_LOCATION ones also indicate a
policy. If I was doing a radical rewrite, I'd probably make them
classes with static/classmethods, e.g.

class ValueScope:

    @classmethod
    def store_value(cls, name, value):
        config = cls.get_config()
        cls._set_value(config, name, value)

    @staticmethod
    def _set_value(config, name, value)
        config.set_value(name, value)

class GlobalScope(ValueScope):

    @staticmethod
    def get_config():
        return GlobalConfig

class PlainLocationScope(ValueScope):

    @staticmethod
    def get_config():
        return LocationConfig

class AppendPathScope(PlainLocationScope):

    @staticmethod
    def _set_value(config, name, value):
        config.set_value(name, value, policy='appendpath')

A less-radical rewrite would be to make the STORE values tuples of
config, POLICY:

STORE_BRANCH = ('branch', POLICY_NONE)
STORE_LOCATION_APPENDPATH = ('location', POLICY_APPENDPATH)

No matter what, I don't think assigning integers to these things is
doing us any good. I'd love to see "STORE_BRANCH = 'branch'". That
would make debugging easier.

> This shows even more in:
>
> 51 - if store not in [STORE_LOCATION,
> 52 - STORE_LOCATION_NORECURSE,
> 53 - STORE_LOCATION_APPENDPATH]:
> 54 + if store not in _policy_name:
>
> Err, wait, checking 'store' in '_policy_name' ? This at least requires a comment...

Yesterday, it looked like a nice way to avoid repeating ourselves.
Today, it looks a bit broken.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1dR4MACgkQ0F+nu1YWqI1IoA...

Read more...

lp:~abentley/bzr/appenddir updated
5668. By Aaron Bentley

Saner checking for valid store locations.

Revision history for this message
Vincent Ladeuil (vila) wrote :

So, I've been working on this from a different angle:
- http://doc.bazaar.canonical.com/devnotes/configuration.html
- lp:~vila/bzr/new-config

This is a work in progress which still doesn't provide a working solution in core.

So basically I can't disapprove this proposal until I'm able to point to a working implementation but on the other hand, this proposal use a feature I'd like to get rid of (option and store policies) so approving it means I'll have to deprecate it in the future and provide a migration path.

On the third hand, I will need to provide a migration path for the existing policies which will require a yet *unknown* amount of work ;)

That being said:
- option expansion is (now) available and using {nickname} is very close to the 'appendir' policy,
- 'appenddir' fooled me (twice) as a policy name: it really add the *basename* of the branch/wt which indeed is a dir but I tend to think in terms of os.path API and read 'appenddir' as 'os.path.dirname'

Regarding policies, the plan is to define them once and only once for a given option, that means *outside* of the config files. This is my main issue with the current implementation.

review: Abstain
Revision history for this message
Vincent Ladeuil (vila) wrote :

Also, aren't there cases where 'appendir' would be empty and led to confusing errors ?

I also want to add that I *like* the feature even if I frown upon the implementation (while understanding that's the only one available ;).

Revision history for this message
Vincent Ladeuil (vila) wrote :

@Aaron: What's the status on this ? I'll mark it wip to clear the queue but I'm open to discussion and don't want to mark it rejected (or should it be just abandoned ?).

Unmerged revisions

5668. By Aaron Bentley

Saner checking for valid store locations.

5667. By Aaron Bentley

Support appenddir configuration policy.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'bzrlib/config.py'
--- bzrlib/config.py 2011-01-20 04:44:14 +0000
+++ bzrlib/config.py 2011-02-17 16:40:48 +0000
@@ -109,25 +109,23 @@
109POLICY_NONE = 0109POLICY_NONE = 0
110POLICY_NORECURSE = 1110POLICY_NORECURSE = 1
111POLICY_APPENDPATH = 2111POLICY_APPENDPATH = 2
112POLICY_APPENDDIR = 5
112113
113_policy_name = {114_policy_name = {
114 POLICY_NONE: None,115 POLICY_NONE: None,
115 POLICY_NORECURSE: 'norecurse',116 POLICY_NORECURSE: 'norecurse',
116 POLICY_APPENDPATH: 'appendpath',117 POLICY_APPENDPATH: 'appendpath',
117 }118 POLICY_APPENDDIR: 'appenddir',
118_policy_value = {119 }
119 None: POLICY_NONE,120_policy_value = dict((v, k) for k, v in _policy_name.iteritems())
120 'none': POLICY_NONE,121_policy_value['none'] = POLICY_NONE
121 'norecurse': POLICY_NORECURSE,
122 'appendpath': POLICY_APPENDPATH,
123 }
124
125122
126STORE_LOCATION = POLICY_NONE123STORE_LOCATION = POLICY_NONE
127STORE_LOCATION_NORECURSE = POLICY_NORECURSE124STORE_LOCATION_NORECURSE = POLICY_NORECURSE
128STORE_LOCATION_APPENDPATH = POLICY_APPENDPATH125STORE_LOCATION_APPENDPATH = POLICY_APPENDPATH
129STORE_BRANCH = 3126STORE_BRANCH = 3
130STORE_GLOBAL = 4127STORE_GLOBAL = 4
128STORE_LOCATION_APPENDDIR = POLICY_APPENDDIR
131129
132_ConfigObj = None130_ConfigObj = None
133def ConfigObj(*args, **kwargs):131def ConfigObj(*args, **kwargs):
@@ -561,6 +559,10 @@
561 if extra_path:559 if extra_path:
562 value = urlutils.join(value, extra_path)560 value = urlutils.join(value, extra_path)
563 return value561 return value
562 elif policy == POLICY_APPENDDIR:
563 if extra_path:
564 value = urlutils.join(value, urlutils.basename(extra_path))
565 return value
564 else:566 else:
565 raise AssertionError('Unexpected config policy %r' % policy)567 raise AssertionError('Unexpected config policy %r' % policy)
566 else:568 else:
@@ -931,7 +933,8 @@
931 """Save option and its value in the configuration."""933 """Save option and its value in the configuration."""
932 if store not in [STORE_LOCATION,934 if store not in [STORE_LOCATION,
933 STORE_LOCATION_NORECURSE,935 STORE_LOCATION_NORECURSE,
934 STORE_LOCATION_APPENDPATH]:936 STORE_LOCATION_APPENDPATH,
937 STORE_LOCATION_APPENDDIR]:
935 raise ValueError('bad storage policy %r for %r' %938 raise ValueError('bad storage policy %r for %r' %
936 (store, option))939 (store, option))
937 self.reload()940 self.reload()
938941
=== modified file 'bzrlib/help_topics/en/configuration.txt'
--- bzrlib/help_topics/en/configuration.txt 2011-01-16 01:12:01 +0000
+++ bzrlib/help_topics/en/configuration.txt 2011-02-17 16:40:48 +0000
@@ -276,6 +276,8 @@
276 appendpath:276 appendpath:
277 for contained locations, any additional path components are277 for contained locations, any additional path components are
278 appended to the value.278 appended to the value.
279 appenddir:
280 for contained locations, the last path component is appended to the value.
279281
280Policies are specified by keys with names of the form "$var:policy".282Policies are specified by keys with names of the form "$var:policy".
281For example, to define the push location for a tree of branches, the283For example, to define the push location for a tree of branches, the
@@ -286,7 +288,21 @@
286 push_location:policy = appendpath288 push_location:policy = appendpath
287289
288With this configuration, the push location for ``/top/location/branch1``290With this configuration, the push location for ``/top/location/branch1``
289would be ``sftp://example.com/location/branch1``.291would be ``sftp://example.com/location/branch1``, and the push location for
292``top/location/subdir/branch2`` would be
293``sftp://example.com/location/subdir/branch2``.
294
295If we want to avoid including subdirectories, we can use the appenddir policy
296instead::
297
298 [/top/location]
299 push_location = sftp://example.com/location
300 push_location:policy = appenddir
301
302With this configuration, the push location for ``/top/location/branch1``
303would be ``sftp://example.com/location/branch1``, and the push location for
304``top/location/subdir/branch2`` would be
305``sftp://example.com/location/branch2``.
290306
291307
292The main configuration file, bazaar.conf308The main configuration file, bazaar.conf
293309
=== modified file 'bzrlib/tests/test_config.py'
--- bzrlib/tests/test_config.py 2011-01-20 04:44:14 +0000
+++ bzrlib/tests/test_config.py 2011-02-17 16:40:48 +0000
@@ -33,7 +33,6 @@
33 errors,33 errors,
34 osutils,34 osutils,
35 mail_client,35 mail_client,
36 mergetools,
37 ui,36 ui,
38 urlutils,37 urlutils,
39 tests,38 tests,
@@ -1216,6 +1215,13 @@
1216 self.assertEqual('normal',1215 self.assertEqual('normal',
1217 self.my_config.get_user_option('appendpath_option'))1216 self.my_config.get_user_option('appendpath_option'))
12181217
1218 def test_get_user_option_appenddir(self):
1219 foo_config = config.LocationConfig('/foo')
1220 foo_config.set_user_option('bar', 'baz',
1221 config.STORE_LOCATION_APPENDDIR)
1222 qux_config = config.LocationConfig('/foo/bar/qux')
1223 self.assertEqual('baz/qux', qux_config.get_user_option('bar'))
1224
1219 def test_get_user_option_norecurse(self):1225 def test_get_user_option_norecurse(self):
1220 self.get_branch_config('http://www.example.com')1226 self.get_branch_config('http://www.example.com')
1221 self.assertEqual('norecurse',1227 self.assertEqual('norecurse',
12221228
=== modified file 'doc/developers/contribution-quickstart.txt'
--- doc/developers/contribution-quickstart.txt 2010-12-02 10:41:05 +0000
+++ doc/developers/contribution-quickstart.txt 2011-02-17 16:40:48 +0000
@@ -58,9 +58,9 @@
5858
59 [/home/USER/bzr]59 [/home/USER/bzr]
60 push_location = lp:~LAUNCHPAD_USER/bzr/60 push_location = lp:~LAUNCHPAD_USER/bzr/
61 push_location:policy = appendpath61 push_location:policy = appenddir
62 public_branch = http://bazaar.launchpad.net/~LAUNCHPAD_USER/bzr/62 public_branch = http://bazaar.launchpad.net/~LAUNCHPAD_USER/bzr/
63 public_branch:policy = appendpath63 public_branch:policy = appenddir
6464
65with your local and Launchpad usernames inserted.65with your local and Launchpad usernames inserted.
6666
6767
=== modified file 'doc/en/release-notes/bzr-2.4.txt'
--- doc/en/release-notes/bzr-2.4.txt 2011-02-16 17:20:10 +0000
+++ doc/en/release-notes/bzr-2.4.txt 2011-02-17 16:40:48 +0000
@@ -26,6 +26,10 @@
26* External merge tools can now be configured in bazaar.conf. See26* External merge tools can now be configured in bazaar.conf. See
27 ``bzr help configuration`` for more information. (Gordon Tyler, #489915)27 ``bzr help configuration`` for more information. (Gordon Tyler, #489915)
2828
29* New "appenddir" configuration policy appends only the last path element to a
30 config value, providing better support for colocated branches and
31 bzr-pipeline. (Aaron Bentley)
32
29Improvements33Improvements
30************34************
3135
@@ -38,7 +42,7 @@
3842
39* Branching, merging and pulling a branch now copies revisions named in43* Branching, merging and pulling a branch now copies revisions named in
40 tags, not just the tag metadata. (Andrew Bennetts, #309682)44 tags, not just the tag metadata. (Andrew Bennetts, #309682)
41 45
42* ``bzr cat-revision`` no longer requires a working tree. (Jelmer Vernooij, #704405)46* ``bzr cat-revision`` no longer requires a working tree. (Jelmer Vernooij, #704405)
4347
44Bug Fixes48Bug Fixes