Merge lp:~parthm/bzr/262450 into lp:bzr

Proposed by Parth Malwankar
Status: Merged
Approved by: Martin Pool
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~parthm/bzr/262450
Merge into: lp:bzr
Diff against target: 124 lines (+50/-3)
5 files modified
NEWS (+4/-0)
bzrlib/bzrdir.py (+0/-2)
bzrlib/tests/blackbox/test_upgrade.py (+14/-0)
bzrlib/tests/features.py (+26/-0)
bzrlib/transport/__init__.py (+6/-1)
To merge this branch: bzr merge lp:~parthm/bzr/262450
Reviewer Review Type Date Requested Status
Vincent Ladeuil Needs Fixing
Martin Pool Approve
Review via email: mp+19483@code.launchpad.net

This proposal supersedes a proposal from 2010-02-16.

Commit message

(mbp, for parthm) check permissions on .bzr are copied to backup.bzr

To post a comment you must log in.
Revision history for this message
Parth Malwankar (parthm) wrote : Posted in a previous version of this proposal

Fix for #262450.
To ensure that permissions for .bzr are copied to backup.bzr,
we do a stat on .bzr and pass the mode bits to target.mkdir.

Test modifies permissions for .bzr to be 0700 and ensures that
the permissions for backup.bzr are 0700 after upgrade.

Revision history for this message
Parth Malwankar (parthm) wrote : Posted in a previous version of this proposal

Just wanted to add that I haven't tested this on windows as
I don't have access to a windows machine.

Revision history for this message
Martin Pool (mbp) wrote : Posted in a previous version of this proposal

On 17 February 2010 01:05, Parth Malwankar <email address hidden> wrote:
> Just wanted to add that I haven't tested this on windows as
> I don't have access to a windows machine.

Thanks for noting that.

You can fairly easily set up python to run under Wine on Linux (I
posted or added docs about this before) and then do

winepython ./bzr selftest ...

and it's pretty good at testing this kind of thing.

--
Martin <http://launchpad.net/~mbp/>

Revision history for this message
Martin Pool (mbp) wrote : Posted in a previous version of this proposal

So probably the permissions stuff just can't work like this on Windows, in which case we should probably:

1- make that test requireFeature something unixy; probably it would be good to declare a new Feature for unix permissions and just say it exists if os.platform=posix
2- check the actual runtime code passes on windows (even if it does nothing)
3- file a followon bug about inheriting acls on windows

Revision history for this message
Martin Pool (mbp) wrote : Posted in a previous version of this proposal

This does indeed fail under WinePython.

------------
Traceback (most recent call last):
  File "Z:\home\mbp\lib\python\testtools\runtest.py", line 128, in _run_user
    return fn(*args)
  File "Z:\home\mbp\lib\python\testtools\testcase.py", line 369, in _run_test_method
    testMethod()
  File "Z:\home\mbp\bzr\integration\bzrlib\tests\blackbox\test_upgrade.py", line 157, in test_upgrade_permission_check
    self.assertTrue(new_perms == old_perms)
AssertionError
------------

This will probably clash with your other patch to use numbered backups.

Also just a note for the future, if you want you can do bugfixes based off and proposed back into bzr.2.0. This kind of thing would be good to put into the stable release; indeed it might be worth backporting from your branch.

Let me know if you want a hand with adding the feature stuff to skip on windows.

review: Needs Fixing
Revision history for this message
Parth Malwankar (parthm) wrote :

Based on Martins comments:

>So probably the permissions stuff just can't work like this on Windows, in which case we should probably:
> 1- make that test requireFeature something unixy; probably it would be good to declare a new Feature for unix permissions and just say it exists if os.platform=posix

Added PosixOSFeature and the permissions test case now requireFeatures this.

> 2- check the actual runtime code passes on windows (even if it does nothing)

Ran tests on Window. It would be good if someone could verify this again.

> 3- file a followon bug about inheriting acls on windows

Filed bug #523187 for Windows

Revision history for this message
Parth Malwankar (parthm) wrote :

Not sure it resubmit of the proposal was the right thing to do as the comments remain with the previous proposal. Oh well.

Martin said:

> This will probably clash with your other patch to use numbered backups.

Yes. Depending on the order in which they land I could look into updating it. It should be minor as only backup_dir needs to be updated in one place in the test case.

>Also just a note for the future, if you want you can do bugfixes based off and proposed back into bzr.2.0. This kind of thing would be good to put into the stable release; indeed it might be worth backporting from your branch.

Makes sense. I will look into backporting this fix.
Do we just want to port the permissions fix or should I also look into porting the numbered backup.bzr naming fix?

Revision history for this message
Martin Pool (mbp) wrote :

On 18 February 2010 00:18, Parth Malwankar <email address hidden> wrote:
> Based on Martins comments:
>
>>So probably the permissions stuff just can't work like this on Windows, in which case we should probably:
>> 1- make that test requireFeature something unixy; probably it would be good to declare a new Feature for unix permissions and just say it exists if os.platform=posix
>
> Added PosixOSFeature and the permissions test case now requireFeatures this.

I think, to be a bit pedantic, the actual check is for unix
permissions; this could conceivably pass under cygwin and fail on unix
with a vfat filesystem.

>> 2- check the actual runtime code passes on windows (even if it does nothing)
>
> Ran tests on Window. It would be good if someone could verify this again.

I'll check

>> 3- file a followon bug about inheriting acls on windows
>
> Filed bug #523187 for Windows

thanks
--
Martin <http://launchpad.net/~mbp/>

Revision history for this message
Martin Pool (mbp) wrote :

On 18 February 2010 00:27, Parth Malwankar <email address hidden> wrote:
> Not sure it resubmit of the proposal was the right thing to do as the comments remain with the previous proposal. Oh well.

Generally you don't need to resubmit, just push to your branch.

>
> Martin said:
>
>> This will probably clash with your other patch to use numbered backups.
>
> Yes. Depending on the order in which they land I could look into updating it. It should be minor as only backup_dir needs to be updated in one place in the test case.
>
>>Also just a note for the future, if you want you can do bugfixes based off and proposed back into bzr.2.0.  This kind of thing would be good to put into the stable release; indeed it might be worth backporting from your branch.
>
> Makes sense. I will look into backporting this fix.
> Do we just want to port the permissions fix or should I also look into porting the numbered backup.bzr naming fix?

Well, they're both bug fixes, but I would say that the permissions one
is more clearly just a bug fix, whereas the numbering one introduces
more of a permissions change. So I would backport only the first.

--
Martin <http://launchpad.net/~mbp/>

Revision history for this message
Parth Malwankar (parthm) wrote :

> On 18 February 2010 00:18, Parth Malwankar <email address hidden> wrote:
> > Based on Martins comments:
> >
> >>So probably the permissions stuff just can't work like this on Windows, in
> which case we should probably:
> >> 1- make that test requireFeature something unixy; probably it would be good
> to declare a new Feature for unix permissions and just say it exists if
> os.platform=posix
> >
> > Added PosixOSFeature and the permissions test case now requireFeatures this.
>
> I think, to be a bit pedantic, the actual check is for unix
> permissions; this could conceivably pass under cygwin and fail on unix
> with a vfat filesystem.
>

Added feature PosixPermissionsFeature which specifically checks for file
permissions support (by create-change perms-verify perms sequence) if
os.name is posix.
Tests are passing on Windows (a borrowed system) and Linux. Windows skips the
test as expected.
Haven't been able to verify on Cygwin or OSX and I don't have those systems.

Removed PosixOSFeature added earlier.

--
Parth

Revision history for this message
Martin Pool (mbp) wrote :

lovely.

I might move the new feature into features.py and mention it in the testing guide as I merge it.

review: Approve
Revision history for this message
Martin Pool (mbp) wrote :

backport to 2.0 sent to pqm

Revision history for this message
Martin Pool (mbp) wrote :

this fails with

^^^^[log from bzrlib.tests.per_transport.TransportTests.test_copy_tree(FTPTestServer)]
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/tests/per_transport.py", line 1442, in test_copy_tree
   transport.copy_tree('from', 'to')
 File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/transport/__init__.py", line 1063, in copy_tree
   source.copy_tree_to_transport(target)
 File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/transport/__init__.py", line 1077, in copy_tree_to_transport
   to_transport.mkdir(dir)
 File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/transport/ftp/__init__.py", line 362, in mkdir
   unknown_exc=errors.FileExists)
 File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/transport/ftp/__init__.py", line 200, in _translate_perm_error
   raise unknown_exc(path, extra=extra)
FileExists: File exists: '/to/dir': 550 error creating directory: Permission denied

Parth, let me know if you want help with it.

Revision history for this message
Parth Malwankar (parthm) wrote :

> this fails with
>
> ^^^^[log from
> bzrlib.tests.per_transport.TransportTests.test_copy_tree(FTPTestServer)]
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/tests/per_transport.py", line
> 1442, in test_copy_tree
> transport.copy_tree('from', 'to')
> File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/transport/__init__.py", line
> 1063, in copy_tree
> source.copy_tree_to_transport(target)
> File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/transport/__init__.py", line
> 1077, in copy_tree_to_transport
> to_transport.mkdir(dir)
> File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/transport/ftp/__init__.py",
> line 362, in mkdir
> unknown_exc=errors.FileExists)
> File "/home/pqm/bzr-pqm-workdir/home/2.0/bzrlib/transport/ftp/__init__.py",
> line 200, in _translate_perm_error
> raise unknown_exc(path, extra=extra)
> FileExists: File exists: '/to/dir': 550 error creating directory: Permission
> denied
>
>
> Parth, let me know if you want help with it.

Fixed.
copy_tree updated to consider only rwx bits during copy i.e. target.mkdir('.', stat.st_mode & 0777)

Revision history for this message
Vincent Ladeuil (vila) wrote :

15 +* ``bzr upgrade`` now creates the ``backup.bzr`` directory with the same

Sorts before:

9 * ``bzr upgrade`` now names backup directory as ``backup.bzr.~N~`` instead

that should address the conflict.

45 +class _PosixPermissionsFeature(Feature):

I think we put new features into bzrlib.test.features instead.

88 + import os, stat

Generally, for tests, we put imports at the top of the file.

109 + stat = self.stat(from_relpath)
110 + target.mkdir('.', stat.st_mode & 0777)

A comment would not hurt :)

review: Needs Fixing
Revision history for this message
Parth Malwankar (parthm) wrote :

> 15 +* ``bzr upgrade`` now creates the ``backup.bzr`` directory with the
> same
>
> Sorts before:
>
> 9 * ``bzr upgrade`` now names backup directory as ``backup.bzr.~N~``
> instead
>
> that should address the conflict.
>

Fixed. I merged the trunk and resolved this.

> 45 +class _PosixPermissionsFeature(Feature):
>
> I think we put new features into bzrlib.test.features instead.
>
> 88 + import os, stat
>
> Generally, for tests, we put imports at the top of the file.
>
> 109 + stat = self.stat(from_relpath)
> 110 + target.mkdir('.', stat.st_mode & 0777)
>
> A comment would not hurt :)

Done.
Thanks for the review comments Vincent.

Revision history for this message
Martin Pool (mbp) wrote :
Revision history for this message
Martin Pool (mbp) wrote :

This fails in pqm in a blackbox.test_upgrade test because of a clash with the change of backup directory name:

Traceback (most recent call last):
 File "/usr/lib/python2.4/site-packages/testtools/runtest.py", line 128, in _run_user
   return fn(*args)
 File "/usr/lib/python2.4/site-packages/testtools/testcase.py", line 368, in _run_test_method
   testMethod()
 File "/home/pqm/bzr-pqm-workdir/home/+trunk/bzrlib/tests/blackbox/test_upgrade.py", line 160, in test_upgrade_permission_check
   new_perms = os.stat(backup_dir).st_mode & 0777
OSError: [Errno 2] No such file or directory: 'backup.bzr'
------------

Revision history for this message
Parth Malwankar (parthm) wrote :

> OSError: [Errno 2] No such file or directory: 'backup.bzr'
> ------------

Fixed.

Revision history for this message
Martin Pool (mbp) wrote :

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'NEWS'
--- NEWS 2010-03-12 19:59:50 +0000
+++ NEWS 2010-03-17 05:39:30 +0000
@@ -95,6 +95,10 @@
95 the kept file on content conflicts where one side deleted the file.95 the kept file on content conflicts where one side deleted the file.
96 (Vincent Ladeuil, #529968)96 (Vincent Ladeuil, #529968)
9797
98* ``bzr upgrade`` now creates the ``backup.bzr`` directory with the same
99 permissions as ``.bzr`` directory on a POSIX OS.
100 (Parth Malwankar, #262450)
101
98* ``bzr upgrade`` now names backup directory as ``backup.bzr.~N~`` instead102* ``bzr upgrade`` now names backup directory as ``backup.bzr.~N~`` instead
99 of ``backup.bzr``. This directory is ignored by bzr commands such as103 of ``backup.bzr``. This directory is ignored by bzr commands such as
100 ``add``.104 ``add``.
101105
=== modified file 'bzrlib/bzrdir.py'
--- bzrlib/bzrdir.py 2010-03-11 11:07:32 +0000
+++ bzrlib/bzrdir.py 2010-03-17 05:39:30 +0000
@@ -602,8 +602,6 @@
602 # already exists, but it should instead either remove it or make602 # already exists, but it should instead either remove it or make
603 # a new backup directory.603 # a new backup directory.
604 #604 #
605 # FIXME: bug 262450 -- the backup directory should have the same
606 # permissions as the .bzr directory (probably a bug in copy_tree)
607 old_path = self.root_transport.abspath('.bzr')605 old_path = self.root_transport.abspath('.bzr')
608 new_path = self.root_transport.abspath(backup_dir)606 new_path = self.root_transport.abspath(backup_dir)
609 ui.ui_factory.note('making backup of %s\n to %s' % (old_path, new_path,))607 ui.ui_factory.note('making backup of %s\n to %s' % (old_path, new_path,))
610608
=== modified file 'bzrlib/tests/blackbox/test_upgrade.py'
--- bzrlib/tests/blackbox/test_upgrade.py 2010-02-15 11:28:35 +0000
+++ bzrlib/tests/blackbox/test_upgrade.py 2010-03-17 05:39:30 +0000
@@ -15,12 +15,15 @@
15# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA15# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
1616
17"""Black box tests for the upgrade ui."""17"""Black box tests for the upgrade ui."""
18import os
19import stat
1820
19from bzrlib import (21from bzrlib import (
20 bzrdir,22 bzrdir,
21 repository,23 repository,
22 )24 )
23from bzrlib.tests import (25from bzrlib.tests import (
26 features,
24 TestCaseInTempDir,27 TestCaseInTempDir,
25 TestCaseWithTransport,28 TestCaseWithTransport,
26 )29 )
@@ -146,6 +149,17 @@
146 self.run_bzr('init-repository --format=metaweave repo')149 self.run_bzr('init-repository --format=metaweave repo')
147 self.run_bzr('upgrade --format=knit repo')150 self.run_bzr('upgrade --format=knit repo')
148151
152 def test_upgrade_permission_check(self):
153 """'backup.bzr' should retain permissions of .bzr. Bug #262450"""
154 self.requireFeature(features.PosixPermissionsFeature)
155 old_perms = stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
156 backup_dir = 'backup.bzr.~1~'
157 self.run_bzr('init --format=1.6')
158 os.chmod('.bzr', old_perms)
159 self.run_bzr('upgrade')
160 new_perms = os.stat(backup_dir).st_mode & 0777
161 self.assertTrue(new_perms == old_perms)
162
149163
150 def test_upgrade_with_existing_backup_dir(self):164 def test_upgrade_with_existing_backup_dir(self):
151 self.make_format_5_branch()165 self.make_format_5_branch()
152166
=== modified file 'bzrlib/tests/features.py'
--- bzrlib/tests/features.py 2010-02-23 07:43:11 +0000
+++ bzrlib/tests/features.py 2010-03-17 05:39:30 +0000
@@ -14,6 +14,8 @@
14# along with this program; if not, write to the Free Software14# along with this program; if not, write to the Free Software
15# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA15# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
1616
17import os
18import stat
1719
18from bzrlib import tests20from bzrlib import tests
19from bzrlib.symbol_versioning import deprecated_in21from bzrlib.symbol_versioning import deprecated_in
@@ -23,3 +25,27 @@
23paramiko = tests.ModuleAvailableFeature('paramiko')25paramiko = tests.ModuleAvailableFeature('paramiko')
24pycurl = tests.ModuleAvailableFeature('pycurl')26pycurl = tests.ModuleAvailableFeature('pycurl')
25subunit = tests.ModuleAvailableFeature('subunit')27subunit = tests.ModuleAvailableFeature('subunit')
28
29class _PosixPermissionsFeature(tests.Feature):
30
31 def _probe(self):
32 def has_perms():
33 # create temporary file and check if specified perms are maintained.
34 import tempfile
35
36 write_perms = stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
37 f = tempfile.mkstemp(prefix='bzr_perms_chk_')
38 fd, name = f
39 os.close(fd)
40 os.chmod(name, write_perms)
41
42 read_perms = os.stat(name).st_mode & 0777
43 os.unlink(name)
44 return (write_perms == read_perms)
45
46 return (os.name == 'posix') and has_perms()
47
48 def feature_name(self):
49 return 'POSIX permissions support'
50
51PosixPermissionsFeature = _PosixPermissionsFeature()
2652
=== modified file 'bzrlib/transport/__init__.py'
--- bzrlib/transport/__init__.py 2010-03-05 05:30:19 +0000
+++ bzrlib/transport/__init__.py 2010-03-17 05:39:30 +0000
@@ -1060,7 +1060,12 @@
1060 """1060 """
1061 source = self.clone(from_relpath)1061 source = self.clone(from_relpath)
1062 target = self.clone(to_relpath)1062 target = self.clone(to_relpath)
1063 target.mkdir('.')1063
1064 # create target directory with the same rwx bits as source.
1065 # use mask to ensure that bits other than rwx are ignored.
1066 stat = self.stat(from_relpath)
1067 target.mkdir('.', stat.st_mode & 0777)
1068
1064 source.copy_tree_to_transport(target)1069 source.copy_tree_to_transport(target)
10651070
1066 def copy_tree_to_transport(self, to_transport):1071 def copy_tree_to_transport(self, to_transport):