bzr-builddeb

unversioned executability issue, perhaps in builddeb

Bug #588060 reported by Monty Taylor on 2010-05-31

This bug affects 2 people

Affects		Status	Importance	Assigned to	Milestone
	bzr-builddeb	Fix Released	High	Unassigned

Bug Description

Trying to do bzr merge-upstream, I got this:

bzr: ERROR: Tree transform is malformed [('unversioned executability', 'new-130')]

Mon 2010-05-31 14:33:49 -0700
0.023 bazaar version: 2.1.1
0.023 bzr arguments: [u'merge-upstream', u'../pandora-build/pandora-build-0.131.6.tar.gz', u'--version=0.131.6']
0.032 looking for plugins in /home/mordred/.bazaar/plugins
0.105 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.106 Plugin name builddeb already loaded
0.140 encoding stdout as sys.stdout encoding 'UTF-8'
0.157 opening working tree '/home/mordred/src/pandora-build/debian'
0.193 Using 'debian/changelog' to get package information
[19833] 2010-05-31 14:33:49.792 INFO: Using distribution unstable
0.199 Using .. for orig-dir, taken from /home/mordred/.bazaar/builddeb.conf
0.214 opening working tree '/home/mordred/src/pandora-build/debian'
0.229 creating branch <bzrlib.branch.BzrBranchFormat7 object at 0x23fbe50> in file:///home/mordred/src/pandora-build/tmprKoLWK/upstream/.bzr/
0.262 created new branch BzrBranch7('file:///home/mordred/src/pandora-build/tmprKoLWK/upstream/')
0.271 trying to create missing lock '/home/mordred/src/pandora-build/tmprKoLWK/upstream/.bzr/checkout/dirstate'
0.271 opening working tree '/home/mordred/src/pandora-build/tmprKoLWK/upstream'
0.379 opening working tree '/home/mordred/src/pandora-build/tmprKoLWK/upstream'
0.400 Importing upstream version 0.131.6 from /tmp/tmpRg_1WZ with parents ['<email address hidden>']
0.668 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 853, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1055, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 661, in run_argv_aliases
    return self.run_direct(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 665, in run_direct
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/home/mordred/.bazaar/plugins/builddeb/cmds.py", line 620, in run
    merge_type=merge_type, force=force)
  File "/home/mordred/.bazaar/plugins/builddeb/import_dsc.py", line 1417, in merge_upstream
    upstream_revision=upstream_revision)
  File "/home/mordred/.bazaar/plugins/builddeb/import_dsc.py", line 911, in import_upstream
    file_ids_from=upstream_trees + [self.tree])
  File "/home/mordred/.bazaar/plugins/builddeb/bzrtools_import.py", line 198, in import_dir
    import_archive(tree, dir_file, file_ids_from=file_ids_from)
  File "/home/mordred/.bazaar/plugins/builddeb/bzrtools_import.py", line 296, in import_archive
    for conflict in cook_conflicts(resolve_conflicts(tt), tt):
  File "/usr/lib/python2.6/dist-packages/bzrlib/transform.py", line 2769, in resolve_conflicts
    raise MalformedTransform(conflicts=conflicts)
MalformedTransform: Tree transform is malformed [('unversioned executability', 'new-130')]

0.669 Transferred: 0KiB (0.0K/s r:0K w:0K)
0.669 return code 3

Related branches

lp:~lifeless/bzr-builddeb/trunk

Merged into lp:bzr-builddeb at revision 463

Bzr-builddeb-hackers: Pending requested 2010-07-04

Merged into lp:~bzr-builddeb-hackers/bzr-builddeb/trunk-old

James Westby: Approve on 2009-12-01

lp:ubuntu/oneiric/bzr-builddeb

Revision history for this message

Monty Taylor (mordred) wrote on 2010-05-31:

From some experimenting, I was able to isolate this cause for me:

<mtaylor> lifeless: I figured out the cause at least - in the tree I had moved a file, license.py, to internal/licensing.py and then added a new file called license.py
<mtaylor> lifeless: this caused merge upstream to become confused
<mtaylor> lifeless: if I re-expressed that in the upstream bzr tree as adding a new file internal/licesnsing.py and modifying license.py with the new contents - all worked fine

So I'm guessing it has to do with move/rename tracking and tarball importing perhaps.

Revision history for this message

Robert Collins (lifeless) wrote on 2010-05-31: Re: [Bug 588060] Re: unversioned executability issue, perhaps in builddeb

tarball imports aren't currently correlating with upstream deltas to
do rename detection, at a guess.

Revision history for this message

James Westby (james-w) wrote on 2010-06-01:

Hi,

Thanks for the bug report. I have seen the same error in "import-dsc" too.

Rob is correct that rename detection isn't currently done, but I don't think that
should be the only fix. I reverted the change to do it in import-dsc because it
caused problems with the importer service. I don't remember the details right
now, but it may be that the same problems will affect merge-upstream, at least
in some cases.

My proposal for fixing this is:

* Stop bzr-builddeb crashing when doing these operations.
* Evaluate rename detection in addition to the above fix.

I have reassigned the bug to bzr-builddeb pending investigation.

Thanks,

James

Changed in bzr:
status:	New → Triaged
importance:	Undecided → High
affects:	bzr → bzr-builddeb

Revision history for this message

Robert Collins (lifeless) wrote on 2010-07-04:

This conflict is added when find_conflicts runs, which finds a executability assignment (which can be False or True), for a transaction id which has no final name.

It would be nice to have a trace-mode for transform, to see how it gets where it gets to.

The poking continues.

Revision history for this message

Robert Collins (lifeless) wrote on 2010-07-04:

Ok, so the testtools change is
R utils.py -> compat.py
N utils.py

the exec change which is blowing up is on utils.py - I think its the new file.

Revision history for this message

Robert Collins (lifeless) wrote on 2010-07-04:

Interestingly,

self.tree_file_id(trans_id)
'utils.py-20080717114508-zreytll5ao80u8kp-2'

Revision history for this message

Robert Collins (lifeless) wrote on 2010-07-04:

(Sorry for all the comment spam, trying to capture everything relevant).

That fileid is the original fileid, which we would expect as rename tracking isn't in use at this point.

And its definitely blowing up on the tar, not later.

(Pdb) print file_ids_from[0].path2id('testtools/utils.py')
utils.py-20100622232940-0binq278qxebanch-1

So the new id is known.

(Pdb) print removed.difference(added)
set([])
(Pdb) print added.difference(removed)
set(['.bzrignore', 'testtools/tests/test_compat.py', 'testtools/compat.py'])

I wonder perhaps if the file-id-from code is effectively moving utils by assigning the file id to compat.py?

yes - this is the issue:

(Pdb) self._duplicate_ids()
[('duplicate id', 'new-33', 'new-35')]

There is a duplicate file id conflict occuring. What we want to do in this case, I think, is to make use of the tree we're importing from, and get a delta from that to start with.

I'm going to unblock myself right now by hacking around this in this tree (adding a rename call at the top :)), but I think there is enough info in this bug to write an automated test now, and to reason about how to fix it.

Revision history for this message

Robert Collins (lifeless) wrote on 2010-07-04:

Ok, so here is what happens:
import deletes the content (not file ids) of everything.
Then it adds all the content again.
Then it selects the file ids to use for things that were not readded.

This is pretty clever, but not helpful for renames.

I've made the file id look up happen earlier, and it accurately handles the rename + add case I'm facing. You might like to try my branch.

James Westby (james-w) on 2010-07-29

Changed in bzr-builddeb:
status:	Triaged → Fix Committed

James Westby (james-w) on 2010-09-14

Changed in bzr-builddeb:
status:	Fix Committed → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.