bzr-builder

Merge lp:~rom1-chal/bzr-builder/reporting_conflict_from_merge into lp:bzr-builder

  1. reporting_conflict_from_merge
  2. Merge into trunk
Proposed by Romain Chalumeau
Status: Work in progress
Proposed branch: lp:~rom1-chal/bzr-builder/reporting_conflict_from_merge
Merge into: lp:bzr-builder
Diff against target: 125 lines (+69/-5)
2 files modified
recipe.py (+64/-2)
tests/test_recipe.py (+5/-3)
To merge this branch: bzr merge lp:~rom1-chal/bzr-builder/reporting_conflict_from_merge
Reviewer Review Type Date Requested Status
Jelmer Vernooij (community) code Needs Fixing
James Westby Pending
Review via email: mp+30127@code.launchpad.net

Description of the change

Hi,

I am release manager for a company with many developers.
I massively use recipes to manage my integration plans of feature branches.

My concern is about reporting on merges producing conflicts.

As a release mgr, I have to know :
 - which feature branch has introduced the conflict
 - and who are the developers that are concerned by the conflicted modifications

To do so, i have written the ConflictsFromMerge exception which can be raised by merge_branch (instead of BzrCommandError).
The exception gets in parameter the ChildBranch.url which has stopped the build.
To detect the developers, I parse conflicted parts on the annotated conflicted files.

Very useful for my integration team and myself.

The tests "test_build_tree_merged_with_conflicts" and "test_pull_or_branch_pull_with_conflicts" have been updated to take into account the

Rgds/Romain

To post a comment you must log in.
Revision history for this message
James Westby (james-w) wrote :

Hi,

Thanks for this, it looks great.

I have a few formatting things that I would like to clean up when I merge,
but they are minor.

The thing that is stopping me doing it right now is that I'm not sure how
generally applicable the output is. I'm trying to think of things that would
both be widely useful, and still give you the information that you need.

Perhaps not showing the developers, but the revisions (like you would get from
bzr log) for each revision involved in the conflict would work?

I'm going to ponder this over the weekend, and we can get it landed next week.

Thanks,

James

Reply
Revision history for this message
Romain Chalumeau (rom1-chal) wrote :

Le vendredi 16 juillet 2010 à 20:10 +0000, James Westby a écrit :
> Hi,
>
 Hi James

> Thanks for this, it looks great.

Thx
>
> I have a few formatting things that I would like to clean up when I merge,
> but they are minor.
>
> The thing that is stopping me doing it right now is that I'm not sure how
> generally applicable the output is. I'm trying to think of things that would
> both be widely useful, and still give you the information that you need.
>
> Perhaps not showing the developers, but the revisions (like you would get from
> bzr log) for each revision involved in the conflict would work?
>

I agree. So far, it is a helper for my integration team.
To explain the context, the development teams wants to go towards
continuous integration (of course in a single branch).
As a release manager, i want feature branches.

The bzr-builder helped making coexist the two methods.
I use daily crons based on recipes to build the integration or release
branch and send it to hudson.
With conflicts, I catch the merge exceptions and put the related msgs in
emails sent only to the targetted developers. It helps me avoiding to
spam a mailing where the emails are not read anymore. That's the reason
why i need the developers concerned by a conflict.

I agree that returning the conflicted revisions objects would be more
flexible to generate custom msg or filter the information.

Yet, the other piece of information i need to communicate is the
branches that are concerned by the conflict so that the developers can
easily reproduce the conflict locally. I did not find a "sexy" way of
listing the branches from a shared repository that contain such or such
revision. As the recipe contains the urls, i add it to the exception
msg.

> I'm going to ponder this over the weekend, and we can get it landed next week.
>
> Thanks,

Thx to you
Romain
>
> James
>
>
>

--

Reply
Revision history for this message
James Westby (james-w) wrote :

On Mon, 19 Jul 2010 08:03:58 -0000, Romain Chalumeau <email address hidden> wrote:
> I agree. So far, it is a helper for my integration team.
> To explain the context, the development teams wants to go towards
> continuous integration (of course in a single branch).
> As a release manager, i want feature branches.
>
> The bzr-builder helped making coexist the two methods.
> I use daily crons based on recipes to build the integration or release
> branch and send it to hudson.
> With conflicts, I catch the merge exceptions and put the related msgs in
> emails sent only to the targetted developers. It helps me avoiding to
> spam a mailing where the emails are not read anymore. That's the reason
> why i need the developers concerned by a conflict.

Interesting approach.

> I agree that returning the conflicted revisions objects would be more
> flexible to generate custom msg or filter the information.
>
> Yet, the other piece of information i need to communicate is the
> branches that are concerned by the conflict so that the developers can
> easily reproduce the conflict locally. I did not find a "sexy" way of
> listing the branches from a shared repository that contain such or such
> revision. As the recipe contains the urls, i add it to the exception
> msg.

I'm perfectly happy to have the urls in there, it's just including the
list of developers that I find a little specific.

Thanks,

James

Reply
Revision history for this message
Romain Chalumeau (rom1-chal) wrote :

OK, i understand.

What do you think of the approach of getting in the exception an array
of revision objects in conflict. In these objects, we can find any
information necessary to build a reporting on the conflict status
(revno, revid, dates, files, developers, etc...)

As the exception is typed, i can easily extend the build command to
catch the branch and revisions in conflict, and so customize my
reporting. The bzr-builder can keep a standard exception msg.

For a future feature, I was thinking about a report template in
a .bazaar directory for instance. If none is present, the standard error
is reported. If a template is present, it replaces the values for the
revisions in conflict.

Keywords might be something like :
revfrom_no / revto_no, revfrom_author / revto_author, etc...

An option --email-report may exist to automatically to email the report
to the one launching the command and the concerned developers.

Rgds/Romain

>
> Thanks,
>
> James

--

Reply
Revision history for this message
Martin Pool (mbp) wrote :

On 19 July 2010 18:43, Romain Chalumeau <email address hidden> wrote:
> OK, i understand.
>
> What do you think of the approach of getting in the exception an array
> of revision objects in conflict. In these objects, we can find any
> information necessary to build a reporting on the conflict status
> (revno, revid, dates, files, developers, etc...)
>
> As the exception is typed, i can easily extend the build command to
> catch the branch and revisions in conflict, and so customize my
> reporting. The bzr-builder can keep a standard exception msg.
>
>
>
>
> For a future feature, I was thinking about a report template in
> a .bazaar directory for instance. If none is present, the standard error
> is reported. If a template is present, it replaces the values for the
> revisions in conflict.
>
> Keywords might be something like :
> revfrom_no / revto_no, revfrom_author / revto_author, etc...

I think this basic concept would be useful to have within bzr itself.

> An option --email-report may exist to automatically to email the report
> to the one launching the command and the concerned developers.

I think we should have a post-merge-conflicts hook, which can be
configured in a particular context to send mail to whoever.
--
Martin

Reply
Revision history for this message
Romain Chalumeau (rom1-chal) wrote :

Le lundi 19 juillet 2010 à 19:27 +0000, Martin Pool a écrit :
> On 19 July 2010 18:43, Romain Chalumeau <email address hidden> wrote:
> > For a future feature, I was thinking about a report template in
> > a .bazaar directory for instance. If none is present, the standard error
> > is reported. If a template is present, it replaces the values for the
> > revisions in conflict.
> >
> > Keywords might be something like :
> > revfrom_no / revto_no, revfrom_author / revto_author, etc...
>
> I think this basic concept would be useful to have within bzr itself.
>
> > An option --email-report may exist to automatically to email the report
> > to the one launching the command and the concerned developers.
>
> I think we should have a post-merge-conflicts hook, which can be
> configured in a particular context to send mail to whoever.

Hi,

I really like the idea of post-merge-conflicts.
Would help a lot for reporting with an automation of continuous
merging.

Shall i open a blue print in bzr ?

--

Reply
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

We don't really use blue prints in Bazaar. Perhaps it's a useful topic to bring up on the mailing list ?

I agree with James that it would be nice to get this into Bazaar itself.

Reply
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Voting Needs Fixing, as I think this is something that is more appropriate for the regular bzr merge code in one form or another - it's not really bzr-builder specific.

review: Needs Fixing (code)
Reply

Unmerged revisions

99. By Romain Chalumeau

Tests updates
Take into account the new exceptions raised ConflictsDetected and ConflictsFromMerge
in tests :
test_build_tree_merged_with_conflicts
test_pull_or_branch_pull_with_conflicts

98. By Romain Chalumeau

ConflictsDetected exception

97. By Romain Chalumeau

ConflictFromMerge exception

When a merge directive produces conflicts, a conflictFromMerge exception is raised.
The msg of the exception gives a complete report to ease the integrator solving the conflict ie :
- branch url which causes the conflict
- files in conflict
- authors who have committed the conflicted modifications
- path where to handle the conflicts

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'recipe.py'
2--- recipe.py 2010-07-09 20:51:37 +0000
3+++ recipe.py 2010-07-16 16:32:03 +0000
4@@ -68,6 +68,68 @@
5 % (base_branch.deb_version, available_tokens))
6
7
8+# Class to handle exception when merge produces conflicts
9+# The msg is targetted to give the necessary information to resolve the conflict
10+class ConflictsFromMerge(errors.BzrError) :
11+ _fmt = "Conflicts while merging\n\nThe build had to be stopped due to conflicts detected while merging %(childbranch)s\nYou will find the conflicted files in %(workingdir)s\n\n%(conflicts)s"
12+
13+
14+ def __init__(self, childbranch, tree):
15+ errors.BzrError.__init__(self)
16+ self.childbranch = childbranch
17+ self.tree = tree
18+ self.conflicts = self.analyse_conflicts()
19+ self.workingdir = tree.basedir
20+
21+ def analyse_conflicts(self):
22+
23+ from bzrlib import annotate, osutils, revision
24+ import time
25+
26+ msg = ''
27+ # Init of the current revision : _expand_annotations does not work with None
28+ current_rev = revision.Revision(revision.CURRENT_REVISION)
29+ current_rev.parent_ids = self.tree.get_parent_ids()
30+ current_rev.committer = self.tree.branch.get_config().username()
31+ current_rev.message = "?"
32+ current_rev.timestamp = round(time.time(), 3)
33+ current_rev.timezone = osutils.local_time_offset()
34+
35+ for conflicted_file in self.tree._iter_conflicts() :
36+ # Annotate conflicted files
37+ fileid = self.tree.path2id(conflicted_file)
38+ annotations = list(self.tree.annotate_iter(fileid))
39+ annotation = list(annotate._expand_annotations(annotations, self.tree.branch, current_rev))
40+
41+ in_conflict_part = False
42+ developers = []
43+ # Retrieve the developpers name from the annotation inside the conflict block
44+ for (revno_str, author, date_str, origin, text) in annotation :
45+ if text.startswith( "=======") :
46+ continue
47+ if text.startswith(">>>>>>> MERGE-SOURCE"):
48+ in_conflict_part = False
49+ continue
50+ if in_conflict_part and len(author) > 0 :
51+ developers.append(author)
52+ if text.startswith("<<<<<<< TREE"):
53+ in_conflict_part = True
54+
55+ developers_set = set(developers)
56+ msg += "The conflicts in %s are related to changes from %s \n" % (conflicted_file, ', '.join(developers_set))
57+
58+ msg+= "You may contact them to correctly handle the resolution."
59+ return msg
60+
61+# Unresolved conflicts are still present in the working dir the build is being done
62+class ConflictsDetected(errors.BzrError) :
63+
64+ _fmt = "Unresolved conflicts are still present in the working directory %(workingdir)s. \nPlease resolve those conflicts before relaunching the build on this working dir."
65+
66+ def __init__(self, tree):
67+ errors.BzrError.__init__(self)
68+ self.workingdir = tree.basedir
69+
70 class CommandFailedError(errors.BzrError):
71
72 _fmt = "The command \"%(command)s\" failed."
73@@ -163,7 +225,7 @@
74 conflicts = tree_to.conflicts()
75 if len(conflicts) > 0:
76 # FIXME: better reporting
77- raise errors.BzrCommandError("Conflicts... aborting.")
78+ raise ConflictsDetected(tree_to)
79 except:
80 if created_br_to:
81 br_to.unlock()
82@@ -215,7 +277,7 @@
83 merger.set_pending()
84 if conflict_count:
85 # FIXME: better reporting
86- raise errors.BzrCommandError("Conflicts from merge")
87+ raise ConflictsFromMerge(child_branch.url, tree_to)
88 tree_to.commit("Merge %s" %
89 urlutils.unescape_for_display(
90 child_branch.url, 'utf-8'))
91
92=== modified file 'tests/test_recipe.py'
93--- tests/test_recipe.py 2010-07-09 20:51:37 +0000
94+++ tests/test_recipe.py 2010-07-16 16:32:03 +0000
95@@ -34,6 +34,8 @@
96 RecipeParser,
97 RecipeBranch,
98 RecipeParseError,
99+ ConflictsFromMerge,
100+ ConflictsDetected,
101 resolve_revisions,
102 RUN_INSTRUCTION,
103 )
104@@ -550,7 +552,7 @@
105 base_branch = BaseRecipeBranch("source1", "1", 0.2)
106 merged_branch = RecipeBranch("merged", "source2")
107 base_branch.merge_branch(merged_branch)
108- self.assertRaises(errors.BzrCommandError, build_tree,
109+ self.assertRaises(ConflictsFromMerge, build_tree,
110 base_branch, "target")
111 self.failUnlessExists("target")
112 tree = workingtree.WorkingTree.open("target")
113@@ -694,10 +696,10 @@
114 source.add(["b"])
115 rev_id = source.commit("two")
116 source.branch.tags.set_tag("one", rev_id)
117- e = self.assertRaises(errors.BzrCommandError,
118+ e = self.assertRaises(ConflictsDetected,
119 pull_or_branch, tree_to, br_to, source.branch,
120 to_transport, rev_id, accelerator_tree=source)
121- self.assertEqual("Conflicts... aborting.", str(e))
122+ self.assertStartsWith(str(e), "Unresolved conflicts are still present in the working directory")
123 tree_to.unlock()
124 br_to.unlock()
125 tree_to = workingtree.WorkingTree.open("target")

Subscribers

People subscribed via source and target branches