Launchpad itself

~wgrant/launchpad:buildd-manager-nicer-retries

Git
lp:~wgrant/launchpad
buildd-manager-nicer-retries

Last commit made on 2023-10-29

Get this branch:: git clone -b buildd-manager-nicer-retries https://git.launchpad.net/~wgrant/launchpad

Only William Grant can upload to this branch. If you are William Grant please log in for upload directions.

Branch merges

Merged into launchpad:master

Colin Watson (community): Approve on 2023-10-27

Branch information

Name:: buildd-manager-nicer-retries

Repository:: lp:~wgrant/launchpad

Recent commits

272d66a... by William Grant on 2023-10-27

Cope more gracefully with intermittent builder glitches

buildd-manager would previously immediately count any single scan
failure against the builder and job. This meant that three glitches --
say, network timeouts -- over the course of job would result in the
build being requeued. A builder's failure count is reset on successful
dispatch, but a job's deliberately isn't since we want to fail builds
that are repeatedly killing builders. This meant that a single network
glitch in the second attempt at a build would cause it to be failed.

This added layer of failure counting substantially reduces the
likelihood of those two scenarios, by requiring five consecutive
unsuccessful scans before a single failure is counted against a builder
or job. This means that brief network interruptions, or indeed temporary
insanity on buildd-manager's part, should no longer cause builds to be
requeued or failed at all.

The only significant downside of this change is that recovery from
legitimate failures will now take a few minutes longer. But that's much
less of a concern with the very large build farm we have nowadays.