zsync meta file generation overwrites .zsync files from previous builds

Might be transient due to multiple calls to ports-daily-live for different arches; let's see if it happens again

It's a nice theory, but no, it happened again with the ubuntu daily-live builds for 20090829:

$ wget -q http://cdimages.ubuntu.com/daily-live/20090828/karmic-desktop-i386.iso.zsync -O - | sha1sum
2ffe48275af51debc3acadd536a095c63a53367b -
$ wget -q http://cdimages.ubuntu.com/daily-live/20090829/karmic-desktop-i386.iso.zsync -O - | sha1sum
2ffe48275af51debc3acadd536a095c63a53367b -

Note that the sha1sum of 20090828's daily-live zsync file 17 hours ago when I filed the bug was 3eb8447edf4f701988381a756f309ff24e28793e as reported above.

(If you can point me at source, I'd be happy to look to see if I can figure out what's going on.)

brz branch http://people.canonical.com/~cjwatson/bzr/cdimage/mainline/

Not sure it has all the deployed bits, but I think it does

Loic, thanks for the link. Based on reading the publish-daily script and confirming with rsync, each iso's daily zsync files are hardlinks of each other, e.g.:

$ rsync -q -avpP --links --hard-links --exclude daily-live/*/*.iso rsync://cdimages.ubuntu.com/cdimage/daily-live .
$ ls -li daily-live/*/*.zsync | sort -n
14098466 -rw-rw-r-- 4 steve steve 1439955 2009-08-30 00:12 daily-live/20090829.1/karmic-desktop-amd64.iso.zsync
14098466 -rw-rw-r-- 4 steve steve 1439955 2009-08-30 00:12 daily-live/20090829.2/karmic-desktop-amd64.iso.zsync
14098466 -rw-rw-r-- 4 steve steve 1439955 2009-08-30 00:12 daily-live/20090829/karmic-desktop-amd64.iso.zsync
14098466 -rw-rw-r-- 4 steve steve 1439955 2009-08-30 00:12 daily-live/20090830/karmic-desktop-amd64.iso.zsync
14098466 -rw-rw-r-- 4 steve steve 1439955 2009-08-30 00:12 daily-live/current/karmic-desktop-amd64.iso.zsync

14098471 -rw-rw-r-- 4 steve steve 1435825 2009-08-30 00:12 daily-live/20090829.1/karmic-desktop-i386.iso.zsync
14098471 -rw-rw-r-- 4 steve steve 1435825 2009-08-30 00:12 daily-live/20090829.2/karmic-desktop-i386.iso.zsync
14098471 -rw-rw-r-- 4 steve steve 1435825 2009-08-30 00:12 daily-live/20090829/karmic-desktop-i386.iso.zsync
14098471 -rw-rw-r-- 4 steve steve 1435825 2009-08-30 00:12 daily-live/20090830/karmic-desktop-i386.iso.zsync
14098471 -rw-rw-r-- 4 steve steve 1435825 2009-08-30 00:12 daily-live/current/karmic-desktop-i386.iso.zsync

(The rsync command I listed skips the isos, so is okay to perform on a relatively slow link.) Experimenting with zsyncmake shows that if the zsync meta file already exists, it merely opens and writes over the prior one; it doesn't delete it and create a new one, even if teh date it's creating the meta file for has changed. Thus, it looks like publish-daily copies via a hardlink the previous builds zsync file (based on lines 108-114 of the version published above) to the directory it's going to build in, builds it, and the rsync to the archive dutifuly keeps those hardlinks intact.

I think that the following change (entirely untested) ought to work around the issue:

=== modified file 'bin/publish-daily'
--- bin/publish-daily 2009-08-07 16:21:27 +0000
+++ bin/publish-daily 2009-08-31 03:14:55 +0000
@@ -235,6 +235,7 @@

        if which zsyncmake >/dev/null 2>&1; then
                echo "Making source $i zsync metafile ..."
+ rm -f "$PUBLISH/source/$DIST-src-$i.iso.zsync"
                zsyncmake -o "$PUBLISH/source/$DIST-src-$i.iso.zsync" -u "$DIST-src-$i.iso" \
                          "$PUBLISH/source/$DIST-src-$i.iso"
        fi

as I don't see any way to tell zsyncmake to automatically create a new meta file even there's a currently existing one.

Thanks!

That only fixes one call to zsyncmake for source isos.

Based on your patch, I have committed the following:

--- bin/publish-daily 2009-08-07 16:21:27 +0000
+++ bin/publish-daily 2009-08-31 05:02:13 +0000
@@ -183,6 +183,7 @@
   # zsync metafiles
   if which zsyncmake >/dev/null 2>&1; then
    echo "Making $arch zsync metafile ..."
+ rm -f "$PUBLISH/$DIST-$TYPE-$arch.iso.zsync"
    zsyncmake -o "$PUBLISH/$DIST-$TYPE-$arch.iso.zsync" \
       -u "$DIST-$TYPE-$arch.iso" \
       "$PUBLISH/$DIST-$TYPE-$arch.iso"
@@ -235,6 +236,7 @@

  if which zsyncmake >/dev/null 2>&1; then
   echo "Making source $i zsync metafile ..."
+ rm -f "$PUBLISH/$DIST-src-$i.iso.zsync"
   zsyncmake -o "$PUBLISH/source/$DIST-src-$i.iso.zsync" -u "$DIST-src-$i.iso" \
      "$PUBLISH/source/$DIST-src-$i.iso"
  fi

And picking on UNR, I have tested it by re-building todays daily:

antimony$ sha1sum 20090831/karmic-netbook-remix-i386.iso.zsync
7e35e7d02f934af3f72649e4411475b284c3fc83 20090831/karmic-netbook-remix-i386.iso.zsync
antimony$ sha1sum 20090831.1/karmic-netbook-remix-i386.iso.zsync
40d1d2283a62555b3bf0f12ce985531350730a71 20090831.1/karmic-netbook-remix-i386.iso.zsync