fdatasync slows the test suite down on PQM

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/08/2011 08:54 AM, Vincent Ladeuil wrote:
> ** Changed in: bzr/2.4
> Status: In Progress => Fix Released
>
> ** Changed in: bzr
> Milestone: None => 2.5b1
>
> ** Changed in: bzr
> Status: In Progress => Fix Released
>

I know the patch for this has landed, but I haven't done any inspection
to see that the test suite is actually faster now. Have you?

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5o/50ACgkQJdeBCYSNAAMJlwCfQqyI5xalK8Niv0p39jEGhRqq
hrwAnAl2gy5NvnnRUTa3gdW5v1EsaO4K
=OxWn
-----END PGP SIGNATURE-----

Sure. Down to ~45 mins on trunk as well as 2.4.

But we've moved to a new host (the log files are not rsync'ed yet but should be soon) so I don't know the times there.

I also asked for /tmp to be mounted as tmpfs so if could to that followup about running *some* fsync tests, that would be great.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/08/2011 09:31 PM, Vincent Ladeuil wrote:
> Sure. Down to ~45 mins on trunk as well as 2.4.
>
> But we've moved to a new host (the log files are not rsync'ed yet but
> should be soon) so I don't know the times there.
>
> I also asked for /tmp to be mounted as tmpfs so if could to that
> followup about running *some* fsync tests, that would be great.
>

Also note that U1 was moved off of that machine as well, so we would be
conflicting less with what they were doing. In their case, it was much
worse than a slow suite, though. Trial defaults to timing out if a test
takes to long, which meant something like 8/10 runs were failing spuriously.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5p27UACgkQJdeBCYSNAAO+iwCbB3n8Rhg2mdlMdjoc31XDodjW
xscAoKMOinFws18U4M0AAtTDNJ+gmQKG
=6z8b
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/08/2011 09:31 PM, Vincent Ladeuil wrote:
> Sure. Down to ~45 mins on trunk as well as 2.4.
>
> But we've moved to a new host (the log files are not rsync'ed yet but
> should be soon) so I don't know the times there.
>
> I also asked for /tmp to be mounted as tmpfs so if could to that
> followup about running *some* fsync tests, that would be great.
>

Well, I can say that a recent submission has:
 https://code.launchpad.net/~gagern/bzr/bug842993-reconfigure/+merge/74613

Has a "sent to pqm by email" at 9:29 and a "Approved => Merged" at
09:54. I think there is a fair amount of extra overhead involved, like
actually routing the email, running the branch scanner, and updating the
MP. But I think it certainly puts an upper bound at 25 minutes. Which is
quite nice.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5p6y8ACgkQJdeBCYSNAAMEQwCgghOKDI07hmagM4+kFYc/BfAi
0BwAn16K2oT2c4zcg/jSdXTrELJxkq+/
=sC2q
-----END PGP SIGNATURE-----

On Tue, 30 Aug 2011 10:44:06 -0000, John A Meinel <email address hidden> wrote:
> I tried a patch that just made 'osutils.fdatasync' a no-op and it
> dropped the run time to about 1.5hours. This isn't sufficient to restore
> bzr-2.3 speeds, but it helps remove PQM from being a blocking factor in
> our workflow.

I recommend using the libeatmydata LD_PRELOAD library for test
runs. fsync still called, code exactly the same, but LD_PRELOAD wrapper
makes it run in "OSX emulation mode" (i.e. a no-op)

--
Stewart Smith