>>>>> "robert" == Robert Collins <email address hidden> writes:
robert> Review: Needs Fixing
robert> I'm fairly sure that this breaks compatibility with
robert> --parallel=ec2.
Right, care to give it a try ? You know it's far easier for you
than for me.
I presume your suspicion comes from the fact that the plugin uses
fork_for_tests right ?
robert> I think the concept is useful but please consider how
robert> to fit in with that plugin, and on windows, where
robert> external processes are much more expensive to start.
Using a process for each slice was simpler than starting one
process by processor and then implementing a way to send the
slices via another socket/pipe, so I tried that approach a bit
and punted.
If you played a bit with -Eslices, you noticed that, indeed, many
processes are spawned (one by slice) and avoiding them can give
even better performances.
Bug given that the test suite is *not* fully running so far on
windows, being able to run even part of it, faster is still a win IMHO.
And even if starting new processes is slow, I doubt it can be slower
to run 8 parallel newly started processes than a single started
once process :).
Hmm, I don't doubt it, I know in fact, the overhead of starting
new processes and handling the results via subunit is high, even
on linux, yet --parallel=fork reduces the *elapsed* time
enormously. Unless that can't reproduced on windows, I far prefer
being able to reduce the elapsed time than delay using -parallel=fork there.
Would you be ok if instead of modifying fork_for_tests I create a
new fork_balanced_for_tests until we can reconcile both ?
>>>>> "robert" == Robert Collins <email address hidden> writes:
robert> Review: Needs Fixing
robert> I'm fairly sure that this breaks compatibility with
robert> --parallel=ec2.
Right, care to give it a try ? You know it's far easier for you
than for me.
I presume your suspicion comes from the fact that the plugin uses
fork_for_tests right ?
robert> I think the concept is useful but please consider how
robert> to fit in with that plugin, and on windows, where
robert> external processes are much more expensive to start.
Using a process for each slice was simpler than starting one
process by processor and then implementing a way to send the
slices via another socket/pipe, so I tried that approach a bit
and punted.
If you played a bit with -Eslices, you noticed that, indeed, many
processes are spawned (one by slice) and avoiding them can give
even better performances.
Bug given that the test suite is *not* fully running so far on
windows, being able to run even part of it, faster is still a win IMHO.
And even if starting new processes is slow, I doubt it can be slower
to run 8 parallel newly started processes than a single started
once process :).
Hmm, I don't doubt it, I know in fact, the overhead of starting
new processes and handling the results via subunit is high, even
on linux, yet --parallel=fork reduces the *elapsed* time
enormously. Unless that can't reproduced on windows, I far prefer
being able to reduce the elapsed time than delay using -parallel=fork there.
Would you be ok if instead of modifying fork_for_tests I create a for_tests until we can reconcile both ?
new fork_balanced_