[concurrency-interest] ForkJoinPool spawning much more workers than its target parallelism
dl at cs.oswego.edu
Fri Oct 1 08:21:05 EDT 2010
On 10/01/10 07:57, Antoine CHAMBILLE wrote:
> We test for performance on an Intel Xeon Nehalem platform (2 sockets, 4-cores
> CPUs, hyperthreading, so 16 hardware threads).
Could you tell me exactly what OS platform and JDK version?
> We have always seen the fork join pool spawning one or two more workers than
> the target parallelism. It has been explained that this is the way the fork
> join pool keeps close to its target parallelism. On our test platform we
> usually see 18 running workers for our target parallelism of 16.
Yes. Creating or restarting enough internal spares to maintain
target parallelism and avoid starvation is intrinsically heuristic
(it is impossible to know for sure whether lack of progress is due
to insufficient threads vs momentary unrelated stalls), so
will often transiently overshoot.
> But the last time we updated from the jsr166y trunk (just after the recent
> openJDK synchronization) we notice a strong change in this behaviour: The
> fork join pool now creates tons of worker, up to 3 times the target
> Is that an expected behaviour? Do you have an idea of the code change that
> creates it?
This is surprising; thanks for reporting it!
It would be helpful if you could send me (off-list)
a test case showing this. The changes last month
mainly entail using a backoff/timeout to smooth over false
alarms due to transient loads, so should (and does for our tests)
generally result in fewer spare threads, not more. However, it does
introduce new dependencies on JVM timed wait mechanics, that might
account for this.
More information about the Concurrency-interest