[concurrency-interest] ForkJoinPool spawning much more workers than its target parallelism

Antoine CHAMBILLE ach at quartetfs.com
Fri Oct 1 07:57:28 EDT 2010


Dear all,

 

We at QuartetFS have been using the ForkJoinPool (JDK6 + jsr166y) for a
while. We write a high throughput aggregation engine that receives tuples
and aggregates them following some group-by clauses. The ForkJoinPool is a
fantastic tool to do that in parallel, because it remains efficient for very
small tasks, and because tasks can resubmit subtasks without any risk of
dead-locking.

 

We test for performance on an Intel Xeon Nehalem platform (2 sockets,
4-cores CPUs, hyperthreading, so 16 hardware threads). We dimension the
target parallelism of the fork join pool to 16.

 

We have always seen the fork join pool spawning one or two more workers than
the target parallelism. It has been explained that this is the way the fork
join pool keeps close to its target parallelism. On our test platform we
usually see 18 running workers for our target parallelism of 16.

 

But the last time we updated from the jsr166y trunk (just after the recent
openJDK synchronization) we notice a strong change in this behaviour: The
fork join pool now creates tons of worker, up to 3 times the target
parallelism. On our platform we see up to 50 running workers with a target
parallelism of 16. And the overall performance is reduced a little (-5%)
compared to our previous jsr166y snapshot (July 2010).

 

 

Is that an expected behaviour? Do you have an idea of the code change that
creates it?

 

 

Thank you very much,

-Antoine CHAMBILLE

QuartetFS

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20101001/f5d7a2f4/attachment.html>


More information about the Concurrency-interest mailing list