[concurrency-interest] ForkJoinPool not designed for nested Java 8 streams.parallel().forEach( ... )

Doug Lea dl at cs.oswego.edu
Tue May 6 07:17:35 EDT 2014


On 05/06/2014 06:47 AM, Paul Sandoz wrote:

> I think the use of Thread.sleep is a little misleading, since it is a blocking
> operation. Try placing all those sleep statements in managed blocks and you will
> get different results e.g. with:

Yes. We ought to provide this. I suppose we could intercept 
java.lang.Thread.sleep to do it automatically,
but just adding ForkJoinTask.delay(timeout) would help.
Similarly for CompletableFuture.


> Assuming *non-blocking* operations an enclosed (nested) parallel stream will
> compete for the same common pool resources with the enclosing parallel stream,
> something i have advised against doing. I don't know if that would be
> significantly different from equivalent work performed by two independent
> parallel streams executed concurrently; need to measure as Russel says.

There's a fun story here. FJ is fully capable of keeping up with
nested parallelism, but some of it is internally disabled, so nested
forms will (only) sometimes run in a slow emulation of sequential
processing. In versions prior to JDK7 release, continuation-thread
compensation generated spare threads to maintain target parallelism.
This works great for correctly expressed nested parallelism but is
prone to positive feedback loops when people make small design errors
like inverting joins. When mixed with blocking IO, we saw cases where
this generated thousands of threads. So for the sake of JVM stability,
compensation is restricted to cases where the pool is saturated. This
could be re-explored.

-Doug




More information about the Concurrency-interest mailing list