[concurrency-interest] ForkJoinPool not designed for nested Java 8streams.parallel().forEach( ... )

David Holmes davidcholmes at aapt.net.au
Mon May 5 16:15:52 EDT 2014


Christian,

You can not use arbitrary synchronization devices within a ForkJoinPool.
Such devices require/assume that you have control over the threads involved
in a computation which is not the case with FJ. The FJP docs state:

"A ForkJoinPool is constructed with a given target parallelism level; by
default, equal to the number of available processors. The pool attempts to
maintain enough active (or available) threads by dynamically adding,
suspending, or resuming internal worker threads, even if some tasks are
stalled waiting to join others. However, no such adjustments are guaranteed
in the face of blocked IO or other unmanaged synchronization. The nested
ForkJoinPool.ManagedBlocker interface enables extension of the kinds of
synchronization accommodated."

Phaser has been written to work with FJP but the other synchronization
constructs have not.

David Holmes
------------

> -----Original Message-----
> From: concurrency-interest-bounces at cs.oswego.edu
> [mailto:concurrency-interest-bounces at cs.oswego.edu]On Behalf Of
> Christian Fries
> Sent: Tuesday, 6 May 2014 5:59 AM
> To: concurrency-interest at cs.oswego.edu
> Subject: [concurrency-interest] ForkJoinPool not designed for nested
> Java 8streams.parallel().forEach( ... )
>
>
>
>
> Dear All.
>
> I am new to this list, so please excuse, if this is not the place
> for the following topic (I did a bug report to Oracle and also
> posted this to Stackoverflow at
> http://stackoverflow.com/questions/23442183 - and it was
> suggested (at SO) that I post this here too).
>
> Java 8 introduced parallel streams which use a common
> ForkJoinPool. I believe that the implementation of ForkJoinTask
> has a problem, which becomes relevant in a situation, which is
> likely to occur if Java 8 parallel streams are used a lot.
> Specifically the problem arises if one uses a nested Java 8
> parallel stream foreach, e.g. as in
>
> 	// Outer loop
>
> IntStream.range(0,numberOfTasksInOuterLoop).parallel().forEach(i -> {
> 		// Inner loop
>
> IntStream.range(0,numberOfTasksInInnerLoop).parallel().forEach(j -> {
> 			// Work done here loop
> 		});
>
> 	});
>
>
> In this situation the inner loop may be started on a workerThread
> of the common ForkJoinPool.
>
> Now note that the implementation of ForkJoinTask check if the
> task is run on the parent thread (current thread) or on a forked
> worked thread via the line
>
> ((t = Thread.currentThread()) instanceof ForkJoinWorkerThread)
>
> Hoever, for the case of the nested loop, it is not sufficient if
> the thread is „instanceof ForkJoinWorkerThread“, because the
> parent thread itself was a ForkJoinWorkerThread. Further in the
> implementation of doInvoke (see
> http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/6be37bafb11a/src/s
hare/classes/java/util/concurrent/ForkJoinTask.java#l393 ) this implied that
we call awaitJoin on the workQueue - I believe this is wrong. Now, that
workQueue belongs to the outer loop and this may result in a DEADLOCK.

For details see
http://stackoverflow.com/questions/23442183/using-a-semaphore-inside-a-java-
8-parallel-stream-action-may-deadlock-is-this-a
For a detailed test code run
http://svn.finmath.net/finmath%20experiments/trunk/src/net/finmath/experimen
ts/concurrency/ForkJoinPoolTest.java

That said, it appears as if the current Java implementation of ForkJoinPool
was written without having „Nesting“ in mind (submitting tasks from a worker
tasks to a common pool), but Java 8 streams makes that kind of nesting a
natural thing.

I believe I know the fix for the problem (we have to store the parent thread
and check if currentThread() equals the parentThread of a ForkJoinTask - in
case you like me to get involved.

Please excuse if this is not the right mailing list for this.

Best
Christian



________________________________________________
http://www.christian-fries.de





_______________________________________________
Concurrency-interest mailing list
Concurrency-interest at cs.oswego.edu
http://cs.oswego.edu/mailman/listinfo/concurrency-interest



More information about the Concurrency-interest mailing list