[concurrency-interest] ForkJoinPool not designed for nested Java 8streams.parallel().forEach( ... )

Christian Fries email at christian-fries.de
Mon May 5 16:40:26 EDT 2014

Dear David.

Thank you! I do understand that the use of that Semaphore is evil - I will refrain from it.

That said, just consider the Semaphore in my demo code as the litmus of a litmus test. (I am using that semaphore just to show that we have an pool.awaitJoin on the wrong thread).

It appears to me as if nested Java 8 parallel streams will create a very subtle „implicit coupling“ between the tasks submitted to the common pool, because tasks of an inner loop are joined with tasks on an outer loop.

So maybe the problem is to have a common FJP at all?

I still have the impression that the line 
	((t = Thread.currentThread()) instanceof ForkJoinWorkerThread)
is not correct in the case of a nested loop.

Thank you for your attention.


Am 05.05.2014 um 22:15 schrieb David Holmes <davidcholmes at aapt.net.au>:

> Christian,
> You can not use arbitrary synchronization devices within a ForkJoinPool.
> Such devices require/assume that you have control over the threads involved
> in a computation which is not the case with FJ. The FJP docs state:
> "A ForkJoinPool is constructed with a given target parallelism level; by
> default, equal to the number of available processors. The pool attempts to
> maintain enough active (or available) threads by dynamically adding,
> suspending, or resuming internal worker threads, even if some tasks are
> stalled waiting to join others. However, no such adjustments are guaranteed
> in the face of blocked IO or other unmanaged synchronization. The nested
> ForkJoinPool.ManagedBlocker interface enables extension of the kinds of
> synchronization accommodated."
> Phaser has been written to work with FJP but the other synchronization
> constructs have not.
> David Holmes
> ------------
>> -----Original Message-----
>> From: concurrency-interest-bounces at cs.oswego.edu
>> [mailto:concurrency-interest-bounces at cs.oswego.edu]On Behalf Of
>> Christian Fries
>> Sent: Tuesday, 6 May 2014 5:59 AM
>> To: concurrency-interest at cs.oswego.edu
>> Subject: [concurrency-interest] ForkJoinPool not designed for nested
>> Java 8streams.parallel().forEach( ... )
>> Dear All.
>> I am new to this list, so please excuse, if this is not the place
>> for the following topic (I did a bug report to Oracle and also
>> posted this to Stackoverflow at
>> http://stackoverflow.com/questions/23442183 - and it was
>> suggested (at SO) that I post this here too).
>> Java 8 introduced parallel streams which use a common
>> ForkJoinPool. I believe that the implementation of ForkJoinTask
>> has a problem, which becomes relevant in a situation, which is
>> likely to occur if Java 8 parallel streams are used a lot.
>> Specifically the problem arises if one uses a nested Java 8
>> parallel stream foreach, e.g. as in
>> 	// Outer loop
>> IntStream.range(0,numberOfTasksInOuterLoop).parallel().forEach(i -> {
>> 		// Inner loop
>> IntStream.range(0,numberOfTasksInInnerLoop).parallel().forEach(j -> {
>> 			// Work done here loop
>> 		});
>> 	});
>> In this situation the inner loop may be started on a workerThread
>> of the common ForkJoinPool.
>> Now note that the implementation of ForkJoinTask check if the
>> task is run on the parent thread (current thread) or on a forked
>> worked thread via the line
>> ((t = Thread.currentThread()) instanceof ForkJoinWorkerThread)
>> Hoever, for the case of the nested loop, it is not sufficient if
>> the thread is „instanceof ForkJoinWorkerThread“, because the
>> parent thread itself was a ForkJoinWorkerThread. Further in the
>> implementation of doInvoke (see
>> http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/6be37bafb11a/src/s
> hare/classes/java/util/concurrent/ForkJoinTask.java#l393 ) this implied that
> we call awaitJoin on the workQueue - I believe this is wrong. Now, that
> workQueue belongs to the outer loop and this may result in a DEADLOCK.
> For details see
> http://stackoverflow.com/questions/23442183/using-a-semaphore-inside-a-java-
> 8-parallel-stream-action-may-deadlock-is-this-a
> For a detailed test code run
> http://svn.finmath.net/finmath%20experiments/trunk/src/net/finmath/experimen
> ts/concurrency/ForkJoinPoolTest.java
> That said, it appears as if the current Java implementation of ForkJoinPool
> was written without having „Nesting“ in mind (submitting tasks from a worker
> tasks to a common pool), but Java 8 streams makes that kind of nesting a
> natural thing.
> I believe I know the fix for the problem (we have to store the parent thread
> and check if currentThread() equals the parentThread of a ForkJoinTask - in
> case you like me to get involved.
> Please excuse if this is not the right mailing list for this.
> Best
> Christian
> ________________________________________________
> http://www.christian-fries.de
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest

More information about the Concurrency-interest mailing list