[concurrency-interest] ForkJoinPool not designed for nested Java 8 streams.parallel().forEach( ... )

Christian Fries email at christian-fries.de
Mon May 5 15:58:44 EDT 2014



Dear All.

I am new to this list, so please excuse, if this is not the place for the following topic (I did a bug report to Oracle and also posted this to Stackoverflow at http://stackoverflow.com/questions/23442183 - and it was suggested (at SO) that I post this here too).

Java 8 introduced parallel streams which use a common ForkJoinPool. I believe that the implementation of ForkJoinTask has a problem, which becomes relevant in a situation, which is likely to occur if Java 8 parallel streams are used a lot. Specifically the problem arises if one uses a nested Java 8 parallel stream foreach, e.g. as in

	// Outer loop
	IntStream.range(0,numberOfTasksInOuterLoop).parallel().forEach(i -> {
		// Inner loop
		IntStream.range(0,numberOfTasksInInnerLoop).parallel().forEach(j -> {
			// Work done here loop
		});

	});


In this situation the inner loop may be started on a workerThread of the common ForkJoinPool.

Now note that the implementation of ForkJoinTask check if the task is run on the parent thread (current thread) or on a forked worked thread via the line

((t = Thread.currentThread()) instanceof ForkJoinWorkerThread)

Hoever, for the case of the nested loop, it is not sufficient if the thread is „instanceof ForkJoinWorkerThread“, because the parent thread itself was a ForkJoinWorkerThread. Further in the implementation of doInvoke (see http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/6be37bafb11a/src/share/classes/java/util/concurrent/ForkJoinTask.java#l393 ) this implied that we call awaitJoin on the workQueue - I believe this is wrong. Now, that workQueue belongs to the outer loop and this may result in a DEADLOCK.

For details see http://stackoverflow.com/questions/23442183/using-a-semaphore-inside-a-java-8-parallel-stream-action-may-deadlock-is-this-a
For a detailed test code run http://svn.finmath.net/finmath%20experiments/trunk/src/net/finmath/experiments/concurrency/ForkJoinPoolTest.java

That said, it appears as if the current Java implementation of ForkJoinPool was written without having „Nesting“ in mind (submitting tasks from a worker tasks to a common pool), but Java 8 streams makes that kind of nesting a natural thing.

I believe I know the fix for the problem (we have to store the parent thread and check if currentThread() equals the parentThread of a ForkJoinTask - in case you like me to get involved.

Please excuse if this is not the right mailing list for this. 

Best
Christian



________________________________________________
http://www.christian-fries.de







More information about the Concurrency-interest mailing list