[concurrency-interest] ForkJoinPool not designed for nested Java 8 streams.parallel().forEach( ... )

Doug Lea dl at cs.oswego.edu
Fri May 9 08:53:16 EDT 2014


On 05/09/2014 08:46 AM, Oleksandr Otenko wrote:
> To be honest, I don't understand why "nested" parallellism occurs at all.

Sorry, I should have led off with lt;dr version:

Nested parallelism issues occur because the stream framework
parallelizes one expression at a time. Currently, the only way you
can express nesting is to use more than one stream expression.

-Doug

> 1. flatten
> 2. recreate the shape
>
> unless you are dealing with infinite streams; in the latter case the order of
> processing is extremely important, and it is no longer "uncontrolled" parallellism.
>
> Alex
>
> On 08/05/2014 09:48, Christian Fries wrote:
>> Dear All.
>>
>> With respect to the improvement of the documentation suggested
>>
>> Am 06.05.2014 um 01:09 schrieb Doug Lea <dl at cs.oswego.edu>:
>>
>>> Sorry that you and a bunch of people at stackOverflow spent so much time
>>> on this because you did not did not notice the java.util.stream documentation
>>> (http://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html)
>>> about the use of independent functions/actions (especially not those including
>>> synchronization). We should do something about this.
>> I would like to add two aspects from a user perspective:
>>
>> First, the linked page (
>> http://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html
>> ) does not mention the ForkJoinPool or Managed Blocker, etc. at all. The words
>> „pool“, „common“, „fork“, „managed“ do not even occur in the document. It is
>> just a discussion of parallel streams from the stream perspective suggesting
>> that code like the one I have used (thread save, no interference, etc.) works
>> fine. (This is not actually my observation, it was a comment on SO).
>> So a footnote that the backend of parallel streams is currently the FJP and
>> that corresponding limitations apply (use of managed blockers) should be added
>> (on the other hand - I would prefer to have the streams framework work without
>> too much knowledge of the underlying implementation).
>>
>> Second, I would like to comment on the idea to „forbid“ nested parallelism at
>> all: From a user perspective I try to (or have to) use parallelism a lot. In
>> my library many parts are functional, stateless, free of interference. If you
>> use parallelism a lot it may be that you have parallelism encapsulated
>> „inside“ many small parts and nested parallelism is a natural thing. It is
>> also a good thing. If I have an outer parallel loop of 5 on our 64 core
>> machine it would be a waste to forbid inner (nested) parallelism. Everyone of
>> the 5 could work on its task with a parallelism of > 10 - so we are talking
>> about a factor of 10. - I have this situation in my lib and it works great.
>>
>> Of course, one aspect of the issue in my previous post is that we are
>> submitting to a COMMON thread pool. A workaround would be to use a dedicated
>> pool for each inner loop. But a common thread pool is an advantage, since it
>> allows to globally control parallelism!
>>
>> Best
>> Christian
>>
>>
>> _______________________________________________
>> Concurrency-interest mailing list
>> Concurrency-interest at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>





More information about the Concurrency-interest mailing list