[concurrency-interest] ForkJoinPool not designed for nested Java 8 streams.parallel().forEach( ... )

√iktor Ҡlang viktor.klang at gmail.com
Tue May 6 13:54:03 EDT 2014


On Tue, May 6, 2014 at 6:52 PM, Doug Lea <dl at cs.oswego.edu> wrote:

> On 05/06/2014 08:34 AM, √iktor Ҡlang wrote:
>
>>
>>   In versions prior to JDK7 release, continuation-thread
>>     compensation generated spare threads to maintain target parallelism.
>>     This works great for correctly expressed nested parallelism but is
>>     prone to positive feedback loops when people make small design errors
>>     like inverting joins. When mixed with blocking IO, we saw cases where
>>     this generated thousands of threads. So for the sake of JVM stability,
>>     compensation is restricted to cases where the pool is saturated. This
>>     could be re-explored.
>>
>>
>> Speaking of which—is there anything that prevents adding the capability to
>> specify the maximum number of threads allowed to be spawned due to managed
>> blocking? (The current default blows up about every JVM there is.)
>>
>
> The current default is to only generate a new thread if saturated,
> which is the most conservative possible policy. If we allowed a
> bound on compensations, then everything could freeze up if past
> some arbitrary limit, which seems even less popular than risking
> OutOfMemoryError (and RejectedExecutionException) thrown by
> attempted Thread construction. Especially since you can use
> ForkJoinWorkerThreadFactories and/or SecurityManagers or to bound
> the total number of Threads in a program, whether by FJ or elsewhere.


Assuming that there are other threads in the system than the pool, it'd be
perfectly fine to have all threads in the pool stalled when waiting for
external threads to complete/signal progress, right? The code that submits
the ManagedBlocker only does so to be polite about his/her blocking, and
won't have a full "global view" of what is currently running, and
definitely does not want to have an OOME due to too many native threads.

How can ForkJoinWorkerThreadFactory be used to cap the maximum number of
threads without blowing something up?


>
>
> -Doug
>
>
>


-- 
Cheers,
√
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20140506/d26f2f97/attachment.html>


More information about the Concurrency-interest mailing list