[concurrency-interest] Improved FJ thread throttling

√iktor Ҡlang viktor.klang at gmail.com
Tue Jul 8 13:16:31 EDT 2014

Hi Doug,

How about allowing a System Property to set the number of spares, with the
default being 256 so one does not have to implement a ThreadFactory to cap
it to something different? (Given that there are System Properties for most
of the other setting for the common pool)

On Tue, Jul 8, 2014 at 7:00 PM, Doug Lea <dl at cs.oswego.edu> wrote:

> ForkJoin extensions and adaptations for JDK8 (streams etc) included
> overly course-grained thread throttling. This was on the to-do-list
> for a while. A new update addresses this.
> Context: By design, ForkJoinPool relies on only a single parameter,
> target parallelism. It takes all responsibility for ensuring that the
> "right" number of threads are running at any given time for
> data-parallel and async applications.  It is impossible to even define
> what the "right number" is, so it is impossible for us or anyone else
> to get this exactly right.  (This is different than for example
> setting up N services using a newFixedThreadPool(N), where the only
> right answer is N.)  One approach to dealing with this would be to
> introduce a zillion controls that would be even harder to use and
> prone to even more policy inconsistency and context-dependence
> problems than seen with ThreadPoolExecutor. This would be a throwback
> to the days when every efficient parallel program had to be custom
> built. Some people think that people should still write parallel
> programs this way (please feel free to do so.)  FJ instead implements
> portable algorithms and internal policies that are rarely optimal for
> any given platform and application but often close to optimal.  As FJ
> is used for increasingly diverse purposes, getting thread throttling
> approximately "right" in all cases gets more challenging. But we like
> challenges.
> Aside: The situation is very similar to that for ConcurrentHashMap,
> that also only accepts only one optional parameter (capacity). If you
> have special requirements, you may be able to create a custom map that
> outperforms CHM. But over time, CHM evolves to benefit from diverse
> usage experiences, so customization becomes less likely to be
> worthwhile.
> More background: Any given parallel computation (including one just
> using FJ for asyncs) might, for good performance, need fewer than the
> target threads, the same number, or, if some dependent computations
> block waiting for others, possibly more threads to compensate for the
> blocked ones. (See below about blocking for other reasons.)  The
> "more" case is now less common than before, but you can't ignore it
> without risk of locking up computations.  And in some cases of mixed
> parallel/clustered systems, this could lead to distributed deadlock.
> (Note: The number of spare threads needed has little to do with the
> target parallelism level, but instead the form of the parallel
> computation dag.)  So, even though creating more than a dozen spare
> threads is rare, FJ itself imposed only a ceiling (32K threads) that
> is so high that programs typically die for other reasons before
> reaching it; and documents only an intentionally vague implementation
> note that "This implementation rejects submitted tasks (that is, by
> throwing RejectedExecutionException) only when the pool is shut down
> or internal resources have been exhausted."
> One disadvantage of this policy is that the ceiling is so high that
> programming mistakes (for example those with infinitely nested joins)
> or intentional abuses are usually not caught in a very nice way. For
> example, on Unix-based systems, people might encounter "No more
> processes" just trying to kill the program.  Especially when
> implementing the JDK8 Common Pool, we should have dropped this limit
> from being a thousand times larger than expected under normal use down
> to a value that far exceeds that needed in any practical program, but
> still gives JVMs a chance to recover. So the update includes an
> absolute ceiling of 256 more threads than the target parallelism (or
> the original total of 32K, whichever is lower.)  This will not impact
> any current practical programs except those that by chance never ran
> long enough to hit higher limits.  The value 256 is somewhat
> arbitrary. It's the highest value for which any multicore JVM is
> expected to always have enough resources to recover from.  By choosing
> a conservatively high value, there is good justification for including
> this in a JDK8 update and additionally being more aggressive about
> killing off spare threads.  To further limit behavior, we still also
> allow users to supply ThreadFactories that throw exceptions after
> hitting some maximum, but still don't particularly recommend use. Most
> implementations of external limits are arbitrarily imprecise in part
> because they cannot tell when threads are really gone: decrementing a
> count does not necessarily mean that the thread has stopped or its
> resources have been recovered.  (The internal bounds have some of the
> same problems, but handle them conservatively.)
> But the main story with this update is improved internal tracking that
> is usually much closer to running the right number of threads than
> before.  (This version also includes more and better internal
> documentation and refactorings that take advantage of JVM improvements
> that have occurred since JDK6, for example being much better than
> before at compiling 64bit logical operations without needing to code
> by splitting into 32bit parts.)
> The only version available is in our jsr166 main (JDK8/JDK9 only)
> repository, with the aim of having some of you try it out before
> considering integration into OpenJDK.  To use it, you can either use
> the jar at http://gee.cs.oswego.edu/dl/concurrent/dist/jsr166.jar and
> run with -Xbootclasspath/p:jsr166.jar; or copy into an OpenJDK and
> build files:
> http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/
> main/java/util/concurrent/ForkJoinPool.java?view=log
> http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/
> main/java/util/concurrent/ForkJoinTask.java?view=log
> http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/
> main/java/util/concurrent/ForkJoinWorkerThread.java?view=log
> ...
> Also, a few notes about blocking threads in any context (in FJ, other
> Executors, even the JVM itself).  Whenever there is some bound on
> thread construction, and threads start blocking, eventually programs
> will freeze or throw exceptions.  We can't/won't forbid all blocking
> because it is often harmlessly transient.  On the other hand, most
> programs deal with saturation effects of long-term blocking about as
> well as they deal with other resource failures (out of memory etc),
> which is not very well. But coping mechanisms do always exist.  FJ
> provides a thread-vs-memory tradeoff hook via ManagedBlocker: If a
> task blocks but you want to ensure liveness for processing other tasks
> use a ManagedBlocker. If you are content to let other work pile up
> unless/until blocked threads resume, don't use it.  This is not always
> an easy decision to make, but cannot be automated because the number
> of ways/reasons that tasks may block is unbounded.  Note:
> ThreadPoolExecutor cannot use this approach, so instead supports
> RejectedExecutionHandlers for use in similar situations.
> -Doug
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20140708/1b37b1e5/attachment.html>

More information about the Concurrency-interest mailing list