[concurrency-interest] Bounded Task Queue for ScheduledThreadPoolExecutor

Volkan Yazıcı volkan.yazici at gmail.com
Thu Apr 25 04:12:59 EDT 2019


Thanks so much for taking time to reply professor.

STPE.DelayWorkQueue ctor does not have any arguments and every access is
synchronized on STPE.DelayWorkQueue#lock ReentrantLock, which I believe
makes the enforcement of a bound "relatively easy". Would you mind sharing
which sort of configurations you have in mind that might potentially lead
to future bugs, please?

I see that you are more inclined to recommend me having my own custom STPE
wrapper overriding ScheduledExecutorService methods with a check on
getQueue().size() rather than creating a feature request in OpenJDK. Is my
interpretation right?

Best.

On Wed, Apr 24, 2019 at 1:15 PM Doug Lea via Concurrency-interest <
concurrency-interest at cs.oswego.edu> wrote:

> On 4/23/19 2:37 AM, Volkan Yazıcı via Concurrency-interest wrote:
> > Hello,
> >
> > Given a ScheduledThreadPoolExecutor (STPE) with a set of slow consumers,
> > the unbounded task queue constitutes the perfect disguise for
> > backpressure-related problems. Producers just keep on pushing tasks to
> > the point that 1) growing queue starts choking memory, 2) almost all
> > pending time-sensitive tasks become obsolete when they are assigned to a
> > thread to get executed, and 3) even relatively cheap tasks starve to
> > death for no good reason. To the best of my knowledge, unfortunately,
> > STPE does accept neither a bound, nor a custom queue in its ctor, even
> > though the internally used ThreadPoolExecutor (TPE) perfectly allows
> > that[1]. To work around the problem, I create my own
> > ScheduledExecutorService where time-sensitive tasks are delegated to an
> > internal STPE and the rest to a TPE. That being said, I would rather
> > prefer STPE to just accept a custom implementation or a bound for the
> > task queue it employs.
>
> STPE started out (long ago, pre-release) allowing custom queues,
> defaulting to DelayQueue. This was changed to use an internal
> specialization of DelayQueue to allow cleanup of cancelled tasks, which
> helps significantly in networking applications in which timeout
> callbacks rarely turn out to be needed.
>
> At this point there is no reasonable path to reinstating a custom queue
> option. However, it would be possible to add yet another configuration
> method setMaximumQueueSize, which would cause a
> RejectedExecutionException if exceeded.
>
> We worry when adding such methods about unknown interactions with other
> configuration methods and constructors, leading to future bug reports.
> Which I think has happened every time we have added one.
>
> In the mean time you could get most of this effect at reasonable cost by
> wrapping STPE within another Executor that rejects when
> stpe.getQueue().size() exceeds a threshold.
>
> -Doug
>
>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20190425/cc468428/attachment.html>


More information about the Concurrency-interest mailing list