[concurrency-interest] Questions about ThreadPoolExecutor

Kwok, Grace (MSCIBARRA) Grace.Kwok at mscibarra.com
Thu Mar 2 13:55:43 EST 2006


>"If you set the maximum pool size on your ThreadPoolExecutor and assign
CallerRunsPolicy() as the rejectedExecutionHandler, then tasks
submitted while the active thread limit has been reached will be
executed in the submitting thread."

Thanks for the above hint.  This should fit part of my needs.  By using
this policy, it should avoid task deadlocks even if my pool is bounded,
correct?

>"In your case, you'd write a BatchExecutor that submitted its tasks to
a master executor. "

Since my original problem is to be

1) - able to specify the number of threads to run each batch of tasks.
(i.e. I have a batch of 100 tasks; I only want to allocate 6 threads to
be running them even if the pool has more than 6 threads available.  The
reasons are one, I don't want one batch to take away all the thread
resources and two, if the tasks are big stress to the db, I simply want
the control to be able to specify less threads to run them.)
2) - bound the size of the pool
3) - avoid task deadlocks
4) - would need something like invokeAll(..) in ExecutorService to wait
and get my result back. 
5) - nice to be able to reuse threads in pool instead of creating an
Executor for each batch since some batch might be small, some are large.

In your suggestion of this BatchExecutor, in esssense, there is really
one Executor, I am not so clear on how I would achieve 1 and 4?  Would
my BatchExecutor has to be wise in figuring out the pace of dispatching
to the master executor so to satisfy X number of threads running the
batch of tasks?

Thank you very much for everyone's valuable time.
Grace


-----Original Message-----
From: Joe Bowbeer [mailto:joe.bowbeer at gmail.com] 
Sent: Wednesday, March 01, 2006 7:14 PM
To: Kwok, Grace (MSCIBARRA)
Cc: concurrency-interest at cs.oswego.edu
Subject: Re: [concurrency-interest] Questions about ThreadPoolExecutor

On 3/1/06, Kwok, Grace (MSCIBARRA) <Grace.Kwok at mscibarra.com> wrote:
>
> Do you mind explain this further?
> "This could be done by having all of the dedicated executors submit 
> their work to one master executor."
>
>

We call this beast a composite executor.  There's an example in the
Executor javadoc:

http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/Executor.ht
ml

The SerialExecutor example serializes the execution of a set of tasks on
another executor.

In your case, you'd write a BatchExecutor that submitted its tasks to a
master executor.  The master executor could be a cached ThreadPool
(Executors.newCachedThreadPool) with a maximumPoolSize to prevent thread
explosion.

Btw, another way to get a handle on thread creation across a set of
executors is to assign the same ThreadFactory to them all.  I wouldn't
make your factory too smart because then you'd be rewriting
ThreadPoolExecutor, but at least you'd be able to name the threads
logically and keep track of the total count, etc.


> -----Original Message-----
> From: Joe Bowbeer [mailto:joe.bowbeer at gmail.com]
> Sent: Wednesday, March 01, 2006 5:17 PM
> To: Kwok, Grace (MSCIBARRA)
> Cc: Brian Goetz; concurrency-interest at cs.oswego.edu
> Subject: Re: [concurrency-interest] Questions about ThreadPoolExecutor
>
> On 3/1/06, Kwok, Grace (MSCIBARRA) <Grace.Kwok at mscibarra.com> wrote:
> > "You can create a custom Executor for each batch of Callables.  You 
> > will probably want to use some other bounding mechanism to ensure 
> > that
>
> > aggregate number of threads created by such executors doesn't exceed

> > some threshold."
> >
>
> This could be done by having all of the dedicated executors submit 
> their work to one master executor.
>
>
> > If I am to create an Executor for each batch of Callables, the 
> > threads
>
> > from Executor 1 could not be reused by threads from Executor 2, is 
> > that correct?  Is this optimal?  Is this like a pool of Executors?
> >
>
> ThreadPoolExecutor per batch seems right to me.  If you want to 
> cleanup when a batch is done, you can call executorService.shutdown().
>
> You can also call executorService.setKeepAliveTime() so that the idle 
> threads will self-terminate more or less automatically.
> --------------------------------------------------------
>
--------------------------------------------------------

NOTICE: If received in error, please destroy and notify sender.  Sender does not waive confidentiality or privilege, and use is prohibited.



More information about the Concurrency-interest mailing list