[concurrency-interest] Questions about ThreadPoolExecutor

David Holmes dcholmes at optusnet.com.au
Wed Mar 1 22:42:19 EST 2006


Joe writes:
> We call this beast a composite executor.  There's an example in the
> Executor javadoc:
>
> http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/Executor.html
>
> The SerialExecutor example serializes the execution of a set of tasks
> on another executor.
>
> In your case, you'd write a BatchExecutor that submitted its tasks to
> a master executor.  The master executor could be a cached ThreadPool
> (Executors.newCachedThreadPool) with a maximumPoolSize to prevent
> thread explosion.

I'm losing the plot here a bit. It seems there were two original
requirements:
 - bound the size of the pool
 - avoid task deadlocks

The suggestion for avoiding deadlock was to use a synchronous queue which
seemed to preclude using a bounded pool, hence an apparent conflict.

But to avoid deadlocks (more a gridlock: each executing task needs to submit
a new subtask but the pool is full so no one executes and we come to
stand-still) you always need sufficient threads for the maximum number of
concurrent tasks, or else revert to an "execute in the current thread"
rejection policy.

By batching things we seem to be reducing the pool size issue somewhat by
requiring that the pool contain as many threads as the maximum number of
concurrent tasks needed by any one batch. This is essence would serialize
batches. If the pool is greater than that number then we can allow some
overlap of batches and in the extreme we have enough threads for every
batch. Is that a reasonable summery?

BTW another option would be to change the task structure. Suppose that
originally we have:

    TaskA {
        // do some work
        spawn TaskB
        spawn TaskC
        join TaskA
        join TaskB
        // do more work
    }

Then instead you factor "do more work" into TaskD and have it submitted by
the last of TaskB and TaskC to complete. Of course this requires some
coordination between B and C, or some coordination infrastructure around
them.

Cheers,
David Holmes



More information about the Concurrency-interest mailing list