[concurrency-interest] Awaiting a set of tasks on an ExecutorService

Josh Humphries jhump at bluegosling.com
Fri Aug 18 16:48:41 EDT 2017


I think the easiest thing would be to decorate each task to call
"latch.countDown()" on a CountDownLatch that is initialized with the total
number of tasks. After they are all submitted, the code that wants to wait
for them to finish would simply await the latch. This decouples completion
of all tasks from the actual ExecutorService that is running them, so you
can share the same ExecutorService for multiple, even overlapping, sets of
tasks.

----
*Josh Humphries*
jhump at bluegosling.com

On Fri, Aug 18, 2017 at 4:42 PM, Shevek <shevek at anarres.org> wrote:

> Hi,
>
> I need to build a complicated object on a multiprocessor system. The
> target object uses synchronization to guarantee thread-safe access.
> However, the individual tasks which build elements of the target object are
> expensive, so I farm them out to an ExecutorService. The question is:
>
> What is the recommended way to submit a stream of tasks to an
> ExecutorService, and then at a point, wait until they're all done.
>
> * I don't know the list of tasks up front. I read them iteratively from a
> set of files or events.
> * A big collection of Future(s) isn't realistic, there are just too many
> tasks.
> * Reaping completed futures on submission means that I end up with
> exceptions in weird places; I really need to gather max 100 suppressed
> exceptions, and discard the rest.
> * ForkJoinPool has invoke() but I think for a million-task job, I still
> end up with a huge list of futures. This also assumes I ignore the note
> about synchronization.
> * CompletionService allows me to wait for _any_ of the submitted tasks,
> but not for _all_ of them.
>
> * Bonus points for sharing a single ExecutorService and having "sets" of
> tasks which can be independently awaited. This starts to sound very like
> FJP, except for the blocking/I/O and the stream-ness of my task set.
>
> Mostly, dear list, I'm expecting a one-liner from some API class that I've
> missed because this is NOT an original request, I just can't find a
> standard solution.
>
> Thank you.
>
> S.
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20170818/c47d2117/attachment.html>


More information about the Concurrency-interest mailing list