[concurrency-interest] Java 8 CompletableFuture.thenCombine() hangs

Paul Sandoz paul.sandoz at oracle.com
Thu Apr 3 04:28:51 EDT 2014


On Apr 2, 2014, at 11:06 PM, Dennis Sosnoski <dms at sosnoski.com> wrote:

> On 04/03/2014 01:06 AM, Paul Sandoz wrote:
>> On Apr 2, 2014, at 1:16 AM, Dennis Sosnoski <dms at sosnoski.com> wrote:
>> 
>>> ...
>>> 
>>> When I run it from the command line it mostly doesn't make it through the first call to run(). Sometimes it does, but it generally only makes a few passes before hanging. In other circumstances (such as running inside Eclipse) I've seen it make it all the way through to the end, though most often it also hangs there in the first few passes. I'm thinking it might be Hotspot related because of the halting pattern.
>>> 
>> My gut feeling is that it is more likely to be a race condition in the CompletableFuture.thenCombine and completion linked list management code.
> 
> What I meant to say in my original email was that it seemed like the failure either occurred in the first few loop executions or not at all, and I thought it might relate to optimizations in the JVM eliminating the window for this to occur. But even if that's the case, there still has to be a problem either in the classes or the VM implementation, and this would show up with "random" failures in other code.
> 
> I first saw the problem with a slightly more complex set of calculations, just reduced it down to what I provided to keep things simple.
> 

Thanks, very much appreciated.

I think i may have found the cause, my initial gut feeling was wrong about a race condition :-)

It appears to be caused by a StackOverflowException being thrown when an async task attempts to complete all its dependent tasks in the completion chain. That exception gets swallowed and does not propagate.

This behaviour is likely exacerbated because the default F/J pool (common pool) is LIFO-based, so precedence will be given for later tasks to execute before earlier tasks, thus setting up the scenario for a StackOverflowException to occur.

See example code below, where the first async task will not complete until subsequent async tasks have done so, which creates a very long completion chain.

Not quite sure how it fix it... might require the use of an explicit stack on the heap, plus it might point to a general weakness in CompletableFuture for handling exceptions thrown by the framework (e.g. OutOfMemoryError).

Paul.

public class Test2 {

    static List<CompletableFuture<Integer>> tasks = new ArrayList<>();

    private Integer run(int n) {
        CountDownLatch cdl = new CountDownLatch(n - 1);

        CompletableFuture<Integer> last = CompletableFuture.supplyAsync(() -> {
            try {
                cdl.await();
            }
            catch (InterruptedException e) { }
            return 0;
        });

        tasks.add(last);
        for (int i = 1; i < n; i++) {
            final int v = i;
            last = CompletableFuture.supplyAsync(() -> {
                cdl.countDown();
                return v;
            }).thenCombine(last, Integer::max);
            tasks.add(last);
        }
        return last.join();
    }

    public static void main(String[] args) {
        Test2 fail = new Test2();

        CountDownLatch cld = new CountDownLatch(1);
        Thread t = new Thread(() -> {
            try {
                cld.await(2, TimeUnit.SECONDS);
            }
            catch (InterruptedException e) {
            }

            if (cld.getCount() == 1) {
                List<CompletableFuture<Integer>> it = tasks.stream().filter(task -> !task.isDone()).collect(toList());
                System.out.printf("%d tasks not completed\n", it.size());
            }
        });

        t.start();
        fail.run(10000);
        cld.countDown();
    }
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20140403/ec2bd8d0/attachment.bin>


More information about the Concurrency-interest mailing list