[concurrency-interest] Concurrency-interest Digest, Vol 162, Issue 3

David Stuebe davidstuebe at upserve.com
Tue Aug 7 12:10:02 EDT 2018


Hi Jonas

$ java --version
java 10.0.2 2018-07-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.2+13)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)

$ uname -a
Darwin dstuebe 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT
2018; root:xnu-3789.73.14~1/RELEASE_X86_64 x86_64

$ java -classpath out/production/efstress com.upserve.NestedParallel
starting
*** application hangs ***

Running jstack should dump a thread stack including a section like:

Found one Java-level deadlock:
=============================
"outer pool1":
  waiting to lock monitor 0x00007fbf02006800 (object 0x00000006cfa288d0, a
java.util.concurrent.ConcurrentHashMap$Node),
  which is held by "outer pool2"
"outer pool2":
  waiting to lock monitor 0x00007fbefe11ac00 (object 0x00000006cfaac488, a
java.util.concurrent.ConcurrentHashMap$Node),
  which is held by "outer pool1"

Please accept my apologies for the code structure. I did my best to
synthesize a simple repro case that demonstrates the issue and allows
experimentation with different implementations for the inner and outer loop.

Best

David


On Tue, Aug 7, 2018 at 12:00 PM <concurrency-interest-request at cs.oswego.edu>
wrote:

> Send Concurrency-interest mailing list submissions to
>         concurrency-interest at cs.oswego.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://cs.oswego.edu/mailman/listinfo/concurrency-interest
> or, via email, send a message with subject or body 'help' to
>         concurrency-interest-request at cs.oswego.edu
>
> You can reach the person managing the list at
>         concurrency-interest-owner at cs.oswego.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Concurrency-interest digest..."
>
>
> Today's Topics:
>
>    1. Re: CHM.compute restrictions (Jonas Konrad)
>    2. Re: CHM.compute restrictions (David Stuebe)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 7 Aug 2018 17:13:24 +0200
> From: Jonas Konrad <me at yawk.at>
> To: concurrency-interest at cs.oswego.edu
> Subject: Re: [concurrency-interest] CHM.compute restrictions
> Message-ID: <97d61d52-22eb-d429-3458-7d459c7f4c94 at yawk.at>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> How often is this supposed to deadlock? And in what environment? Because
> I can't get it to. Also, the code is *very* confusing.
>
> - Jonas
>
> On 08/07/2018 04:39 PM, David Stuebe via Concurrency-interest wrote:
> > Hi Doug, Concurrency Interest
> >
> > Sorry I lost the subject when I reposted.
> >
> > I understand that updating other keys or recursively initializing the
> > same key is illegal in a CHM.compute. I don't understand how this
> > example code could be recursive though?
> >
> > It seems to be a defect in parallel stream. Using a for loop to submit
> > the tasks in parallel for the out loop has no issue. These should be
> > effectively the same. There is some interaction between the outer
> > parallel stream that I don't understand.
> >
> > I don't think Stream.parallel is expected to result in stream operators
> > executing recursively on stream elements?
> >
> > Thanks
> >
> > On Mon, Aug 6, 2018 at 12:01 PM
> > <concurrency-interest-request at cs.oswego.edu
> > <mailto:concurrency-interest-request at cs.oswego.edu>> wrote:
> >
> >     Send Concurrency-interest mailing list submissions to
> >     concurrency-interest at cs.oswego.edu
> >     <mailto:concurrency-interest at cs.oswego.edu>
> >
> >     To subscribe or unsubscribe via the World Wide Web, visit
> >     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
> >     or, via email, send a message with subject or body 'help' to
> >     concurrency-interest-request at cs.oswego.edu
> >     <mailto:concurrency-interest-request at cs.oswego.edu>
> >
> >     You can reach the person managing the list at
> >     concurrency-interest-owner at cs.oswego.edu
> >     <mailto:concurrency-interest-owner at cs.oswego.edu>
> >
> >     When replying, please edit your Subject line so it is more specific
> >     than "Re: Contents of Concurrency-interest digest..."
> >
> >
> >     Today's Topics:
> >
> >         1. (no subject) (David Stuebe)
> >         2. Re: CHM.compute restrictions (was: no     subject) (Doug Lea)
> >
> >
> >
>  ----------------------------------------------------------------------
> >
> >     Message: 1
> >     Date: Mon, 6 Aug 2018 10:10:33 -0400
> >     From: David Stuebe <davidstuebe at upserve.com
> >     <mailto:davidstuebe at upserve.com>>
> >     To: "concurrency-interest at cs.oswego.edu
> >     <mailto:concurrency-interest at cs.oswego.edu>"
> >              <concurrency-interest at cs.oswego.edu
> >     <mailto:concurrency-interest at cs.oswego.edu>>
> >     Subject: [concurrency-interest] (no subject)
> >     Message-ID:
> >
> >     <CAJh0pqEVuzteLqiijAsFiVMFm1XL+KO7o12MzH5wu=tSqyZDVQ at mail.gmail.com
> >     <mailto:tSqyZDVQ at mail.gmail.com>>
> >     Content-Type: text/plain; charset="utf-8"
> >
> >     Hey folks
> >
> >     Attempted to post this last week while I was joining the list. I
> >     think it
> >     bounced. Please ignore if it is a repeat. I have not seen any
> >     responses and
> >     I am sad.
> >
> >     I found some great discussion of a previous issue* with nested
> >     Stream.parallel operations here and hoping I might find some answers
> >     to a
> >     related question.
> >     *
> >
> http://cs.oswego.edu/pipermail/concurrency-interest/2014-May/012652.html
> >
> >     I am using a ConcurrentHashMap.compute operation in an outer
> >     Stream.parallel forEach operation. A library used in the compute
> method
> >     also uses Stream.parallel.
> >
> >     I have written a test that illustrates the issue and explores
> different
> >     implementations in 215 lines of code.
> >
> https://gist.github.com/dstuebe/89361f64dc44a935e53d0a49f149317c#file-nestedparallel-java
> >
> >     The code deadlocks with the following stack trace:
> >
> https://gist.github.com/dstuebe/89361f64dc44a935e53d0a49f149317c#file-stacktrace-txt
> >
> >     I do not understand why line 87 (the compute block) appears to be
> called
> >     recursively leading to deadlock when I use Stream.parallel for the
> outer
> >     loop.
> >
> >     Best
> >
> >     David
> >     -------------- next part --------------
> >     An HTML attachment was scrubbed...
> >     URL:
> >     <
> http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20180806/543a8fb9/attachment-0001.html
> >
> >
> >     ------------------------------
> >
> >     Message: 2
> >     Date: Mon, 6 Aug 2018 10:31:55 -0400
> >     From: Doug Lea <dl at cs.oswego.edu <mailto:dl at cs.oswego.edu>>
> >     To: concurrency-interest at cs.oswego.edu
> >     <mailto:concurrency-interest at cs.oswego.edu>
> >     Subject: Re: [concurrency-interest] CHM.compute restrictions (was: no
> >              subject)
> >     Message-ID: <45511b57-3bb5-d3ec-d696-81086e7b389f at cs.oswego.edu
> >     <mailto:45511b57-3bb5-d3ec-d696-81086e7b389f at cs.oswego.edu>>
> >     Content-Type: text/plain; charset=utf-8
> >
> >     On 08/06/2018 10:10 AM, David Stuebe via Concurrency-interest wrote:
> >      > Hey folks
> >      >
> >      > Attempted to post this last week while I was joining the list.
> >
> >     To reduce spam, non-member posts are silently dropped. Sorry.
> >
> >      >
> >      > I am using a ConcurrentHashMap.compute operation in an outer
> >      > Stream.parallel forEach operation. A library used in the compute
> >     method
> >      > also uses Stream.parallel.
> >      >
> >      > I have written a test that illustrates the issue and explores
> >     different
> >      > implementations in 215 lines of code.
> >      >
> >
> https://gist.github.com/dstuebe/89361f64dc44a935e53d0a49f149317c#file-nestedparallel-java
> >      >
> >      > The code deadlocks with the following stack trace:
> >      >
> >
> https://gist.github.com/dstuebe/89361f64dc44a935e53d0a49f149317c#file-stacktrace-txt
> >      >
> >      > I do not understand why line 87 (the compute block) appears to be
> >     called
> >      > recursively leading to deadlock when I use Stream.parallel for
> >     the outer
> >      > loop.
> >
> >     A recursive CHM.compute call appears to invoked while trying to
> >     initialize the contents of an element in the same map, which is
> >     disallowed in general, but sometimes works anyway. In most other
> cases,
> >     CHM successfully detects this and throws an exception, but it cannot
> >     catch all of them.
> >
> >     -Doug
> >
> >
> >
> >
> >
> >
> >     ------------------------------
> >
> >     Subject: Digest Footer
> >
> >     _______________________________________________
> >     Concurrency-interest mailing list
> >     Concurrency-interest at cs.oswego.edu
> >     <mailto:Concurrency-interest at cs.oswego.edu>
> >     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
> >
> >
> >     ------------------------------
> >
> >     End of Concurrency-interest Digest, Vol 162, Issue 1
> >     ****************************************************
> >
> >
> >
> > _______________________________________________
> > Concurrency-interest mailing list
> > Concurrency-interest at cs.oswego.edu
> > http://cs.oswego.edu/mailman/listinfo/concurrency-interest
> >
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 7 Aug 2018 11:32:05 -0400
> From: David Stuebe <davidstuebe at upserve.com>
> To: Peter Levart <peter.levart at gmail.com>
> Cc: concurrency-interest at cs.oswego.edu
> Subject: Re: [concurrency-interest] CHM.compute restrictions
> Message-ID:
>         <
> CAJh0pqHYncVSJeXbXsmkjqx7Vppyybr7yMYHfsYX8sMcvcfDJg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Peter, Doug
>
> That is very interesting!
>
> I am not sure I fully grok your explanation, but it did suggest changing
> the inner task pool to be a fixed thread pool rather than a forkjoin pool.
> This appears to alleviate the issue!
>
> More experiments to follow.
>
> David
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20180807/19501b59/attachment-0001.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
> ------------------------------
>
> End of Concurrency-interest Digest, Vol 162, Issue 3
> ****************************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20180807/7141d102/attachment-0001.html>


More information about the Concurrency-interest mailing list