[concurrency-interest] Deadlock with when no threads have the lock

Stuart Monteith stuart.monteith at linaro.org
Mon Jun 19 06:53:00 EDT 2017


I've encountered a similar issue before with park/Unpark. There was
bug where class loading had a hand in causing Unparks to be missed:
    https://bugs.openjdk.java.net/browse/JDK-8074773

There can only be one outstanding unpark at a time, so subsequent
unparks can be lost.


BR,
   Stuart

On 17 June 2017 at 08:34, Viktor Klang <viktor.klang at gmail.com> wrote:
> There's the possibility that a task which unlocks the lock is stuck in the
> executor's submission queue because all the workers are blocked on obtaining
> the lock.
>
> --
> Cheers,
>>
> On Jun 17, 2017 8:13 AM, "Kirk Pepperdine" <kirk at kodewerk.com> wrote:
>>
>> There are 94 threads that appear to be waiting on 2 different locks. 47
>> are waiting on a ForkJoinExecutor and are simply idle. From this one would
>> hope that you were on a 48 core machine.
>>
>> As an example….
>>
>> "marathon-akka.actor.default-dispatcher-35" #271 prio=5 os_prio=0
>> tid=0x00007ff868004000 nid=0x11fd4 runnable [0x00007ff8a9be0000]
>>    java.lang.Thread.State: WAITING (parking)
>> 	at sun.misc.Unsafe.park(Native Method)
>> 	- parking to wait for  <0x00000005c0581aa0> (a
>> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
>> 	at scala.concurrent.forkjoin.ForkJoinPool.scan(ForkJoinPool.java:2075)
>> 	at
>> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>> 	at
>> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>>
>>
>> 26 are waiting on a ReentantLock. These are probably the threads of
>> interest.
>>
>> "ForkJoinPool-2-worker-33" #25346 daemon prio=5 os_prio=0
>> tid=0x00007ff80c11c000 nid=0x5747 waiting on condition [0x00007ff8a91d8000]
>>    java.lang.Thread.State: WAITING (parking)
>> 	at sun.misc.Unsafe.park(Native Method)
>> 	- parking to wait for  <0x00000005c0af6298> (a
>> java.util.concurrent.locks.ReentrantLock$FairSync)
>> 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>> 	at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>> 	at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
>> 	at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
>> 	at
>> java.util.concurrent.locks.ReentrantLock$FairSync.lock(ReentrantLock.java:224)
>> 	at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
>> 	at mesosphere.marathon.util.RichLock$.apply$extension(Lock.scala:7)
>> 	at mesosphere.marathon.util.Lock.apply(Lock.scala:25)
>> 	at mesosphere.marathon.util.KeyedLock.apply(WorkQueue.scala:94)
>>
>>
>> Are you waiting on an external signal?
>>
>>
>> — Kirk
>>
>>
>>
>>
>>
>> On Jun 17, 2017, at 6:03 AM, Henri Tremblay <henri.tremblay at gmail.com>
>> wrote:
>>
>> If you keep doing thread dumps, is the lock and the waiting threads the
>> same? So a real deadlock. If yes, Kirk suggestion is interesting. And I
>> would also do a heap dump to have a look at the threads holding a reference
>> to the reentrant lock.
>>
>> On 16 June 2017 at 18:55, Kirk Pepperdine <kirk at kodewerk.com> wrote:
>>>
>>> Hi Ken,
>>>
>>> If you have a lock with nothing holding on to it then either a VM thread
>>> has the lock. The most common culprit are GC threads. I’ve not taken a deep
>>> look at the code but you could set -XX:+PrintConcurrentLocks which would
>>> give you an idea if the problem is with the re-enterant lock. Not looked too
>>> deeply at the code but I don’t see anything obvious at the moment.
>>>
>>> — Kirk
>>>
>>> On Jun 16, 2017, at 11:36 PM, Ken Sipe <kensipe at gmail.com> wrote:
>>>
>>> We have encountered a strange deadlock scenario in which it appears that
>>> all threads are waiting on acquiring a ReentrantLock, but no thread has the
>>> lock.
>>>
>>> Any thoughts on this?
>>>
>>> https://gist.github.com/timcharper/9ab4fea9da4669e620507e85e764d94a
>>>
>>>
>>> Thanks,
>>> ken
>>>
>>>
>>> _______________________________________________
>>> Concurrency-interest mailing list
>>> Concurrency-interest at cs.oswego.edu
>>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>
>>>
>>>
>>> _______________________________________________
>>> Concurrency-interest mailing list
>>> Concurrency-interest at cs.oswego.edu
>>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>>
>>
>>
>>
>> _______________________________________________
>> Concurrency-interest mailing list
>> Concurrency-interest at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>


More information about the Concurrency-interest mailing list