[concurrency-interest] ReentrantReadWriteLock in inconsistent state

Gregg Wonderly gregg at cytetech.com
Wed Aug 1 10:09:27 EDT 2012


One of the comments hints at an order of instructions issue where the head can 
be replaced, without being configured for wakeup to work.  Then, inbetween 
instructions for linking and then setting the wakeup state, a wakeup might find 
the head not ready for wakeup?

A long long time ago, in Java1.3, I encountered a wakeup problem with 
Object.wait() and Object.notify() in one of my applications, and just started 
using Object.wait(long) instead.  That's a pretty dependable way to manage 
notification/wakeup issues so that bugs from all ends of the spectrum don't 
result in loss of services.   Of course, using some instrumentation at that 
point, to log that you had wait (count retries) longer than expected, can help 
track down bugs which actually result in undesirable pauses.

Gregg Wonderly

On 8/1/2012 8:27 AM, √iktor Ҡlang wrote:
> Hi Phil,
>
> Related to this?
> http://bugs.sun.com/view_bug.do?bug_id=6822370
>
> Cheers,
>>
> On Wed, Aug 1, 2012 at 3:20 PM, Phil Harvey <phil at philharveyonline.com
> <mailto:phil at philharveyonline.com>> wrote:
>
>     We had a deadlock-like failure of our application recently.
>
>     I initially reported it on the BDB JE forum (
>     https://forums.oracle.com/forums/thread.jspa?messageID=10480988) but further
>     analysis of the heap and thread dumps has exposed a problem that looks like
>     a Java locking bug. I'm hoping you can offer advice on whether this is the case.
>
>     We’re using Oracle JVM 1.6.0_25-b06, running on Linux version:
>     2.6.18-194.32.1.el5.
>
>     We are launching Java as follows: java -server -XX:+UseConcMarkSweepGC
>     -XX:+HeapDumpOnOutOfMemoryError -Xmx1024m ...
>
>     Several consecutive thread dumps showed that Thread t at 41101 was blocked
>     indefinitely in ReentrantReadWriteLock. writeLock().lock().
>
>     We know from code inspection that nothing ever takes a read lock on this
>     ReentrantReadWriteLock, so started trying to find out what has got its write
>     lock.
>
>     The output of "jstack -l" should list which thread holds this exclusive lock
>     in the "locked ownable synchronizers" section but does not.
>
>     Our first theory was that the owning thread might have terminated.
>
>     We wrote a simple test program to explore this. We found from heap dump
>     analysis that even if the owning thread terminates, the lock itself still
>     refers to it via the ReentrantReadWriteLock.WriteLock.sync.
>     exclusiveOwnerThread field. Looking in the java.util.concurrent source code,
>     it seems that this field only gets null'ed when the lock is released.
>
>     However, looking in the heap dump taken following our "deadlock", we were
>     surprised to find that the lock in question has a null
>     sync.exclusiveOwnerThread field.
>
>     Surely a write lock should be in one of two states (except possibly for a
>     tiny instant when its state is being non-atomically switched):
>
>     1) The lock is available, and sync.exclusiveOwnerThread is null 2) The lock
>     is unavailable, and sync.exclusiveOwnerThread is populated
>
>     But our lock was indefinitely in this state:
>
>     3) The lock is unavailable and sync.exclusiveOwnerThread is null
>
>     Does anyone know whether this represents a bug? If not, can you explain what
>     it means for a lock to be in this counterintuitive state?
>
>     Thanks, Phil
>
>
>     _______________________________________________
>     Concurrency-interest mailing list
>     Concurrency-interest at cs.oswego.edu <mailto:Concurrency-interest at cs.oswego.edu>
>     http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
>
>
> --
> Viktor Klang
>
> Akka Tech Lead
> Typesafe <http://www.typesafe.com/>- The software stack for applications that scale
>
> Twitter: @viktorklang
>
>
>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



More information about the Concurrency-interest mailing list