[concurrency-interest] ReentrantReadWriteLock in inconsistent state

Ariel Weisberg ariel at weisberg.ws
Wed Aug 1 10:04:58 EDT 2012


Hi,

I remember that. That was fixed Oracle JDK 1.6.0_18. It hasn't been
reproducing for us since 1.6.0_18, but I am not sure if we are using
ReentrantLock in the same way anymore.

The reproducer we used
was [1]https://github.com/VoltDB/voltdb/tree/master/tools/lbd_lock_test
If I remember correctly it prints '.' as it goes and when it hangs it
stops printing dots.

Regards,
Ariel

On Wed, Aug 1, 2012, at 09:27 AM, √iktor Ҡlang wrote:

Hi Phil,



  Related to this?

[2]http://bugs.sun.com/view_bug.do?bug_id=6822370


Cheers,
√
On Wed, Aug 1, 2012 at 3:20 PM, Phil Harvey
<[3]phil at philharveyonline.com> wrote:

  We had a deadlock-like failure of our application recently.

  I initially reported it on the BDB JE forum (
  [4]https://forums.oracle.com/forums/thread.jspa?messageID=10480988)
  but further analysis of the heap and thread dumps has exposed a
  problem that looks like a Java locking bug. I'm hoping you can offer
  advice on whether this is the case.

  We’re using Oracle JVM 1.6.0_25-b06, running on Linux version:
  2.6.18-194.32.1.el5.

  We are launching Java as follows: java -server
  -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -Xmx1024m
  ...

  Several consecutive thread dumps showed that Thread t at 41101 was
  blocked indefinitely in ReentrantReadWriteLock. writeLock().lock().

  We know from code inspection that nothing ever takes a read lock on
  this ReentrantReadWriteLock, so started trying to find out what has
  got its write lock.

  The output of "jstack -l" should list which thread holds this
  exclusive lock in the "locked ownable synchronizers" section but
  does not.

  Our first theory was that the owning thread might have terminated.

  We wrote a simple test program to explore this. We found from heap
  dump analysis that even if the owning thread terminates, the lock
  itself still refers to it via the
  ReentrantReadWriteLock.WriteLock.sync. exclusiveOwnerThread field.
  Looking in the java.util.concurrent source code, it seems that this
  field only gets null'ed when the lock is released.

  However, looking in the heap dump taken following our "deadlock", we
  were surprised to find that the lock in question has a null
  sync.exclusiveOwnerThread field.

  Surely a write lock should be in one of two states (except possibly
  for a tiny instant when its state is being non-atomically switched):

  1) The lock is available, and sync.exclusiveOwnerThread is null 2)
  The lock is unavailable, and sync.exclusiveOwnerThread is populated

  But our lock was indefinitely in this state:

  3) The lock is unavailable and sync.exclusiveOwnerThread is null

  Does anyone know whether this represents a bug? If not, can you
  explain what it means for a lock to be in this counterintuitive
  state?

  Thanks, Phil
  _______________________________________________
  Concurrency-interest mailing list
  [5]Concurrency-interest at cs.oswego.edu
  [6]http://cs.oswego.edu/mailman/listinfo/concurrency-interest



--
Viktor Klang
Akka Tech Lead
[7]Typesafe - The software stack for applications that scale
Twitter: @viktorklang

_______________________________________________

Concurrency-interest mailing list

[8]Concurrency-interest at cs.oswego.edu

[9]http://cs.oswego.edu/mailman/listinfo/concurrency-interest

References

1. https://github.com/VoltDB/voltdb/tree/master/tools/lbd_lock_test
2. http://bugs.sun.com/view_bug.do?bug_id=6822370
3. mailto:phil at philharveyonline.com
4. https://forums.oracle.com/forums/thread.jspa?messageID=10480988
5. mailto:Concurrency-interest at cs.oswego.edu
6. http://cs.oswego.edu/mailman/listinfo/concurrency-interest
7. http://www.typesafe.com/
8. mailto:Concurrency-interest at cs.oswego.edu
9. http://cs.oswego.edu/mailman/listinfo/concurrency-interest
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20120801/532435a7/attachment-0001.html>


More information about the Concurrency-interest mailing list