[concurrency-interest] Hung progress in ThreadPoolExecutor ExecutorCompletionService when slave threads killed.

Dawid Weiss dawid.weiss at gmail.com
Sun May 13 16:08:31 EDT 2012


Hi there.

I wrote a JUnit runner that attempts to annihilate any threads that
leaked out of a given test's scope. The process starts by setting an
interrupt flag, then after some delay it just calls Thread.stop()
(let's not go into why this particular method was chosen for now, I
realize the consequences).

Anyway. I've encountered an interesting scenario where a
ThreadPoolExecutor is used in combination with
ExecutorCompletionService (in Apache Lucene). What happens is that the
thread pool's threads are first interrupted (which doesn't terminate
thread pool threads, they are reused) and then stopped, which does
seem to kill them (I see uncaught stacks:

java.lang.IllegalMonitorStateException
	at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:155)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
	at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:460)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:449)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)

and then the thread pool's pool is refilled with fresh threads.

Interestingly, any subsequent use of the executor completion service
deadlocks. I would assume the problem is somewhere in my code but this
happens only on Java 1.7, on 1.6 the execution proceeds normally with
those refilled threads. A stack trace of the hung process shows all
executor service threads parked on:

"LuceneTestCase-84-thread-10" prio=6 tid=0x00000000087a3800 nid=0x19a8
waiting on condition [0x000000000e00e000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000f92a0318> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)

and the main thread utilizing completion service is parked on the queue:

"TEST-TestScope-org.apache.lucene.search.TestPrefixInBooleanQuery.testTermBooleanQuery-seed#[10A1C9CC1F06B49B]"
prio=6 tid=0x0000000008798000 nid=0x1cbc waiting on condition
[0x000000000aebd000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000f94614c0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ExecutorCompletionService.take(ExecutorCompletionService.java:193)
	at org.apache.lucene.search.IndexSearcher$ExecutionHelper.next(IndexSearcher.java:756)
        ...

The completion service has submitted tasks, they're just not executed.
This in fact is a deadlock state.

Any hints/ ideas what might have changed in between 1.6 and 1.7 that
may be causing this? I couldn't reproduce on a small example but the
above scenario is 100% reproducible (well, on my machine and Java
1.7.0_03-b05).

Thanks,
Dawid


More information about the Concurrency-interest mailing list