[concurrency-interest] Hung progress in ThreadPoolExecutor ExecutorCompletionService when slave threads killed.

David Holmes davidcholmes at aapt.net.au
Sun May 13 17:44:26 EDT 2012


Dawid,

Once you have used Thread.stop (regardless of what exception it throws) on
the thread pool then you have to discard the thread pool. The code is not
async-exception safe (and can't be) and so you can have corrupted the
internal state of the pool, leading to subsequent erroneous behaviour/hangs
etc.

David

> -----Original Message-----
> From: concurrency-interest-bounces at cs.oswego.edu
> [mailto:concurrency-interest-bounces at cs.oswego.edu]On Behalf Of Dawid
> Weiss
> Sent: Monday, 14 May 2012 6:09 AM
> To: concurrency-interest
> Subject: [concurrency-interest] Hung progress in ThreadPoolExecutor
> ExecutorCompletionService when slave threads killed.
>
>
> Hi there.
>
> I wrote a JUnit runner that attempts to annihilate any threads that
> leaked out of a given test's scope. The process starts by setting an
> interrupt flag, then after some delay it just calls Thread.stop()
> (let's not go into why this particular method was chosen for now, I
> realize the consequences).
>
> Anyway. I've encountered an interesting scenario where a
> ThreadPoolExecutor is used in combination with
> ExecutorCompletionService (in Apache Lucene). What happens is that the
> thread pool's threads are first interrupted (which doesn't terminate
> thread pool threads, they are reused) and then stopped, which does
> seem to kill them (I see uncaught stacks:
>
> java.lang.IllegalMonitorStateException
> 	at
> java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(Reentrant
> Lock.java:155)
> 	at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(Abst
ractQueuedSynchronizer.java:1260)
> 	at
> java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:460)
> 	at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.
> java:449)
> 	at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor
> .java:1043)
> 	at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
> or.java:1103)
> 	at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
> tor.java:603)
> 	at java.lang.Thread.run(Thread.java:722)
>
> and then the thread pool's pool is refilled with fresh threads.
>
> Interestingly, any subsequent use of the executor completion service
> deadlocks. I would assume the problem is somewhere in my code but this
> happens only on Java 1.7, on 1.6 the execution proceeds normally with
> those refilled threads. A stack trace of the hung process shows all
> executor service threads parked on:
>
> "LuceneTestCase-84-thread-10" prio=6 tid=0x00000000087a3800 nid=0x19a8
> waiting on condition [0x000000000e00e000]
>    java.lang.Thread.State: WAITING (parking)
> 	at sun.misc.Unsafe.park(Native Method)
> 	- parking to wait for  <0x00000000f92a0318> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> 	at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObj
> ect.await(AbstractQueuedSynchronizer.java:2043)
> 	at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.
> java:442)
> 	at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor
> .java:1043)
> 	at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut
> or.java:1103)
> 	at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu
> tor.java:603)
> 	at java.lang.Thread.run(Thread.java:722)
>
> and the main thread utilizing completion service is parked on the queue:
>
> "TEST-TestScope-org.apache.lucene.search.TestPrefixInBooleanQuery.
> testTermBooleanQuery-seed#[10A1C9CC1F06B49B]"
> prio=6 tid=0x0000000008798000 nid=0x1cbc waiting on condition
> [0x000000000aebd000]
>    java.lang.Thread.State: WAITING (parking)
> 	at sun.misc.Unsafe.park(Native Method)
> 	- parking to wait for  <0x00000000f94614c0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> 	at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObj
> ect.await(AbstractQueuedSynchronizer.java:2043)
> 	at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.
> java:442)
> 	at
> java.util.concurrent.ExecutorCompletionService.take(ExecutorComple
> tionService.java:193)
> 	at
> org.apache.lucene.search.IndexSearcher$ExecutionHelper.next(IndexS
> earcher.java:756)
>         ...
>
> The completion service has submitted tasks, they're just not executed.
> This in fact is a deadlock state.
>
> Any hints/ ideas what might have changed in between 1.6 and 1.7 that
> may be causing this? I couldn't reproduce on a small example but the
> above scenario is 100% reproducible (well, on my machine and Java
> 1.7.0_03-b05).
>
> Thanks,
> Dawid
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



More information about the Concurrency-interest mailing list