[concurrency-interest] Thread pools threads exhausted?

robert lazarski robertlazarski at gmail.com
Mon Jan 19 14:17:43 EST 2009


Hi all,

I have an app that reads a lot of url's via apache's httpclient, which
is documented to be thread safe. About once a week or so, we can no
longer query the url's without restarting the app server. We've done a
lot of debugging, but have not isolated the cause yet. One of the
things we are unsure of is the state of our ExecutorService. How can
we tell if all the threads we assigned to the pool are still alive?
Maybe we get timeouts because no threads are available? Our code is
below, which is a springframework bean and therefore deployed in
singleton mode by default. When it fails, the Future times out
repeatably until we restart the app server.  We don't see any uncaught
exception logs.

public class URLReader {
	
	private static final ThreadFactory factory = new
ONExceptionThreadFactory(new ONExceptionHandler());
	private static final ExecutorService executorService =
Executors.newFixedThreadPool(10, factory);
	// See the HttpClient options if need be in HttpConnectionManager the
associated Parameters class.
	private HttpClient objHttp;

	/** commons logging declaration. */
    private static Log logger = LogFactory.getLog(
    		URLReader.class);

    private int timeout_in_seconds;
    private int max_tries;
    private String appInstance;
	
	/**
     * Connect to given URL and return response, or timeout. See
http://commons.apache.org/httpclient/threading.html
     * for programming guidelines and default settings.
     *
     * The behavior of this method is to encode the url.
     */
    public String readURL(final String url) throws Exception {

    	// lazy init
    	if (objHttp == null) {
    		MultiThreadedHttpConnectionManager mgr = new
MultiThreadedHttpConnectionManager();
    		HttpConnectionManagerParams params = new HttpConnectionManagerParams();
    		params.setDefaultMaxConnectionsPerHost(10);
    		mgr.setParams(params);
    		objHttp = new HttpClient(mgr);
    	}
		Callable<String> task = new Callable<String>() {
		     public String call() {
	        	 logger.debug("appInstance: "+appInstance+ " , says ...
readURL call() invoked with url: " + url);
	             // Send the POST request.
	             PostMethod objPost = new PostMethod(url);
		         try {
		             objHttp.executeMethod(objPost);
	                     // Read the response.
		             byte[] byteResponse = objPost.getResponseBody();
		             return new String(byteResponse);
		         } catch (Exception ex) {
		        	 logger.error(ex.getMessage(), ex);
		         } finally {
		        	 objPost.releaseConnection();
		         }
                         // couldn't connect!
		         return null;
		     }
		};

		Future<String> future = executorService.submit(task);
		try {
		    // wait and timeout or return result
		    return future.get(timeout_in_seconds, TimeUnit.SECONDS);	
		} catch (InterruptedException ex) {
                      // Re-assert the thread's interrupted status
                     Thread.currentThread().interrupt();
                     // We don't need the result, so cancel the task too
                     future.cancel(true);
                     logger.error("Thread interrupted, connection to
url: " + url + " is cancelled: " + ex.getMessage(), ex);
                } catch (Exception ex) {
                     // We don't need the result, so cancel the task
                     future.cancel(true);
        	     logger.error(ex.getMessage(), ex);
                }
               // couldn't connect!
               return null;
	   }
       }

/** In long running applications, always use uncaught exception handlers
 *  for all threads that at least log the exception. */
public class ONExceptionThreadFactory implements ThreadFactory  {
	
	private static final ThreadFactory defaultFactory =
Executors.defaultThreadFactory();
	
    private final Thread.UncaughtExceptionHandler handler;

    public ONExceptionThreadFactory(Thread.UncaughtExceptionHandler handler) {
        this.handler = handler;
    }

    public Thread newThread(Runnable r) {
        Thread t = defaultFactory.newThread(r);
        t.setUncaughtExceptionHandler(handler);
        return t;
    }

}

public class ONExceptionHandler implements Thread.UncaughtExceptionHandler {
	
   /** commons logging declaration. */
    private static Log logger = LogFactory.getLog(
                ONExceptionHandler.class);

    public void uncaughtException(Thread t, Throwable ex) {
                logger.error("Unhandled exception caught by threadname: "
                                + t.getName() + "\n" + ex.getMessage(), ex);
    }

}


More information about the Concurrency-interest mailing list