[concurrency-interest] Thread pools threads exhausted?

Joe Bowbeer joe.bowbeer at gmail.com
Mon Jan 19 14:36:12 EST 2009


Note that executorService.submit can throw RejectedExecutionException but
this is not handled in readUrl.

Recommend:

Configure the rejected execution handler on your executor service to log
rejections.

And/or handle rejected executions in readUrl.

Also, are you only submitting to one host?  I notice maxConnectionsPerHost
is the same as the fixed size of the thread pool.

Joe

On Mon, Jan 19, 2009 at 11:17 AM, robert lazarski wrote:

> Hi all,
>
> I have an app that reads a lot of url's via apache's httpclient, which
> is documented to be thread safe. About once a week or so, we can no
> longer query the url's without restarting the app server. We've done a
> lot of debugging, but have not isolated the cause yet. One of the
> things we are unsure of is the state of our ExecutorService. How can
> we tell if all the threads we assigned to the pool are still alive?
> Maybe we get timeouts because no threads are available? Our code is
> below, which is a springframework bean and therefore deployed in
> singleton mode by default. When it fails, the Future times out
> repeatably until we restart the app server.  We don't see any uncaught
> exception logs.
>
> public class URLReader {
>
>        private static final ThreadFactory factory = new
> ONExceptionThreadFactory(new ONExceptionHandler());
>        private static final ExecutorService executorService =
> Executors.newFixedThreadPool(10, factory);
>        // See the HttpClient options if need be in HttpConnectionManager
> the
> associated Parameters class.
>        private HttpClient objHttp;
>
>        /** commons logging declaration. */
>    private static Log logger = LogFactory.getLog(
>                URLReader.class);
>
>    private int timeout_in_seconds;
>    private int max_tries;
>    private String appInstance;
>
>        /**
>     * Connect to given URL and return response, or timeout. See
> http://commons.apache.org/httpclient/threading.html
>     * for programming guidelines and default settings.
>     *
>     * The behavior of this method is to encode the url.
>     */
>    public String readURL(final String url) throws Exception {
>
>        // lazy init
>        if (objHttp == null) {
>                MultiThreadedHttpConnectionManager mgr = new
> MultiThreadedHttpConnectionManager();
>                HttpConnectionManagerParams params = new
> HttpConnectionManagerParams();
>                params.setDefaultMaxConnectionsPerHost(10);
>                mgr.setParams(params);
>                objHttp = new HttpClient(mgr);
>        }
>                Callable<String> task = new Callable<String>() {
>                     public String call() {
>                         logger.debug("appInstance: "+appInstance+ " , says
> ...
> readURL call() invoked with url: " + url);
>                     // Send the POST request.
>                     PostMethod objPost = new PostMethod(url);
>                         try {
>                             objHttp.executeMethod(objPost);
>                             // Read the response.
>                             byte[] byteResponse =
> objPost.getResponseBody();
>                             return new String(byteResponse);
>                         } catch (Exception ex) {
>                                 logger.error(ex.getMessage(), ex);
>                         } finally {
>                                 objPost.releaseConnection();
>                         }
>                         // couldn't connect!
>                         return null;
>                     }
>                };
>
>                Future<String> future = executorService.submit(task);
>                try {
>                    // wait and timeout or return result
>                    return future.get(timeout_in_seconds, TimeUnit.SECONDS);
>                } catch (InterruptedException ex) {
>                      // Re-assert the thread's interrupted status
>                     Thread.currentThread().interrupt();
>                     // We don't need the result, so cancel the task too
>                     future.cancel(true);
>                     logger.error("Thread interrupted, connection to
> url: " + url + " is cancelled: " + ex.getMessage(), ex);
>                } catch (Exception ex) {
>                     // We don't need the result, so cancel the task
>                     future.cancel(true);
>                     logger.error(ex.getMessage(), ex);
>                }
>               // couldn't connect!
>               return null;
>           }
>       }
>
> /** In long running applications, always use uncaught exception handlers
>  *  for all threads that at least log the exception. */
> public class ONExceptionThreadFactory implements ThreadFactory  {
>
>        private static final ThreadFactory defaultFactory =
> Executors.defaultThreadFactory();
>
>    private final Thread.UncaughtExceptionHandler handler;
>
>    public ONExceptionThreadFactory(Thread.UncaughtExceptionHandler handler)
> {
>        this.handler = handler;
>    }
>
>    public Thread newThread(Runnable r) {
>        Thread t = defaultFactory.newThread(r);
>        t.setUncaughtExceptionHandler(handler);
>        return t;
>    }
>
> }
>
> public class ONExceptionHandler implements Thread.UncaughtExceptionHandler
> {
>
>   /** commons logging declaration. */
>    private static Log logger = LogFactory.getLog(
>                ONExceptionHandler.class);
>
>    public void uncaughtException(Thread t, Throwable ex) {
>                logger.error("Unhandled exception caught by threadname: "
>                                + t.getName() + "\n" + ex.getMessage(), ex);
>    }
>
> }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20090119/13d9fc43/attachment.html>


More information about the Concurrency-interest mailing list