[concurrency-interest] Exception handling & Executors

Jan Nielsen jnielsen at sungardsct.com
Tue Oct 25 15:38:10 EDT 2005


With respect to your comment below about not being able to do anything 
with an out-of-memory condition, could you not reliably release, for 
example, cached data in an OutOfMemoryError handler:

  Map m = new HashMap();
  Object o = new PreAllocatedMemoryInBytes( 16 );
    // blah, blah, blah...allocating all the memory to 'm' values...
  catch( OutOfMemoryError e )
      o = null;
      // only log after we have enough memory

The m.clear() method may require some allocation to work, so it would be 
susceptible to failure without having that much available memory, 

    public void clear() {
        Entry[] tab = table;
        for (int i = 0; i < tab.length; i++)
            tab[i] = null;
        size = 0;

In the single-threaded case, pre-allocating sufficient memory to perform 
this operation should address this, should it not?



Joe Bowbeer wrote:

>A few ideas:
>1. If you execute Runnable instead of FutureTask then any
>RuntimeExceptions and Errors thrown can escape the Executor.
>2. You can hook the ThreadPoolExecutor.afterExecute method to handle
>the RuntimeExceptions thrown by your task.  (Note: the Throwable
>signature is a little misleading.)
>3. You can use the Thread.set[Default]UncaughtExceptionHandler to
>handle uncaught exceptions and errors of all types.  (To associate the
>exception handling with your executor, you can provide your own thread
>4. If you want to use FutureTask then you can hook its done() method
>and deal with the exception there.
>A comment:
>I don't know how you expect to do anything with an
>OutOfMemoryException, except perhaps log it.  I don't know of any way
>to reliably recover from these.
>On 10/25/05, John Adams <jadams3 at nortel.com> wrote:
>>First off, thanks for maintaining this list, it has proven invaluable to me
>>as I learn to use the concurrency utilities in Java 5.0.
>>The reason I'm posting is that I've had a lot of problems using the Executor
>>framework to deal with exceptions from errant tasks.  I've complained about
>>this to Sun, whose response was essentially that exceptions belong with the
>>task, so they should be try / caught at that level.
>>I don't agree.  I don't, for example, believe that it is the responsibility
>>of the task to handle OutOfMemoryError's, or in fact most of the classes
>>that occur within the Error class of throwables.   Many of those exceptions
>>imply that the tasks should have some knowledge about how to respond to
>>events are within the domain of the platform the tasks are executing on.
>>Now you could reasonably argue --> What is there to do anyways in that case
>>?   All I want to do is a System.exit(), since we have a watchdog process
>>that does "the right thing" when the VM goes down.  In theory that would
>>have been easy for me to do with our custom threadpools pre-1.5, however to
>>get it working with the concurrent utilities has been a painful experience.
>>I thought it may be helpful to note what we tried in what order, so that
>>maybe we could get some support beyond the custom Future task support going
>>in later VM releases.  So, in order ....
>>1. Override the ThreadGroup before creating the Executor.    We did this
>>initially expecting things to be easy, however we were quickly thwarted by
>>the DefaultThreadFactory that does this ...
>>       DefaultThreadFactory() {
>>            SecurityManager s = System.getSecurityManager();
>>            group = (s != null)? s.getThreadGroup() :
>>         }
>>I can see some security reasons for this, however I was not expecting it -->
>>I have stages that I wanted to log & name according to threadgroups.
>>However, that's not insurmountable ... I created my own threadfactory which
>>just took the currentThread().getThreadGroup() ... no luck.
>>  I also used the setExceptionHandler() method on Thread & ThreadGroup, but
>>to no avail.  I left this code in since I wanted this code to be independent
>>of our security manager choices, and explicitly setting the exception
>>handler seemed like a reasonable thing to do.  (and maybe relevant to ppl on
>>this list)
>>2.  So then we went to look at the Future and realized what was happening,
>>the future task was catching throwable. (Yes, I know all about custom
>>futures, they don't help me now)  The only way we could rationally seem to
>>do this is to override afterExecute() in the executor and do the following
>>try {
>>                // note I'd rather cast than use temporary objects
>>                if ( !((FutureTask) task).isCancelled()){  // I don't care
>>about CancellationExceptions
>>                    ((FutureTask) task).get();
>>                }
>>             } catch (ExecutionException ee) {
>>                 // ie if (ee == OutOfMemoryError) {System.exit()}
>>            }
>>3. Ok, so now for every task we're checking if it is cancelled, casting 2x,
>>and getting a result we almost never want, all of which have a performance
>>impact.   I tried throwing an OutOfMemoryError .... didn't trip the exit().
>>More than a little aggravated, I looked at the exception, and realized that
>>ExecutionException wraps Throwable, which means I need to inspect the cause.
>> So I changed that bit of code to
>>try {
>>               if ( !((FutureTask) task).isCancelled()){ // I still don't
>>care about CancellationExceptions
>>                    ((FutureTask) task).get();
>>                }
>>             } catch (ExecutionException ee) {
>>                 //  ie if (ee.getCause() == OutOfMemoryError)
>>            }
>>4.  The fourth problem I had was a bit subtle, and didn't come up until
>>later.   We have subsequent tasks that queue up on other executors.  This
>>was resulting in a chain of ExecutionExceptions being set up, which meant we
>>had to recursively inspect all exceptions in the chain to make sure there
>>was no out of memory error.   Think SEDA, and you'll be able to picture why
>>we have multiple stages and chained execution exceptions.  It also makes us
>>more vulnerable to StackOverflow errors.
>>Once you pull that all together, you can catch the OutOfMemoryError cleanly,
>>I think.  The moral of this story may be to not use the Executors if you
>>want to be a framework and don't trust your tasks, but I don't think it has
>>to be this hard.  There are several places in the Java, and patterns that I
>>can think of that have a setExceptionHandler() pattern going on, wouldn't it
>>be reasonable to add similar functionality instead of forcing us to go
>>through these hoops to find an Error ?
>>Again, all of this can be avoided if you trust your tasks to do a try /
>>catch (Throwable t) , but as part of a framework I don't exactly have that
>>luxury.  Thanks again for all the information, I hope this is useful to
>>others on this list.
>>John Adams
>Concurrency-interest mailing list
>Concurrency-interest at altair.cs.oswego.edu

Jan Nielsen
System Architect
Luminis Solutions
SunGard SCT

jnielsen at sungardsct.com

+1 801 257 4155 (voice)
+1 801 485 6606 (facsimile)

90 South 400 West
Salt Lake City, UT 84101

More information about the Concurrency-interest mailing list