[concurrency-interest] Memory sensitive memoization

Attila Szegedi szegedia at gmail.com
Mon Jun 27 13:27:08 EDT 2011

I used to be a big fan of using soft refs for memory-sensitive caching, but in most cases nowadays I prefer an explicitly sized LRU cache instead. 

As far as tuning for soft references is concerned, you should be aware of the following issues (specific to garbage collectors in Sun/Oracle HotSpot JVM):

- All references are cleared during a GC cycle. Therefore:
- if you're using a throughput collector (SerialGC, ParallelGC, or ParallelOldGC), regardless of your free memory ratio, refs in old generation are only cleared when memory fills up completely, as that's the only time old generation collector is run. For practical purposes, this means they're always cleared when free memory drops to zero. Not much granularity there; all soft caches in VM accumulate elements, and then when it's time for a full GC, they're all jettisoned. 
- The exception to above is when you have a different -Xms and -Xmx; at least on Server VM, the -XX:SoftRefLRUPolicyMSPerMB is interpreted against -Xmx, so when a full GC is triggered before your heap has reached its -Xmx size, some soft refs will actually survive.
- with CMS, you'll see a more fine-grained clearing of soft references, but as the additional objects in memory increase the pressure on the GC you'll usually need to add more memory, or set a more aggressive initiating occupancy threshold to make sure CMS can clear enough memory without hitting a concurrent mode failure. Basically, you need to find a balance between at least -XX:CMSInitiatingOccupancyFraction,  -Xmx, and -XX:SoftRefLRUPolicyMSPerMB such that you don't hit a full GC. That's hard. In practical cases, it often turns out easier to use an LRU internally and tune its size. YMMV.

- I have no practical experience with G1 collector, so can't speak about its interaction with soft refs, nor can I speak of soft ref behavior on non-HotSpot VMs.


On Jun 27, 2011, at 12:12 AM, Dhanji R. Prasanna wrote:

> […]
> This is the nature of soft references vs weak references. Soft refs are kept around until the JVM requires more memory (this is somewhat tuneable with a free memory ratio, so softrefs can be claimed before hitting memory limits). Your observation is actually the intended, documented behavior.
> I would caution against using soft/weak references as an LRU caching mechanism, having built many Java caches in heavily loaded environments, my experience tells me that out-of-process caches (like memcached) are generally better. Others may disagree, of course.
> Dhanji.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20110627/34597205/attachment.html>

More information about the Concurrency-interest mailing list