[concurrency-interest] Memory sensitive memoization

Roman Elizarov elizarov at devexperts.com
Tue Jun 28 09:05:19 EDT 2011

We at Devexperts have come through the same experience. We were using SoftReferences for caching for a while and had found then inadequate for exactly the same reason. Now we use only size-limited LRU caches (sometimes they are also time-limited). We rarely use true LRU, though (due to cost of its maintenance in concurrent scenario). Usually it is some kind of approximate-LRU algorithm.

Roman Elizarov

From: concurrency-interest-bounces at cs.oswego.edu [mailto:concurrency-interest-bounces at cs.oswego.edu] On Behalf Of Dhanji R. Prasanna
Sent: Tuesday, June 28, 2011 9:49 AM
To: Jed Wesley-Smith
Cc: Sanchay Harneja; concurrency-interest at cs.oswego.edu; Tim Peierls
Subject: Re: [concurrency-interest] Memory sensitive memoization

On Tue, Jun 28, 2011 at 2:50 PM, Jed Wesley-Smith <jwesleysmith at atlassian.com<mailto:jwesleysmith at atlassian.com>> wrote:
We have found the hard way that using SoftReferences for what is essentially an application level concern (caching) is actually counter-productive for the most part. The problem being that they clear at the time when the application is under serious GC pressure, and there is no way to prioritise which ones get cleared. This means that caches start missing and expensive operations are performed right at the time when the application is under the most load, usually increasing the pressure on it.

Exactly, this was one of our most serious problems. Not all data LRU across the same priority level. The problem with using SoftRefs for caching is that it treats all memory as equal even across caches. There are also many many other issues with softrefs, such as clearing overhead, clock timestamping, the overhead of finalization (done in a single thread) and so on.

I have heard several others (Cliff Click's "The JVM does what?" for instance, as well as various Google sources) express the same opinion, that SoftReferences are at best a somewhat illusory benefit.

Yep. There are others at Google who (reasonably) argue that efficient caches can be built in Java but I was quite burned after deploying some of them in production and argue against them often. I worked with a number of teams (while at Google) managing very large Java deployments who had similar conclusions to mine.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20110628/f26aa57d/attachment-0001.html>

More information about the Concurrency-interest mailing list