[concurrency-interest] Memory sensitive memoization

Sanchay Harneja sanchay.h at gmail.com
Tue Jun 28 11:00:18 EDT 2011


For the time being we are thinking of going with the following approach -
1. Use MapMaker with hard keys, soft values.
2. At regular intervals poll process heap used ratio (using
java.lang.management). If more than certain threshold, manually clear out
some entries in the cache.

This solution gives us good amount of flexibility. We cannot afford to go
LRU and put a size cap on our caches - since we cannot predict the size of
our caches, or the number of caches for that matter !

Thanks!
Sanchay

On Tue, Jun 28, 2011 at 6:35 PM, Roman Elizarov <elizarov at devexperts.com>wrote:

>  We at Devexperts have come through the same experience. We were using
> SoftReferences for caching for a while and had found then inadequate for
> exactly the same reason. Now we use only size-limited LRU caches (sometimes
> they are also time-limited). We rarely use true LRU, though (due to cost of
> its maintenance in concurrent scenario). Usually it is some kind of
> approximate-LRU algorithm.****
>
> ** **
>
> Sincerely,****
>
> Roman Elizarov****
>
> ** **
>
> *From:* concurrency-interest-bounces at cs.oswego.edu [mailto:
> concurrency-interest-bounces at cs.oswego.edu] *On Behalf Of *Dhanji R.
> Prasanna
> *Sent:* Tuesday, June 28, 2011 9:49 AM
> *To:* Jed Wesley-Smith
> *Cc:* Sanchay Harneja; concurrency-interest at cs.oswego.edu; Tim Peierls
> *Subject:* Re: [concurrency-interest] Memory sensitive memoization****
>
> ** **
>
> ** **
>
> On Tue, Jun 28, 2011 at 2:50 PM, Jed Wesley-Smith <
> jwesleysmith at atlassian.com> wrote:****
>
> We have found the hard way that using SoftReferences for what is
> essentially an application level concern (caching) is actually
> counter-productive for the most part. The problem being that they clear at
> the time when the application is under serious GC pressure, and there is no
> way to prioritise which ones get cleared. This means that caches start
> missing and expensive operations are performed right at the time when the
> application is under the most load, usually increasing the pressure on it.
> ****
>
> ** **
>
> Exactly, this was one of our most serious problems. Not all data LRU across
> the same priority level. The problem with using SoftRefs for caching is that
> it treats all memory as equal even across caches. There are also many many
> other issues with softrefs, such as clearing overhead, clock timestamping,
> the overhead of finalization (done in a single thread) and so on.****
>
>  ****
>
>  I have heard several others (Cliff Click's "The JVM does what?" for
> instance, as well as various Google sources) express the same opinion, that
> SoftReferences are at best a somewhat illusory benefit.****
>
>  ** **
>
> Yep. There are others at Google who (reasonably) argue that efficient
> caches can be built in Java but I was quite burned after deploying some of
> them in production and argue against them often. I worked with a number of
> teams (while at Google) managing very large Java deployments who had similar
> conclusions to mine.****
>
> ** **
>
> Dhanji.****
>
> ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20110628/caf8ce35/attachment.html>


More information about the Concurrency-interest mailing list