[concurrency-interest] LinkedBlockingQueue extract and memory"waste"

Francois Valdy francois.valdy at gmail.com
Wed Feb 11 10:54:22 EST 2009


Same road here, we're currently replacing all our LinkedBlockingQueues
by the "so slightly" modified version.
I'll try to follow-up anyway (here first then on sun bug tracking) as
it's definitely something to fix (I might not be able to replace queue
use in third party libraries).

On Wed, Feb 11, 2009 at 4:30 PM, Shaffer, Darron
<Darron_Shaffer at stercomm.com> wrote:
> I saw this issue in an earlier version of the libraries on one
> particular JVM.  We are now using an in-house copy of the class with the
> change described by Francois.
>
> Our testing at the time showed millions of garbage nodes sitting around
> and being collected very slowly.
>
> -- Darron Shaffer
>
> -----Original Message-----
> From: concurrency-interest-bounces at cs.oswego.edu
> [mailto:concurrency-interest-bounces at cs.oswego.edu] On Behalf Of
> Francois Valdy
> Sent: Tuesday, February 10, 2009 3:27 PM
> To: concurrency-interest at cs.oswego.edu
> Subject: [concurrency-interest] LinkedBlockingQueue extract and
> memory"waste"
>
> Hi,
>
> I'm having memory issues with LinkedBlockingQueue (don't worry, I
> won't come along saying there's a memory leak in the JDK).
>
> The issue is that the method extract() doesn't "help" the GC by
> dereferencing head.next prior to changing the head node (with
> head.next = null;).
>
> What comes then is that once a node has been put in old gen for
> whatever reason, all generated nodes from this LinkedBlockingQueue
> will be as well.
> You'll tell me that memory will be recovered upon a full gc, but my
> point is exactly to avoid full gc.
>
> Easiest way to reproduce issue is for force a gc (System.gc(), or
> JConsole/JVisualVM) during queue usage, as it'll promote the current
> used nodes to old gen.
> Prior to this step memory was recovered smoothly by minor gc's, after
> this step you're good for a full gc of only LinkedBlockingQueue nodes
> (16bytes in JRE32, 32bytes in JRE64).
>
> Doug, I understand your attempt at removing extraneous code, but maybe
> this one wasn't the best choice :)
> LinkedList for instance doesn't follow the same strategy, and
> explicitly dereferences next/previous (even if unnecessary from a
> memory leak pov).
>
> I'll now look for a workaround (might end up "recoding"
> LinkedBlockingQueue with one added line :) )
>
> Any thoughts ? (I won't consider moving to realtime java for just that
> :) )
>
> Cheers.
>
> (this might be the reason for all those unresolved threads about
> LinkedBlockingQueue memory consumption, just a guess)
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>



More information about the Concurrency-interest mailing list