[concurrency-interest] Unbounded blocking queues - memory consumption

Bob Lee crazybob at crazybob.org
Thu Oct 20 14:54:44 EDT 2005


Calum,

I might be able to help. I've implemented an object pool for which I
had the same problem, i.e. creating Node objects effectively canceled
out the pool's benefits when the pooled objects were small.

I solved the problem by using ASM to dynamically extend the class of
the object I was pooling at runtime effectively turning it into a
node. For example, given:

class Foo {
}

I dynamically generate this class at runtime:

class PooledFoo extends Foo {
  PooledFoo next;
}

You can get the source here: http://crazybob.org/pool.zip

Bob

---------- Forwarded message ----------
From: Doug Lea <dl at cs.oswego.edu>
To: Calum MacLean <cnmaclean at hotmail.com>
Date: Fri, 07 Oct 2005 09:23:06 -0400
Subject: Re: [concurrency-interest] Unbounded blocking queues - memory
consumption
Calum MacLean wrote:
> Hi
>
> My application is producing a large unbounded quantity of objects which are
> being put into a BlockingQueue for consumption by some other thread.
> I'm currently using LinkedBlockingQueue, as it's unbounded.
> However, I'm seeing that the LinkedBlockingQueue.Node objects take up a fair
> bit of memory - around about 16 bytes each according to my profiler.
> So, while I'm not 100% sure, I'm a bit worried about the memory consumption,
> as it's a large number of objects which are being produced.
>
> Are there any other alternatives for unbounded BlockingQueue implementations
> which maybe take up less memory than LinkedBlockingQueue?
>

I don't believe there is any way to reduce overhead for any
linked structure down and further -- the nodes only have item
and next fields; plus the usual Java per-Object header etc.
If you really need the space and are in full control of the
kinds of elements, you might be able to make a custom version.
If for example, each element is of class Element, you can add
a "next" link to the Element class, used only by the queue, and
then copy-paste-hack LinkedBlockingQueue to directly use it
rather than wrapping each item in a Node. This is not recommended
unless you are desparate though.

-Doug



_______________________________________________
Concurrency-interest mailing list
Concurrency-interest at altair.cs.oswego.edu
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest



More information about the Concurrency-interest mailing list