[concurrency-interest] More control on 'full' with blockingqueue.

Peter Veentjer - Anchor Men p.veentjer at anchormen.nl
Sun Aug 7 04:42:17 EDT 2005


For a searchengine platform I`m currently developing (based on Lucene) I need better control on BlockingQueues when they are full. If I only check on the number of items in the queue it could happen that I would have 10 documents of 100 mb in a Queue (and this is a bad thing). But if I restrict the size of the Queue too much, processing 100 documents of 1kb would take too long. Therefore I need better control if it is allowed to add an item to the Queue and that is why I have written the ConditionalBlockingQueue and it uses a strategy: IsFull (the names have to be improved) and this allows better control. The behaviour of BlockingQueues that block on the capacity (the number of items in the Queue) could be made with the IsFull strategy.
 
My question is what you think of this solution and if there are others that had the same problem.
 
At the moment (I haven`t added the new ConditionalBlockingQueue implementation) I have solved the problem by creating 2 LinkedBlockingQueues: small document queue (with a big capacity) and a big document queue (with a small capacity) where I have some control on the total amount of documents being processed but in some cases it could bring the system down to its knees (if there are a lot of big documents).
 
Another thing I was wondering about, is why final class fields are places in local variables? Functionally they are equivalent so the only reason I can think of is that it would be faster. But isn`t this an optimalisation the compiler could do (if it is faster)?
 
example from the LinkedBlockingQueue:
 
 public E take() throws InterruptedException {
        E x;
        int c = -1;
        final AtomicInteger count = this.count;//Why??
        final ReentrantLock takeLock = this.takeLock;//Why??
        takeLock.lockInterruptibly();
        try {
            try {
                while (count.get() == 0)
                    notEmpty.await();
            } catch (InterruptedException ie) {
                notEmpty.signal(); // propagate to a non-interrupted thread
                throw ie;
            }
            x = extract();
            c = count.getAndDecrement();
            if (c > 1)
                notEmpty.signal();
        } finally {
            takeLock.unlock();
        }
        if (c == capacity)
            signalNotFull();
        return x;
    }



More information about the Concurrency-interest mailing list