[concurrency-interest] Single producer, single consumer: unexpected delays for producer

David Holmes dcholmes at optusnet.com.au
Fri Aug 8 21:49:59 EDT 2008


Dan,

The simple answer is that the consumer is using a resource that the producer
needs. It could be processor cycles, or network access, or contention on
some shared object at the application, VM or OS level. Assuming you're
multi-processor I'd suspect contention somewhere in the TCP/IP stack. What
OS are you on? How many processors do you have?

At 100,000 messages per second you only have 10 microseconds to fully
process each one. That's a tall order on a non-real-time system. Even if
your best processing time is within 10us, it isn't a sustainable rate, so
you should expect to see a lot of variation in the message rate that you can
handle. When GC kicks in you're going to drop a lot of messages.

Cheers,
David Holmes

> -----Original Message-----
> From: concurrency-interest-bounces at cs.oswego.edu
> [mailto:concurrency-interest-bounces at cs.oswego.edu]On Behalf Of Daniel
> Harvey
> Sent: Saturday, 9 August 2008 1:11 AM
> To: concurrency-interest at cs.oswego.edu
> Subject: [concurrency-interest] Single producer,single consumer:
> unexpected delays for producer
>
>
> Hi,
> I'm writing a program that receives a very high message rate
> (~100,000 per second) over a TCP socket, filters out the useful
> information (about 1/5 of the above), and forwards that information
> over another TCP socket. I have been having trouble keeping up with
> the incoming messages, and I think its an oddity of the
> LinkedBlockingQueue I am using to queue the useful updates for later
> sending. The code looks something like this:
>
> public class MessageForwarder {
> 	SendThread	sendThread = new SendThread();
> 	 ...
>
> 	public void onMessage(RapidMessage message) {
> 		long start = System.nanoTime();
>
> 		// Do a bunch of checks to decide if useful...
> 		boolean useful = ...
>
> 		if (useful) {
> 			sendThread.send(throttled.toMessageString())
> 		}
> 		processingTime += (System.nanoTime() - start);
> 	}
>
> 	// inner class
> 	public class SendThread extends Thread {
>
> 		private final BlockingQueue<String> queued = new
> LinkedBlockingQueue<String>(10000);
> 		private final CharsetEncoder              encoder =
> Charset.forName
> ("ISO-8859-1").newEncoder();
>
> 		public void send(String message) {
> 			long start = System.nanoTime();
> 	 		boolean success = queued.offer(message);
> 			sendingTime +=  (System.nanoTime() - start);
>
> 			if (!success) {
> 				log.error("Failed to send message:
> queue is full");
> 			}
> 		}
>
> 		public void run() {
> 			while (true) {
> 				String message = queued.take();
> 				ByteBuffer buffer =
> encoder.encode(CharBuffer.wrap(message));
> 				socketChannel.write(buffer);
> 			}
> 		}
> 	}
> ...
> }
>
> When I print out periodically the value of the "processingTime" and
> the "sendingTime", they are similar: the majority of the processing
> time is taking place in the call to sendThread.send(). The checks to
> see if the message is useful are fast enough that it could run
> 100,000 messages per second, but the calls to "send" are much slower,
> only 20,000 messages per second at best. Note that the queue is not
> filling up (not seeing the error message anywhere). What I really
> don't understand is that if I have the consumer (Sending) thread
> throw away the message after it takes it from the queue (ie just do
> repeated "take"s), then the call to sendThread.send() speeds up by a
> factor of around 6. I don't understand how if the queue isn't filling
> up, why would the actions of the consumer matter to the speed of the
> producer?
>
> Any help or advice you can give would be much appreciated!
>
> Thanks,
> Dan Harvey
>
> _______________________________________________
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest



More information about the Concurrency-interest mailing list