[concurrency-interest] Target CPU utilization

David Holmes dcholmes at optusnet.com.au
Thu Feb 15 18:20:38 EST 2007


> > - You might not be the only application trying to run on the system.
> I thought of that...but most modern operating systems will do a good
> job scheduling the running applications fairly. (Even with a single
> threaded application, I will not include sleeps in order to yield CPU
> time to other apps in the system.)

Yes but the response time will suffer. Two systems "tuned" for 100%
utilization won't meet the throughput/response times they were tuned for if
they only get 50% of the expected CPU.

> > - You might want to allow room for handling transient overload.
> I am not sure I understand this one. Please, could you elaborate?
> (Assume my target is full CPU utilization. Assume I also allow for a
> copious wait time ratio. I will create a superfluous number of
> threads... Hmm... I'd instinctively think that just the opposite is
> true: the less the CPU utilization target, the less I am able to
> handle transient overload. Isn't it?)

I'm thinking about this from a queuing theory perspective. You have an
expected load on your application due to the expected number of tasks and
the work those tasks perform. Assuming you can quantify both then you can
work out expected response times, average queue length etc for a given
number of worker threads and CPUs. If more work comes in than expected - a
transient overload - then the queue length will grow and the response time
will increase. If this queue growth triggers growth of the pool (ie blocking
queue becomes full so pool creates threads from core size up to maximum
size) then without spare CPU cycles you won't help with response times (just
queue length). But if you've allowed for some CPU cycles then they are
available to assist with dealing with the transient overload.

Or to put it more simply. Your normal processing mode allows for idle time
on some CPU's and meets throughput goal X. Then under overload you can
utilize the idle time to try and meet a revised throughput goal Y < X.
Without the spare capacity under overload your throughput might drop to Z <<

If you are providing pools as part of a library you need to provide the
necessary tuning knobs as part of the API.


More information about the Concurrency-interest mailing list