[concurrency-interest] ThreadLocal vs ProcessorLocal
nathan.reynolds at oracle.com
Wed Oct 17 19:18:07 EDT 2012
Linux and Windows thread schedulers will migrate threads among cores on
the same processor and to different processors very readily. Solaris
thread scheduler on Sparc really has to be pushed to move a thread to
another processor. The default for Xen virtual guests is to have the
vCPUs assigned to a particular processor and the vCPUs can't migrate off
that processor. The vCPUs will bounce around the different cores on the
same processor very readily, though. Yes, load balancing can be
problematic on Xen. A test just bumped into that problem.
When building concurrent algorithms, data structures, locks, etc, one
should definitely consider the core that the thread is currently running
on. However, the design should assume that the thread won't be running
on that core or processor for very long.
When I built a NUMA-aware reader writer in C++, each reader acquires the
lock tied to a particular core. When the reader is done, it has to
release that same lock even though it might be running on a different
core. This gave a huge scalability boost. We had 3 Nehalem processors
100% utilized. When we plugged in the 4^(t)^(h) Nehalem processor, it
was 100% utilized but no increase in throughput. When I make the reader
writer lock NUMA-aware, then we got a significant boost in throughput
from the 4^(t)^(h) Nehalem processor.
As for hot adding processors, this isn't too difficult. Most data
structures, locks, etc can handle it. For example, simply add more
leaves to the striping. The changes don't have to be done efficiently
or be concerned about scalability. This is because hot adding
processors should be a rare event in the lifetime of the process.
As for hot removing processors, this can be more difficult. The only
thing I could figure out is that the application has to notify the data
structure when it is okay for it to deal with the removal.
For example, a NUMA-aware Exchanger wouldn't be bothered by hot plugged
processors. Sure, it will see an impact at the moment of the change,
but it will then stabilize to the new processor configuration fairly
quickly. A NUMA-aware reader writer lock will require a bit more work
but again it is doable.
Consulting Member of Technical Staff | 602.333.9091
Oracle PSR Engineering <http://psr.us.oracle.com/> | Server Technology
On 10/17/2012 2:30 PM, Dr Heinz M. Kabutz wrote:
> [newbie question]
> Is it possible for a thread to ever be moved between processors,
> if for example the is an idle processor and another that is really
> busy? If not, then load balancing could be problematic. If yes, then
> this means that we need to take this into consideration and not build
> static structures based on the value of which processor the thread is
> /currently/ running on.
> Again, if no, then what is the guarantee that it won't be available on
> some VMs or some hardware architectures or operating systems?
> [/newbie question]
> Dr Heinz M. Kabutz (PhD CompSci)
> Author of "The Java(tm) Specialists' Newsletter"
> Sun Java Champion
> IEEE Certified Software Development Professional
> Tel: +30 69 75 595 262
> Skype: kabutz
> On 10/17/12 11:05 AM, Romain Colle wrote:
>> On Wed, Oct 17, 2012 at 9:15 AM, Kirk Pepperdine <kirk at kodewerk.com
>> <mailto:kirk at kodewerk.com>> wrote:
>> SOT, right now we are using CPUID for a number of reasons. Very
>> easy to get to on Linux as we can do this in pure Java. However
>> other platforms require C++/assembler so having <gulp> an unsafe
>> operation in the JVM would be a win from my POV.
>> I strongly second that. It is next to impossible to implement most
>> NUMA-aware algorithms without knowing on which NUMA node the current
>> thread is running.
>> Having this information available as a Java class (in Thread, Unsafe,
>> ...) would be a huge win for people like us that need NUMA-aware
>> Romain Colle
>> R&D Project Manager
>> 2 rue Jean Lantier, 75001 Paris, France
>> http://www.quartetfs.com <http://www.quartetfs.com/>
>> Concurrency-interest mailing list
>> Concurrency-interest at cs.oswego.edu
> Concurrency-interest mailing list
> Concurrency-interest at cs.oswego.edu
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Concurrency-interest