[concurrency-interest] ThreadLocal vs ProcessorLocal

David Holmes davidcholmes at aapt.net.au
Wed Oct 17 22:31:12 EDT 2012

Well the nanoTime() behaviour is not a JVM bug, though the JVM can try to
account for the underlying buggy OS and/or configuration and/or hardware.

As Dave states on LInux we will use CLOCK_MONOTONIC if available (which is
pretty much always these days), else we fall back to gettimeofday. Now as
you can infer from the name CLOCK_MONOTONIC is supposed to be monotonic and
if it isn't that is a bug in the OS or a system configuration error (using
an unreliable clocksource such as the TSC on MP systems). In contrast we
make no pretense that gettimeofday is expected to be monotonic.

Also as Dave states we don't try to guard against a buggy CLOCK_MONOTONIC on
linux by ensuring it never reports a value less than any previous value
reported. We could, and probably should, but it is one of many things on a
long list.

But if you see big problems with nanoTime then either your system is using
the TSC as a clocksource when it should not, OR you are running in a virtual
environment and the host system is not providing a stable time source to the
guest OS.

Note: for the TSC to be usable it must be both stable (frequency invariant)
and synchronized across all "processors". While many processors now provide
a stable TSC they don't provide a synchronized TSC. For the OS to be able to
use the TSC as a monotonic clocksource it needs to do its own very accurate
synchronization. Solaris actually attempts this, where most operating
systems simply stopped using the TSC, but because of that there has been a
very long bug-tail on Solaris.

David Holmes
  -----Original Message-----
  From: concurrency-interest-bounces at cs.oswego.edu
[mailto:concurrency-interest-bounces at cs.oswego.edu]On Behalf Of David Dice
  Sent: Thursday, 18 October 2012 12:11 PM
  To: concurrency-interest at cs.oswego.edu
  Subject: Re: [concurrency-interest] ThreadLocal vs ProcessorLocal

    Date: Wed, 17 Oct 2012 16:55:50 -0500
    From: Jacy Odin Grannis <jacyg at alumni.rice.edu>
    To: "Dr Heinz M. Kabutz" <heinz at javaspecialists.eu>
    Cc: concurrency-interest at cs.oswego.edu, David Dice
            <david.dice at gmail.com>
    Subject: Re: [concurrency-interest] ThreadLocal vs ProcessorLocal
            <CAESiqEqAteCAsCDLWXM-3-89bJS2nBRrLtOBmHrJkkcX2=Sh4g at mail.gmail.
    Content-Type: text/plain; charset=ISO-8859-1

    Yes, definitely.  I've seen this happen.  One easy way you can see
    this is System.nanoTime will suddenly start returning wildly different
    values.  nanoTime is only consistent on a single processor, it can
    vary widely between processors (at least on Linux).

    I think what's really needed is a set of language level constructs for
    really addressing the problem.  I know there are experimental projects
    looking to do that (
    ).  I am not sure to what extent it would be possible to build support
    for the various constructs in the JVM; and then aside from that, how
    you would add language support is another matter.

  The nanoTime() behavior sounds like a JVM bug.   nanoTime() values should
be non-retrograde and causal in the sense that if one thread calls nanoTime
and stores the observed value T into a variable, and then some 2nd thread
reads that variable and observes the store of T and then calls nanoTime and
sees value U, we should have U >= T.  (Volatiles are assumed, obviously).
I first ran into this non-monotonic time problem on large SPARC systems
where the HW clock underlying the native gethrtime() API exhibited drift
between CPUs.   The drift was minimal as the kernel syncs the clocks
periodically, so we tracked the the maximum value returned by nanoTime() and
would return the maximum of that tracking value and the value we got via
gethrtime().   This works, but creates its own cache coherence hot-spot as
we're updating that variable frequently, which means that concurrent and
unrelated nanoTime() calls don't scale as well as we might like.   (There
are ways to avoid the coherence hot spot but they usually entail reduced

  It's been years since I've looked at the code, but I think we use
CLOCK_MONOTONIC if it's available on linux.   (David Holmes could best
answer this part of the question regarding linux time sources).  But the
guards against returning a smaller value aren't in place in the linux
platform-specific code as they are on Solaris.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20121018/0c35c5c0/attachment.html>

More information about the Concurrency-interest mailing list