[concurrency-interest] DirectByteBuffers and reachabilityFence

Alexandre De Champeaux adc at quartetfs.com
Mon Dec 7 12:08:15 EST 2015


I have actually written a test class that reproduces a segfault. It uses an
structure similar to DBB and an updated version of the Bits.copyToArray
method which forces a GC and waits a bit from within it to simulate a
safepoint stop and a GC, and leaves time to the reference handler to clean
up.
Is this test too far away from the DBB code to actually simulate a possible
behavior of DBB? Or is hotspot only able to not mark as unreachable vanilla
java DBB, but not customer classes?

Here is the test class:


import sun.misc.Cleaner;
import sun.misc.Unsafe;

/**
 * This test checks whether or not the JVM will dereference "this" and
destroy the buffer while
 * accessing it.
 * <p>
 * Run the main method of this class to run the test.
 * <p>
 * Note that the test segfaults each time if using a JNI call to
mmap/munmap to do the memory
 * management but not as often using Unsafe (malloc recycling memory?).
 * <p>
 * Example output of a failure:
 *
 * <pre>
TestGcThisAndDestroyBuffer.main() 0 -- 0
TestGcThisAndDestroyBuffer.main() 0 -- 1
TestGcThisAndDestroyBuffer.main() 0 -- 2
TestGcThisAndDestroyBuffer.main() 0 -- 3
TestGcThisAndDestroyBuffer.main() 0 -- 4
TestGcThisAndDestroyBuffer.main() 0 -- 5
TestGcThisAndDestroyBuffer.main() 0 -- 6
TestGcThisAndDestroyBuffer.main() 0 -- 7
TestGcThisAndDestroyBuffer.main() 0 -- 8
TestGcThisAndDestroyBuffer.main() 0 -- 9
TestGcThisAndDestroyBuffer.main() 0 -- 10
TestGcThisAndDestroyBuffer.main() 0 -- 11
TestGcThisAndDestroyBuffer.main() 0 -- 12
TestGcThisAndDestroyBuffer.main() 0 -- 13
TestGcThisAndDestroyBuffer.main() 0 -- 14
Must be compiled now, start doing some GCs 30001
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f489fb86220, pid=16949, tid=139949914818304
#
# JRE version: Java(TM) SE Runtime Environment (8.0_60-b27) (build
1.8.0_60-b27)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode
linux-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.so+0x7f7220]
#
# Failed to write core dump. Core dumps have been disabled. To enable core
dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /SSDhome/alexandre/ActivePivot5/scripts/adc/java_tests/hs_err_pid16949.log
Compiled method (nm)   10180   18     n 0       sun.misc.Unsafe::copyMemory
(native)
total in heap  [0x00007f488910f2d0,0x00007f488910f640] = 880
relocation     [0x00007f488910f3f8,0x00007f488910f440] = 72
main code      [0x00007f488910f440,0x00007f488910f640] = 512
Compiled method (nm)   10180   18     n 0       sun.misc.Unsafe::copyMemory
(native)
total in heap  [0x00007f488910f2d0,0x00007f488910f640] = 880
relocation     [0x00007f488910f3f8,0x00007f488910f440] = 72
main code      [0x00007f488910f440,0x00007f488910f640] = 512
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#
 * </pre>
 */
@SuppressWarnings("restriction")
public class TestGcThisAndDestroyBuffer {

/** The threshold found in Bits.copyMemory */
static final long UNSAFE_COPY_THRESHOLD = 1024L * 1024L;
private static final Unsafe unsafe = retrieveUnsafe();

static long ctr = 0;

// The tested method
// See the method in Bits.copyToArray
/* For Bits.copyToArray to segfault we need:
*
* 1) The buffer to be garbage collected while calling the method, for that
we need:
* - A safepoint exist in the method
* - The buffer needs to not be held, so JIT compil must have occured,
* and JIT must have dereferenced the DBB at the time of the call
* 2) The reference handler thread to have run the associated cleaner
*/
static void copyToArray(long srcAddr, Object dst, long dstBaseOffset, long
dstPos, long length) {
long offset = dstBaseOffset + dstPos;
while (length > 0) {
// Code differing from Bits.
if (ctr++ >= 30_000) {
// Counter is there to wait for JIT compilation
System.out.println("Must be compiled now, start doing some GCs " + ctr);
// Here we simulate a safepoint, and a GC at this safepoint
System.gc();
try {
// And here we wait for the reference handler to perform the cleaning to
simulate a fast cleaning.
Thread.sleep(1000);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}

long size = (length > UNSAFE_COPY_THRESHOLD) ? UNSAFE_COPY_THRESHOLD :
length;
unsafe.copyMemory(null, srcAddr, dst, offset, size);
length -= size;
srcAddr += size;
offset += size;
}
}

// A basic class inspired from java.nio.DirectByteBuffer
static class SimpleBuffer {
static int SIZE = (int) (2 * UNSAFE_COPY_THRESHOLD);
static final long arrayBaseOffset = (long)
unsafe.arrayBaseOffset(byte[].class);

final long addr;

public SimpleBuffer() {
addr = unsafe.allocateMemory(SIZE);
unsafe.setMemory(addr, SIZE, (byte) 0);
Cleaner.create(this, new Deallocator(addr));
}

public byte[] copy() {
final byte[] a = new byte[SIZE];
copyToArray(addr, a, arrayBaseOffset, 0, SIZE);
return a;
}

private static class Deallocator implements Runnable {

private long address;

private Deallocator(long address) {
assert (address != 0);
this.address = address;
}

@Override
public void run() {
if (address == 0) {
// Paranoia
return;
}
unsafe.freeMemory(address);
address = 0;
}

}

}

static byte[] aBunchOfCopies() {
byte[] res = null;
for (int i = 0; i < 1000; ++i) {
res = new SimpleBuffer().copy();
if (res[0] == -1) {
return res;
}
}
return res;
}

public static void main(String[] args) {
int i = 0;
while (true) {
final byte[] res = aBunchOfCopies();
System.out.println("TestGcThisAndDestroyBuffer.main() " + res[0] + " -- " +
i++);
}
}

private static final sun.misc.Unsafe retrieveUnsafe() {
try {
return sun.misc.Unsafe.getUnsafe();
} catch (SecurityException se) {
try {
return java.security.AccessController
.doPrivileged(new
java.security.PrivilegedExceptionAction<sun.misc.Unsafe>() {
@Override
public sun.misc.Unsafe run() throws Exception {
java.lang.reflect.Field f =
sun.misc.Unsafe.class.getDeclaredField("theUnsafe");
f.setAccessible(true);
return (sun.misc.Unsafe) f.get(null);
}
});
} catch (java.security.PrivilegedActionException e) {
throw new RuntimeException("Could not initialize intrinsics", e.getCause());
}
}
}

}


On Mon, Dec 7, 2015 at 4:15 PM, Vitaly Davidovich <vitalyd at gmail.com> wrote:

> JIT knows about Unsafe operations, and it also knows the type of memory
> being accessed (or sometimes knows it doesn't know :)).  So I don't think
> it'll mark a DBB as unreachable while these operations are in-flight.
>
> Peter's scenario is unique to WeakReference since it's intentionally not
> considered a strong reference and there's otherwise plain java code in his
> example (that JIT can reason about easily otherwise).
>
> sent from my phone
> On Dec 7, 2015 8:10 AM, "Alexandre De Champeaux" <adc at quartetfs.com>
> wrote:
>
>> Hi all,
>>
>> I recently had a look at the discussion started by Peter Levart on
>> October 21 (
>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-October/014493.html
>> ).
>>
>> It was a very insightful discussion, and made me aware that the "this"
>> object could be garbage collected while being inside a call of one of its
>> method.
>>
>> However, this got me concerned about the java.nio.DirectByteBuffer read
>> and write methods:
>> If the "this" object is garbage collected when making a call like
>> ByteBuffer.allocateDirect(int).someGetOrPutMethod(), the native pointer
>> that is passed to sun.misc.Unsafe might be freed, and accessing it will
>> cause the read/write to occur in an invalid memory area, which might lead
>> to a segfault, or other major issues.
>> This would be quite unlikely: JIT compilation needs to occur while
>> keeping a safepoint, then a GC must happen, and finally the
>> ReferenceHandler thread needs to perform its cleanup fast enough.
>>
>> I am particularly concerned by the get(byte[] dst, int offset, int
>> length) method, that in turns calls Bits.copyToArray, which purposely
>> splits its calls to Unsafe.copyMemory to allow for safepoints to sneak in.
>>
>> Am I correct, or does the JVM performs specific protection for instances
>> of DirectByteBuffer?
>>
>> Regards,
>>
>> Alexandre de Champeaux
>>
>> _______________________________________________
>> Concurrency-interest mailing list
>> Concurrency-interest at cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>>


-- 
Alexandre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://cs.oswego.edu/pipermail/concurrency-interest/attachments/20151207/46d94bb0/attachment-0001.html>


More information about the Concurrency-interest mailing list