Why does sun.misc.Unsafe exist, and how can it be used in the real world?

I came across the sun.misc.Unsafe package the other day and was amazed at what it could do.

Of course, the class is undocumented, but I was wondering if there was ever a good reason to use it. What scenarios might arise where you would need to use it? How might it be used in a real-world scenario?

Furthermore, if you do need it, does that not indicate that something is probably wrong with your design?

Why does Java even include this class?

examples

1. VM "intrinsification." ie CAS (Compare-And-Swap) used in Lock-Free Hash Tables eg:sun.misc.Unsafe.compareAndSwapInt it can make real JNI calls into native code that contains special instructions for CAS

   read more about CAS here http://en.wikipedia.org/wiki/Compare-and-swap

2. The sun.misc.Unsafe functionality of the host VM can be used to allocate uninitialized objects and then interpret the constructor invocation as any other method call.

3. One can track the data from the native address.It is possible to retrieve an object’s memory address using the java.lang.Unsafe class, and operate on its fields directly via unsafe get/put methods!

4. Compile time optimizations for JVM. HIgh performance VM using "magic", requiring low-level operations. eg: http://en.wikipedia.org/wiki/Jikes_RVM

5. Allocating memory, sun.misc.Unsafe.allocateMemory eg:- DirectByteBuffer constructor internally calls it when ByteBuffer.allocateDirect is invoked

6. Tracing the call stack and replaying with values instantiated by sun.misc.Unsafe, useful for instrumentation

7. sun.misc.Unsafe.arrayBaseOffset and arrayIndexScale can be used to develop arraylets,a technique for efficiently breaking up large arrays into smaller objects to limit the real-time cost of scan, update or move operations on large objects

8. http://robaustin.wikidot.com/how-to-write-to-direct-memory-locations-in-java

more on references here - http://bytescrolls.blogspot.com/2011/04/interesting-uses-of-sunmiscunsafe.html

Just from running a search in some code search engine I get the following examples:

• Java Object Notation - use it for more efficient array processing, quoting the javadoc

> Simple class to obtain access to the {@link Unsafe} object. {@link Unsafe}

• is required to allow efficient CAS operations on arrays. Note that the versions in {@link java.util.concurrent.atomic}, such as {@link java.util.concurrent.atomic.AtomicLongArray}, require extra memory ordering guarantees which are generally not needed in these algorithms and are also expensive on most processors.

• SoyLatte - java 6 for osx javadoc excerpt

> /** Base class for sun.misc.Unsafe-based FieldAccessors for static fields. The observation is that there are only nine types of fields from the standpoint of reflection code: the eight primitive types and Object. Using class Unsafe instead of generated bytecodes saves memory and loading time for the dynamically-generated FieldAccessors. */

• SpikeSource

> /* FinalFields that are sent across the wire .. how to unmarshall and recreate the object on the receiving side? We don't want to invoke the constructor since it would establish values for final fields. We have to recreate the final field exactly like it was on the sender side. The sun.misc.Unsafe does this for us. */

There are many other examples, just follow the above link...

Interesting, I'd never even heard of this class (which is probably a good thing, really).

One thing that jumps to mind is using Unsafe#setMemory to zeroize buffers that contained sensitive information at one point (passwords, keys, ...). You could even do this to fields of "immutable" objects (then again I suppose plain old reflection might do the trick here too). I'm no security expert though so take this with a grain of salt.

Based on a very brief analysis of the Java 1.6.12 library using eclipse for reference tracing, it seems as though every useful functionality of Unsafe is exposed in useful ways.

CAS operations are exposed through the Atomic* classes. Memory manipulations functions are exposed through DirectByteBuffer Sync instructions (park,unpark) are exposed through the AbstractQueuedSynchronizer which in turn is used by Lock implementations.

Unsafe.throwException - allows to throw checked exception without declaring them.

This is useful in some cases where you deal with reflection or AOP.

Assume you Build a generic proxy for a user defined Interface. And the user can specify which exception is thrown by the implmentation in a special case just by declaring the exception in the interface. Then this is the only way I know, to rise a checked exception in the Dynamic Implementation of the Interface.

import org.junit.Test;
/** need to allow forbidden references! */ import sun.misc.Unsafe;

/**
 * Demonstrate how to throw an undeclared checked exception.
 * This is a hack, because it uses the forbidden Class {@link sun.misc.Unsafe}.
 */
public class ExceptionTest {

	/**
	 * A checked exception.
	 */
	public static class MyException extends Exception {
		private static final long serialVersionUID = 5960664994726581924L;
	}

	/**
	 * Throw the Exception.
	 */
	@SuppressWarnings("restriction")
	public static void throwUndeclared() {
		getUnsafe().throwException(new MyException());
	}

	/**
	 * Return an instance of {@link sun.misc.Unsafe}.
	 * @return THE instance
	 */
	@SuppressWarnings("restriction")
	private static Unsafe getUnsafe() {
		try {
			
			Field singleoneInstanceField = Unsafe.class.getDeclaredField("theUnsafe");
			singleoneInstanceField.setAccessible(true);
			return (Unsafe) singleoneInstanceField.get(null);
		
		} catch (IllegalArgumentException e) {
			throw createExceptionForObtainingUnsafe(e);
		} catch (SecurityException e) {
			throw createExceptionForObtainingUnsafe(e);
		} catch (NoSuchFieldException e) {
			throw createExceptionForObtainingUnsafe(e);
		} catch (IllegalAccessException e) {
			throw createExceptionForObtainingUnsafe(e);
		}
	}
	
	private static RuntimeException createExceptionForObtainingUnsafe(final Throwable cause) {
		return new RuntimeException("error while obtaining sun.misc.Unsafe", cause);
	}
		

	/**
	 * scenario: test that an CheckedException {@link MyException} can be thrown
	 * from an method that not declare it.
	 */
	@Test(expected = MyException.class)
	public void testUnsingUnsaveToThrowCheckedException() {
		throwUndeclared();
	}
}

>### Class Unsafe >A collection of methods for performing low-level, unsafe operations. Although the class and all methods are public, use of this class is limited because only trusted code can obtain instances of it.

One use of it is in java.util.concurrent.atomic classes:

• AtomicIntegerArray

• AtomicLongArray

For efficient memory copy (faster to copy than System.arraycopy() for short blocks at least); as used by Java LZF and Snappy codecs. They use 'getLong' and 'putLong', which are faster than doing copies byte-by-byte; especially efficient when copying things like 16/32/64 byte blocks.

I was recently working on reimplementing the JVM and found that a surprising number of classes are implemented in terms of Unsafe. The class is mostly designed for the Java library implementers and contains features that are fundamentally unsafe but necessary for building fast primitives. For example, there are methods for getting and writing raw field offsets, using hardware-level synchronization, allocating and freeing memory, etc. It is not intended to be used by normal Java programmers; it's undocumented, implementation-specific, and inherently unsafe (hence the name!). Moreover, I think that the SecurityManager will disallow access to it in almost all cases.

In short, it mainly exists to allow library implementers access to the underlying machine without having to declare every method in certain classes like AtomicInteger native. You shouldn't need to use or worry about it in routine Java programming, as the whole point is to make the rest of the libraries fast enough that you wouldn't need that sort of access.

Off-heap collections may be useful for allocating huge amounts of memory and deallocating it immediately after use without GC interference. I wrote a library for working with off-heap arrays/lists based on sun.misc.Unsafe.

Use it to access and allocate large amounts of memory efficiently, such as in your very own voxel engine! (i.e. Minecraft-style game.)

In my experience, the JVM is often unable to eliminate bounds-checking in place you truly need it. For example, if you're iterating over a large array, but the actual memory access is tucked underneath a non-virtual* method call in the loop, the JVM may still perform a bounds check with each array access, rather than once just before the loop. Thus, for potentially large performance gains, you can eliminate JVM bounds-checking inside the loop via a method which employs sun.misc.Unsafe to access the memory directly, making sure to do any bounds-checking yourself at the correct places. (You are gonna bounds check at some level, right?)
_{*by non-virtual, I mean the JVM shouldn't have to dynamically resolve whatever your particular method is, because you've correctly guaranteed that class/method/instance are some combination of static/final/what-have-you.}

For my home-grown voxel engine, this resulted in a dramatic performance gain during chunk generation and serialization (iow places where I was reading/writing to the entire array at once). Results may vary, but if a lack of bounds-elimination is your problem, then this will fix it.

There are some potentially major problems with this: specifically, when you provide the ability to access memory without bounds-checking to clients of your interface, they will probably abuse it. (Don't forget that hackers can also be clients of your interface... especially in the case of a voxel engine written in Java.) Thus, you should either design your interface in a way such that memory access cannot be abused, or you should be extremely careful to validate user-data before it can ever, ever mingle with your dangerous interface. Considering the catastrophic things a hacker can do with unchecked memory access, it's probably best to take both approaches.

We have implemented huge collections like Arrays,HashMaps,TreeMaps using Unsafe.
And to avoid/minimize the fragmentation, we implemented memory allocator using the concepts of dlmalloc over unsafe.
This helped us to gain the performance in concurrency.

Unsafe.park() and Unsafe.unpark() for the construction of custom concurrency control structures and cooperative scheduling mechanisms.

Haven't used it myself, but I suppose if you have a variable that is only occasionally read by more than one thread (so you don't really want to make it volatile) you could use the putObjectVolatile when writing it in the main thread and readObjectVolatile when doing the rare reads from other threads.

One example of its use is the random method, which calls the unsafe to change the seed.

This site also has also some uses of it.

You need it if you need to replace functionality provided by one of the classes which uses it currently.

This can be custom/faster/more compact serialization/deserialization, a faster/larger buffer/resizable version of ByteBuffer, or adding an atomic variable e.g. one not supported currently.

I have used it for all of these at some time.

The object appears to be availability to work at a lower level than what Java code typically allows for. If you're coding a high level application then the JVM abstracts memory handling and other operations away from the code level so its easier to program. By using the Unsafe library you're effectively completing low-level operations that would typically be done for you.

As woliveirajr stated "random()" uses Unsafe to seed just as many other operations will use the allocateMemory() function included in Unsafe.

As a programmer you probably could get away with never needing this library but having strict control over low-level elements does come in handy (that's why there is still Assembly and (to a lesser extent) C code drifting around in major products)