final transient fields and serialization
JavaSerializationFinalJava Problem Overview
Is it possible to have final transient fields that are set to any non-default value after serialization in Java? My usecase is a cache variable — that's why it is transient. I also have a habit of making Map fields that won't be changed (i.e. contents of the map is changed, but object itself remains the same) final. However, these attributes seem to be contradictory — while compiler allows such a combination, I cannot have the field set to anything but null after unserialization.
I tried the following, without success:
- simple field initialization (shown in the example): this is what I normally do, but the initialization doesn't seem to happen after unserialization;
- initialization in constructor (I believe this is semantically the same as above though);
- assigning the field in
readObject()— cannot be done since the field isfinal.
In the example cache is public only for testing.
import java.io.*;
import java.util.*;
public class test
{
public static void main (String[] args) throws Exception
{
X x = new X ();
System.out.println (x + " " + x.cache);
ByteArrayOutputStream buffer = new ByteArrayOutputStream ();
new ObjectOutputStream (buffer).writeObject (x);
x = (X) new ObjectInputStream (new ByteArrayInputStream (buffer.toByteArray ())).readObject ();
System.out.println (x + " " + x.cache);
}
public static class X implements Serializable
{
public final transient Map <Object, Object> cache = new HashMap <Object, Object> ();
}
}
Output:
test$X@1a46e30 {}
test$X@190d11 null
Java Solutions
Solution 1 - Java
The short answer is "no" unfortunately - I've often wanted this. but transients cannot be final.
A final field must be initialized either by direct assignment of an initial value or in the constructor. During deserialization, neither of these are invoked, so initial values for transients must be set in the 'readObject()' private method that's invoked during deserialization. And for that to work, the transients must be non-final.
(Strictly speaking, finals are only final the first time they are read, so there are hacks that are possible that assign a value before it is read, but for me this is going one step too far.)
Solution 2 - Java
You can change the contents of a field using Reflection. Works on Java 1.5+. It will work, because serialization is performed in a single thread. After another thread access the same object, it shouldn't change the final field (because of weirdness in the memory model & reflaction).
So, in readObject(), you can do something similar to this example:
import java.lang.reflect.Field;
public class FinalTransient {
private final transient Object a = null;
public static void main(String... args) throws Exception {
FinalTransient b = new FinalTransient();
System.out.println("First: " + b.a); // e.g. after serialization
Field f = b.getClass().getDeclaredField("a");
f.setAccessible(true);
f.set(b, 6); // e.g. putting back your cache
System.out.println("Second: " + b.a); // wow: it has a value!
}
}
Remember: Final is not final anymore!
Solution 3 - Java
Yes, this is easily possible by implementing the (apparently little known!) readResolve() method. It lets you replace the object after it is deserialized. You can use that to invoke a constructor that will initialize a replacement object however you want. An example:
import java.io.*;
import java.util.*;
public class test {
public static void main(String[] args) throws Exception {
X x = new X();
x.name = "This data will be serialized";
x.cache.put("This data", "is transient");
System.out.println("Before: " + x + " '" + x.name + "' " + x.cache);
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
new ObjectOutputStream(buffer).writeObject(x);
x = (X)new ObjectInputStream(new ByteArrayInputStream(buffer.toByteArray())).readObject();
System.out.println("After: " + x + " '" + x.name + "' " + x.cache);
}
public static class X implements Serializable {
public final transient Map<Object,Object> cache = new HashMap<>();
public String name;
public X() {} // normal constructor
private X(X x) { // constructor for deserialization
// copy the non-transient fields
this.name = x.name;
}
private Object readResolve() {
// create a new object from the deserialized one
return new X(this);
}
}
}
Output -- the string is preserved but the transient map is reset to an empty (but non-null!) map:
Before: test$X@172e0cc 'This data will be serialized' {This data=is transient}
After: test$X@490662 'This data will be serialized' {}
Solution 4 - Java
The general solution to problems like this is to use a "serial proxy" (see Effective Java 2nd Ed). If you need to retrofit this to an existing serialisable class without breaking serial compatibility, then you will need to do some hacking.
Solution 5 - Java
Five years later, I find my original answer unsatisfactory after I stumbled across this post via Google. Another solution would be using no reflection at all, and use the technique suggested by Boann.
It also makes use of the GetField class returned by ObjectInputStream#readFields() method, which according to the Serialization specification must be called in the private readObject(...) method.
The solution makes field deserialization explicit by storing the retrieved fields in a temporary transient field (called FinalExample#fields) of a temporary "instance" created by the deserialization process. All object fields are then deserialized and readResolve(...) is called: a new instance is created but this time using a constructor, discarding the temporary instance with the temporary field. The instance explicitly restores each field using the GetField instance; this is the place to check any parameters as would any other constructor. If an exception is thrown by the constructor it is translated to an InvalidObjectException and deserialization of this object fails.
The micro-benchmark included ensures that this solution is not slower than default serialization/deserialization. Indeed, it is on my PC:
Problem: 8.598s Solution: 7.818s
Then here is the code:
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InvalidObjectException;
import java.io.ObjectInputStream;
import java.io.ObjectInputStream.GetField;
import java.io.ObjectOutputStream;
import java.io.ObjectStreamException;
import java.io.Serializable;
import org.junit.Test;
import static org.junit.Assert.*;
public class FinalSerialization {
/**
* Using default serialization, there are problems with transient final
* fields. This is because internally, ObjectInputStream uses the Unsafe
* class to create an "instance", without calling a constructor.
*/
@Test
public void problem() throws Exception {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
WrongExample x = new WrongExample(1234);
oos.writeObject(x);
oos.close();
ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
ObjectInputStream ois = new ObjectInputStream(bais);
WrongExample y = (WrongExample) ois.readObject();
assertTrue(y.value == 1234);
// Problem:
assertFalse(y.ref != null);
ois.close();
baos.close();
bais.close();
}
/**
* Use the readResolve method to construct a new object with the correct
* finals initialized. Because we now call the constructor explicitly, all
* finals are properly set up.
*/
@Test
public void solution() throws Exception {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(baos);
FinalExample x = new FinalExample(1234);
oos.writeObject(x);
oos.close();
ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
ObjectInputStream ois = new ObjectInputStream(bais);
FinalExample y = (FinalExample) ois.readObject();
assertTrue(y.ref != null);
assertTrue(y.value == 1234);
ois.close();
baos.close();
bais.close();
}
/**
* The solution <em>should not</em> have worse execution time than built-in
* deserialization.
*/
@Test
public void benchmark() throws Exception {
int TRIALS = 500_000;
long a = System.currentTimeMillis();
for (int i = 0; i < TRIALS; i++) {
problem();
}
a = System.currentTimeMillis() - a;
long b = System.currentTimeMillis();
for (int i = 0; i < TRIALS; i++) {
solution();
}
b = System.currentTimeMillis() - b;
System.out.println("Problem: " + a / 1000f + "s Solution: " + b / 1000f + "s");
assertTrue(b <= a);
}
public static class FinalExample implements Serializable {
private static final long serialVersionUID = 4772085863429354018L;
public final transient Object ref = new Object();
public final int value;
private transient GetField fields;
public FinalExample(int value) {
this.value = value;
}
private FinalExample(GetField fields) throws IOException {
// assign fields
value = fields.get("value", 0);
}
private void readObject(ObjectInputStream stream) throws IOException,
ClassNotFoundException {
fields = stream.readFields();
}
private Object readResolve() throws ObjectStreamException {
try {
return new FinalExample(fields);
} catch (IOException ex) {
throw new InvalidObjectException(ex.getMessage());
}
}
}
public static class WrongExample implements Serializable {
private static final long serialVersionUID = 4772085863429354018L;
public final transient Object ref = new Object();
public final int value;
public WrongExample(int value) {
this.value = value;
}
}
}
A note of caution: whenever the class refers to another object instance, it might be possible to leak the temporary "instance" created by the serialization process: the object resolution occurs only after all sub-objects are read, hence it is possible for subobjects to keep a reference to the temporary object. Classes can check for use of such illegally constructed instances by checking that the GetField temporary field is null. Only when it is null, it was created using a regular constructor and not through the deserialization process.
Note to self: Perhaps a better solution exists in five years. See you then!
Solution 6 - Java
This question is about the Java default serializer, but I landed here from searching about Gson. This answer doesn't apply to the default serializer, but it does apply to Gson and maybe others. I wasn't a fan of (manually) using Reflection or readResolve, so here's something else.
When deserializing, Gson calls the default constructor to create the object. You can move your transient final assignments to the default constructor, and they will be assigned properly. If you only have a non-default constructor that assigns final variables (for example, an ID), it won't matter what you assign them to as they will be overwritten by Gson with Reflection.
This does mean that if your transient final assignments relies on constructor arguments, this won't work.
Here is some example code:
import com.google.gson.Gson;
import java.util.HashMap;
public class Test {
public static void main(String[] args) {
BrokenTestObject broken = new BrokenTestObject("broken");
FixedTestObject fixed = new FixedTestObject("fixed");
broken = serializeAndDeserialize(broken, BrokenTestObject.class);
fixed = serializeAndDeserialize(fixed, FixedTestObject.class);
System.out.println(broken.id + ": " + broken.someCache);
System.out.println(fixed.id + ": " + fixed.someCache);
}
public static <O> O serializeAndDeserialize(O object, Class<O> c) {
Gson gson = new Gson();
String json = gson.toJson(object);
return gson.fromJson(json, c);
}
public static class BrokenTestObject {
public final String id;
public transient final HashMap<String, String> someCache = new HashMap<>();
public BrokenTestObject(String id) {
this.id = id;
}
}
public static class FixedTestObject {
public final String id;
public transient final HashMap<String, String> someCache;
public FixedTestObject(String id) {
this.id = id;
this.someCache = new HashMap<>();
}
//only used during deserialization
private FixedTestObject() {
this.id = null; //doesn't matter, will be overwritten during deserialization
this.someCache = new HashMap<>();
}
}
}
Prints:
broken: null
fixed: {}