Do Java 8 default methods break source compatibility?

It has generally been the case the Java source code has been forward compatible. Until Java 8, as far as I know, both compiled classes and source have been forward compatible with later JDK/JVM releases. [Update: this is not correct, see comments re 'enum', etc, below.] However, with the addition of default methods in Java 8 this appears to no longer be the case.

For example, a library I have been using has an implementation of java.util.List which includes a List<V> sort(). This method returns a copy of the contents of the list sorted. This library, deployed as a jar file dependency, worked fine in a project being built using JDK 1.8.

However, later I had occasion to recompile the library itself using JDK 1.8 and I found the library no longer compiles: the List-implementing class with its own sort() method now conflicts with the Java 8 java.util.List.sort() default method. The Java 8 sort() default method sorts the list in place (returns void); my library's sort() method - since it returns a new sorted list - has an incompatible signature.

So my basic question is:

• Doesn't JDK 1.8 introduce a forward incompatibility for Java source code due to default methods?

Also:

• Is this the first such forward incompatible change?
• Was this considered or discussed when default methods where designed and implemented? Is it documented anywhere?
• Was the (admittedly small) inconvenience discounted versus the benefits?

The following is an example of some code that compiles and runs under 1.7 and runs under 1.8 - but does not compile under 1.8:

import java.util.*;

public final class Sort8 {

    public static void main(String[] args) {
        SortableList<String> l = new SortableList<String>(Arrays.asList(args));
        System.out.println("unsorted: "+l);
        SortableList<String> s = l.sort(Collections.reverseOrder());
        System.out.println("sorted  : "+s);
    }

    public static class SortableList<V> extends ArrayList<V> {
	
        public SortableList() { super(); }
        public SortableList(Collection<? extends V> col) { super(col); }
	
        public SortableList<V> sort(Comparator<? super V> cmp) {
            SortableList<V> l = new SortableList<V>();
            l.addAll(this);
            Collections.sort(l, cmp);
            return l;
        }

    }

}

The following shows this code being compiled (or failing to) and being run.

> c:\tools\jdk1.7.0_10\bin\javac Sort8.java

> c:\tools\jdk1.7.0_10\bin\java Sort8 this is a test
unsorted: [this, is, a, test]
sorted  : [this, test, is, a]

> c:\tools\jdk1.8.0_05\bin\java Sort8 this is a test
unsorted: [this, is, a, test]
sorted  : [this, test, is, a]

> del Sort8*.class

> c:\tools\jdk1.8.0_05\bin\javac Sort8.java
Sort8.java:46: error: sort(Comparator<? super V>) in SortableList cannot implement sort(Comparator<? super E>) in List
                public SortableList<V> sort(Comparator<? super V> cmp) {
                                       ^
  return type SortableList<V> is not compatible with void
  where V,E are type-variables:
    V extends Object declared in class SortableList
    E extends Object declared in interface List
1 error

Doesn't JDK 1.8 introduce a forward incompatibility for Java source code due to default methods?

Any new method in a superclass or interface can break compatibility. Default methods make it less likely that a change in an interface will break compatibility. In the sense that default methods open the door to adding methods to interfaces, you could say that default methods may contribute to some broken compatibility.

Is this the first such forward incompatible change?

Almost certainly not, since we've been subclassing classes from the standard library since Java 1.0.

Was this considered or discussed when default methods were designed and implemented? Is it documented anywhere?

Yes, it was considered. See Brian Goetz's August 2010 paper "Interface evolution via “public defender” methods":

>10. Source compatibility > >It is possible that this scheme could introduce source incompatibilities to the extent that library interfaces are modified to insert new methods that are incompatible with methods in existing classes. (For example, if a class has a float-valued xyz() method and implements Collection, and we add an int-valued xyz() method to Collection, the existing class will no longer compile.)

Was the (admittedly small) inconvenience discounted versus the benefits?

Before, changing an interface would definitely break compatibility. Now, it might. Going from 'definitely' to 'might' can be seen either positively or negatively. On the one hand, it makes it feasible to add methods to interfaces. On the other hand, it opens the door to the kind of incompatibility you saw, not just with classes, but with interfaces too.

The benefits are larger than the inconveniences, though, as cited at the top of Goetz's paper:

>1. Problem statement > >Once published, it is impossible to add methods to an interface without breaking existing implementations. The longer the time since a library has been published, the more likely it is that this restriction will cause grief for its maintainers. > >The addition of closures to the Java language in JDK 7 place additional stress on the aging Collection interfaces; one of the most significant benefits of closures is that it enables the development of more powerful libraries. It would be disappointing to add a language feature that enables better libraries while at the same time not extending the core libraries to take advantage of that feature.

>Doesn't JDK 1.8 introduce a forward incompatibility for Java source code due to default methods?

Yes as you've seen your self.

>Is this the first such forward incompatible change?

No. Java 5 enumkeyword was also breaking because before that you could have variables named that which would no longer compile in Java 5 +

>Was this considered or discussed when default methods where designed and implemented? Is it documented anywhere?

Yes Orcale Java 8 source incompatibility description

> Was the (admittedly small) inconvenience discounted versus the benefits?

Yes

We can draw a parallel with abstract class. An abstract class is intended to be subclassed so that the abstract methods can be implemented. The abstract class itself contains concrete methods that invoke the abstract methods. The abstract class is free to evolve by adding more concrete methods; and this practice may break subclasses.

Therefore the exact problem you described existed even before Java8. The problem is much more manifested on Collection APIs because there are a lot of subclasses out in the wild.

While the leading motivation of default method was to add some useful methods to existing Collection APIs without breaking subclasses, they had to exercise great self-control of doing it too much, for fear of breaking subclasses. A default method is added only if it's absolutely necessary. The real question here is, why List.sort is considered absolutely necessary. I think that is debatable.

Regardless of why default method was introduced in the 1st place, it is now a great tool for API designers, and we ought to treat it the same as concrete methods in abstract classes - they need to be designed carefully up front; and new ones must be introduced with great caution.

Ironically default methods in interfaces were introduced to allow existing libraries using those interfaces not to break, while introducing massive new functionality in the interfaces. (backward compatibility.)

Conflicts like that sort method might arise. Something to pay for the extra functionality. In your case also something to investigate (should new functionality be used instead?).

Java forward compatibility breaks are little, more in its typing system, which was constantly enlarged. First with generic types and now with inferred types from functional interfaces. From version to version and from compiler to compiler there were slight differences.

Reading this issue, I was thinking of its solution.
Default methods have solved the backward compatibility problems but forward compatibility issues will exist.
I think instead of extending existing classes, in such cases, we can have our application specific interfaces to add some desired behaviour to our class. We can implement this application specific interface and use it.