Simple explanation of clojure protocols

I'm trying to understand clojure protocols and what problem they are supposed to solve. Does anyone have a clear explanation of the whats and whys of clojure protocols?

The purpose of Protocols in Clojure is to solve the Expression Problem in an efficient manner.

So, what's the Expression Problem? It refers to the basic problem of extensibility: our programs manipulate data types using operations. As our programs evolve, we need to extend them with new data types and new operations. And particularly, we want to be able to add new operations which work with the existing data types, and we want to add new data types which work with the existing operations. And we want this to be true extension, i.e. we don't want to modify the existing program, we want to respect the existing abstractions, we want our extensions to be separate modules, in separate namespaces, separately compiled, separately deployed, separately type checked. We want them to be type-safe. [Note: not all of these make sense in all languages. But, for example, the goal to have them type-safe makes sense even in a language like Clojure. Just because we can't statically check type-safety doesn't mean that we want our code to randomly break, right?]

The Expression Problem is, how do you actually provide such extensibility in a language?

It turns out that for typical naive implementations of procedural and/or functional programming, it is very easy to add new operations (procedures, functions), but very hard to add new data types, since basically the operations work with the data types using some sort of case discrimination (switch, case, pattern matching) and you need to add new cases to them, i.e. modify existing code:

func print(node):
  case node of:
    AddOperator => print(node.left) + '+' + print(node.right)
    NotOperator => '!' + print(node)

func eval(node):
  case node of:
    AddOperator => eval(node.left) + eval(node.right)
    NotOperator => !eval(node)

Now, if you want to add a new operation, say, type-checking, that's easy, but if you want to add a new node type, you have to modify all the existing pattern matching expressions in all operations.

And for typical naive OO, you have the exact opposite problem: it is easy to add new data types which work with the existing operations (either by inheriting or overriding them), but it is hard to add new operations, since that basically means modifying existing classes/objects.

class AddOperator(left: Node, right: Node) < Node:
  meth print:
    left.print + '+' + right.print

  meth eval
    left.eval + right.eval

class NotOperator(expr: Node) < Node:
  meth print:
    '!' + expr.print

  meth eval
    !expr.eval

Here, adding a new node type is easy, because you either inherit, override or implement all required operations, but adding a new operation is hard, because you need to add it either to all leaf classes or to a base class, thus modifying existing code.

Several languages have several constructs for solving the Expression Problem: Haskell has typeclasses, Scala has implicit arguments, Racket has Units, Go has Interfaces, CLOS and Clojure have Multimethods. There are also "solutions" which attempt to solve it, but fail in one way or another: Interfaces and Extension Methods in C# and Java, Monkeypatching in Ruby, Python, ECMAScript.

Note that Clojure actually already has a mechanism for solving the Expression Problem: Multimethods. The problem that OO has with the EP is that they bundle operations and types together. With Multimethods they are separate. The problem that FP has is that they bundle the operation and the case discrimination together. Again, with Multimethods they are separate.

So, let's compare Protocols with Multimethods, since both do the same thing. Or, to put it another way: Why Protocols if we already have Multimethods?

The main thing Protocols offer over Multimethods is Grouping: you can group multiple functions together and say "these 3 functions together form Protocol Foo". You cannot do that with Multimethods, they always stand on their own. For example, you could declare that a Stack Protocol consists of both a push and a pop function together.

So, why not just add the capability to group Multimethods together? There's a purely pragmatic reason, and it is why I used the word "efficient" in my introductory sentence: performance.

Clojure is a hosted language. I.e. it is specifically designed to be run on top of another language's platform. And it turns out that pretty much any platform that you would like Clojure to run on (JVM, CLI, ECMAScript, Objective-C) has specialized high-performance support for dispatching solely on the type of the first argument. Clojure Multimethods OTOH dispatch on arbitrary properties of all arguments.

So, Protocols restrict you to dispatch only on the first argument and only on its type (or as a special case on nil).

This is not a limitation on the idea of Protocols per se, it is a pragmatic choice to get access to the performance optimizations of the underlying platform. In particular, it means that Protocols have a trivial mapping to JVM/CLI Interfaces, which makes them very fast. Fast enough, in fact, to be able to rewrite those parts of Clojure which are currently written in Java or C# in Clojure itself.

Clojure has actually already had Protocols since version 1.0: Seq is a Protocol, for example. But until 1.2, you couldn't write Protocols in Clojure, you had to write them in the host language.

I find it most helpful to think of protocols as being conceptually similar to an "interface" in object oriented languages such as Java. A protocol defines an abstract set of functions that can be implemented in a concrete way for a given object.

An example:

(defprotocol my-protocol 
  (foo [x]))

Defines a protocol with one function called "foo" that acts on one parameter "x".

You can then create data structures that implements the protocol, e.g.

(defrecord constant-foo [value]  
  my-protocol
    (foo [x] value))

(def a (constant-foo. 7))

(foo a)
=> 7

Note that here the object implementing the protocol is passed as the first parameter x - somewhat like the implicit "this" parameter in object oriented languages.

One of the very powerful and useful features of protocols is that you can extend them to objects even if the object was not originally designed to support the protocol. e.g. you can extend the protocol above to the java.lang.String class if you like:

(extend-protocol my-protocol
  java.lang.String
    (foo [x] (.length x)))

(foo "Hello")
=> 5