(defmacro resolution [] `x)
Viewing the expansion of this macro is illuminating in understanding how Clojure
macros resolve symbols:
(macroexpand '(resolution))
;=> user/x
The expansion of the macro resolves the namespace of the syntax-quoted symbol x.
This behavior is useful in Clojure by helping to avoid free name capturing problems
that are possible in a macro system such as that found in Common Lisp.8 Here’s an
example that would trip up a lesser implementation of syntax-quote, but which does
just what we want in Clojure:
(def x 9)
(let [x 109] (resolution))
;=> 9
8
Among one of the ways that Common Lisp works to alleviate this kind of problem is the use of gensym. The
key difference is that in Common Lisp, you have to be careful to avoid name capturing, whereas Clojure avoids
it by default.
Download from Wow! eBook <www.wowebook.com>
170
CHAPTER 8 Macros
The x defined in the let isn’t the same as the namespace-qualified user/x referred to
by the macro resolution. As you might expect, the macro would’ve thrown an
unbound Var exception had we not first executed the call to def.
Clojure does provide a way to defer symbolic resolution for those instances where
it may be useful to resolve it within the execution context, which we’ll show now.
8.5.1
Anaphora
Anaphora9 in spoken language is a term used in a sentence referring back to a previ-
ously identified subject or object. It helps to reduce repetition in a phrase by replac-
ing “Jim bought 6,000 Christmas lights and hung all of the Christmas lights,” with “Jim
bought 6,000 Christmas lights and hung them all.” In this case, the word them is the
anaphora. Some programming languages use anaphora, or allow for their simple defi-
nition. Scala has a rich set of anaphoric (Odersky 2008) patterns primarily focused
around its _ operator:
Array(1, 2, 3, 4, 5).map(2 * _)
//=> res0: Array[Int] = Array(2, 4, 6, 8, 10)
In this Scala example, the underscore serves to refer back to an implicitly passed
argument to the map function, which in this case would be each element of the array
in succession. The same expression could be written with (x) => 2 * x—the syntax for
an anonymous function—in the body of the map call, but that would be unnecessarily
verbose.
Anaphora don’t nest, and as a result are generally not employed in Clojure. Within
a nested structure of anaphoric macros, you can only refer to the most immediate ana-
phoric binding, and never those from outer lexical contours, as demonstrated in list-
ing 8.7. For example, the Arc programming language (Graham Arc) contains a macro
named awhen similar to Clojure’s when, save that it implicitly defines a local named it
used within its body to refer to the value of the checked expression.
Listing 8.7 An example of anaphora and its weakness
(defmacro awhen [expr & body]
`(let [~'it ~expr]
Define anaphora
(when ~'it
Check its truth
(do ~@body))))
Inline body
(awhen [:a :b :c] (second it)
Use it in body
;=> :b
(awhen nil (println "Will never get here"))
;=> nil
(awhen :outer (awhen :inner [it]))
Fail to nest
;=> [:inner]
9 Anaphora is pronounced un-NAF-er-uh.
Download from Wow! eBook <www.wowebook.com>
Using macros to manage resources
171
Clojure provides similar macros that do nest and replace the need for anaphora: if-
let and when-let. When designing your own macros, it’s preferred that you build
them along these lines so that the macro itself takes the name to be bound. But just
because typical anaphorics are limited, that’s not to say that they’re entirely useless.
Instead, for your own libraries you may find that their usage is intuitive. You’ll see the
pattern ~'symbol at times in Clojure macros, because this is the idiomatic way to selec-
tively capture a symbolic name within the body of a macro. The reason for this bit of
awkwardness10 is because Clojure’s syntax-quote attempts to resolve symbols in the
current context, resulting in fully qualified symbols. Therefore, ~' avoids that resolu-
tion by unquoting a quote.
8.5.2
(Arguably) useful selective name capturing
We contend that there’s only one case to be made for selective name capturing in Clo-
jure macros—the case when you’re forced to embed third-party macros and functions
in your macros that rely on the existence of anaphora. One such macro is the proxy
macro in Clojure’s core libraries, which provides an anaphoric symbol named this
within its body for use therein. We’ll cover the proxy macro in depth in section 9.1, so
there’s no need to discuss it here. But bear in mind that should this macro ever be
embedded within your own macros, you may be forced to use the ~'this pattern.
HYGIENE A hygienic macro is one that doesn’t cause name capturing at macro
expansion time. Clojure macros help to ensure hygiene by namespace-
resolving symbols within the body of syntax-quote at macro-definition time.
As you saw, symbols are expanded into the form user/a-symbol within the
body of syntax-quote. To close this hygienic loop, Clojure also disallows the
definition of qualified locals within the body of a macro. In order to selec-
tively capture names within Clojure macros, you must explicitly do so via the
~'a-symbol pattern.
Clojure prefers that symbols be either declared or bound at macro-definition time.
But using the resolution deferment strategy outlined earlier, you can relax this
requirement for those instances where doing so would be useful.
8.6
Using macros to manage resources
Managing scarce resources or those with a finite lifetime is often viewed as a sweet
spot for macro usage. In Java, such activities are almost always performed using the
try/catch/finally idiom (Bloch 2008), as shown:
try {
// open the resource
}
catch (Exception e) {
// handle any errors
10 Awkwardness is good since it’s a strong signal to make the user aware he is drifting away from the true path to clojure enlightenment. —Christophe Grand
Download from Wow! eBook <www.wowebook.com>
172
CHAPTER 8 Macros
}
finally {
// in any case, release the resource
}
We showed in section 1.5.8 that Clojure also has a try/catch/finally form that can
be used in the same way, but like the Java idiom, you must remember to explicitly
close the resource within the finally block. Clojure provides a generic with-open
macro, demonstrated in listing 8.8, that when given a “closeable” object bound to a
name, will automatically call its .close method (assuming that one exists) within a
finally block.
Listing 8.8 An example of with-open
(import [java.io BufferedReader InputStreamReader]
[java.net URL])
(defn joc-www []
(-> "http://joyofclojure.com/hello" URL.
.openStream InputStreamReader. BufferedReader.))
(let [stream (joc-www)]
(with-open [page stream]
Begin IO block
(println (.readLine page))
(print "The stream will now close... "))
Close implicitly
(println "but let's read from it anyway.")
(.readLine stream))
Use illegally after close
; Hello Cleveland
; The stream will now close... but let's read from it anyway.
; java.io.IOException: Stream closed
Not all instances of resources in your own programs will be closeable. In these
instances, we present a generic template for resource allocating macros that can be
used for many cases, shown in the following listing.
Listing 8.9 A more general template for with-open-like macros
(defmacro with-resource [binding close-fn & body]
`(let ~binding
(try
(do ~@body)
(finally
(~close-fn ~(binding 0))))))
(let [stream (joc-www)]
(with-resource [page stream]
#(.close %)
(.readLine page)))
The macro with-resource is generic enough and so generally ubiquitous across dif-
fering flavors (Symbolics Inc.11) to almost be considered a Lisp design pattern. The
11 The spirit of this section was inspired by a similar discussion of “Writing Macros to Surround Code.” If you
can get your hands on the original Symbolics manuals, do so—they contain a wealth of information.
Download from Wow! eBook <www.wowebook.com>
Putting it all together: macros returning functions
173
macro with-resource differs from with-open in that it does not assume that its
resource is closeable but instead delegates the task of closing the resource to a close-
fn function taken as an argument. One final point is that with-resource avoids the
nesting problem of anaphoric macros because it requires that the resource be named
explicitly a la [stream (joc-www)]. This approach allows for the proper nesting of
with-resource macros; and in fact, the use of named bindings marked by vectors is
ubiquitous and idiomatic in Clojure.
8.7
Putting it all together: macros returning functions
In section 7.1, we introduced Clojure’s constraint facility that uses pre- and post-
condition checks on function arguments and return values respectively to ensure
some assertions about said function. In that section, we talked briefly about how sepa-
rating the constraints from the functions they’re constraining allows you to more flex-
ibly apply different assertion templates based on need and context.
CLOJURE APHORISM Clojure programmers don’t write their apps in Clojure.
They write the language that they use to write their apps in Clojure.
In this section, we’re going to take this idea one step further by introducing a macro
named contract that implements a simple DSL to describe function constraints. For
example, a proposed DSL should be nameable and describe its pre- and post-
conditions in an intuitive way, building a higher-order function that will be used to
apply its constraints later. The following sketches a contract specifying that a function
should take only a positive number and return its value multiplied by 2:
(contract doubler
[x]
(:require
(pos? x))
(:ensure
(= (* 2 x) %)))
The contract’s :require list (Meyer 2000) refers to preconditions, and the :ensure
list the postconditions. Given this description, how would you start to implement a
macro to realize this sketch? If you haven’t already gathered from the section title and
the initial problem statement, the macro must return a function, so we’ll start there
with the following listing.
Listing 8.10 The contract top-level macro
(declare collect-bodies)
(defmacro contract [name & forms]
(list* `fn name (collect-bodies forms)))
The contract macro calls a function collect-bodies that hasn’t been written yet, so
we had to use declare to avoid a compilation error. Hold fast, because we’re going to
implement that necessary function soon. But first, imagine what the form of the
returned function will be when it finally comes out of contract:
Download from Wow! eBook <www.wowebook.com>
174
CHAPTER 8 Macros
(fn doubler
([f x]
{:post [(= (* 2 x) %)],
:pre [(pos? x)]}
(f x)))
We also want to allow for the multi-arity function definition form so that the con-
tract can take more than one specification per arity function, each separated by a
vector of symbols. The first step down that path starts with an implementation of
collect-bodies:
(declare build-contract)
(defn collect-bodies [forms]
(for [form (partition 3 forms)]
(build-contract form)))
The primary task of collect-bodies is to build a list of the body portion of the
contract, each partitioned into three segments. These partitions represent the arg-
list, requires, and ensures of the contract, which we’ll then pass along to another
function named build-contract, that will build the arity bodies and corresponding
constraint maps. This is shown next.
Listing 8.11 The contract auxiliary function build-contract
(defn build-contract [c]
(let [args (first c)]
Grab args
(list
Build list...
(into '[f] args)
Include HOF + args
(apply merge
(for [con (rest c)]
(cond (= (first con) :require)
Process requires
(assoc {} :pre (vec (rest con)))
(= (first con) :ensure)
Process ensures
(assoc {} :post (vec (rest con)))
:else (throw (Exception. (str "Unknown tag " (first con)))))))
(list* 'f args))))
Build call to f
The function build-contract is where the heart of contract construction lies, build-
ing the arity bodies that contain constraint maps. The difference is that each body is a
higher-order function that takes an additional function as an argument, which the
arguments are then delegated to. This allows us to compose the contract function
with a constrained function, as shown in the next listing.
Listing 8.12 Composition of contract function and constrained function
(def doubler-contract
Define contract
(contract doubler
[x]
(:require
(pos? x))
(:ensure
(= (* 2 x) %))))
Download from Wow! eBook <www.wowebook.com>
Putting it all together: macros returning functions
175
(def times2 (partial doubler-contract #(* 2 %)))
Test correct fn
(times2 9)
;=> 18
(def times3 (partial doubler-contract #(* 3 %)))
Test incorrect fn
(times3 9)
; java.lang.AssertionError: Assert failed: (= (* 2 x) %)
As you might expect, times2 fulfills the contract, whereas times3 doesn’t. We could
extend doubler-contract to handle extended arities, as shown here.
Listing 8.13 Contract for multiple-arity functions
(def doubler-contract
(contract doubler
[x]
Define 1-arg contract
(:require
(pos? x))
(:ensure
(= (* 2 x) %))
[x y]
Define 2-arg contract
(:require
(pos? x)
(pos? y))
(:ensure
(= (* 2 (+ x y)) %))))
((partial doubler-
contract #(* 2 (+ %1 %2))) 2 3)
Test a correct fn
;=> 10
((partial doubler-
contract #(+ %1 %1 %2 %2)) 2 3)
Test another correct fn
;=> 10
((partial doubler-contract #(* 3 (+ %1 %2))) 2 3)
Test an incorrect fn
; java.lang.AssertionError:
; Assert failed: (= (* 2 (+ x y)) %)
We could extend the contract to cover any number of expected function arities using
contract, independent of the functions themselves. This provides a nice separation
of the work to be done from the expected work to be done. By using the contract
macro, we’ve provided a way to describe the expectations of a function, including but
not limited to
The possible types of its inputs and output
The relationship of the function output to its inputs
The expected function arities
The “shape” of the inputs and output
The contract macro could be extended in many complementary ways. For example,
Clojure’s function constraints are verified using logical and—the implications being
that any additional pre- or postcondition works to tighten the requirements. But there
may be times when loosening the constraints on the inputs and tightening them on the
Download from Wow! eBook <www.wowebook.com>
176
CHAPTER 8 Macros
output makes more sense. In any case, this section isn’t about the nuances of contracts
programming, and to dig deeper would elude the point that using macros to return
functions is an extremely powerful way to extend the capabilities of Clojure itself.
8.8
Summary
We’ve explored various use cases for macros and given examples of each. Though
instructive to the point under discussion, we also tried to show how macros can be
used to mold Clojure into the language that shortens the gap between your problem
space and solution space. In your own unique programs, you should try to do the
same. But the most important skill that you can learn on your path toward macro mas-
tery is the ability to recognize when to avoid using them. The general answer of course
is whenever, and as often as you can.
In the next chapter, we’ll cover various powerful way to organize and categorize
data types and functions using Clojure’s namespaces, multimethods, types, and
protocols.
Download from Wow! eBook <www.wowebook.com>
Combining
data and code
This chapter covers
Namespaces
Exploring Clojure multimethods with the Universal
Design Pattern
Types, protocols, and records
Putting it all together: a fluent builder for chess moves
Clojure provides powerful features for grouping and partitioning logical units of
code and data. Most logical groupings occur within namespaces, Clojure’s ana-
logue to Java packages. We explore how to build, manipulate, and reason about
them. Also, in this chapter we’ll play with Clojure’s powerful multimethods that
provide polymorphism based on arbitrary dispatch functions. We then uncover
recent additions to Clojure supporting abstraction-oriented programming —types, pro-
tocols, and records. Finally, the chapter concludes with the creation of a fluent
chess-move facility, comparing a Java approach to solving the problem with a Clo-
jure approach.
177
Download from Wow! eBook <www.wowebook.com>
178
CHAPTER 9 Combining data and code
9.1
Namespaces
Newcomers to Clojure have a propensity to hack away at namespace declarations until
they appear to work. This may work sometimes, but it delays the process of learning
how to leverage namespaces more effectively.
From a high-level perspective, namespaces can be likened to a two-level mapping,
where the first level is a symbol to a namespace containing mappings of symbols to
Vars, as shown in figure 9.1. This conceptual model1 is slightly complicated by the fact
that namespaces can be aliased, but even in these circumstances the model holds true.
In the simplest possible terms, qualified symbols of the form joy.ns/authors
cause a two-level lookup: a symbol joy.ns used to lookup a namespace map and a
symbol authors used to retrieve a Var, as shown in the following listing.
Listing 9.1 Namespace navigation
(in-ns 'joy.ns)
(def authors ["Chouser"])
(in-ns 'your.ns)
(clojure.core/refer 'joy.ns)
joy.ns/authors
;=> ["Chouser"]
(in-ns 'joy.ns)
(def authors ["Chouser" "Fogus"])
(in-ns 'your.ns)
joy.ns/authors
;=> ["Chouser" "Fogus"]
Because a symbolic name refers to a Var in the current namespace or another, it fol-
lows that any referred Var always evaluates to the current value and not the value pres-
ent at referral time.
joy.nc/authors
pre
r fix
fi e
x s
s
jo
j y
o .
y n
. s
n
:use
'authors
'joy.ns
:as
'j
:import
'Date
Figure 9.1 The logical layout of
(var authors)
namespaces. The process to resolve
a Var joy.ns/authors includes a
j/authors
symbolic resolution of the namespace
and the Var name. The result is the
Var itself. Aliases created with :use
java.util.Date
work as expected.
1 As always, we’re trying to keep the level of discussion limited to abstractions rather than implementation
details.
Download from Wow! eBook <www.wowebook.com>
Namespaces
179
9.1.1
Creating namespaces
There are a number of ways to create a new namespace; each has its advantages and
use cases. The choice of one namespace-creation mechanism over another amounts
to choosing the level of control over the default symbolic mappings.
NS
In idiomatic Clojure source code, you’ll see the ns macro used almost exclusively. By
using the ns macro, you automatically get two sets of symbolic mappings—all classes
in the java.lang package and all of the functions, macros, and special forms in the
clojure.core namespace:
(ns chimp)
(reduce + [1 2 (Integer. 3)])
;=> 6
Using the ns macro creates a namespace if it doesn’t already exist, and switches to that
namespace. The ns macro is intended for use in source code files and not in the REPL,
although there’s nothing preventing it.
IN-NS
Using the in-ns function also imports the java.lang package like ns; but it doesn’t
create any mappings for functions or macros in clojure.core. The in-ns function
also takes an explicit symbol used as the namespace qualifier, as in
(in-ns 'gibbon)
(reduce + [1 2 (Integer. 3)])
; java.lang.Exception: Unable to resolve symbol: reduce in this context
(clojure.core/refer 'clojure.core)
(reduce + [1 2 (Integer. 3)])
;=> 6
The in-ns function is more amenable to REPL experimentation when dealing with
namespaces than ns.
CREATE-NS
The finest level of control for creating namespaces is provided through the create-ns
function, which when called takes a symbol and returns a namespace object:
(def b (create-ns 'bonobo))
b
;=> #<Namespace bonobo>
((ns-map b) 'String)
;=> java.lang.String
The call to create-ns doesn’t switch to the named namespace, but it does create Java
class mappings automatically. When given a namespace object (retrieved using the
find-ns function also), you can manipulate its bindings programmatically using the
functions intern and ns-unmap:
Download from Wow! eBook <www.wowebook.com>
180
CHAPTER 9 Combining data and code
(intern b 'x 9)
;=> #'bonobo/x
bonobo/x
;=> 9
In the preceding code, we bound the symbol x to the value 9 in the namespace
bonobo, and then referenced it directly using its qualified name bonobo/x. We can do
the same thing for any type of Var binding:
(intern b 'reduce clojure.core/reduce)
;=> #'bonobo/reduce
(intern b '+ clojure.core/+)
;=> #'bonobo/+
(in-ns 'bonobo)
(reduce + [1 2 3 4 5])
;=> 15
Because only Java class mappings are created by create-ns, you’ll have to intern any
Clojure core functions, as we did with + and reduce. You can even inspect the map-
pings within a namespace programmatically, and likewise remove specific mappings:
(in-ns 'user)
(get (ns-map 'bonobo) 'reduce)
;=> #'bonobo/reduce
(ns-unmap 'bonobo 'reduce) ;=> nil
(get (ns-map 'bonobo) 'reduce)
;=> nil
Finally, you can wipe a namespace using remove-ns:
(remove-ns 'bonobo)
;=> #<Namespace bonobo>
(all-ns)
;=> (#<Namespace clojure.set> #<Namespace clojure.main>
#<Namespace clojure.core> #<Namespace clojure.zip>
#<Namespace chimp> #<Namespace gibbon>
#<Namespace clojure.xml>)
You should be careful when populating namespaces
using create-ns and intern, because they cause
potentially confusing side-effects to occur. Their use is
intended only for advanced techniques, and even then
they should be used cautiously.
9.1.2
Expose only what’s needed
Knowing that namespaces operate as a two-level map-
ping will only get you so far in creating and using them
effectively. You must understand other practical matters
to use namespaces to their fullest. For example, for a
Figure 9.2 Namespace private
directories: the directories layout
given namespace joy.contracts, your directory struc-
for an illustrative joy.contracts
ture could look like that in figure 9.2.
namespace
Download from Wow! eBook <www.wowebook.com>
Namespaces
181
This directory structure is fairly
straightforward, but there are a couple
items to note. First, though the
namespace is named joy.contracts,
the corresponding Clojure source file is
located in the contracts-lib/src/joy
directory. This is a common technique
in organizing Java and Clojure projects,
where the actual source directories and
files are located in a common src subdi-
Figure 9.3 Namespace private source: the top of
rectory in the main project directory.
the source file for the joy.contracts namespace
The additional files build.xml, pom.xml,
and project.clj correspond to the build scripts for Apache Ant, Maven, and Leiningen,
respectively. These build scripts will know, through either configuration or conven-
tion, that the src directory contains the directories and source files for Clojure
namespaces and not part of the namespace logical layout. If you were to open the con-
tracts.clj file located in contracts-lib/src/joy in your favorite editor, then you might
see something like that shown in figure 9.3.
The file contracts.clj defines the namespace joy.contracts and defines the func-
tion build-contract using the defn- macro. The use of defn- in this way indicates to
Clojure that the build-contract function is private to the joy.contracts
namespace. The defn- macro is provided for convenience and simply attaches privi-
leged metadata to the Var containing the function. You could attach the same
namespace privacy metadata yourself, as shown:
(ns hider.ns)
(defn ^{:private true} answer [] 42)
(ns seeker.ns
(:refer hider.ns))
(answer)
; java.lang.Exception: Unable to resolve symbol: answer in this context
The use of ^{:private true} in this way will also work within a def and a defmacro,
and for these cases it’s required, because there’s no corresponding def- and def-
macro- in Clojure’s core.
HYPHENS/UNDERSCORES If you decide to name your namespaces with
hyphens, à la my-cool-lib, then the corresponding source file must be
named with underscores in place of the hyphens (my_cool_lib.clj).
Because Clojure namespace names are tied to the directory in which they reside, you
can also create a certain directory structure conducive to hiding implementation
details, as seen in figure 9.4.
By creating another subdirectory to contracts-lib/src/joy named impl, you can
effectively hide implementation details for your code. The public-facing API would be
Download from Wow! eBook <www.wowebook.com>
182
CHAPTER 9 Combining data and code
located in contracts.clj and the “hidden” implemen-
tation details in impl.clj. Your clients would be
expected to refer only to the elements in con-
tracts.clj, whereas your library could refer to ele-
ments in impl.clj, as shown in figure 9.5.
Of course, nothing’s stopping you from also ref-
erencing the joy.contracts.impl namespace, but
you do so at their own peril. There are never any
guarantees that implementation details will remain
Figure 9.4 Private API directories:
the same shape from one release to the next.
using the folder layout to hide
namespace implementation details
9.1.3
Declarative inclusions and exclusions
When defining namespaces, it’s important to include only the references that are
likely to be used. Clojure prefers a fine-grained Var mapping via a set of directives on
the ns macro: :exclude, :only, :as, :refer-clojure, :import, :use, :load, and
:require.
We’ll describe a namespace named joy.ns-ex first in prose and then using ns and
its directives. In this namespace, we want to exclude the defstruct macro from
Figure 9.5 Private API source: the client-facing API is located in
contracts.clj and the private API in impl.clj.
Download from Wow! eBook <www.wowebook.com>
Exploring Clojure multimethods with the Universal Design Pattern
183
clojure.core. Next, we want to use everything in clojure.set and clojure.xml
without namespace qualification. Likewise, we wish to use only the functions are and
is from the clojure.test namespace without qualification. We then want to load the
clojure.zip namespace and alias it as z. Finally, we want to import the Java classes
java.util.Date and java.io.File. By providing directives, the problem of
namespace inclusions and exclusions become a declarative matter, as shown:
(ns joy.ns-ex
(:refer-clojure :exclude [defstruct])
(:use (clojure set xml))
(:use [clojure.test :only (are is)])
(:require (clojure [zip :as z]))
(:import (java.util Date)
(java.io File)))
We’ll touch on further uses of namespaces throughout the rest of the book, with an
extensive example explaining their use as JVM class specifications in section 10.3.
AVOID NAKED :USE One point of note that we should mention is that the
(:use (clojure set xml)) statement is considered a promiscuous operation
and therefore discouraged. The :use directive without the :only option pulls
in all of the public Vars in clojure.set and clojure.xml indiscriminately.
Though this practice is useful when incrementally building your code, it
shouldn’t endure into the production environment. When organizing your
code along namespaces, it’s good practice to export and import only those
elements needed.
We now turn our focus to Clojure’s multimethods, a way of defining polymorphic
functions based on the results of arbitrary functions, which will get you halfway toward
a system of polymorphic types.
9.2
Exploring Clojure multimethods
with the Universal Design Pattern
The most specific event can serve as a general example of a class of events.
—Douglas R. Hofstadter
In Douglas Hofstadter’s Pulitzer prize winning work Gödel, Escher, Bach: An Eternal
Golden Braid, he describes a notion of the Prototype Principle —the tendency of the
human mind to use specific events as models for similar but different events or things.
He presents the idea “that there is generality in the specific” (Hofstadter 1979). Build-
ing on this idea, programmer Steve Yegge coined the term The Universal Design Pattern
(UDP), extrapolating on Hofstadter’s idea (Yegge 2008) and presenting it in terms of
prototypal inheritance (Ungar 1987).
The UDP is built on the notion of a map or map-like object. Though not ground-
breaking, the flexibility in the UDP derives from the fact that each map contains a
reference to a prototype map used as a parent link to inherited fields. You might won-
der how anyone could model a software problem in this way, but we assure you that
Download from Wow! eBook <www.wowebook.com>
184
CHAPTER 9 Combining data and code
countless programmers do so every day when they choose JavaScript (Flanagan 2006).
In this section, we’ll implement a subset of Yegge’s UDP and discuss how it might be
used as the basis for abstraction-oriented programming and polymorphism using Clo-
jure’s multimethods and ad hoc hierarchies.
9.2.1
The parts
In addition to the aforementioned prototype reference, the UDP requires a set of sup-
porting functions to operate: beget, get, put, has?, and forget. The entire UDP is
built on these five functions, but we’ll need the first three for this section.
BEGET
The beget function performs a simple task. It takes a map and associates its prototype
reference to another map, returning a new map:
(ns joy.udp
(:refer-clojure :exclude [get]))
(defn beget [o p] (assoc o ::prototype p))
(beget {:sub 0} {:super 1})
;=> {:joy.udp/prototype {:super 1}, :sub 0}
To participate in the UDP, maps must have a :joy.udp/prototype entry.
PUT
The function put takes a key and an associated value and puts them into the supplied
map, overwriting any existing key of the same name:
(def put assoc)
The put function is asymmetric to the functionality of get. The get method retrieves
values anywhere along the prototype chain, whereas put only ever inserts at the level
of the supplied map.
GET
Because of the presence of the prototype link, get requires more than a simple one-
level lookup. Instead, whenever a value isn’t found in a given map, the prototype
chain is followed until the end:
(defn get [m k]
(when m
(if-let [[_ v] (find m k)]
v
(recur (::prototype m) k))))
(get (beget {:sub 0} {:super 1})
:super)
;=> 1
We don’t explicitly handle the case of “removed ”properties, but instead treat them
like any other associated value. This is fine because the “not found” value of nil is
falsey. Most of the time, it’s sufficient to rely on the fact that looking up a nonexistent
Download from Wow! eBook <www.wowebook.com>
Exploring Clojure multimethods with the Universal Design Pattern
185
key will return nil. But in cases where you want to allow users of your functions to
store any value at all, including nil, you’ll have to be careful to distinguish nil from
“not found,” and the find function is the best way to do this.
9.2.2
Usage
Using only beget, put, and get, you can leverage the UDP in some simple, yet power-
ful ways. Assume that at birth cats like dogs and only learn to despise them when
goaded. Morris the cat has spent most of his life liking 9-Lives cat food and dogs, until
the day comes when a surly Shih Tzu leaves him battered and bruised. We can model
this unfortunate story as shown:
(def cat {:likes-dogs true, :ocd-bathing true})
(def morris (beget {:likes-9lives true} cat))
(def post-traumatic-morris (beget {:likes-dogs nil} morris))
(get cat :likes-dogs)
;=> true
(get morris :likes-dogs)
;=> true
(get post-traumatic-morris :likes-dogs)
;=> nil
The map post-traumatic-morris is like the old morris in every way except for the
fact that he has learned to hate dogs. Modeling cat and dog societal woes is interesting
but far from the only use case for the UDP, as you’ll see next.
NO NOTION OF SELF
Our implementation of the UDP contains no notion of self-awareness via an implicit
this or self reference. Though adding such a feature would probably be possible,
we’ve intentionally excluded it in order to draw a clear separation between the proto-
types and the functions that work on them (Keene 1989). A better solution, and one
that follows in line with a deeper Clojure philosophy, would be to access, use, and
manipulate these prototypes using Clojure’s multimethods.
9.2.3
Multimethods to the rescue
Adding behaviors to the UDP can be accomplished easily using Clojure’s multimethod
facilities. Multimethods provide a way to perform function polymorphism based on the
result of an arbitrary dispatch function. Coupled with our earlier UDP implementa-
tion, we can implement a prototypal object system with differential inheritance similar
to (although not as elegant as) that in the Io language (Dekorte Io). First, we’ll need
to define a multimethod compiler that dispatches on a key :os:
(defmulti compiler :os)
(defmethod compiler ::unix [m] (get m :c-compiler))
(defmethod compiler ::osx [m] (get m :c-compiler))
Download from Wow! eBook <www.wowebook.com>
186
CHAPTER 9 Combining data and code
The multimethod compiler describes a simple scenario: if the function compiler is
called with a prototype map, then the map is queried for an element :os, which has
methods defined on the results for either ::unix or ::osx. We’ll create some proto-
type maps to exercise compiler:
(def clone (partial beget {}))
(def unix {:os ::unix, :c-compiler "cc", :home "/home", :dev "/dev"})
(def osx (-> (clone unix)
(put :os ::osx)
(put :c-compiler "gcc")
(put :home "/Users")))
(compiler unix)
;=> "cc"
(compiler osx)
;=> "gcc"
That’s all there is (Foote 2003) to creating behaviors that work against the specific
“type” of a prototype map. But a problem of inherited behaviors still persists. Because
our implementation of the UDP separates state from behavior, there’s seemingly no
way to associate inherited behaviors. But as we’ll now show, Clojure does provide a way
to define ad hoc hierarchies that we can use to simulate inheritance within our model.
9.2.4
Ad hoc hierarchies for inherited behaviors
Based on the layout of the unix and osx prototype maps, the property :home is over-
ridden in osx. We could again duplicate the use of get within each method defined
(as in compiler), but instead we prefer to say that the lookup of :home should be a
derived function:
(defmulti home :os)
(defmethod home ::unix [m] (get m :home))
(home unix)
;=> "/home"
(home osx)
; java.lang.IllegalArgumentException:
; No method in multimethod 'home' for dispatch value: :user/osx
Clojure allows you to define a relationship stating “::osx is a ::unix” and have the
derived function take over the lookup behavior using Clojure’s derive function:
(derive ::osx ::unix)
Now the home function works:
(home osx)
;=> "/Users"
You can query the derivation hierarchy using the functions parents, ancestors,
descendants, and isa? as shown:
(parents ::osx)
;=> #{:user/unix}
Download from Wow! eBook <www.wowebook.com>
Exploring Clojure multimethods with the Universal Design Pattern
187
(ancestors ::osx)
;=> #{:user/unix}
(descendants ::unix)
;=> #{:user/osx}
(isa? ::osx ::unix)
;=> true
(isa? ::unix ::osx)
;=> false
The result of the isa? function defines how multimethods dispatch. In the absence of
a derivation hierarchy, isa? can be likened to pure equality, but with it traverses a der-
ivation graph.
9.2.5
Resolving conflict in hierarchies
What if we interject another ancestor into the hierarchy for ::osx and want to again
call the home method? Observe the following:
(derive ::osx ::bsd)
(defmethod home ::bsd [m] "/home")
(home osx)
; java.lang.IllegalArgumentException: Multiple methods in multimethod
; 'home' match dispatch value: :user/osx -> :user/unix and
; :user/bsd, and neither is preferred
As shown in figure 9.6, ::osx derives from both ::bsd and ::unix, so there’s no way
to decide which method to dispatch, because they’re both at the same level in the der-
ivation hierarchy. Fortunately, Clojure provides a way to assign favor to one method
over another using the function prefer-method:
(prefer-method home ::unix ::bsd)
(home osx)
;=> "/Users"
In this case, we used prefer-method to explicitly state that for the multimethod home,
we prefer the method associated with the dispatch value ::unix over the one for
::bsd, as illustrated in figure 9.5. As you’ll recall, the home method for ::unix explic-
itly used get to traverse the prototype chain, which is the preferred behavior.
As you might expect, removing the home method for the ::bsd dispatch value
using remove-method will remove the preferential lookup for ::osx:
(remove-method home ::bsd)
(derive ::osx ::unix)
(home osx)
::unix
;=> "/Users"
::osx
All of these functions manipulate and
Figure 9.6 Hierarchy conflict:
most languages allowing type
operate off of the global hierarchy map
(derive ::osx ::bsd)
derivations use a built-in
directly. If you prefer to reduce these
::unix
::bsd
conflict-resolution strategy. In
potentially confusing side-effects, then
the case of CLOS, it’s fully
::osx
you can define a derivation hierarchy
customizable. Clojure requires
(home osx)
conflicts to be resolved with
using make-hierarchy and derive:
???
prefer-method.
Download from Wow! eBook <www.wowebook.com>
188
CHAPTER 9 Combining data and code
(derive (make-hierarchy) ::osx ::unix)
;=> {:parents {:user/osx #{:user/unix}},
:ancestors {:user/osx #{:user/unix}},
:descendants {:user/unix #{:user/osx}}}
Once you have a separate hierarchy in hand, you can provide it to defmulti to specify
the derivation context, thus preserving the global hierarchy map.
9.2.6
Arbitrary dispatch for true maximum power
Until now, we’ve only exercised multimethods using a single privileged :os property,
but this doesn’t accentuate their true power. Instead, multimethods are fully open and
can dispatch on the result of an arbitrary function, even one that can pull apart and/
or combine its inputs into any form:
(defmulti compile-cmd (juxt :os compiler))
(defmethod compile-cmd [::osx "gcc"] [m]
(str "/usr/bin/" (get m :c-compiler)))
(defmethod compile-cmd :default [m]
(str "Unsure where to locate " (get m :c-compiler)))
The dispatch values for the new compile-cmd methods are vectors composed of the
results of looking up the :os key and calling the compiler function defined earlier.
You can now observe what happens when compile-cmd is called:
(compile-cmd osx)
;=> "/usr/bin/gcc"
(compile-cmd unix)
;=> "Unsure where to locate cc"
Using multimethods and the UDP is an interesting way to build abstractions. Multi-
methods and ad hoc hierarchies are open systems, allowing for polymorphic dispatch
based on arbitrary functions. Clojure also provides a simpler model for creating
abstractions and gaining the benefits of polymorphism—types, protocols, and
records—which we’ll cover next.
The handy-dandy juxt function
The juxt function is highly useful in defining multimethod dispatch functions. In a
nutshell, juxt takes a bunch of functions and composes them into a function return-
ing a vector of its argument(s) applied to each given function, as shown:
(def each-math (juxt + * - /))
(each-math 2 3)
;=> [5 6 -1 2/3]
((juxt take drop) 3 (range 9))
[(0 1 2) (3 4 5 6 7 8)]
Having a convenient and succinct way to build vectors of applied functions is powerful
for defining understandable multimethods—although that’s not the limit of juxt’s
usefulness.
Download from Wow! eBook <www.wowebook.com>
Types, protocols, and records
189
9.3
Types, protocols, and records
We showed in the previous section that Clojure multimethods provide a way to
achieve runtime polymorphism based on arbitrary dispatch functions. Though
extremely powerful, multimethods are sometimes less than ideal. Interposing a dis-
patch function into the polymorphism machinery isn’t always conducive to raw speed.
Likewise, dispatching on an arbitrary function is often overkill. Therefore, Clojure
provides facilities for creating logically grouped polymorphic functions that are both
simple and performant—types, records, and protocols. We’ll delve into these topics in
this section and introduce the concept of abstraction-oriented programming, predi-
cated on the creation of logical groupings. But first, we’ll discuss the simplest of the
three topics, records, which you might recognize.
9.3.1
Records
Using maps as data objects is perfectly acceptable and has several lovely features.
Chief among these is that maps require no declaration of any sort: you just use literal
syntax to build them right on the spot. We showed this in section 7.2 when we built an
object like this:
{:val 5, :l nil, :r nil}
This is handy but is missing things that are often desirable, the most significant of
which is a type of its own. The object constructed here is some kind of map, but it
isn’t, as far as Clojure is concerned, a TreeNode. That means that when used in its sim-
ple form as we did here, there’s no clean way2 to determine whether any particular
map is a TreeNode or not.
In such circumstances, records become a compelling3 solution. You define a
record type with a defrecord form. For example, a defrecord for TreeNode looks like
this:
(defrecord TreeNode [val l r])
This creates a new Java class with a constructor that takes a value for each of the fields
listed. It also imports that class into your current namespace so you can easily use it to
create new instances.
Here’s how to create an instance of the TreeNode record:
(TreeNode. 5 nil nil)
;=> #:user.TreeNode{:val 5, :l nil, :r nil}
2
You could test a map for the existence of the keys :val, :l, and :r, a sort of duck-typing but on fields instead
of methods. But because there exists a real possibility than some other kind of object may happen to have
these keys but use them in a different way, undesirable complexity and/or unexpected behavior is likely. For-
tunately, you can mitigate this risk by using namespace-qualified keywords. Despite the general agreement of
experts that ducks are Kosher, we’d definitely classify this particular duck as unclean.
3
There was a pre-Clojure 1.2 convention of attaching :type metadata to an object, which can be looked up
with the type function, but this approach is rarely if ever needed moving forward.
Download from Wow! eBook <www.wowebook.com>
190
CHAPTER 9 Combining data and code
Explicit importing of defrecord and deftype classes
It’s important to note that when you define a defrecord and deftype, corresponding
classes are generated. These classes are automatically imported into the same
namespace where the defrecord and deftype declarations occur, but not in any
other namespace. Instead, you must explicitly import defrecord and deftype
classes using the import function or :import namespace declaration:
(ns my-cool-ns
(:import joy.udp.TreeNode))
Loading a namespace via :require or :use won’t be enough to import defrecord
and deftype classes.
The use of defrecord buys you several important benefits. First of all, it provides a
simple and specific idiom for documenting the expected fields of the object. But it
also delivers several important performance improvements. A record will be created
more quickly, consume less memory, and look up keys in itself more quickly than the
equivalent array map or hash map. Data types can also store primitive values (byte, int,
long, and so on), which take up considerably less memory than the equivalent boxed
objects (Byte, Integer, Long, and so on) supported by other collection types.
That’s a lot of benefit, so what does it cost you? The first cost we already men-
tioned—you must define the record type before using it. Another is that currently,
records aren’t printed in a way that the Clojure reader can read, unlike hash maps.
This can be a problem if you’re using Clojure’s print functions to save off or transmit
data. Here’s what it looks like if we try, successfully, with a literal map and then again,
unsuccessfully, with a record:
The downfall of defstructs
Clojure provides a defstruct mechanism, which can be viewed as a way to define a
map that acts as an ad hoc class mechanism. These structs defined a set of keys
that were required to exist in the map and could therefore not be removed via dissoc.
With the advent of defrecord, the need for structs has been nearly eliminated, and
therefore structs aren’t covered in this book. But if you have a code base reliant on
structs, then a record can replace them with minimal code changes, as highlighted
here:
(defn change-age [p] (assoc p :age 286))
(defstruct person :fname :lname)
(change-age (struct person "Immanuel" "Kant"))
;=> {:fname "Immanuel", :lname "Kant", :age 286}
(defrecord Person [fname lname])
(change-age (Person. "Immanuel" "Kant"))
;=> #:user.Person{:fname "Immanuel", :lname "Kant", :age 286}
Note that the change-age function works with either structs or records—no change
is required. Only the definition and the mechanism of instantiation need to be
updated.
Download from Wow! eBook <www.wowebook.com>
Types, protocols, and records
191
(read-string (pr-str {:val 5, :l nil, :r nil}))
;=> {:val 5, :l nil, :r nil}
(read-string (pr-str (TreeNode. 5 nil nil)))
; java.lang.RuntimeException: java.lang.Exception: No dispatch macro for:
This may change eventually, but there are some tricky problems yet to be worked out
before records can be printed and read back in.
Other noteworthy differences between maps and records include
Records, unlike maps, can’t serve as functions.
Records are never equal to maps with the same key/value mappings.
You still look up values in records by doing (:keyword obj); it’s just that if obj is a
record, this code will run dramatically faster. By the way, that means destructuring will
still work as well. Records support metadata using with-meta and meta just like other
Clojure collections, and you can even redefine a record if desired to have different
fields giving you the compiled performance of Java dynamically. All of these together
mean you can build a lot of code on top of simple hash-map objects and then make
minimal changes to switch to using records instead, gaining all the performance ben-
efits we already covered.
You should understand records well enough to be able to reimplement the persis-
tent binary tree from chapter 5 using defrecord instead of maps. This is shown in the
following listing. Note that we had to add the defrecord and change the expressions
in xconj where objects are created, but the xseq function is defined identically to how
it was before.
Listing 9.2 Persistent binary tree built of records
(defrecord TreeNode [val l r])
Define record type
(defn xconj [t v]
Add to tree
(cond
(nil? t) (TreeNode. v nil nil)
(< v (:val t)) (TreeNode. (:val t) (xconj (:l t) v) (:r t))
:else (TreeNode. (:val t) (:l t) (xconj (:r t) v))))
(defn xseq [t]
Convert trees
(when t
to seqs
(concat (xseq (:l t)) [(:val t)] (xseq (:r t)))))
(def sample-tree (reduce xconj nil [3 5 2 4 6]))
Try it all out
(xseq sample-tree)
;=> (2 3 4 5 6)
You can assoc and dissoc any key you want—adding keys that weren’t defined in the
defrecord works, though they have the performance of a regular map. Perhaps more
surprisingly, dissocing a key given in the record works but returns a regular map
rather than a record. In this example, note that the return value is printed as a plain
map, not with the #:user.TreeNode prefix of a record:
Download from Wow! eBook <www.wowebook.com>
192
CHAPTER 9 Combining data and code
(dissoc (TreeNodePlus 5 nil nil) :l)
;=> {:val 5, :r nil}
A final benefit of records is how well they integrate with Clojure protocols. But to fully
understand how they relate, we must first explore what protocols are.
9.3.2
Protocols
The establishment of protocols ... creates an obvious way for two people who are
not directly communicating to structure independently developed code so that it
works in a manner that remains coherent when such code is later combined.
—Kent M. Pitman (Pitman 2001)
A protocol in Clojure is simply a set of function signatures, each with at least one param-
eter, that are given a collective name. They fulfill a role somewhat like Java interfaces
or C++ pure virtual classes—a class that claims to implement a particular protocol
should provide specialized implementations of each of the functions in that protocol.
Then, when any of those functions is called, the appropriate implementation is poly-
morphic on the type of the first parameter, just like Java. In fact, the first parameter to
a protocol function corresponds to the target object (the thing to the left of the dot
for a method call used in Java source) of a method in object-oriented parlance.
For example, consider what collections such as stacks (First In, Last Out: FILO)
and queues (First In, First Out: FIFO) have in common. Each has a simple function for
inserting a thing (call it push), a simple function for removing a thing (pop), and usu-
ally a function to see what would be removed if you removed a thing (peek). What we
just gave you was an informal description of a protocol; all that’s missing is the name.
We can replace the changing third item of the acronym with an X and call objects that
provide these functions FIXO. Note that besides stacks and queues, FIXO could
include priority queues, pipes, and other critters.
So now let’s look at that informal description rewritten as a formal Clojure
definition:
(defprotocol FIXO
(fixo-push [fixo value])
(fixo-pop [fixo])
(fixo-peek [fixo]))
...and that’s it. The only reason we prefixed the function names with fixo- is so that
they don’t conflict with Clojure’s built-in functions.4 Besides that, it’s hard to imagine
how there could be much less ceremony, isn’t it?
But in order for a protocol to do any good, something must implement it. Proto-
cols are implemented using one of the extend forms: extend, extend-type,5 or
extend-protocol. Each of these does essentially the same thing, but extend-type and
4
It would be better to fix this problem by defining FIXO in a new namespace and excluding from it the simi-
larly named clojure.core functions, except this would be a distraction from the point of this section. We’ll dis-
cuss interesting interactions between namespaces and protocols later in this chapter.
5
Records are a specialized kind of data type, so extend-type is used for both. We’ll look at data types later
in this section.
Download from Wow! eBook <www.wowebook.com>
Types, protocols, and records
193
extend-protocol are convenience macros for when you want to provide multiple
functions for a given type. For example, the binary TreeNode from listing 9.2 is a
record, so if we want to extend it, extend-type would be most convenient. Because
TreeNode already has a function xconj that works just like push should, we’ll start by
implementing that:
(extend-type TreeNode
FIXO
(fixo-push [node value]
(xconj node value)))
(xseq (fixo-push sample-tree 5/2))
;=> (2 5/2 3 4 5 6)
The first argument to extend-type is the class or interface that the entire rest of the
form will be extending. Following the type name are one or more blocks, each starting
with the name of the protocol to be extended and followed by one or more functions
from that protocol to implement. So in the preceding example, we’re implementing a
single function fixo-push for TreeNode objects, and we call the existing xconj func-
tion. Got it? The reason this is better than simply defining a regular function named
fixo-push is that protocols allow for polymorphism. That same function can have a
different implementation for a different kind of object. Clojure vectors can act like
stacks by extending FIXO to vectors:
(extend-type clojure.lang.IPersistentVector
FIXO
(fixo-push [vector value]
(conj vector value)))
(fixo-push [2 3 4 5 6] 5/2)
;=> [2 3 4 5 6 5/2]
Here we’re extending FIXO to an interface instead of a concrete class. This means that
fixo-push is now defined for all classes that inherit from IPersistentVector. Note
that we can now call fixo-push with either a vector or a TreeNode, and the appropri-
ate function implementation is invoked.
Clojure-style mixins
As you proceed through this section, you’ll notice that we extend the FIXO protocol’s
fixo-push function in isolation. This works fine for our purposes, but you might want
to take note of the implications of this approach. Consider the following:
(use 'clojure.string)
(defprotocol StringOps (rev [s]) (upp [s]))
(extend-type String
StringOps
(rev [s] (clojure.string/reverse s)))
(rev "Works")
;=> "skroW"
Download from Wow! eBook <www.wowebook.com>
194
CHAPTER 9 Combining data and code
(continued)
Defining the StringOps protocol and extending its rev function to String seems to
work fine. But observe what happens when the protocol is again extended to cover
the remaining upp function:
(extend-type String
StringOps
(upp [s] (clojure.string/upper-case s)))
(upp "Works")
;=> "WORKS"
(rev "Works?")
; IllegalArgumentException No implementation of method: :rev
; of protocol: #'user/StringOps found for
; class: java.lang.String
The reason for this exception is that for a protocol to be fully populated (all of its func-
tions callable), it must be extended fully, per individual type. Protocol extension is at
the granularity of the entire protocol and not at a per-function basis. This behavior
seems antithetical to the common notion of a mixin—granules of discrete functional-
ity that can be “mixed into” existing classes, modules, and so on. Clojure too has
mixins, but it takes a slightly different approach:
(def rev-mixin {:rev clojure.string/reverse})
(def upp-mixin {:upp (fn [this] (.toUpperCase this))})
(def fully-mixed (merge upp-mixin rev-mixin))
(extend String StringOps fully-mixed)
(-> "Works" upp rev)
;=> SKROW
Mixins in Clojure refer to the creation of discrete maps containing protocol function
implementations that are combined in such a way as to create a complete implemen-
tation of a protocol. Once mixed together (as in the Var fully-mixed), only then are
types extended to protocols. As with many of Clojure’s features, mixins and protocol
extension are fully open.
What we’ve just done is impossible with Java interfaces or C++ classes, at least in the
order we did it. With either of those languages, the concrete type (such as TreeNode
or vector) must name at the time it’s defined all the interfaces or classes it’s going to
implement. Here we went the other way around—both TreeNode and vectors were
defined before the FIXO protocol even existed, and we easily extended FIXO to each of
them. This matters in the real world because the concrete types and even the protocol
could be provided by third-party libraries—possibly even different third-party librar-
ies—and we could still match them up, provide implementations for the appropriate
functions, and get back to work. All this without any adapters, wrappers, monkey-
patching, or other incidental complexity getting in the way. In fact, Clojure polymor-
phism lives in the protocol functions, not in the classes, as shown in figure 9.7.
Download from Wow! eBook <www.wowebook.com>
Types, protocols, and records
195
Monkey Patching
Wrapping
Pr
P o
r to
t co
c ls
TreeNode
TreeNodeWrapper
FIXO
fixo-pop
(fixo-pop TreeNode)
fixo-pop
fixo-push
(fixo-pop Vector)
fixo-push
fixo-peek
(fixo-push TreeNode)
(fixo-push Vector)
TreeNode
fixo-peek
(fixo-peek TreeNode)
(fixo-peek Vector)
Figure 9.7 As opposed to the notion of monkey-patching and wrapping,
the polymorphism in Clojure resides in the functions themselves and not
in the classes worked with.
You can even extend a protocol to nil itself. You’d be forgiven for not immediately
seeing why you’d want to do this; but consider how TreeNode implements fixo-push,
and yet the sample-tree we’re using was built using xconj instead. Trying to build up
a tree the same way with fixo-push runs into a problem:
(reduce fixo-push nil [3 5 2 4 6 0])
; java.lang.IllegalArgumentException:
; No implementation of method: :fixo-push
; of protocol: #'user/FIXO found for class: nil
The xconj implementation specifically handled the initial nil case, but because pro-
tocol methods dispatch on the first argument, we need special support from extend to
get fixo-push to behave similarly. This is done by extending a protocol to the value
nil, like this:
(extend-type nil
FIXO
(fixo-push [t v]
(TreeNode. v nil nil)))
(xseq (reduce fixo-push nil [3 5 2 4 6 0]))
;=> (0 2 3 4 5 6)
All the options and arrangements of code allowed by extend can be disorienting, but
one thing you can keep firmly in mind is that extend is always about a protocol. Each
method listed in an extend form is implementing an intersection between a protocol
and something else. That something else can be a concrete class, an interface, a
record type, or even nil, but it’s always being connected to a protocol.
See the following listing for complete implementations of FIXO for TreeNode and
vectors. As mentioned in the sidebar, in order for the FIXO protocol to be fully realiz-
able, each of its functions should be mixed in. But you might not always require that a
protocol be fully realizable.
Download from Wow! eBook <www.wowebook.com>
196
CHAPTER 9 Combining data and code
Listing 9.3 Complete implementations of FIXO for TreeNode and vector
(extend-type TreeNode
FIXO
Delegate
(fixo-push [node value]
to xconj
(xconj node value))
(fixo-peek [node]
Walk down left nodes
(if (:l node)
to find smallest
(recur (:l node))
(:val node)))
Build new path down
(fixo-pop [node]
left to removed item
(if (:l node)
(TreeNode. (:val node) (fixo-pop (:l node)) (:r node))
(:r node))))
(extend-type clojure.lang.IPersistentVector
FIXO
fixo-push is
(fixo-push [vector value]
vector's conj
(conj vector value))
(fixo-peek [vector]
peek is peek
(peek vector))
(fixo-pop [vector]
pop is pop
(pop vector)))
If you’ve done six impossible things this morning, why not round it off with breakfast
at Milliways, the Restaurant at the End of the Universe?
—Douglas Adams
Each of the function bodies in the previous example have either had no code in com-
mon with each other, or called out to another function such as xconj for implementa-
tion details that they have in common. These techniques work well when there’s a low
level of commonality between the methods being implemented, but sometimes you
have many methods of a protocol or even whole protocol implementations that you
want to extend to multiple classes. In these cases, some languages would encourage
you to create a base class that implements some or all of the methods and then inherit
from that. Clojure has a different approach.
SHARING METHOD IMPLEMENTATIONS
Clojure doesn’t encourage implementation inheritance, so although it’s possible to
inherit from concrete classes as needed for Java interoperability,6 there’s no way to use
extend to provide a concrete implementation and then build another class on top of
that. There are important reasons why Clojure intentionally avoids this, but regardless
of the reasons, we’re left with the question of how best to avoid repeating code when
similar objects implement the same protocol method.
The simplest solution is to write a regular function that builds on the protocol’s
methods. For example, Clojure’s own into takes a collection and uses the conj imple-
mentation provided by the collection. We can write a similar function for FIXO objects
like this:
6 Mechanisms that support something like Java-style implementation inheritance include gen-class, proxy,
and extending protocol methods to Java abstract classes and interfaces.
Download from Wow! eBook <www.wowebook.com>
Types, protocols, and records
197
(defn fixo-into [c1 c2]
(reduce fixo-push c1 c2))
(xseq (fixo-into (TreeNode. 5 nil nil) [2 4 6 7]))
;=> (2 4 5 6 7)
(seq (fixo-into [5] [2 4 6 7]))
;=> (5 2 4 6 7)
But this is only an option when your function can be defined entirely in terms of the
protocol’s methods. If this isn’t the case, you may need the more nuanced solution
provided by the extend function. We mentioned it earlier but so far have only given
examples of a macro built on top of it, extend-type. Though this and extend-
protocol are frequently the most convenient way to implement protocol methods,
they don’t provide a natural way to mix in method implementations. The extend func-
tion takes a map for each protocol you want to implement, and you can build that
map however you’d like, including by merging in implementations that are already
defined. In the following listing, you should note how a FIXO implementation could
be defined early using a map and extended to a protocol/record type later (while still
maintaining every benefit of using the original map).
Listing 9.4 Using a map to extend FIXO to TreeNode
(def tree-node-fixo
Define map of names
{:fixo-push (fn [node value]
to functions
(xconj node value))
:fixo-peek (fn [node]
(if (:l node)
(recur (:l node))
(:val node)))
:fixo-pop (fn [node]
(if (:l node)
(TreeNode. (:val node) (fixo-pop (:l node)) (:r node))
(:r node)))})
Extend protocol
(extend TreeNode FIXO tree-node-fixo)
using map
(xseq (fixo-into (TreeNode. 5 nil nil) [2 4 6 7]))
;=> (2 4 5 6 7)
These record objects and the way protocols can be extended to them result in rather
differently shaped code than the objects built out of closures that we showed in sec-
tion 7.2. Often this ability to define the data and implementation separately is desir-
able, but you’re likely to find yourself occasionally in a circumstance where closures
may feel like a better fit than records, and yet you want to extend a protocol or inter-
face, not just provide ad hoc method names as in section 7.2.
REIFY
The reify macro brings together all the power of function closures and all the perfor-
mance and protocol participation of extend into a single form. For example, say you
want a stack-like FIXO that’s constrained to a certain fixed size. Any attempt to push
items onto one of these fixed-fixos when it’s already full will fail, and an unchanged
Download from Wow! eBook <www.wowebook.com>
198
CHAPTER 9 Combining data and code
object will be returned. The wrinkle in the requirements that makes reify a reason-
able option is that you’ll want this size limit to be configurable. Thus you’ll need a
constructor or factory function, shown next, that takes the size limit and returns an
object that will obey that limit.
Listing 9.5 Size-limited stack FIXO using reify
(defn fixed-fixo
([limit] (fixed-fixo limit []))
([limit vector]
(reify FIXO
(fixo-push [this value]
(if (< (count vector) limit)
(fixed-fixo limit (conj vector value))
this))
(fixo-peek [_]
(peek vector))
(fixo-pop [_]
(pop vector)))))
Just like the extend forms, reify has method arglists that include the object itself. It’s
idiomatic to use name the argument this in methods where you need to use it and _
in methods where you ignore its value. But both these conventions should only be fol-
lowed where natural.
NAMESPACED METHODS
A rough analogy can be drawn between protocols and Java interfaces.7 We’ve noted
some of the differences already, but it can be a useful analogy nonetheless. In such a
comparison, where record types are concrete classes, you might see that Java packages
and C++ namespaces are each like Clojure namespaces. It’s normal in all three of
these environments for the interface and the class to each be in a namespace, and not
necessarily the same one. For example, probably few readers were surprised to see
that when we made the class IPersistentVector extend the protocol user/FIXO, they
were each from a different namespace or package.
One way this analogy breaks down is that methods of the protocol itself are
namespaced in a way that Java and C++ interfaces aren’t. In those languages, all meth-
ods of a class share the same effective namespace, regardless of interfaces they’re
implementing. In Clojure, the methods always use the same namespace as the proto-
col itself, which means a record or type can extend (via extend, extend-type, and so
on) identically named methods of two different protocols without any ambiguity. This
is a subtle feature, but it allows you to avoid a whole category of issues that can come
up when trying to combine third-party libraries into a single codebase.
Note that because the methods share the namespace of their protocol, you can’t
have identically named methods in two different protocols if those protocols are in
the same namespace. Because both are under the control of the same person, it’s easy
7 Those of you familiar with Haskell might recognize analogies to its typeclasses in our discussion.
Download from Wow! eBook <www.wowebook.com>
Types, protocols, and records
199
to resolve this by moving one of the protocols to a different namespace or using more
specific method names.
METHOD IMPLEMENTATIONS IN DEFRECORD
We’ve already shown how both protocols and interfaces can be extended to record
types using the various extend forms, but there’s another way to achieve similar
results. Protocol and interface method implementations can be written directly inside
a defrecord form, which ends up looking like the following.
Listing 9.6 Method implementations in defrecord
(defrecord TreeNode [val l r]
FIXO
Implement FIXO
(fixo-push [t v]
methods inline
(if (< v val)
(TreeNode. val (fixo-push l v) r)
(TreeNode. val l (fixo-push r v))))
(fixo-peek [t]
(if l
(fixo-peek l)
Call method instead
val))
of using recur
(fixo-pop [t]
(if l
(TreeNode. val (fixo-pop l) r)
r)))
(def sample-tree2 (reduce fixo-push (TreeNode. 3 nil nil) [5 2 4 6]))
(xseq sample-tree2)
;=> (2 3 4 5 6)
This isn’t only more convenient in many cases, but it can also produce dramatically
faster code. Calling a protocol method like fixo-peek on a record type that imple-
ments it inline can be several times faster than calling the same method on an object
that implements it via an extend form. Also note that the fields of the object are now
available as locals—we use val instead of (:val t).
Polymorphism and recur
Throughout this section, we’ve implemented the fixo-peek function using different
methodologies, but a more subtle difference is worth noting. The first implementation
uses recur for its recursive call as shown:
(fixo-peek [node]
(if (:l node)
(recur (:l node))
(:val node)))
Because of the nature of recur, the first implementation of fixo-peek isn’t polymor-
phic on the recursive call. But the second version of fixo-peek uses a different
approach:
Download from Wow! eBook <www.wowebook.com>
200
CHAPTER 9 Combining data and code
(continued)
(fixo-peek [t]
(if l
(fixo-peek l)
val))
You’ll notice that the recursive call in the second implementation is direct (mundane)
and as a result is polymorphic. In the course of writing your own programs, this dif-
ference will probably not cause issues, but it’s worth storing in the back of your mind.
Putting method definitions inside the defrecord form also allows you to implement
Java interfaces and extend java.lang.Object, which isn’t possible using any extend
form. Because interface methods can accept and return primitive values as well as
boxed objects, implementations of these in defrecord can also support primitives.
This is important for interoperability and can provide ultimate performance parity
with Java code.
We do need to note one detail of these inline method definitions in relation to
recur. Specifically, uses of recur in these definitions can’t provide a new target object:
the initial argument will get the same value as the initial (non-recur) call to the
method. For example, fixo-push takes args t and v, so if it used recur, only a single
parameter would be given: the new value for the v arg.
9.3.3
Building from a more primitive base with deftype
You may have noticed we’ve been using our own function xseq throughout the exam-
ples in this section, instead of Clojure’s seq. This shouldn’t be necessary, as Clojure
provides an ISeqable interface that its seq function can use—all we need to do is to
have our own type implement ISeqable. But an attempt to do this with defrecord is
doomed:
(defrecord InfiniteConstant [i]
clojure.lang.ISeq
(seq [this]
(lazy-seq (cons i (seq this)))))
; java.lang.ClassFormatError: Duplicate method
; name&signature in class file user/InfiniteConstant
This is because record types are maps and implement everything maps should—seq
along with assoc, dissoc, get, and so forth. Because these are provided for us, we
can’t implement them again ourselves, and thus the preceding exception. For the rare
case where you’re building your own data structure instead of just creating applica-
tion-level record types, Clojure provides a lower-level deftype construct that’s similar
to defrecord but doesn’t implement anything at all, so implementing seq won’t con-
flict with anything:
(deftype InfiniteConstant [i]
clojure.lang.ISeq
(seq [this]
Download from Wow! eBook <www.wowebook.com>
Types, protocols, and records
201
(lazy-seq (cons i (seq this)))))
(take 3 (InfiniteConstant. 5))
;=> (5 5 5)
But that also means that keyword lookups, assoc, dissoc, and so on will remain unim-
plemented unless we implement them ourselves:
(:i (InfiniteConstant. 5))
;=> nil
The fields we declared are still public and accessible (although you should try to avoid
naming them the same at the methods in java.lang.Object); they just require nor-
mal Java interop forms to get at them:
(.i (InfiniteConstant. 5))
;=> 5
With all that in mind, the following listing is a final implementation of TreeNode using
deftype, which lets us implement not only ISeq so that we can use seq instead of
xseq, but also IPersistentStack so we can use peek, pop, and conj as well as the
fixo- versions.
Listing 9.7 Implementing map interfaces with deftype
(deftype TreeNode [val l r]
FIXO
Implement FIXO
(fixo-push [_ v]
methods inline
(if (< v val)
(TreeNode. val (fixo-push l v) r)
(TreeNode. val l (fixo-push r v))))
(fixo-peek [_]
(if l
(fixo-peek l)
Call method instead
val))
of using recur
(fixo-pop [_]
(if l
(TreeNode. val (fixo-pop l) r)
r))
clojure.lang.IPersistentStack
Implement interfaces
(cons [this v] (fixo-push this v))
(peek [this] (fixo-peek this))
(pop [this] (fixo-pop this))
clojure.lang.Seqable
(seq [t]
(concat (seq l) [val] (seq r))))
(extend-type nil
FIXO
(fixo-push [t v]
Redefine to use
(TreeNode. v nil nil)))
new TreeNode
(def sample-tree2 (into (TreeNode. 3 nil nil) [5 2 4 6]))
(seq sample-tree2)
;=> (2 3 4 5 6)
Download from Wow! eBook <www.wowebook.com>
202
CHAPTER 9 Combining data and code
One final note about deftype—it’s the one mechanism by which Clojure lets you cre-
ate classes with volatile and mutable fields. We won’t go into it here because using
such classes is almost never the right solution. Only when you’ve learned how Clojure
approaches identity and state, how to use reference types, what it means for a field to
be volatile, and all the pitfalls related to that, should you even consider creating
classes with mutable fields. By then, you’ll have no problem understanding the official
docs for deftype, and you won’t need any help from us.
None of the examples we’ve shown in this section come close to the flexibility of
multimethods. All protocol methods dispatch on just the type of the first argument.
This is because that’s what Java is good at doing quickly, and in many cases it’s all the
polymorphism that’s needed. Clojure once again takes the practical route and makes
the highest-performance mechanisms available via protocols, while providing more
dynamic behavior than Java does and leaving multimethods on the table for when ulti-
mate flexibility is required.
9.4
Putting it all together: a fluent builder for chess moves
People have been known to say that Java is a verbose programming language. This
may be true when compared to the Lisp family of languages, but considerable mind-
share has been devoted to devising ways to mitigate its verbosity. One popular tech-
nique is known as the fluent builder (Fowler 2005) and can be summed up as the
chaining of Java methods to form a more readable and agile instance construction
technique. In this section, we’ll show a simple example of a fluent builder supporting
the construction of chess move descriptions. We’ll then explain how such a technique
is unnecessary within Clojure and instead present an alternative approach that’s sim-
pler, concise, and more extensible. We’ll leverage Clojure’s records in the final solu-
tion, illustrating that Java’s class-based paradigm is counter to Clojure’s basic
principles and often overkill for Java programs.
9.4.1
Java implementation
We’ll start by identifying all of the component parts of a Move class including from and
to squares, a flag indicating whether the move is a castling move, and also the desired
promotion piece if applicable. In order to constrain the discussion, we’ll limit our
idea of a Move to those elements listed. The next step would be to create a simple class
with its properties and a set of constructors, each taking some combination of the
expected properties. We’d then generate a set of accessors for the properties, but not
their corresponding mutators, because it’s probably best for the move instances to be
immutable.
Having created this simple class and rolled it out to the customers of the chess
move API, we begin to notice that our users are sending into the constructor the to
string before the from string, which is sometimes placed after the promotion, and so
on. After some months of intense design and weeks of development and testing, we
release the following elided chess move class:
Download from Wow! eBook <www.wowebook.com>
Putting it all together: a fluent builder for chess moves
203
public class FluentMove {
String from, to, promotion = "";
boolean castlep;
public static MoveBuilder desc() { return new MoveBuilder(); }
public String toString() {
return "Move " + from +
" to " + to +
(castlep ? " castle" : "") +
(promotion.length() != 0 ? " promote to " + promotion : "");
}
public static final class MoveBuilder {
FluentMove move = new FluentMove();
public MoveBuilder from(String from) {
move.from = from; return this;
}
public MoveBuilder to(String to) {
move.to = to; return this;
}
public MoveBuilder castle() {
move.castlep = true; return this;
}
public MoveBuilder promoteTo(String promotion) {
move.promotion = promotion; return this;
}
public FluentMove build() { return move; }
}
}
For brevity’s sake, our code has a lot of holes, such as missing checks for fence post
errors, null, empty strings, assertions, and invariants; it does allow us to illustrate that
the code provides a fluent builder given the following main method:
public static void main(String[] args) {
FluentMove move = FluentMove.desc()
.from("e2")
.to("e4").build();
System.out.println(move);
move = FluentMove.desc()
.from("a1")
.to("c1")
.castle().build();
System.out.println(move);
move = FluentMove.desc()
.from("a7")
.to("a8")
.promoteTo("Q").build();
System.out.println(move);
}
Download from Wow! eBook <www.wowebook.com>
204
CHAPTER 9 Combining data and code
// Move e2 to e4
// Move a1 to c1 castle
// Move a7 to a8 promote to Q
The original constructor ambiguities have disappeared, with the only trade-off being a
slight increase in complexity of the implementation and the breaking of the common
Java getter/setter idioms—both of which we’re willing to live with. But if we’d started
the chess move API as a Clojure project, the code would likely be a very different expe-
rience for the end user.
9.4.2
Clojure implementation
In lieu of Java’s class-based approach, Clojure provides a core set of collection types,
and as you might guess, its map type is a nice candidate for move representation:
{:from "e7", :to "e8", :castle? false, :promotion \Q}
Simple, no?
In a language like Java, it’s common to represent everything as a class—to do oth-
erwise is either inefficient, non-idiomatic, or outright taboo. Clojure prefers simplifi-
cation, providing a set of composite types perfect for representing most categories of
problems typically handled by class hierarchies. Using Clojure’s composite types
makes sense for one simple reason: existing functions, built on a sequence abstrac-
tion, just work:
(defn build-move [& pieces]
(apply hash-map pieces))
(build-move :from "e7" :to "e8" :promotion \Q)
;=> {:from "e7", :to "e8", :promotion \Q}
In two lines, we’ve effectively replaced the Java implementation with an analogous, yet
more flexible representation. The term domain-specific language (DSL) is often thrown
around to describe code such as build-move, but to Clojure (and Lisps in general) the
line between DSL and API is blurred. In the original FluentMove class, we required a
cornucopia of code in order to ensure the API was agnostic of the ordering of move ele-
ments; using a map, we get that for free. Additionally, FluentMove, though relatively
concise, was still bound by fundamental Java syntactical and semantic constraints.
There’s one major problem with our implementation—it doesn’t totally replace
the Java solution. If you recall, the Java solution utilized the toString method to print
its representative form. The existence of a polymorphic print facility in Java is nice,
and it allows a class creator to define a default print representation for an object when
sent to any Java print stream. This means that the same representation is used on the
console, in log files, and so on. Using raw maps can’t give us this same behavior, so
how can we solve this problem?
Download from Wow! eBook <www.wowebook.com>
Putting it all together: a fluent builder for chess moves
205
USING RECORDS
If we instead use a record, then the solution is as simple as that shown next.
Listing 9.8 A chess move record
(defrecord Move [from to castle? promotion]
Object
(toString [this]
(str "Move " (:from this)
" to " (:to this)
(if (:castle? this) " castle"
(if-let [p (:promotion this)]
(str " promote to " p)
"")))))
As we mentioned in the previous section, within the body of a record we can take up
to two actions: participate in a protocol, or override any of the methods in the
java.lang.Object class. For the Move record, we override toString in order to allow
it to participate in Java’s overarching polymorphic print facility, as shown:
(str (Move. "e2" "e4" nil nil))
;=> "Move e2 to e4"
(.println System/out (Move. "e7" "e8" nil \Q))
; Move e7 to e8 promote to Q
We’ve once again gone back to positional construction using records, but as we’ll
show, Clojure even has an answer for this.
SEPARATION OF CONCERNS
Both FluentMove and build-move make enormous assumptions about the form of the
data supplied to them and do no validation of the input. For FluentMove, object-
oriented principles dictate that the validation of a well-formed move (not a legal
move, mind you) should be determined by the class itself. There are a number of
problems with this approach, the most obvious being that to determine whether a
move is well-formed, the class needs information about the rules of chess. We can
rewrite FluentMove to throw an exception to prevent illegal moves from being con-
structed, but the root problem still remains—FluentMove instances are too smart. Per-
haps you don’t see this as a problem, but if we were to extend our API to include other
aspects of the game of chess, then we’ll find that bits of overlapping chess knowledge
would be scattered throughout the class hierarchy. By viewing the move structure as a
value, Clojure code provides some freedom in the implementation of a total solution,
as shown:
(defn build-move [& {:keys [from to castle? promotion]}]
{:pre [from to]}
(Move. from to castle? promotion))
(str (build-move :from "e2" :to "e4"))
;=> "Move e2 to e4"
Download from Wow! eBook <www.wowebook.com>
206
CHAPTER 9 Combining data and code
By wrapping the Move constructor in a build-move function, we put the smarts of con-
structing moves there instead of in the type itself. In addition, using a precondition,
we specified the required fields, and by using Clojure’s named parameters and argu-
ment destructuring we’ve again ensured argument order independence. As a final
added advantage, Clojure’s records are maps and as a result can operate in almost
every circumstance where a map would. As author Rich Hickey proclaimed, any new
class in general is itself an island, unusable by any existing code written by anyone,
anywhere. So our point is this: consider throwing the baby out with the bath water.
9.5
Summary
Clojure disavows the typical object-oriented model of development. But that’s not to
say that it completely dismisses all that OOP stands for. Instead, Clojure wholeheartedly
touts the virtues of interface-oriented programming (or abstraction-oriented program-
ming, as we’ve called it), in addition to runtime polymorphism. But in both cases, the
way that Clojure presents these familiar topics is quite different from what you might
be accustomed to. In almost every circumstance, Clojure’s abstraction-oriented facili-
ties will sufficiently represent your problem domain, but there may be times when they
simply can’t. We’ll preach the virtues of abstractions more throughout the rest of the
book, but for now we’re compelled to take a side path into an explorations of Java
interoperability.
Download from Wow! eBook <www.wowebook.com>
Java.next
This chapter covers
Generating objects on the fly with proxy
Clojure gen-class and GUI programming
Clojure’s relationship to Java arrays
All Clojure functions implement...
Using Clojure data structures in Java APIs
definterface
Be wary of exceptions
Regardless of your views on the Java language itself, it’s difficult to deny that the JVM
is a stellar piece of software. The confluence of the just-in-time (JIT) compiler, gar-
bage collection, HotSpot, and the flexible bytecode have created an environment
that many programmers have chosen to grow their alternative programming lan-
guages. Additionally, the deluge of library options hosted on the JVM further make
the JVM the language target of choice. From Clojure to Groovy to Scala to Fantom to
Frink to Ioke to Jess to JRuby to Jython, there seems to be no lack of options for the
enthusiastic polyglot programmer. We may soon see job listings for “JVM program-
mers.” But where does that leave Java the programming language?
Java the language isn’t dead.
207
Download from Wow! eBook <www.wowebook.com>
208
CHAPTER 10 Java.next
The JVM is optimized for running Java bytecode, and only recently1 have Java.next
languages been a consideration. You may ask yourself whether JVM bytecode is equiva-
lent to Java source code, and the answer is no. Instead, languages such as Clojure and
Scala compile directly to bytecode and can access Java compiled libraries as needed.
Because of their reliance on the JVM as the runtime environment, Clojure and the
other Java.next languages will be fundamentally constrained by the limitations of the
JVM itself. The limitations of the JVM as defined by the limitations of the Java language
specification set the beat by which the Java.next languages dance. Java isn’t dead; it’s
alive and well, and it runs the show.
THE JAVA.NEXT MANTRA The apprentice avoids all use of Java classes. The jour-
neyman embraces Java classes. The master knows which classes to embrace
and which to avoid.
An expert understanding of the Java Virtual Machine isn’t required for writing power-
ful applications in Clojure, but it’ll help when issues stemming from host limitations
arise. Thankfully, Clojure does a good job of mitigating many of the limitations inher-
ent in its host, but some are too deeply embedded in the fibers of the JVM to avoid. Clo-
jure provides a specific set of interoperability tools: gen-class, proxy, definterface,
its exceptions facility, and a host of array functions. We’ll touch on each of these in
turn, but we’ll begin with the creation of anonymous objects using proxy.
10.1
Generating objects on the fly with proxy
There’s a saying within the Clojure community stating (Halloway 2009) that Clojure
does Java better than Java. This is a bold statement, but not one without merit, as we’ll
show throughout this chapter. Java programmers are accustomed to drawing a severe
distinction between development time and runtime. Using Clojure’s proxy feature
allows you to blur this distinction.
CLOJURE APHORISM Many software projects require a lot of planning because
their implementation languages don’t foster change. Clojure makes it a lot
easier to plan for change.
Clojure’s proxy mechanism is meant strictly for interoperability purposes. In section
9.3, we discussed how reify was intended to realize a single instance of a type, proto-
col, or interface—in other words, abstractions. But when dealing with Java libraries,
you’re at times required to extend concrete classes, and it’s in this circumstance where
proxy shines. Be aware that by using proxy, you bring a lot of Java’s semantics into
your Clojure programs. Though extending concrete classes is seen often in Java,
doing so in Clojure is considered poor design, leading to fragility, and should there-
fore be restricted to those instances where interoperability demands it.
1 More details can be found in JSR-000292, “Supporting Dynamically Typed Languages on the Java Platform.”
Download from Wow! eBook <www.wowebook.com>
Generating objects on the fly with proxy
209
10.1.1 A simple dynamic web service
Using Clojure breaks the ponderous code/compile/run development cycle by adding
an element of dynamism into the fold. Take for example a scenario where we want to
develop a web service using an existing Java 1.5 API.
Listing 10.1 A simple dynamic web service
(ns joy.web
(:import (com.sun.net.httpserver HttpHandler HttpExchange HttpServer)
(java.net InetSocketAddress HttpURLConnection)
(java.io IOException FilterOutputStream)
(java.util Arrays)))
(defn new-server [port path handler]
Create service
(doto (HttpServer/create (InetSocketAddress. port) 0)
(.createContext path handler)
(.setExecutor nil)
(.start)))
(defn default-handler [txt]
Print hello message
(proxy [HttpHandler] []
(handle [exchange]
(.sendResponseHeaders exchange HttpURLConnection/HTTP_OK 0)
(doto (.getResponseBody exchange)
(.write (.getBytes txt))
Close over txt
(.close)))))
(def server (new-server 8123
"/joy/hello"
(default-handler "Hello Cleveland")))
After entering the code in listing 10.1, you should see the message “Hello Cleveland”
in your web browser at address http://localhost:8123/joy/hello. This is only margin-ally interesting, especially because the source is organized in a way that doesn’t take
advantage of Clojure’s flexibility.
If we instead organize the code to bind the return of default-handler, we can
manipulate the handler independently and update its behavior at runtime, as shown:
(.stop server 0)
(def p (default-handler
"There's no problem that can't be solved
with another level of indirection"))
(def server (new-server 8123 "/joy/hello" p))
At this point, visiting the aforementioned URL will show the new message, making this
simple server more compelling. But we can take it one step further by making changes
without taking the server instance down in such a clumsy fashion. Ideally, we’d like to
be able to call a function to change the message at any time:
(change-message p "Our new message")
The implementation of change-message is given in the following listing.
Download from Wow! eBook <www.wowebook.com>
210
CHAPTER 10 Java.next
Listing 10.2 Convenience functions for changing the web service message
(defn make-handler-fn [fltr txt]
(fn [this exchange]
Name explicit this
(let [b (.getBytes txt)]
(-> exchange
.getResponseHeaders
(.set "Content-Type" "text/html"))
(.sendResponseHeaders exchange
HttpURLConnection/HTTP_OK
0)
(doto (fltr (.getResponseBody exchange))
Pass through filter
(.write b)
(.close)))))
(defn change-message
"Convenience method to change a proxy's output message"
([p txt] (change-message p identity txt))
Use identity filter
([p fltr txt]
(update-proxy p
{"handle" (make-handler-fn fltr txt)})))
We’ve added a few extras to the implementation that will be useful later, but for now
concentrate on the fact that change-message calls the function update-proxy with
the proxy object p and a map containing an anonymous function keyed on a string
referencing a method name to override. The anonymous function looks similar to the
handle method defined in the returned proxy from the original default-handler
function, with some extras added for flexibility’s sake. You can test this by entering the
following function call:
(change-message p "Hello Dynamic!")
Refreshing your browser will reflect the change made by displaying the string "Hello
Dynamic!". If so inclined, you can also inspect the current proxy mappings using the
function proxy-mappings. The question remains—how does update-proxy change
the behavior of a previously generated proxy class?
IT’S CALLED PROXY FOR A REASON
As we mentioned, the proxy function generates the bytecode for an actual class on
demand, but it does so in such a way to provide a more dynamic implementation.
Instead of inserting the bytecode for the given function bodies directly into the proxy
class, Clojure instead generates a proper proxy in which each method looks up the
function implementing a method in a map. This trades highly useful dynamic behav-
ior for some runtime cost, but in many cases this is a fair trade.
Based on the method name, the corresponding function is retrieved from a map
and invoked with the this reference and the argument(s).
PROXIES FOR TRUE POWER DYNAMISM
Working from the abstract model in figure 10.1, observe how Clojure updates the
mapped functions within a proxy at runtime. This web service is a humble example,
but there’s a point to take away from this exercise: to perform this same task in Java
Download from Wow! eBook <www.wowebook.com>
Generating objects on the fly with proxy
211
isn’t impossible but would require an enormous
p
amount of scaffolding to implement properly,
this = HttpHandler.
whereas in Clojure it’s built into the language.
handle(exchange)
PROXIES AS PROPER CITIZENS
In the original change-message function, we
provided a hook named fltr that took the result
(fn [this exchange])
of the call to the .getResponseBody method.
Because the result of this method call is a
java.io. OutputStream, we can use that infor-
mation to our advantage when creating a filtering
function. The use of the identity function as the
Figure 10.1 Proxy lookup: the
default filter ensures that the usage doesn’t break
instance returned by proxy is a proper
proxy that does method dispatch to
in the default case; but if we’re to utilize our own
functions in a lookup table. These
filtering function, we must ensure that we prop-
functions can therefore be swapped out
erly wrap the original, which again is a perfect use
with replacements as needed.
case for proxy. A simple implementation of a
screaming-filter would be implemented as such:
(defn screaming-filter [o]
(proxy [FilterOutputStream] [o]
(write [b]
(proxy-super write (.getBytes (str "<strong>"
(.toUpperCase (String. b))
"</strong>"))))))
The proxy returned by screaming-filter extends the Java class java.io.Filter-
OutputStream to the superclass constructor (via the [o] vector). It passes the argument
o, which corresponds to the OutputStream obtained from the .getResponseBody
method.
ANAPHORIC PROXY In section 8.5, we mentioned that it’s non-idiomatic to
write anaphoric macros, yet you might’ve noticed that proxy is a contradic-
tion of that statement. The use of the anaphora this is subject to the same
nesting limitations as previously mentioned and is a good candidate for
change in later versions of Clojure. You might notice that the reify macro,
though similar to proxy, doesn’t use an anaphoric this but instead requires
that it be named explicitly—the preferred approach for your own, and likely
the way forward for all future Clojure core macros.
The call to the proxy-super function is similar to Java’s super.method() semantics. If
we now execute the call to change-message passing in screaming-filter, we’ll see
the expected filtered message in all caps and bold on a browser refresh:
(change-message p screaming-filter "whisper")
Note that in a break from almost every other construct in Clojure, proxy-super is not
thread-safe. If some other thread were to call this proxy instance’s write method
while proxy-super was still running, the base class’s method would be called directly,
Download from Wow! eBook <www.wowebook.com>
212
CHAPTER 10 Java.next
incorrectly skipping the proxy implementation. So be careful using proxy-super and
multiple threads in close proximity to each other.
FINAL POINTS ABOUT PROXY
Clojure’s proxy capabilities are truly dynamic, allowing you to create fully stubbed
proxies using either construct-proxy, get-proxy-class, or init-proxy. In both
cases, a partially to fully realized proxy will be constructed, allowing programmatic
customization using update-proxy and arbitrary mixin maps.
There’s a universe of difference between the code outlined in this subsection and
systems employing true code hot-loading, but it’s a reasonable facsimile. Using proxy
is powerful, but doing so creates unnamed instances unavailable for later extension. If
you instead wish to create named classes then you’ll need to use Clojure’s gen-class
mechanism, which we’ll discuss next.
10.2
Clojure gen-class and GUI programming
In section 9.1, we mentioned that Clojure namespaces can be used as the basis for
generating a named class. In this section, we’ll address this topic and others related to
Clojure’s gen-class function and :gen-class namespace directive in the context of
writing a simple graphical user interface (GUI) library.
10.2.1 Namespaces as class specifications
Similarly to the ns example in section 9.1, the explanation of gen-class begs a declar-
ative approach for a namespace defining a class named joy.gui.DynaFrame. We’d like
this class to extend javax.swing.JFrame and declare the Vars providing its overriding
method implementations to be prefixed2 by the symbol df-. In addition, we’d like the
class to implement the clojure.lang.IMeta interface. We’d also like a place to store
information about instances of this class in state and would like the initialization func-
tion called on construction to be named df-init. We’d like to define a single construc-
tor, taking a string and passing it onto the superclass constructor also taking a string.
We then want to declare two public methods: the first named display taking a
java.awt.Container and returning void, and the second static method version tak-
ing no arguments and returning a string. Finally, we’ll declare the required imports
needed.
The worded DynaFrame class declaration is complex but has the advantage of hav-
ing a direct code translation, as shown next.
Listing 10.3 The DynaFrame class namespace declaration
(ns joy.gui.DynaFrame
(:gen-class
:name joy.gui.DynaFrame
:extends javax.swing.JFrame
Superclass
:implements [clojure.lang.IMeta]
Interface
:prefix df-
2 If you don’t specify a :prefix, then the default - will be used.
Download from Wow! eBook <www.wowebook.com>
Clojure gen-class and GUI programming
213
:state state
Instance state
:init init
Init function
Constructor
:constructors {[String] [String]}
mapping
:methods [[display [java.awt.Container] void]
Public
^{:static true} [version [] String]])
method
(:import (javax.swing JFrame JPanel)
Static method
(java.awt BorderLayout Container)))
You can compile this namespace by saving it in a directory joy/gui, located on the
classpath, in a file named DynaFrame.clj and executing the function (compile
'joy.gui.DynaFrame) in a fresh REPL. This allows a compiled class to be immediately
available. But trying to create an instance in the same REPL will prove fruitless:
(joy.gui.DynaFrame. "1st")
; java.lang.UnsupportedOperationException:
; joy.gui.DynaFrame/df-init not defined
Clearly we haven’t defined the df-init function, so we’ll do that now by switching to
the joy.gui.DynaFrame namespace, defining it outright:
(in-ns 'joy.gui.DynaFrame)
(defn df-init [title]
[[title] (atom {::title title})])
Now run the following in your REPL:
(joy.gui.DynaFrame. "2nd")
; java.lang.UnsupportedOperationException:
; meta (joy.gui.DynaFrame/df-meta not defined?)
Because we told the Clojure compiler that the class should implement the IMeta inter-
face, we should’ve provided a concrete implementation, which you can do at the REPL:
(defn df-meta [this] @(.state this))
(defn version [] "1.0")
As an added bonus, we implemented the static method version. To see the effects of
these functions, execute the following:
(meta (joy.gui.DynaFrame. "3rd"))
;=> {:joy.gui.DynaFrame/title "3rd"}
(joy.gui.DynaFrame/version)
;=> "1.0"
We’ve filled in most of the implementation of the DynaFrame class except for the
display function, which you can implement as follows:
(defn df-display [this pane]
(doto this
(-> .getContentPane .removeAll)
(.setContentPane (doto (JPanel.)
(.add pane BorderLayout/CENTER)))
(.pack)
(.setVisible true)))
Download from Wow! eBook <www.wowebook.com>
214
CHAPTER 10 Java.next
Figure 10.2 A simple use of DynaFrame: now
Figure 10.3 A simple dynamic update of
that you’ve compiled the DynaFrame class,
DynaFrame: we can update the
you can start using it to display simple GUIs.
DynaFrame on the fly without restarting.
You can see df-display in action within the REPL by running the following:
(def gui (joy.gui.DynaFrame. "4th"))
(.display gui (doto (javax.swing.JPanel.)
(.add (javax.swing.JLabel. "Charlemagne and Pippin"))))
This will now display the GUI frame seen in figure 10.2.
And because it’s a DynaFrame we should be able to change it on the fly, right? Right:
(.display gui (doto (javax.swing.JPanel.)
(.add (javax.swing.JLabel. "Mater semper certa est." ))))
This will change the view to that in figure 10.3.
But now that you have this interesting little frame, what can you do with it? Next,
we’ll experiment with DynaFrame as the foundation for agile GUI prototyping.
THE GUTS OF NAMESPACE COMPILATION
So what exactly does the :gen-class directive provide in terms of generated class
files? With or without :gen-class, Clojure will generate a set of classes correspond-
ing to each function in a namespace. For the function joy.gui.DynaFrame/df-dis-
play, a class file will be generated on the classpath of joy.gui.DynaFrame
$df_display containing (at least) a method invoke, at the location CLASSPATH/
joy/gui/DynaFrame$df_display.class, as shown:
package joy.gui;
public class DynaFrame$df_display extends AFunction {
. . .
public Object invoke(Object that, Object container) {
. . . display actions . . .
}