7.2 Day 1: Training Luke

In Star Wars, the apprentice Luke joined with Yoda for advanced training in the ways of the Jedi. He had started his training under another. Like Luke, you have already started your training for functional languages. You used closures in Ruby and graduated to higher-order functions in Scala and Erlang. In this chapter, you’re going to learn to apply some of those concepts in Clojure.

Go to the Clojure home site at http://www.assembla.com/wiki/show/clojure/Getting_Started. Follow the instructions to install Clojure on your platform and with your preferred development environment. I’m using a prerelease version of Clojure 1.2, and it should be fully ready by the time this book is in your hands. You may first need to install the Java platform, though today, most operating systems come with Java installed. I’m using the leiningen tool[18] to manage my Clojure projects and Java configuration. That tool lets me build a project and insulates me from the Java details such as classpaths. Once that’s installed, I can then create a new project:

Then, I can start the Clojure console, called the repl:

...and I’m off to the races. Underneath, leiningen is installing some dependencies and calling Java with a few Clojure Java archives (jars) and options. Your installation may require you to start the repl some other way. From here on out, I’ll just tell you to start the repl.

After all of that work, you have a primitive console. When I ask you to evaluate code, you can use this repl, or you can use one of any number of IDEs or editors that have Clojure support.

Let’s type some code:

So, the console is working. In Clojure, you’ll enclose any function call entirely in parentheses. The first element is the function name, and the rest are the arguments. You can nest them, too. Let’s demonstrate the concept with a little math.

Calling Basic Functions

That’s just basic math. Division is a little more interesting:

Clojure has a basic data type called a ratio. It’s a nice feature that will allow delaying computation to avoid the loss of precision. You can still easily work in floats if you so choose. You can easily calculate remainders:

That’s short for modulus operator. This notation is called prefix notation. Languages to this point have supported infix notation, with the operator between the operands, like 4 + 1 - 2. Many people prefer infix notation because we’re used to it. We’re used to seeing math expressed in this way. After warming up, you should be getting into the flow of prefix notation. It’s a little awkward doing math in this way, but it works. Prefix notation with parentheses does have advantages, though. Consider this expression:

There’s no ambiguity. Clojure will evaluate this statement in parenthetical order. And check out this expression:

You can easily add elements to the calculation, if you so choose. You can even use this style when working with subtraction or division:

We’ve evaluated (8 - 1) - 2 and (8 / 2) / 2 in conventional (infix) notation. Or, if you’d like to see the Clojure equivalent using only two operands at a time, it is (- (- 8 1) 2) and (/ (/ 8 2) 2). You can also get some surprisingly powerful results out of simple operator evaluation:

Nice. We can see whether a list is in order with a single operator and an arbitrary number of arguments.

Aside from the prefix notation and the extra wrinkle of multiple parameter lists, Clojure’s syntax is very simple. Let’s try to push the typing system a little, probing for strong typing and coerced types:

Clojure works to coerce types for us. In general, you’ll find that Clojure supports strong, dynamic typing. Let’s get a little more focused and look at some of Clojure’s basic building blocks, called forms.

Think of a form as a piece of syntax. When Clojure parses code, it first breaks the program down into pieces called forms. Then, Clojure can compile or interpret the code. I’m not going to distinguish between code and data, because in Lisp, they are one and the same. Booleans, characters, strings, sets, maps, and vectors are all examples of forms that you’ll see throughout this chapter.

Strings and Chars

You’ve already been introduced to strings, but we can take it a little deeper. You’ll enclose strings in double quotes, and use C-style escaping characters, as in Ruby:

No surprises. As an aside, so far, we’ve used a single argument with println, but the function also works well with zero or more arguments to print a blank line or several values concatenated together.

In Clojure, you can convert things to a string with the str function:

If the target underneath is a Java class, str will call the underlying toString function. This function takes more than one argument, like this:

As a result, Clojure developers use str to concatenate strings together. Conveniently, you can concatenate items that are not strings:

You can even concatenate different types together. To represent a character outside of double quotes, you can precede it with a \ character, like this:

And as usual, you can concatenate them together with str:

Let’s make some comparisons:

So, characters are not strings of length 1.

But you can easily convert characters to strings. That’s enough string manipulation for now. Let’s move on to some boolean expressions.

Booleans and Expressions

Clojure has strong, dynamic typing. Recall that dynamic typing means types are evaluated at run time. You’ve already seen a few of the types in action, but let’s focus that discussion a little bit. A boolean is the result of an expression:

As with most other languages in this book, true is a symbol. But it is also something else. Clojure’s types are unified with the underlying Java type system. You can get the underlying class with the class function. Here’s the class of a boolean:

So, you can see the JVM peeking through. This type strategy will make things far more convenient for you as you move along. You can use the result of booleans in many expressions. Here is a simple if:

Like Io, we passed in code as the second argument to the if. Conveniently, Lisp lets us treat the data as code. We can make it prettier by breaking the code across multiple lines:

We can add an else as the third argument:

Now, let’s see what else we can use as booleans. First, what passes for a nil in Clojure?

Ah. That’s simple. The symbol called nil.

0 and "" are true, but nil is not. We’ll introduce other boolean expressions as we need them. Now, let’s look at some more complex data structures.

Lists, Maps, Sets, and Vectors

As with all functional languages, core data structures such as lists and tuples do the heavy lifting. In Clojure, three big ones are lists, maps, and vectors. We’ll start things off with the collection you’ve spent most of your time with so far, the list.

Lists

A list is an ordered collection of elements. These elements can be anything, but in idiomatic Clojure, lists are used for code and vectors for data. I’ll walk you through lists with data, though, to prevent confusion. Since lists evaluate as functions, you can’t do this:

If you really want a list made up of 1, 2, and 3, you need to do one of these things, instead:

Then, you can manipulate the lists as usual. The second form is called quoting. The four main operations for a list are first (the head), rest (the list minus the head), last (the last element), and cons (construct a new list given a head and a tail):

Of course, you can combine these with higher-order functions, but we’ll wait to do so until we encounter sequences. Now, let’s move to a close cousin of the list, the vector.

Vectors

Like a list, a vector is an ordered collection of elements. Vectors are optimized for random access. You’ll surround vectors with square brackets, like this:

Use lists for code and vectors for data. You can get various elements like this:

Note that vectors are functions, too, taking an index as an argument. You can combine two vectors like this:

You may have noticed that repl printed out a list instead of a vector. Many of the functions that return collections use a Clojure abstraction called sequences. We’ll learn more about them in day 2. For now, just understand that Clojure is returning a sequence and rendering it as a list in the repl.

Clojure lets you get the typical head and tail of a vector, of course:

We’ll use both of these features as we do pattern matching. Both lists and vectors are ordered, so let’s see some unordered collections, sets and maps.

Sets

A set is an unordered collection of elements. The collection has a stable order, but that order is implementation-dependent, so you shouldn’t count on it. You’ll wrap a set in #{}, like this:

We can assign those to a variable called spacecraft and then manipulate them:

We can also create a sorted set that takes items in any order and returns them in sorted order:

You can merge two sets, like this:

Or compute the difference:

I’ll give you one last convenient oddity before I move on. Sets are also functions. The set #{:jar-jar, :chewbacca} is an element but also a function. Sets test membership, like this:

When you use a set as a function, the function will return the first argument if that argument is a member of the set. That covers the basics for sets. Let’s move on to maps.

Maps

As you know, a map is a key-value pair. Using Clojure, you’ll represent a map with curly braces, like this:

These are examples of maps, key-value pairs, but they are tough to read. It would be hard to see an odd number of keys and values, leading to an error:

Clojure solves this problem by allowing commas as whitespace, like this:

A word preceded with : is a keyword, like symbols in Ruby or atoms in Prolog or Erlang. Clojure has two kinds of forms that are used to name things in this way, keywords and symbols. Symbols point to something else, and keywords point to themselves. true and map are symbols. Use keywords to name domain entities such as a property in a map as you would use an atom in Erlang.

Let’s define a map called mentors:

Now, you can retrieve a value by passing a key as the first value:

Maps are also functions. Keywords are also functions:

:luke, the function, looks itself up in a map. It’s odd but useful. As with Ruby, you can use any data type as a key or value. And you can merge two maps with merge:

You can also specify an operator to use when a hash exists in both maps:

In this case, we combined the 4 and the 3 values associated with the x-wing keys with +. Given an association, you can create a new association with a new key-value pair, like this:

You can create a sorted map that takes items in any order and spits them out sorted, like this:

We’re gradually adding more structure to data. Now, we can move on to the form that adds behavior, the function.

Defining Functions

Functions are the centerpiece of all kinds of Lisps. Use defn to define a function.

The simplest form is (defn [params] body). We defined a function called force-it with no parameters. The function simply concatenated two strings together. Call the function like any other:

If you want, you can specify an extra string describing the function:

Then, you can recall documentation for the function with doc:

Let’s add a parameter:

By the way, you can use this doc feature on any other function that specifies a documentation line:

Now that you can define a basic function, let’s move on to the parameter list.

Bindings

As in most of the other languages we’ve looked at so far, the process of assigning parameters based on the inbound arguments is called binding. One of the nice things about Clojure is its ability to access any portion of any argument as a parameter. For example, say you had a line, represented as a vector of points, like this:

You could build a function to access the end of the line, like this:

But we don’t really need the whole line. It would be nicer to bind our parameter to the second element of the line. With Clojure, it’s easy:

The concept is called destructuring. We’re taking a data structure apart and grabbing only the pieces that are important to us. Let’s take a closer look at the bindings. We have [[_ second]]. The outer brackets define the parameter vector. The inner brackets say we’re going to bind to individual elements of a list or vector. _ and second are individual parameters, but it’s idiomatic to use _ for parameters you want to ignore. In English, we’re saying “The parameters for this function are _ for the first element of the first argument, and second for the second element of the first argument.”

We can also nest these bindings. Let’s say we have a tic-tac-toe board, and we want to return the value of the middle square. We’ll represent the board as three rows of three, like this:

Now, we want to pick up the second element of the second row of three, like this:

Beautiful! We’re essentially nesting the same concept. Let’s break it down. The bindings are [[_ [_ c _] _]]. We’ll bind one parameter to the inbound argument, [_ [_ c _] _]. That parameter says we’re going to ignore the first and third elements, which are the top and bottom rows in our tic-tac-toe board. We’ll focus on the middle row, which is [_ c _]. We’re expecting another list and grabbing the center one:

We can simplify that function in a couple of different ways. First, we don’t have to list any of the wildcard arguments that come after the target arguments:

Next, destructuring can happen in the argument list or also in a let statement. In any Lisp, you’ll use let to bind a variable to a value. We could use let to hide the destructuring from the clients of the center function:

let takes two arguments. First is a vector with the symbol you want to bind ([[_ [_c]]]) followed by the value (board) that you want bound. Next is some expression that presumably uses that value (we just returned c). Both forms produce equivalent results. It all depends on where you want the destructuring to happen. I’ll show you a couple of brief examples using let, but understand that you could also use them within an argument list.

You can destructure a map:

You can also combine maps and vectors:

We bound to a vector, skipping the first and picking out the name of the second map. You can see the influence of Lisp on Prolog and, by extension, Erlang. Destructuring is simply a form of pattern matching.

Anonymous Functions

In Lisp, functions are just data. Higher-order functions are built into the language from the ground up because code is just like any other kind of data. Anonymous functions let you create unnamed functions. It’s a fundamental capability for every language in this book. In Clojure, you’ll define a higher-order function with the fn function. Typically, you’ll omit the name, so the form looks like (fn [parameters*] body). Let’s take an example.

Let’s use an argument to a higher-order function to build a list of word counts, given a list of words. Let’s say we have a list of people’s names:

We can compute the length of one word like this:

We can build a list of the lengths of the names, like this:

You’ve seen these concepts before. count in this context is a higher-order function. In Clojure, that concept is easy because a function is a list, just like any other list element. You can use the same building blocks to compute a list having twice the lengths of person names, like this:

Since that function is so simple, we can write it as an anonymous function, like this:

We can also use a shorter form:

With the short form, # defines an anonymous function, with % bound to each item of the sequence. # is called a reader macro.

Anonymous functions give you convenience and leverage to create a function that doesn’t need a name right now. You’ve seen them in other languages. Here are some of the functions on collections that use higher-order functions. For all of these functions, we’re going to use a common vector, called v:

We’ll use that list with several anonymous functions in the following examples.

apply

apply applies a function to an argument list. (apply f ’(x y)) works like (f x y), like this:

filter

The filter function works like find_all in Ruby. It takes a function that is a test and returns the sequence of elements that pass the test. For example, to get only the odd elements or the elements less than 3, use this:

We’ll take a deeper look at some of the anonymous functions as we look deeper into Clojure sequences. For now, let’s take a break and see what Rich Hickey, creator of Clojure, has to say.

Interview with Rich Hickey, Creator of Clojure

Rich Hickey answered some questions for the readers of this book. He had a special interest in why this Lisp version could be more broadly successful than other Lisps, so this interview is a little more extensive than most. I hope you find his answers as fascinating as I did.

Bruce Tate:

Why did you write Clojure?

Rich Hickey:

I’m just a practitioner who wanted a predominantly functional, extensible, dynamic language, with a solid concurrency story, for the industry-standard platforms of the JVM and CLR[19] and didn’t find one.

Bruce Tate:

What do you like about it the most?

Rich Hickey:

I like the emphasis on abstractions in the data structures and library and the simplicity. Maybe that’s two things, but they are related.

Bruce Tate:

What feature would you change if you could start over again?

Rich Hickey:

I’d explore a different approach to numbers. Boxed numbers are certainly a sore spot on the JVM. This is an area I am actively working on.

Bruce Tate:

What’s the most interesting problem you’ve seen solved with Clojure?

Rich Hickey:

I think Flightcaster[20] (a service that predicts flight delays in real time) leverages many aspects of Clojure—from using the syntactic abstraction of macros to make a DSL for the machine learning and statistical inference bits to the Java interop for working with infrastructure like Hadoop and Cascading.

Bruce Tate:

But how can Clojure be more broadly successful when so many other dialects of Lisp have failed?

Rich Hickey:

This is an important question! I’m not sure I would characterize the major dialects of Lisp (Scheme and Common Lisp) as having failed at their missions. Scheme was an attempt at a very small language that captured the essentials of computation, while Common Lisp strove to standardize the many variant dialects of Lisp being used in research. They have failed to catch on as practical tools for general-purpose production programming by developers in industry, something that neither was designed to be.

Clojure, on the other hand, is designed to be a practical tool for general-purpose production programming by developers in industry and as such adds these additional objectives to the Lisps of old. We work better in teams, we play well with other languages, and we solve some traditional Lisp problems.

Bruce Tate:

How does Clojure work better in a team setting?

Rich Hickey:

There is a sense in which some Lisps are all about maximally empowering the individual developer, but Clojure recognizes that development is a team effort. For example, it doesn’t support user-defined reader macros, which could lead to code being written in small incompatible micro-dialects.

Bruce Tate:

Why did you choose to run on existing virtual machines?

Rich Hickey:

The existence of large and valuable code bases written in other languages is a fact of life today and wasn’t when the older Lisps were invented. The ability to call and be called by other languages is critical, especially on the JVM and CLR.

The whole idea of standard multilanguage platforms abstracting away the host OS barely existed when the older Lisps were invented. The industry is orders of magnitude larger now, and de facto standards have arisen. Technically, the stratification that supports the reuse of core technologies like sophisticated garbage collectors and dynamic compilers like HotSpot is a good thing. So, Clojure emphasizes language-on-platform rather than language-is-platform.

Bruce Tate:

Fair enough. But how is this Lisp any more approachable?

Rich Hickey:

Lots of reasons. For example, we wanted to deal with the parentheses “problem.” Lisp programmers know the value of code-is-data, but it is wrong to simply dismiss those who are put off by the parentheses. I don’t think moving from foo(bar, baz) to (foo bar baz) is difficult for developers. But I did take a hard look at the parentheses use in the older Lisps to see whether the story could be better, and it could. Older Lisps use parentheses for everything. We don’t. And in older Lisps, there are simply too many parentheses. Clojure takes the opposite approach, doing away with the grouping parentheses, making it slightly harder for macro writers but easier for users.

The combination of fewer parentheses and almost no overloading of parentheses renders Clojure much easier to scan, visually parse, and understand than older Lisps. Leading double parentheses are more common in Java code, the horrid ((AType)athing).amethod(), than in Clojure code.

What We Learned in Day 1

Clojure is a functional language on the JVM. Like Scala and Erlang, this Lisp dialect is a functional language but not a pure functional language. It allows limited side effects. Unlike other Lisp dialects, Clojure adds a little syntax, preferring braces for maps and brackets for vectors. You can use commas for whitespace and omit parentheses in some places.

We learned to use the basic Clojure forms. The simpler forms included booleans, characters, numbers, keywords, and strings. We also broke into the various collections. Lists and vectors were ordered containers with vectors optimized for random access and lists optimized for ordered traversal. We used sets, or unordered collections, and maps, which were key-value pairs.

We defined some named functions, providing a name, parameter list, and function body with an optional documentation string. Next, we used deconstruction with bindings, allowing us to bind any parameter to any part of an inbound argument. The feature was reminiscent of Prolog or Erlang. Finally, we defined some anonymous functions and used them to iterate over a list with the map function.

On day 2, we’ll look at recursion in Clojure, a basic building block in most functional languages. We’ll also look at sequences and lazy evaluations, some cornerstones of the Clojure models that help layer a common, powerful abstraction on top of collections.

Now, we’ll take a break to put into practice what you’ve learned so far.

Day 1 Self-Study

Clojure is a new language, but you’ll still find a surprisingly active, and growing, community. They were among the best that I found as I researched this book.

Find:

• Examples using Clojure sequences

• The formal definition of a Clojure function

• A script for quickly invoking the repl in your environment

Do:

• Implement a function called (big st n) that returns true if a string st is longer than n characters.

• Write a function called (collection-type col) that returns :list, :map, or :vector based on the type of collection col.