3.4 Day 3: The Parade and Other Strange Places

My first few days with Io were frustrating, but after a couple of weeks, I found myself giggling like a school girl at the unexpected places the language would take me. It’s like Ferris showing up on the news, at the ball park, in the parade—everywhere you don’t expect him. In the end, I got out of Io exactly what I wanted, which was a language that changed the way I think.

Domain-Specific Languages

Just about everyone who is deeply involved with Io appreciates the power that Io gives you in the area of DSLs. Jeremy Tregunna, one of the core committers for Io, told me about an implementation of a subset of C in Io that took around 40 lines of code! Since that example is just a little too deep for us to consider, here’s another one of Jeremy’s gems. This one implements an API that provides an interesting syntax for phone numbers.

Say you want to represent phone numbers in this form:

There are many approaches to the problem of managing such a list. Two that come to mind are parsing the list or interpreting it. Parsing it means that you would write a program to recognize the various elements of the syntax, and then you could place the code in a structure that Io understands. That’s another problem for another day. It would be much more fun to interpret that code as an Io hash. To do this, you will have to alter Io. When you’re done, Io will accept this list as a valid syntax for building hashes!

Here’s how Jeremy attacked the problem, with an assist from Chris Kappler, who brought this example up to the current version of Io:

That code is slightly more complex than anything you’ve seen so far, but you know the basic building blocks. Let’s deconstruct it:

The first line adds an operator to Io’s assignment operator table. Whenever : is encountered, Io will parse that as atPutNumber, understanding that the first argument is a name (and thus a string), and the second is a value. So, key : value will be parsed as atPutNumber("key", value). Moving on:

The parser calls the curlyBrackets method whenever it encounters curly brackets ({}). Within this method, we create an empty map. Then, we execute call message arguments foreach(arg, r doMessage(arg)) for each argument. That’s a seriously dense line of code! Let’s take it apart.

From left to right, we take the call message, which is the part of the code between the curly brackets. Then, we iterate through each of the phone numbers in the list with forEach. For each phone name and phone number, we execute r doMessage(arg). For example, the first phone number will execute as r "Bob Smith": "5195551212". Since : is in our operator table as atPutNumber, we’ll execute r atPutNumber("Bob Smith", "5195551212"). That brings us to the following:

Remember, key : value will parse as atPutNumber("key", value). In our case, the key is already a string, so we strip the leading and trailing quotes. You can see that atPutNumber simply calls atPut on the target range, which is self, stripping the quotes off the first argument. Since messages are immutable, to strip the quotes, we have to translate the message to a mutable value for it to work.

You can use the code like this:

Understanding Io’s syntax is trivial. You just have to know what’s going on in the libraries. In this case, you see a few new libraries. The doString message evaluates our phone book as code, File is a prototype for working with files, with specifies a filename and returns a file object, openForReading opens that file and returns the file object, and contents returns the contents of that file. Taken together, this code will read the phone book and evaluate it as code.

Then, the braces define a map. Each line in the map "string1" : "string2" does a map atPut("string1", "string2"), and we’re left with a hash of phone numbers. So, in Io, since you can redefine anything from operators to the symbols that make up the language, you can build DSLs to your heart’s content.

So, now you can begin to see how you would change Io’s syntax. How would you go about dynamically changing the language’s behavior? That’s the topic of the next section.

Io’s method_missing

Let’s review the flow of control. The behavior for what happens in a given message is all baked into Object. When you send an object a message, it will do the following:

1. Compute the arguments, inside out. These are just messages.

2. Get the name, target, and sender of the message.

3. Try to read the slot with the name of the message on the target.

4. If the slot exists, return the data or invoke the method inside.

5. If the slot doesn’t exist, forward the message to the prototype.

These are the basic mechanics of inheritance within Io. You normally wouldn’t mess with them.

But you can. You can use the forward message in the same way that you would use Ruby’s method_missing, but the stakes are a little higher. Io doesn’t have classes, so changing forward also changes the way you get any of the basic behaviors from object. It’s a bit like juggling hatchets on the high wire. It’s a cool trick if you can get away with it, so let’s get started!

XML is a pretty way to structure data with an ugly syntax. You may want to build something that lets you represent XML data as Io code. For example, you might want to express this:

like this:

Let’s call the new language LispML. We’re going to use Io’s forward like a missing method. Here’s the code:

Let’s carve it up. The Builder prototype is the workhorse. It overrides forward to pick up any arbitrary method. First, it prints an open tag. Next, we use a little message reflection. If the message is a string, Io will recognize it as a sequence, and Builder prints the string without quotes. Finally, Builder prints a closing tag.

The output is exactly what you’d expect:

I have to say, I’m not sure whether LispML is that much of an improvement over traditional XML, but the example is instructive. You’ve just completely changed the way inheritance works in one of Io’s prototypes. Any instance of Builder will have the same behavior. Doing this, you can create a new language with Io’s syntax but none of the same behaviors by defining your own Object and basing all of your prototypes on that new object. You can even override Object to clone your new object.

Concurrency

Io has outstanding concurrency libraries. The main components are coroutines, actors, and futures.

Coroutines

The foundation for concurrency is the coroutine. A coroutine provides a way to voluntarily suspend and resume execution of a process. Think of a coroutine as a function with multiple entry and exit points. Each yield will voluntarily suspend the process and transfer to another process. You can fire a message asynchronously by using @ or @@ before a message. The former returns a future (more later), and the second returns nil and starts the message in its own thread. For example, consider this program:

fezzik and vizzini are independent instances of Object with coroutines. We fire asynchronous talk and rhyme methods. These run concurrently, voluntarily yielding control to the other at specified intervals with the yield message. The last pause waits until all async messages complete and then exits. Coroutines are great for solutions requiring cooperative multitasking. With this example, two processes that need to coordinate can easily do so, to read poetry, for example:

Java and C-based languages use a concurrency philosophy called preemptive multitasking. When you combine this concurrency strategy with objects that have changeable state, you wind up with programs that are hard to predict and nearly impossible to debug with the current testing strategies that most teams use. Coroutines are different. With coroutines, applications can voluntarily give up control at reasonable times. A distributed client could relinquish control when waiting for the server. Worker processes could pause after processing queue items.

Coroutines are the basic building blocks for higher levels of abstractions like actors. Think of actors as universal concurrent primitives that can send messages, process messages, and create other actors. The messages an actor receives are concurrent. In Io, an actor places an incoming message on a queue and processes the contents of the queue with coroutines.

Next, we’ll look into actors. You won’t believe how easy they are to code.

Actors

Actors have a huge theoretical advantage over threads. An actor changes its own state and accesses other actors only through closely controlled queues. Threads can change each other’s state without restriction. Threads are subject to a concurrency problem called race conditions, where two threads access resources at the same time, leading to unpredictable results.

Here’s the beauty of Io. Sending an asynchronous message to any object makes it an actor. End of story. Let’s take a simple example. First, we’ll create two objects called faster and slower:

Now, we’ll add a method called start to each:

We can call both methods sequentially on one line of code with simple messages, like this:

They start in order, because the first message must finish before the second can begin. But we can easily make each object run in its own thread by preceding each message with @@, which will return immediately and return nil:

We add an extra wait to the end so that all threads finish before the program terminates, but that’s a great result. We are running in two threads. We made both of these objects actors, just by sending an asynchronous message to them!

Futures

I will finish up the concurrency discussion with the concept of futures. A future is a result object that is immediately returned from an asynchronous message call. Since the message may take a while to process, the future becomes the result once the result is available. If you ask for the value of a future before the result is available, the process blocks until the value is available. Say we have a method that takes a long time to execute:

I can execute the method and do something else immediately until the result is available:

Then, I can use the future value:

The futureResult code fragment will return a future object, immediately. In Io, a future is not a proxy implementation! The future will block until the result object is available. The value is a Future object until the result arrives, and then all instances of the value point to the result object. The console is going to print the string value of the last statement returned.

Futures in Io also provide automatic deadlock detection. It’s a nice touch, and they are easy to understand and use.

Now that you’ve had a flavor of Io’s concurrency, you have a sound foundation for evaluating the language. Let’s wrap up day 3 so you can put what you know into practice.

What We Learned in Day 3

In this section, you learned to do something nontrivial in Io. First, we bent the rules of syntax and built a new hash syntax with braces. We added an operator to the operator table and wired that into operations on a hash table. Next, we built an XML generator that used method_missing to print XML elements.

Next, we wrote some code that used coroutines to manage concurrency. The coroutines differed from concurrency in languages like Ruby, C, and Java because threads could only change their own state, leading to a more predictable and understandable concurrency model and less of a need for blocking states that become bottlenecks.

We sent some asynchronous messages that made our prototypes actors. We didn’t have to do anything beyond changing the syntax of our messages. Finally, we looked briefly at futures and how they worked in Io.

Day 3 Self-Study

Do:

• Enhance the XML program to add spaces to show the indentation structure.

• Create a list syntax that uses brackets.

• Enhance the XML program to handle attributes: if the first argument is a map (use the curly brackets syntax), add attributes to the XML program. For example:

  book({"author": "Tate"}...) would print <book author="Tate">: