PragPub 2013-09: Issue #51

Table of Contents

Features

- Playing with the Play Framework

by Nilanjan Raychaudhuri

Time to play? Get the scoop on Play Framework, which has been getting a lot of attention for how easy it makes it to build web applications with Java and Scala.

- Concurrent Programming in Clojure

by Michael Bevilacqua-Linn

Michael continues his exploration of Clojure with a look at its advantages for concurrent programming.

- The Automated Tester

by Jonathan Rasmussen

Don’t let yourself be limited by yesterday’s job definitions.

- Hadoop

by Jesse Anderson

What you don’t know about Hadoop, and should.

Departments

- On Tap

by Michael Swaine

Putting the fun back in Web development.

- Choice Bits

We follow Twitter so you don’t have to.

- Rothman & Lester

by Johanna Rothman and Andy Lester

The key to a successful interview is preparation. Johanna and Andy discuss how to walk in the room prepared to nail it.

- Puzzle

by Michael Swaine

A short session in mental calisthenics.

- Calendar

Want to meet one of the Pragmatic Bookshelf authors face-to-face? Here’s where they’ll be in the coming months.

- Bookshelf

What’s new and what’s hot from the Pragmatic Bookshelf.

- Guest Column

by James Iry

John Shade is on vacation and James Iry is filling in.

- Rear Window

Ted Nelson’s Computer Lib, the inimitable book that served as the manifesto for a revolution.

Except where otherwise indicated, entire contents copyright © 2013 The Pragmatic Programmers.

Feel free to distribute this magazine (in whole, and for free) to anyone you want. However, you may not sell this magazine or its content, nor extract and use more than a paragraph of content in some other publication without our permission.

Published monthly in PDF, mobi, and epub formats by The Pragmatic Programmers, LLC, Dallas, TX, and Raleigh, NC. E-Mail support@pragprog.com, phone +1-800-699-7764. The editor is Michael Swaine (mailto:michael@pragprog.com). Visit us at http://pragprog.com

ISSN: 1948-3562

On Tap

The Play Issue

by Michael Swaine

Play is a web framework that brings fun back to web development for Java and Scala developers. Chad Fowler and Nilanjan Raychaudhuri are working on a book for the Pragmatic Bookshelf on Play, and in this issue Nilanjan takes you on an exploration of this framework, which has been getting a lot of attention among Web developers.

Last month, Michael Bevilaqua-Linn explained Clojure’s take on value, state, and identity. This month, he takes it a bit further and shows how Clojure’s philosophy makes it ideal for concurrent programming.

Frequent contributor Jonathan Rasmussen explores the role of the automated tester, showing how it is different from traditional Quality Assurance, and shares a few ideas on how you can get from here to there.

You know something about Hadoop, but probably not enough. Jesse Anderson wants to tell you about a few good uses for Hadoop.

Rothman and Lester are back with advice on preparing for an interview. James Iry steps in for John Shade with a brief, incomplete, and mostly wrong history of programming languages. Plus we have some selected nuggests of wit and wisdom in Choice Bits, Pragmatic Bookshelf news and author events, another puzzle, and in our back-page parting shot that we call Rear Window, a fond look back at a book that had a lot to do with starting the personal computer revolution: Ted Nelson’s Computer Lib.

What’s Next?

We’re definitely in exploratory mode with the magazine and we have been listening carefully to you about the features you want in PragPub. You’ve made it clear that you want more delivery models: direct to Kindle, to your Dropbox, and other options. You also want to be able to subscribe without PayPal.

All perfectly reasonable requests, and we’re exploring implementing all those things. We expect to have some of them in place by next issue.

And in that next issue you can look forward to further exploration of Clojure by Michael Bevilaqua-Linn, career development planning with Rothman and Lester, the return of John Shade, the launch of a new personal computer history series, more features and maybe a new regular department.

Photo Credits: cover and p. 10, istockphoto.com/rappensuncle; p. 22, istockphoto.com/stefanschurr; p. 28, istockphoto.com/Alexander Bedrin; p. 33, istockphoto.com/Andreas Rodriguez.

Choice Bits

From the Twitterstream

We follow Twitter so you don’t have to.

Programmers tweet the darnedest things.

Sometimes You Punt

Reading error messages with unfamiliar vocabulary is stressful. That’s why many devs don’t read them. — @jeffcohen

September is “National Preparedness Month.” I’m not ready for that. — @PragmaticAndy

That moment when the salt shaker lid falls off, ruining your perfectly good sandwich. TODAY IS FAIL — @rit

2pm and brain finally has found first gear so I could get some code written. Going to the comic book store instead. :D — @adamgoucher

This Coding Life

San Francisco rents have become so high that even tech entrepreneurs are living in their cars. — @mims

“The beach is full of nerds arguing.” - @moritzheiber :) #eurucamp — @chadfowler

I suppose now that nearly everyone has a Mac, cmd-W can become shorthand for “what I’m looking at is displeasing.” — @timbray

Apparently, Adobe Photoshop can’t load my photograph until it initializes its “Type Tool.” That makes sense—IF YOU'RE CRAZY. — @MrAlanCooper

How many conference talks have you seen which have XKCD comics on their slides? They are the unsung LOLcats. — @chadfowler

Using a photo of a cat in a box to illustrate the box model in a CSS lecture achievement unlocked. — @antiheroine

Fame Is Fleeting but a Tattoo Is Forever

Perhaps Steve Ballmer can become a media darling again by twerking onstage at the next #Microsoft event? — @scottwambler

Trending: Everything that’s wrong with Microsoft, as told by veterans who abandoned the company. — @mims

Getting interviewed by David Frost? At least he went easier on me than when he interviewed a certain other Richard. — @richardbranson

Market Cap Doesn’t Equal Box Office Earnings: “Jobs” behind “Social Network” & “The Internship” via @marketingland. — @dannysullivan

“Project Runway” designers are so much more efficient at expressing a concept than I am. — @Ihnatko

Bradley Manning was the most searched for Bradley, now she’s the third most searched for Chelsea. — @dannysullivan

Sound and Fury

Most Americans are on the Stockholm Diet. We love food that's hurting us. — some idiot

Being motivated by competition means grading yourself on the curve. — some idiot

Whoever came up with the placebo effect should have patented it. — some idiot

We need more exciting examples than fib and factorial for teaching recursion. I suggest Fukushima. — some idiot

Brad Silverberg for CEO, I say. And refocus the company around Windows 95. — some idiot

Microsoft has to go thru the motions so the inhouse execs think they have a shot, but you know they're going with Ben Afleck. — some idiot

Bugged

Time to fix bug = 1 min. Time to write good test = 1 hour. It’s like that sometimes. — @jrasmusson

I didn’t plan it this way, but my summer project became writing about Snowden, the surveillance state, and the press. — @jayrosen_nyu

It’s the NSA’s world, we just live in it. — @davewiner

Why encrypt? Because (for now) we can. — @judell

Error messages are like paramedics knocking at your door when you’ve had a heart attack. Don’t ignore them. They’re trying to help — @jeffcohen

Who Are Those Guys?

First, they’re not all guys. Second, we have to confess that we cleaned up their punctuation and stuff a little. OK, who we followed this month: Scott W. Ambler, Richard Branson, Tim Bray, Jeff Cohen, Alan Cooper, Chad Fowler, Adam Goucher, Andy Hunt, Andy Ihnatko, Brendan W. McAdams, Christopher Mims, Jen Myers, Jonathan Rasmusson, Jay Rosen, Danny Sullivan, Jon Udell, Dave Winer, and the usual gang of idiots.

Fair’s fair. You can follow us at www.twitter.com/pragpub.

Rothman & Lester

Preparing for an Interview

by Johanna Rothman and Andy Lester

The key to a successful interview is preparation. Johanna and Andy discuss how to walk in the room prepared to nail it.

Johanna: This month, we’re talking about preparing for an interview. What do you recommend, Andy?

Andy: Probably the most common recommendation is that you read up on the company and find out as much as you can about it. Learn about the industry they’re in, read their website of public-facing material, and so on. I agree that this makes sense, but I also think that this research should have been done before you ever fired off a résumé.

This kind of research is key to figuring out if it’s likely that you’re interested in working for them in the first place. Better that you find that out before spending time tailoring a résumé to the company, sending it off, and going through the interview process with a company that you may not have any interest in working with anyway.

Johanna: If you’re looking for a job, you should research the company on LinkedIn, Glassdoor, on the web, through your friends and colleagues, all those ways, to see whether you might like working there. You can’t tell whether you might like the job or the culture if you don’t do your research first. Check out my blog post.

Andy: All this research is also in service of being able to talk directly to the problems that the company is looking to solve. It’s so much better if you can come in prepared to answer questions and talk coherently about the specific needs of the company and the hiring manager.

For example, say you see from LinkedIn that the hiring manager has only been on the job for six months. Now you have a hook. “How has it been since coming on in April? What challenges did you face, and how do think things have gone?”

Or, “I see from your LinkedIn profile that you’ve been managing this department for nine years. What sorts of changes has the department undergone in that time?”

Johanna: If you have an idea about the company’s products and services, you have a great opportunity of explaining how you have addressed similar things in the past.

Andy: Sometimes I hear people say “That’s creepy to know that,” but it absolutely is not.

Johanna: Why would people think that would be creepy? Are they candidates?

Andy: I don’t ask them that. I just assure them it’s not. And I also assure them that the company is doing the same sort of research on you before they even consider bringing you in for an interview.

Johanna: Right. The company has searched for you on LinkedIn. Someone might have already asked for a “backdoor” reference on you. If you’ve applied through a recruiter, they have more research on you. If you have a Twitter account or a blog, you can bet that they have read some of your tweets or your blog entries. They have done their research. Why wouldn’t you do your research? Being specific about how you can help the company and meet their needs is a good idea.

Andy: Explain what you mean by a “backdoor reference.”

Johanna: A backdoor reference is one where you’re connected to me and you see I’m connected to the candidate. You call or email me, and say, “Hey, Johanna, I see you’re connected to Connie Candidate. What can you tell me about her?” I might ask a couple of questions, such as, “In what dimension?” because your original question was a little open, even for me.

But asking someone who knows the candidate about when they worked with the candidate, what they did, would they work with the candidate again? Those are all great questions, and even someone who hasn’t worked with a candidate in years might be asked, as a backdoor reference.

Andy: And that backdoor reference works in your favor as well. You look up the hiring manager on LinkedIn and say “Hey, she used to work at Yoyodyne, and I’m pretty sure my buddy Bob worked at Yoyodyne, too. I wonder if he knows her.” There’s a lot to be discovered out there if you do the legwork.

Johanna: That’s why you need to keep updating your network and keep networking when you are looking for a job. Even when you’re not looking for a job, nurturing your network is something you need to do.

Researching the company and researching the hiring manager are the first steps. What next?

Andy: Preparing a list of questions to ask in the interview. “What’s a day like here?” “What will I be working on the first week? The first month?” “What are the biggest challenges facing the department in the next year?”

These are questions that help you get a better understanding of the company as well as show the interviewer that you have a head for business, and that you’re not just coming in to get a paycheck. (That’s why you don’t ask about salary in the interview. Check out this post.)

Johanna: It depends on who you’re talking to, right? Those are terrific questions for a hiring manager. You also want to ask questions of peers. Those questions might be the type of “What kinds of meetings do you have?” If you think you’re going to an agile shop, you can ask about the standups: “How long are the standups? When are they? Who attends? Tell me about the most recent standup....”

You want to learn what’s normal here.

You can also ask, “How often do you work closely with other people? What is teamwork like? Is there teamwork?”

You want to learn about recognition, “What kind of recognition is there here? What was the best recognition you ever received?”

Andy: The one thing to be really careful of is that your questions don’t make it sound like you bring a lot of baggage to the place. “Have you ever had a problem where, like, one person was slacking off and never got anything done and everyone else had to cover for him? How did they deal with that?” A question like that makes you sound like you have an axe to grind.

Johanna: Culture is about three things: What we can discuss, how we are rewarded, and how we treat each other. You want to ask questions that discover culture. You do not want to sound like a whiner in the interview. You do want to discover culture. It’s a fine line. Check out this post for more information.

That’s why it’s important to think about what you want to ask in advance. You can write down your questions.

Andy: I think you should write down your questions.

Johanna: That way, you know how to ask them and what to ask. You won’t get stuck and blurt them out wrong.

Andy: Another thing to do: Block out lots of time in your schedule. Say you’re scheduled for a one-hour interview. Block out at least three. The last thing you want to do is have an interview going really well, where you’ve talked well over an hour, and maybe the hiring manager has brought in Dave from down the hall, and the three of you are just chatting up a storm talking about the problems of the company, and the things that you’ve done in the past and that time you did that Oracle-to-PostgreSQL database conversion and then you say “Hey, I have to go.”

Johanna: Yes! You can ask in advance, “How much time do you expect to take for this interview?” And then add more time. You’ll need commuting time to and from. And if the conversation is going really well, you want the flexibility to remain and be able to talk to more people.

Andy: And speaking of the commute, drive to the place the day before so you know where it is and exactly how to get there and exactly how long it takes to get there. You don’t want to be driving around at 1:56 going “Where is this place? Is that 452 North or South Shorewood?”

Johanna: Or, you can do what I do, which is have paper directions and GPS directions. I do not rely on GPS only. Usually, the company has directions on their website, which I use. I print those. I use directions from the web, on paper, and GPS. It’s too important to get there on time, to be worried about wasting paper. If I need the backup of paper, I use it.

Andy: Also: Google Maps has been wrong before.

Johanna: So has Apple Maps.

Andy: Thumbs up on the paper back-up. What are you going to do if you’re without signal inexplicably? Go into the place and say “Sorry I’m late, I lost signal.”

Also, don’t go in there saying “Hi, I’m supposed to be meeting with... uh... someone in HR?” Commit to memory the names of the people you’re going to be meeting.

Johanna: I had a workshop a couple of years ago, at a place outside of Boston. I knew sort of where it was. It should have taken me 45 minutes to get there. I left my house with just over an hour in advance. But it was raining, and the roads were all backed up. It took me 90 minutes to get there. And, of course, I had no cell part of the way through my trip. My only saving grace was that everyone else was late!

Know who you will be meeting with first. And then you can ask for a list of the rest of the people. They may have changed who you will meet with.

Andy: Figure out the day before (or multiple days before) what it is you’re going to wear. Try it on. Know that it fits.

One of the things that I have discovered in my life is that as I grow older, the longer I leave clothes in the closet the more likely they are to... have shrunk unexpectedly.

Maybe the morning of the interview you realize that you thought you’d taken the suit to the cleaners after Aunt Marge’s wedding back in 2011, but apparently you didn’t. Oops.

Johanna: No open-toed shoes, ladies and gentlemen. No flip flops. No sandals. You should look presentable for a business meeting. That is what an interview is. Do you need a suit? Maybe, maybe not. You can ask. A jacket? A tie? Ask what is acceptable at that office.

Dress clothes/business casual might be fine. But you need to wear clean clothes that do not look like they need an iron. An xkcd T-shirt is not acceptable interview attire. Anything you wear to the beach is not acceptable.

Andy: All of this prep stuff may sound silly, and maybe even a little overly worrisome, but if you’re heading into a job interview, you want absolutely everything perfect because you want to have to worry about nothing. You’re going to be nervous enough as it is without realizing five minutes before you walk out the door that you don’t know where your nice dress shoes are.

Johanna: Once you know what you’re going to wear, you can relax about how you look. Now, it’s on to how you present yourself professionally.

One of the ways you can avoid worrying about your preparation is to think about the “stories” of your career. These aren’t made-up stories. These are real stories. These are how you will articulate the value of your career to potential hiring managers and interviewers.

Why do we call them stories? Because you need to think about them that way, so you can recall the particulars. In that sense, they are stories. They are your stories of your projects, your events of your career.

Andy: Right. If one of the big achievements in your past few years is doing that Oracle-to-PostgreSQL conversion project, then that’s a great story to tell. You talk about what brought it about, and how you prepared for it, and the problems that came up and how you solved them, and then the outcome of the project. If you can cap it off with “Going with the open source solution saved us $80,000 a year in licensing,” that’s golden.

Johanna: How you did it is a big piece of it.

Did you have the initiative for it? Did you fight for it? You want to show that initiative—that’s part of your story. You want to showcase that initiative in your interview. That’s what we mean by your stories. You have a wonderful accomplishment, with terrific value to the organization. Not only was it valuable, but the way you sold it to the organization was valuable.

Maybe you have some other quality, such as being able to coach others. You want to showcase that skill. Maybe you are able to refactor code so that it’s simple. Maybe you are able to help a team timebox their work. Whatever your quality, preference, or non-technical skill is, that added value on your technical skill is what helps you create all your value at work.

Andy: And details. If it’s a tech job you’re looking for, then don’t skimp on the tech details. Talking details shows that you know what you’re talking about. Putting “PostgreSQL” on a résumé is something that anyone can do. Being able to talk clearly about how you decided how to size your PostgreSQL server’s WAL logs is something that only comes from experience.

Johanna: Those added details separate the wannabes from the real doers.

When you base this on your most recent five years or so of experience, you have the stories of your career. You can go past the most recent five years of experience—that’s fine. But you want to ground the interview in your most recent experience. Why? Because your most recent behavior is more predictive of future behavior. Check out this post.

Andy: Here’s one final bit of prep. Come up with an answer to the question “tell me about yourself,” as it applies to the job for which you’re interviewing. Then, practice saying it. You don’t have to memorize it, but the words should come freely when you say them. It shouldn’t be longer than about a minute.

You may not be asked the exact words “tell me about yourself,” but somewhere in the start of the interview you’ll probably be given an opportunity to introduce yourself and an overview of the skills you bring to the company. The words you say and how you say them can influence the rest of the interview. Check out this post.

Johanna: We hope this has helped you all think about how to prepare for job interviews. Next month, we’ll talk about your professional development, and provide action plans and specifics on developing your career.

Johanna Rothman helps leaders solve problems and seize opportunities. She consults, speaks, and writes on managing high-technology product development. She enables managers, teams, and organizations to become more effective by applying her pragmatic approaches to the issues of project management, risk management, and people management. She writes the Pragmatic Manager email newsletter and two blogs on www.jrothman.com.

Andy Lester has developed software for more than twenty years in the business world and on the Web in the open source community. Years of sifting through résumés, interviewing unprepared candidates, and even some unwise career choices of his own spurred him to write his nontraditional book on the new guidelines for tech job hunting. Andy is an active member of the open source community, and lives in the Chicago area. He blogs at petdance.com, and can be reached by email at andy@petdance.com.

Send the authors your feedback or discuss the article in the magazine forum.

Playing with the Play Framework

Bring the Fun Back into Web Development

by Nilanjan Raychaudhuri

Time to play? Get the scoop on Play Framework, which has been getting a lot of attention for how easy it makes it to build web applications with Java and Scala.

Play is a web framework that brings fun back to web development for Java and Scala developers. I know what you are thinking: “Oh no, another web framework?” Before you turn to the next article, give me five minutes to make the case that developers building web applications for the JVM cannot avoid Play Framework. I am certain that it will impress you.

Why Play?

So what is wrong with building web applications for the JVM? It’s not fun anymore. Something is missing from the experience. The type of web applications we are building has changed. Today’s web applications are integrating more real-time data, have more scalability requirements, and need to be more user-friendly. However, the web frameworks that we use to build web applications for the JVM haven’t improved very much.

So what do we need? We need a framework that makes web development fun and easy by combining the new client-side technologies such as HTML5, LESS, and CoffeeScript, with server-side tools that provide scalability and development ease so that you can focus on solving business problems using tools that meets today’s demand. Play Framework combines the best tools from the Java and Scala worlds with the latest client-side technologies so that you can be productive from the start. Play Framework is a complete web development stack that takes care of your Web 3.0 application requirements.

I think that an example would be handy right about now. The best way to explore Play Framework is to build an application using it. It is also an opportunity for you to find out what Play Framework offers.

In this article we will build a small sample application called find_events using the REST API provided by Last.fm that finds events and concerts for various artists. In the process we will explore the framework and learn. If you just want to follow along without getting your hands dirty, you can access the codebase on GitHub.

Home screen of the find_events application

Let’s rock and roll!

Installing and Running Play

To install Play Framework, simply follow the setup guide found on the Play Framework website. If you use a Mac and have Homebrew, then you can use the command found in Listing NR-1 to install Play Framework.

Listing NR-1: Installing Play Framework via Homebrew

brew install play

After the installation process, check that you have the play command available by typing play help at the shell.

Play Framework Provides Fast Feedback

Play Framework is an MVC framework written in Scala. It provides both Scala and Java APIs to developers. You can mix both Java and Scala when building your application. One of the key features of Play Framework is type safety. It uses the Scala compiler to type check everything so that you can catch errors earlier in the process. It also provides a fast feedback cycle throughout which you can edit any file in your application and then refresh your browser to see the results. No more lengthy build-deploy cycle!

Now we are ready to take it for a spin. The easiest way to create an application using Play Framework is to use the play new command (Listing NR-2).

Listing NR-2: Creating the find_events application

play new find_events

This command will take you through a couple of interactive steps to gather information (Figure NR-2) about the project before it creates the project structure for you. For this article we will select Scala as our language of choice but you can still mix in Java code to the project whenever you need to.

Steps for creating the project

To run the application, change your directory to find_events and execute the command found in Listing NR-3.

Listing NR-3: Running the find_events application

play run

Now, if you point your browser to http://localhost:9000, you will see the welcome page of our application. Congratulations! You just created your first Play Framework application without writing a single line of code (Figure NR-3).

Welcome page

The welcome page contains enough information to get you started with how the page is rendered, various components of Play Framework, setting up an IDE, etc. If you are seeing this page for the first time, I would recommend that you spend some time on it. If you prefer to use an IDE to build this application, you could use one of the commands mentioned in the welcome page. The command found in Listing NR-4 will generate the project files for Eclipse.

Listing NR-4: Generating Eclipse project files

play eclipse

Writing Our First Controller Action

Before we write our first controller action, we need to understand how requests are served by Play Framework. Play Framework ships with its own embedded HTTP server that works as a network interface with clients. When a client sends an HTTP request to a Play Framework application, the request is handled by the embedded HTTP server, and then the request is forwarded to Play Framework to generate the appropriate response. Finally, the server sends the response back to the client.

Play Framework is Built for Asynchronous Web Programming

Non-blocking I/O is the name of the game these days. The philosophy behind non-blocking I/O is to not consume resources when waiting for something to happen. For example, traditionally web servers dedicate one thread per request. That thread is blocked until the request-response cycle is complete. On the other hand, non-blocking, asynchronous I/O allows servers to handle multiple requests and responses with a single thread. This has a big impact on server performance. Now you can handle more requests with a smaller number of fixed threads.

Play Framework uses Netty as its HTTP server. Netty is one of the popular NIO servers for the JVM and is included in the Play Framework distribution by default. This design decision has a couple of consequences. First, the web programming model supported by Play Framework is completely asynchronous. You are encouraged to handle requests in a completely non-blocking fashion (which we’ll also see in the find_events application). Second, it is different from the servlet programming model. In fact, Play Framework doesn’t natively understand servlets. This also has an impact on the way you deploy Play Framework applications. Usually Netty does such a wonderful job of handling requests that most Play Framework applications use Netty in production (check the Play Framework documentation for details).

So, when you point your browser to http://localhost:9000/, the following events occur:

Netty intercepts the request and sends an asynchronous message to Play Framework.

Play Framework finds the action mapped to the URL by looking at the routes file (conf/routes). The routes file is the configuration file that maps all of the URLs supported by the application to their corresponding actions. You’ll find an example in Listing NR-5.

Listing NR-5: routes file

GET     /  controllers.Application.index

Any HTTP GET request that ends with / in the URL will invoke the index action of the controller.Application object.

The controller action is responsible for producing the response. The response could be in any format: text, JSON, or HTML. The index action found in Listing NR-6 is using the app/views/index.scala.html view to produce the response.

Listing NR-6: index action

def index = Action {

Ok(views.html.index("Your new application is ready."))

}

Action is a simple abstraction over a function that takes a request and produces a response. Here Ok produces an HTTP 200 OK response with the contents of the index view.

Everything in Play Framework is a Function

This is one of those simple ideas that makes Play Framework a joy with which to work. Building an application using Play Framework involves writing multiple functions. A controller is nothing but a singleton object with collections of functions. Each action is a function that takes an HTTP request as a parameter and produces an HTTP response as the output. The view that you render (e.g., index.scala.html) is also compiled to a function that takes one or more parameters and produces HTML, text, or some other type of response. So views.html.index("Your new application is ready.") is essentially invoking the apply method of the index object by passing "Your new application is ready." as a parameter. In Scala these are called function objects.

That’s enough theory for now. Let’s write our first controller action. Copy and paste the action found in Listing NR-7 to your Application controller and then refresh your browser (make sure your Play Framework application is still running).

Listing NR-7: find controller action with syntax error

def find = Action {

Ok(views.html.index("Coming soon))

}

I made a deliberate mistake in the code, so if everything goes well you will see the error page found in Figure NR-4.

Error page

I am not sure about you, but I don’t know of any web framework that does such a good job of showing errors with line numbers (this only happens in development mode). Since almost everything is statically type checked, you will see these helpful error messages even if you make mistakes in views or route files. For example, if you point to a non-existent action in your route file, you will get a compilation error.

First of all, we need to create a new route for the find action so that the client can invoke it using a URL (Listing NR-8).

Listing NR-8: new find router

GET /find/events/:name  controllers.Application.find(name: String)

The routes file not only maps URLs to actions, but it can also parse the URL and pass portions of it as parameters to the action. Here the URL part after /find/events/ is parsed into an identifier called name and passed as a parameter to the find action of the Application controller. The find action will take the name of the artist and return all of the events associated with the artist using the REST API provided by Last.fm. But first you need to create an API account with Last.fm to access their API. Once you acquire the API key, you will need to add it to the application.conf file of your application. From configuring log levels to databases, this file is used to configure all aspects of Play Framework applications. We will add our API key as shown in Listing NR-9.

Listing NR-9: Last.fm API key in application.conf

lastfm.api.key="<your api key goes here>"

The name of the key is important, because we will use it in the code to access the API key. To fetch the events from Last.fm, we will use the artist.getEvents API method by passing the API key and the artist name. For example, the URL shown in Listing NR-10 will fetch all of the events associated with artist name "Fun" in JSON format (make sure to add your API key).

Listing NR-10: Fetching artist events using curl

curl -v "http://ws.audioscrobbler.com/2.0/

method=artist.getevents&artist=Fun.&api_key=<your api key>&format=json"

Now, from the find action, if we can construct the URL using the artist’s name and the API key, we can use the web service client provided by Play Framework to make the REST service call to Last.fm. A possible Play Framework implementation of that logic is found in Listing NR-11.

Listing NR-11: Application.scala

package controllers

import play.api._

import play.api.mvc._

import play.api.libs.ws.WS

import java.net.URLEncoder

import play.api.Play._

import scala.concurrent._

import ExecutionContext.Implicits.global

object Application extends Controller {

def index = Action { implicit request =>

Ok(views.html.index("Your new application is ready."))

}

def find(artistName: String) = Action { request =>

val rootUrl = "http://ws.audioscrobbler.com/2.0/"

val apiMethod = "artist.getEvents"

val apiKey = configuration

.getString("lastfm.api.key")

.getOrElse("no key")    <A>

val req = rootUrl +

"?method=" + apiMethod +

"&artist=" + URLEncoder.encode(artistName, "UTF-8") +

"&api_key=" + apiKey +

"&format=json"     <B>

Async {                        <C>

WS.url(req).get().map { response =>

Ok(response.json)

}

}

}

}

The find action retrieves the API Key (line A) from the application.conf file using the configuration object provided by the play.api.Play object. The configuration object is created automatically when a Play Framework application starts and can be used to access any configuration information provided in the application.conf file.

Next, the URL is constructed (line B) using the artist name and the API key. Finally (line C), we are making the REST web service call using the WS client. In Play Framework, WS client provides a nice and easy abstraction for making web service calls. Here the WS.url(req).get() call creates an instance of the web service client for the given URL and then makes an HTTP GET type request. The biggest benefit of WS client over any other HTTP client is that it is completely asynchronous. So the get() call in Listing NR-11 will immediately return with a Promise of future results. In the context of Play Framework think of Promise as a read handle to some value that will be fulfilled some time in the future. It can be thought of as a proxy object for the actual result. The map method in Listing NR-11 is invoked on Promise and will be called when the Promise is completed, like a callback. Using the map method we are registering a callback with the Promise that will be invoked when the Promise is complete (in other words, when the response is returned from the web service call). Inside the callback we are extracting the JSON response from the service and wrapping it as an HTTP 200 OK response.

Everything is Non-blocking

Play Framework is tuned to handle thousands of HTTP requests with a limited number of threads. Without any tuning, Play Framework applications are known to handle thousands of requests per second. With additional tuning based on your application needs, you can scale even further. So how is Play Framework handling the incoming HTTP requests? The main engine behind it is a concurrency framework called Akka. Akka makes writing concurrent and distributed applications easy for developers. Akka provides higher-level concurrency abstractions over threads. Play Framework uses the future-based concurrency model to handle all incoming requests. Futures are a nice way to perform many operations in parallel and in non-blocking fashion. Netty asynchronously handles all of the incoming packets and fires an event to the Play Framework request handler whenever a request is ready. The Play Framework request handler uses Scala Futures to manage all of the incoming requests by invoking appropriate controller actions. In Play Framework, actions should finish as soon as possible without blocking anything. Play Framework developers use abstractions like Future and Promise to offload work asynchronously, without waiting for it to finish.

But we still have a problem. We still don’t have the actual response. We only have the Future for the response, and we cannot return it from the action because Play Framework expects a response. The Async combinator in Play Framework allows us to return the Future of a result from an action. Internally, Play Framework will take the necessary steps to extract the result when ready and send it to Netty so that the response can be sent back to the client. So, the find action is completely non-blocking.

Testing and Debugging Play

As we continue to build our application, we need to verify our code is working. Traditionally this is done through writing automated unit tests. In the world of Play Framework it’s not any different. Play Framework provides very useful helpers to write unit tests and functional tests. It comes bundled with specs2 for unit testing and Selenium for functional testing. Please take a look at the Play Framework documentation for details. Additionally, Play Framework provides something called the Play console, which allows you to interactively work with Play Framework components. It is a REPL (Read-Evaluate-Print-Loop) for Play Framework applications. Traditionally a REPL is used to interactively work with a programming language, but here we will use it to interactively work with controller actions. To start the REPL, fire up the command found in Listing NR-12 from your project directory.

Listing NR-12: Starting the Play console

play test:console

This will start the Play console in test mode. Test mode allows us to use the test helpers inside the REPL to make our life easier. Now, without wasting time, let’s invoke our find action to check that it returns the JSON response from Last.fm. The commands found in Listing NR-13 will import all of the necessary classes and types we need to invoke the action.

Listing NR-13: Importing dependencies at the Play console

scala> import play.api.test._

import play.api.test._

scala> import play.api.mvc._

import play.api.mvc._

The test package is imported so that we can create a fake HTTP request and application to invoke our action.

Listing NR-14: Creating a fake application

?ng (see below what he had was the same as NR-13, I made up Creating a false application delete below?)

scala> play.api.Play.start(FakeApplication())

This will start an instance of the Play Framework application. You don’t have to start any server or process. Just invoking Play.start by passing an instance of Application starts the fully functional Play Framework application. In this case, FakeApplication creates an instance of our application in test mode. The code found in Listing NR-15 invokes the find action by passing "Fun." as an artist name with a FakeRequest.

Listing NR-15: invoking find at the console

scala> val AsyncResult(future) =

controllers.Application.find("Fun.").apply(FakeRequest("GET", "/"))

In Play Framework, action is simply a function that takes an HTTP request and returns an HTTP response. In this case, we are passing an instance of FakeRequest (again, because it’s easy to create) to invoke the action. The content of the request here is not important because we are not using the request inside of our action. Finally, we get the Future for which we are looking. Now we will wait for the future (await waits for the future to complete) and then print the result (Listing NR-16).

Listing NR-16: the Future returns

scala> val result = Helpers.await(future)

result: play.api.mvc.Result = SimpleResult(200, Map(Content-Type ->

application/json; charset=utf-8))

scala> Helpers.contentAsString(result)

res2: String = {"events":{"event"

[{"id":"3283420","title":"fun.","artists"

{"artist":"fun.","headliner":"fun."},"venue":{"id":"8779401",

"name":"La Maroquinerie","location":{"geo:point"

{"geo:lat":"48.868348","geo:long":"2.392217"},"city":"Paris",

"country":"France","street":"23 Rue Boyer","postalcode":"75020"},

"url":"http://www.last.fm/venue/8779401+La+Maroquinerie",

"website":"http://www.lamaroquinerie.fr",

"phonenumber":"+33-(0)1-40333505",

"image":[{"#text":"http://userserve-ak.last.fm/serve/34/5492666.png",

"size":"small"},{"#text":"http://userserve-ak.last.fm/serve/64/5492666.png",

"size":"medium"},{"#text":"http://userserve-ak.last.fm/serve/126/5492666.png",

"size":"large"},{"#text":"http://userserve-ak.last.fm/serve/252/5492666.png",

"size":"extralarge"},{"#text":"http://userserve-ak.la...

We got the result we were looking for. I don’t know about you, but I think that it’s pretty cool to invoke controllers interactively so that you can get quicker feedback. This feature is not restricted to controllers; you can literally invoke any part of the application from the console. The design decision to make everything a function in Play Framework has a benefit: we can invoke them from the REPL without the help of any server process or tools. Please note that await is only applicable when you are testing code. In the real world, your code should be completely non-blocking.

Play Framework Comes with a Build Tool

To build, compile, and distribute your code, Play Framework bundles a build tool called Simple Build Tool (SBT). SBT is the de facto build tool used in Scala projects. This build tool provides features like incremental compilation, continuous testing, running tests, parallel task execution, and very powerful plug-in support. Most of the magic of auto-reloading your changes, tasks like console and run are provided through by SBT. Every Play Framework project needs to define its own project and library dependencies using the SBT build file. This build file is automatically created with every Play Framework project. Take a look at the project/Build.scala file of your project.

Now that we are convinced that our action is producing the expected JSON response, let’s move on to building the UI of our application.

Adding UI Using Scala, LESS, and CoffeeScript

No web application is complete without a nice looking UI. Thanks to tools like Twitter Bootstrap, LESS, and CoffeeScript, writing UIs has never been so easy. Amazingly, and in fact, amateurs like myself can come up with a nice looking UI. Most of these new tools scrape the boilerplate out of the work. LESS extends CSS with dynamic behavior so that you can create variables and functions to reuse CSS code, while CoffeeScript makes writing JavaScript a nicer experience. So where do all of these fit into our Play Framework discussion? Play Framework natively supports all of these tools. You don’t have to do anything extra to reap these benefits.

Type Safety for the Win!

Anything that can be compiled is compiled by Play Framework. This includes your Scala files, routes file, and managed resources. Not only does Play Framework compile LESS and CoffeeScript files, it also checks them for errors. Go ahead and try to introduce errors in one of these files and you will see the result. This means that you catch more errors during compilation and get faster feedback. Internally Play Framework uses the Google Closure Compiler to compile JavaScript and catch errors.

Play Framework provides two types of resources: managed and unmanaged. Managed dependencies like LESS and CoffeeScripts are automatically compiled to CSS and JavaScript, respectively, whenever they are changed, while unmanaged resources are all of the files that you put under the public folder. These are mainly your static content.

To save some time, I have already created the less and coffee files for the find_events project. So go ahead and download find.less and find.coffee from the GitHub project and copy them into app/assets/stylesheets and app/assets/javascripts folders, respectively.

Any resources that you add under assets are automatically managed by Play Framework. Play Framework will compile these files and produce find.css and find.js files, which you can reference in your HTML.

HTML is generated by view templates in Play Framework. The view template is a function that takes some parameters and produces HTML content. But it’s not restricted to only HTML; you can produce other types of content as well. The templating language used in these templates is Scala. The good news is that it is quite easy to pick up the templating language even if you are Java developer. Usually the name of the view template is named after the action.

Typically the view templates share a common view template that provides the layout of the application. In our case it is main.scala.html. This file includes all of the CSS and JavaScript, and creates the layout for the application. We will modify the file to add our find.css and find.js files (Listing NR-17).

Listing NR-17: main.scala.html

@(title: String)(content: Html)

<A>

<!DOCTYPE html>

<html>

<head>

<title>@title</title>

<link rel="stylesheet" media="screen"

href="@routes.Assets.at("stylesheets/main.css")">

<link rel="stylesheet" media="screen"

href="@routes.Assets.at("stylesheets/find.css")">

<B>

<link rel="shortcut icon" type="image/png"

href="@routes.Assets.at("images/favicon.png")">

<script src="@routes.Assets.at("javascripts/jquery-1.7.1.min.js")"

type="text/javascript"></script>

<script src="@routes.Assets.at("javascripts/find.js")"

type="text/javascript"></script>

<C>

</head>

<body>

@content

</body>

</html>

The first line of the view template declares the parameters that it accepts. The main.scala.html template takes two parameters: the title of the page and the body of the page. Behind the scenes, Play Framework will compile this template into a function that also takes two parameters. Line B includes the generated find.css from the find.less file using the Assets.at function, which provides additional support for Etags, caching, and JavaScript minification.

You can use Assets.at to serve any public asset. Similarly, line C includes the generated find.js file from find.coffee so that other view templates can use it.

Now let’s modify index.scala.html to give it the same look as Figure NR-1. Since we are doing asynchronous programming on the server side, why not take it all the way through by using Ajax to asynchronously submit the request to the server? We will use the built-in JQuery library to make an Ajax request from the client. Listing NR-18 shows index.scala.html after the change.

Listing NR-18: index.scala.html

@(message: String)(implicit request: RequestHeader)

@helper.javascriptRouter("jsRoutes")

(routes.javascript.Application.find)

<A>

@main("Find Events for an artist") {

<div id="container">

<h1>Find Events</h1>

<form onsubmit="return false;">

<input type="text" name="artist"

placeholder="Enter artist name" id="artistName">

<input type="button" class="button search"

value="find" id="findButton"/>

</form>

</div>

<div id="results">&nbsp;</div>

}

In find.coffee, we register the click event handler for findButton. When the button is clicked, an Ajax request will be made to our Application.find action. But wait a minute: where is the url? In Play Framework, application URLs are specified in only one place: in the routes file. Instead, whenever anything needs a URL, it uses something called reverse routing. In line A, we are using the JavaScript reverse routing. The helper.javascriptRouter generates the necessary JavaScript functions that you can use to invoke their server-side counterparts. Now, if you refresh, you should see a UI very similar to the one shown in Figure NR-1. Find out whether your favorite band is playing near you!

Summary

Play Framework brings fun back into building web applications for the JVM. We have just scratched the surface of the Play Framework in this article. One article is simply not enough to explain the breadth of functionality provided by Play Framework. Many details are deliberately left out so that you can get a high-level overview of the framework. You should not stop here. Explore the framework, and I guarantee you that you will have your “wow” moments. I certainly had fun building the sample application, and I hope you had fun too.

About the Author

Nilanjan is a consultant/trainer and core member of Play framework team. He works for Typesafe. He has more than 12 years of experience managing and developing software solutions in Java, Ruby, Groovy, and also in Scala. He is zealous about programming in Scala ever since he got introduced to this beautiful language. He enjoys sharing his experience via talks in various conferences and he is also the author of the Scala in Action and upcoming Play Recipes books.

Send the author your feedback or discuss the article in the magazine forum.

Concurrent Programming in Clojure

Running in Parallel

by Michael Bevilacqua-Linn

Michael continues his exploration of Clojure with a look at its advantages for concurrent programming.

Welcome back once more to the series of Pragmatic articles on Clojure.

Last month, we learned about Clojure’s take on value, state, and identity. This time, we’ll take things a bit further and see how Clojure’s philosophy makes it ideal for concurrent programming. As we do so, we’ll take a closer look at Clojure’s managed references—atoms, agents, and refs—as well as the software transactional memory system they work with.

A concurrent program is one where multiple processes, or threads, may be run in parallel. The main difficulty with concurrent programming is coordinating access to shared resources. As we’ll see, this can have strange effects even in seemingly simple situations.

Let’s start off with a look at a simple concurrent programming problem in Java. We just want to increment counters from multiple threads running concurrently. We use two counters: the first is a plain old int , and the second is a much more impressive sounding AtomicInteger.

Next, we create a thread pool with five threads in it using Java’s ExecutorService, and we submit 10,000 tasks to the service. Each task increments both our normal counter and our Atomic counter. The full code snippet is below.

public class ConcurrencyExamples {

private static int counter = 0;

private static AtomicInteger atomicCounter =

new AtomicInteger(0);

public static void main(String[] args)

throws InterruptedException {

ExecutorService executors = Executors.newFixedThreadPool(5);

for (int i = 0; i < 10000; i++) {

executors.execute(new Runnable() {

public void run() {

counter++;

atomicCounter.incrementAndGet();

}

});

}

// Shutdown the pool and block until all tasks have executed.

executors.shutdown();

executors.awaitTermination(60, TimeUnit.SECONDS);

System.out.println(

String.format("Normal int counter: %s",

counter));

System.out.println(

String.format("AtomicInteger counter: %s",

atomicCounter));

}

}

If we’ve handled our concurrent counter manipulation properly, both counters should have the value 10,000. As we can see from the following output, that’s not the case.

Normal int counter: 9815

AtomicInteger counter: 10000

So what happened? It turns out that incrementing an integer isn’t an atomic operation. It’s actually broken up into multiple steps when compiled into bytecode, so the thread doing an increment can see the int it’s incrementing in an inconsistent state.

To make this a bit more clear, let’s take a look at a simpler example which just increments an int a single time.

public class JustIncAnInt {

private static int i;

public static void main(String[] args){

i++;

}

}

Once we compile the above code, we can use a command line tool that comes with most JDKs to get a readable representation of the bytecode it’s compiled to, as we do below.

> javap -c JustIncAnInt

Compiled from "JustIncAnInt.java"

public class JustIncAnInt extends java.lang.Object{

public JustIncAnInt();

Code:

0:  aload_0

1:  invokespecial  #1; //Method java/lang/Object."<init>":()V

4:  return

public static void main(java.lang.String[]);

Code:

0:  getstatic  #2; //Field i:I

3:  iconst_1

4:  iadd

5:  putstatic  #2; //Field i:I

8:  return

}

As we can see, that single increment is actually compiled to multiple bytecode operations. Without going too far into the weeds, the 0: getstatic #2; //Field i:I is responsible for loading the current value of the counter. It’s not until 4: iaddthat the increment is performed, and 5: putstatic #2; //Field i:Ithat it’s written back into the variable.

This, combined with subtleties involving when a write that one thread makes become visible to other threads, leaves plenty of opportunity for two threads to read and increment the variable at the same time. One solution would be to use one of Java’s locking mechanisms to protect the counter variable to ensure that only one thread can access the counter at a time, and that any modifications made to it are visible when the lock is relinquished.

None of this is a surprise to any developer versed in concurrency on the JVM, but it’s a nice illustration of the difficulties in concurrent programming. Programming with locks is low level and fraught with peril; a fantastic treatment of lock-based programming and other options for concurrent programming can be found in "". ?ed: Citing a book; just give me the info and I'll do it.

Another, higher level solution, is the one that we demonstrated using AtomicInteger. The AtomicInteger class provides an incrementAndGet which will atomically increment an internal counter, ensuring that no thread can see it in an inconsistent state.

Two main downsides to the AtomicInteger solution are that it’s very specific: it can only be used for integers, and it can only deal with coordinating changes to a single object. What if, for instance, we needed to change a map and an integer in lock step? To do so with Java, we’d probably need to fall back to locks.

Clojure’s immutable data structures, managed reference types, and software transactional memory system combine to provide a model that’s both high level and much easier to use than locking concurrency. Let’s dig in with a look at an atom, which I find to be the simplest to understand of the reference types Clojure provides.

Atoms are good for managing state that’s independent and to which we want to make synchronous changes. Here, independent means we can make the change without coordinating a change to any other piece of state. Synchronous means we want to block the thread that triggered the change until it’s done.

We can create an atom out of any immutable data structure using atom, and make a change to the atom’s state by passing the atom and function into swap! function. The swap! function applies the passed in function to the value wrapped by the atom, swaps the old value for the new one and returns it. Here, we create a simple atom counter, and a function to increment it.

(def atom-counter (atom 0))

(defn increment-atom-counter []

(swap! atom-counter inc))

To dereference an atom, or any of the Clojure’s managed references, we can use the @, as we show below.

=> @atom-counter

0

=> (increment-atom-counter)

1

So far, so good, but how does that help us with concurrency? When multiple threads attempt to modify an atom concurrently using swap!, Clojure first reads the current value out of the atom. Then it uses a lower level function over atoms called compare-and-set!.

The compare-and-set! function takes an old value and a new value, and it atomically sets the atom to the new value only if the current value of the atom equals the passed in old value. If swap! is unable to set the value of the atom using compare-and-set!, it will continue to retry until it is successful.

Here, we’ve rewritten our original Java example to something similar in Clojure, which takes advantage of atoms.

(defn concurrent-atom-modification []

(let [executors (Executors/newFixedThreadPool 5)

counter (atom 0)]

(dotimes [_ 10000]

(.execute executors (fn [] (swap! counter inc))))

(.shutdown executors)

(.awaitTermination executors 60 TimeUnit/SECONDS)

@counter))

Running concurrent-atom-modification sets our counter to the correct value.

=> (concurrent-atom-modification)

10000

This gives us a high level approach to concurrency much like the one we saw with Java’s AtomicInteger, but we could wrap any immutable data structure, like a Clojure vector or map in one so it’s more general.

One consequence of the way that atoms work is that the function passed into swap can’t have any side effects, or at least no side effects that can’t be repeated. This is because the function may be retried if the compare-and-set! fails the first time around.

Next up, let’s take a look at Clojure’s agents. Like atoms, agents let us manage changes to independent state, but they’re designed to do so in an asynchronous way. To modify the value referred to by an agent, we can use the send function.

Like swap!, it takes an atom to modify and a function that modifies it. Unlike swap!, it returns immediately. Operations are queued if necessary and applied to the agent serially.

Here we’ve create an atom version of our counter, and a function to increment it.

(def agent-counter (agent 0))

(defn increment-agent-counter []

(send agent-counter inc))

Calling increment-agent-counter doesn’t return the value of incrementing the counter, instead it returns a reference to the agent. At some point in the future, the counter will be incremented. Since incrementing the counter a single time is very fast, by the time we dereference the counter it’s already been done.

=> (increment-agent-counter)

#<Agent@64889c4e: 1>

=> @agent-counter

1

Since agents are guaranteed to be applied serially, any side effects that they execute will only execute once, unlike atoms.

The final managed reference type we’ll take a look at is the ref. These can be used when we need to make coordinated changes across more than one data structure.

In the example we’ll see below, we’ll take a look at a tiny system that keeps track of television series and episodes. An episode of a series is represented as a map with an id, a name, and a nested map that represents the series it’s attached to. The nested series has a series id and name, as we show below.

{:id 42

:name "Fragged"

:series {:id 10 :name "Battlestar Galactica"}}

As episodes are added to the system, we want to populate two maps, one of episodes keyed off of episode id and one of series keyed off of series id. To add the series to our series map, we’ll pick the embedded series out of an episode when it’s added, and add it to the series map if it’s not already there.

We’d like to ensure that we never see our data in an inconsistent state, where we’ve added an episode but not its corresponding series, and we’ll do so using Clojure’s refs and software transactional memory system.

Let’s start by creating a couple of refs to hold our maps.

(def all-episodes (ref {}))

(def all-series (ref {}))

The meat of our solution involves adding episodes using assoc. Much like the swap!function does for atoms, and the send function does for agents, alter takes a reference, function, and arguments.

Unlike swap! and send!, ?ed (send! or send without exclamation point?) alter must be called inside of a transaction. To create a transaction, we wrap our calls to alter inside of dosync. This acts a lot like a database transaction, and it turns our multiple calls to alter into one atomic unit.

If one thread was to modify either map while another was in the process of doing so, one thread would win and its transaction would commit. The other would roll its transaction back and try it again. This means that, just like with the atoms we saw earlier, we need to avoid side effects in our transactions.

(defn add-new-episode [current-episode]

(let [current-series (:series current-episode)]

(dosync

(alter all-episodes

#(assoc % (:id current-episode) current-episode))

(alter all-series

#(assoc % (:id current-series) current-series)))))

Now we can create a few test episodes, and add them to our system.

(def e1 {:id 1

:name "33"

:series {:id 10 :name "Battlestar Galactica"}})

(def e2 {:id 2

:name "Water"

:series {:id 10 :name "Battlestar Galactica"}})

(def e3 {:id 3

:name "The Target"

:series {:id 11 :name "The Wire"}})

=> (add-new-episode e1)

{10 {:name "Battlestar Galactica", :id 10}}

=> (add-new-episode e2)

{10 {:name "Battlestar Galactica", :id 10}}

=> (add-new-episode e3)

{11 {:name "The Wire", :id 11},

10 {:name "Battlestar Galactica", :id 10}}

Both of our maps contain the correct value, and they got there in a thread-safe way!

=> @all-episodes

{3 {:name "The Target",

:series {:name "The Wire", :id 11},

:id 3},

2 {:name "Water",

:series {:name "Battlestar Galactica", :id 10},

:id 2},

1 {:name "33",

:series {:name "Battlestar Galactica", :id 10},

:id 1}}

=> @all-series

{11 {:name "The Wire", :id 11},

10 {:name "Battlestar Galactica", :id 10}}

That wraps up our look at Clojure’s concurrency. We’ve only scratched the surface here. Clojure’s API docs present an excellent, if concise, description of the reference types we’ve seen here and they’re a great place to look for more information.

Thanks for reading! Join us again next month as we introduce Clojure’s take on polymorphism without object orientation.

Michael Bevilacqua-Linn has been programming computers ever since he dragged an Apple IIGS that his parents got for opening a bank account into his fifth grade class to explain loops and variables to a bunch of pre-teenagers. He currently works for Comcast, where he builds distributed systems that power infrastructure for their next generation services, and wrote Functional Programming Patterns for the Pragmatic Bookshelf. He tweets occasionally at @NovusTiro.

Send the author your feedback or discuss the article in the magazine forum.

The Automated Tester

Write. Code. Automate.

by Jonathan Rasmussen

Don’t let yourself be limited by yesterday’s job definitions.

In the last couple of years I’ve seen a huge spike in demand for people who can write, code, and automate their own acceptance tests. I call these folks automated testers. Not only do these people automate their own tests, they also do their own analysis and enjoy talking with customers.

In this article I would like to explore the role of the automated tester, show how it is different than traditional Quality Assurance (QA), along with a few ideas for how people can get from here to there.

Software is Eating the World

In his seminal article, Why Software Is Eating the World, Mark Andreessen makes a strong case for how software is taking over huge swathes of the modern economy and eating the world. Amazon. Apple. Google. Twitter. All these companies are building highly defensible, highly profitable businesses, with software at the heart of what they do.

This is great for the disruptors, but painful for the incumbents.

- Amazon killed Barnes and Noble.

- Netflix ended BlockBuster.

- And AirBnb and Uber are redefining the meaning of hotel and taxi.

And it’s not just companies that are getting disrupted. It’s people.

With the rise of Agile, and the popularity of running projects with a more entrepreneurial startup spirit, traditional software development roles are being challenged. None more so than traditional QA.

Everyone is a Tester

Today everyone is a tester.

- Analysts test.

- Developers test.

- Designers test.

- Customers test.

Everyone tests. Quality is no longer the sole domain or responsibility of a single group or entity. It’s a team responsibility. And that’s a big change for the traditional tester. Everyone is in their house.

This wouldn’t be a big deal if testers could show up on a project, be given a well written spec, and then be left alone to create comprehensive test plans to be executed every time someone changed the software. But those days are gone.

That role doesn’t exist anymore.

So if analysts are testing. And developers are testing. What’s traditional QA doing?

Leveling Up

I see two options. Traditional QA testers can dig in, defend their turf, and fight for the mantle of sole custodians of quality (in other words try and keep the status quo). Or they can branch out and get into analysis and development the way BAs and dev’s have gotten into testing.

To do this, the trick is to stop viewing analysis, development, and testing as solo roles. These are things that need to be done. Who does them doesn’t really matter.

I know many testers who are wonderful analysts. The bring an attention to detail and an outside-the-box way of thinking that’s invaluable in fleshing out requirements and necessary for doing thorough analysis.

I also know testers who are great at automating their own test cases. I don’t mean simple click-and-record playback bloat-ware.... I mean actually getting into code, using open source frameworks such as Watir, Selenium, Cucumber, and RSpec.

By expanding their skills and getting into these other areas testers can quickly become doubly valuable on projects. Not only can they do their own analysis and write great acceptance tests, they can also code them up, verify things work, and reduce their reliance on developers for automation.

Shorter queues. Fewer handoffs. Less waste. More value.

Enter The Automated Tester

This is where the automated tester comes in. These are testing experts with a passion for business and a thirst for automation.

They like knowing the business because it helps them write better test cases. And the more they automate, the more time they have for edge cases and exploratory testing.

They also like technology because a good understanding of the tech gives them insight into the limits and capabilities of their testing frameworks.

The experienced ones know they can’t (and shouldn’t) automate everything. Every test has a cost, and they think carefully about what to automate. They also work closely with developers to see what unit tests the developers have written, to ensure they aren’t duplicating efforts.

And of course they still love exploratory testing. Automation is fine. But it’s not perfect. To truly be effective, you still can’t beat the intuition and creativity that comes from firing up the browser, clicking through the app, and looking for ways to bust things along the way.

Getting There

Now if all this sounds easy—I assure you it isn’t. This is a tough, demanding role. You’ve got know the business. You’ve got to have a passion for automation and technology. And you have to enjoy continuously learning and retraining yourself.

But it’s a huge opportunity. It’s a big role. And it’s not for the faint of heart. But for those who are looking to do more than test, I think it can be a hugely rewarding role, and one that will soon be sought after by many a software team.

Here are some ideas on how to get there.

1. Build apps.

You don’t have to be a developer. The goal is to get just deep enough to understand at a high level how things work.

If you are testing web apps, you should know what HTML/CSS and JavaScript are. If you are building iPhone applications, you should have XCode and be really familiar with the emulator.

If you build a app with a table that contains data from a database, give yourself a pat on the back. You’ve just replicated the functionality of 99 percent of business applications out there today.

Building simple apps and understanding the technology is going to give you insight. Insight into how these testing frameworks work, their limitations, and where, if necessary, they need to be abandoned.

2. Master one testing automation framework.

There are a lot of automated tested frameworks to choose from. Don’t sweat it. Just pick one. Learn it really well. Then move on to another.

They all work pretty much the same, and once you’ve got one figured out, you’ve pretty much got them all figured out.

If you are testing the web, start with Selenium. Then try Watir, RSpec, Cucumber, or whatever else your team likes. If you are on the iPhone try Frank. The framework doesn’t really matter. Just pick one and go.

3. Hug a developer.

A good developer can be your best friend. The good ones are already doing what you are getting into and the best will be more than happy to show you how it all works. They can set you up, help get you going, and even augment the frameworks you are using by making them easier to use.

Study what they do. Learn how they set things up. And soon you will be able to do it on your own.

4. Automate everything.

I know I just said the good testers don’t automate everything before their eyes, but in the beginning you should try, just to get the practice and see what’s possible.

You are going to write a lot of bad tests when you first start out. Don’t worry about it. We all do. Just automate everything you can, and you’ll soon start to get a feel for which tests add value, which ones don’t, and where to draw that line.

5. Fear nothing.

It’s going to be scary when you start (I am talking to you traditional QA testers). You are going to be leaving a space of tradition, comfort, strength, responsibility, and entering into something completely foreign and new. I’ll let you in on a little secret. It’s like this for everybody.

Developers go through this every time they learn a new language. BA’s go through it when learning a new business domain. And project managers go through it when they drop the Gantt chart in favor of the burndown.

You have nothing to fear. Everyone is going through it. Just be humble. Ask for help when you need it. And you will fit right in.

One Challenge

One challenge of becoming an automated tester is people are going to look at you a little bit funny. Companies don’t like things that don’t fall neatly into slots, and you are going to be an anomaly.

You are going to go to conferences, explain what you do to people, and they are going ask:

“So are you an analyst, tester, or developer?”

Conferences, papers, titles, pay scales, expectations, and bodies of knowledge are all not going to be ready for you. Nobody is expecting you to work this way. We’ve got 50 years of history working against us.

All I can say is hang in there. Do whatever feels right. Start slow. Start fast. Look for the opportunity and seize it when it comes.

In the meantime, practice, be prepared, and keep taking on more responsibility. Once you demonstrate the value this role brings (through action—not words) project managers will soon be getting into fistfights for your services, and you’ll have the freedom to contribute however you wish.

Remember the goal isn’t to test or even build great software. It’s to build something wonderful for your customers. And you are just doing whatever it takes to do that.

Summary

There’s a new role for testers out there. One that blends the art of traditional testing with analysis, development, and automation.

If you have a passion for testing, are curious about technology, and are looking for more ways to contribute to your project, consider becoming an automated tester. It’s an exciting role, one that nearly every Agile project needs, and one I hope we see a lot more people entering into in the near future.

Jonathan Rasmusson is the author of The Agile Samurai. As an experienced entrepreneur and former agile coach for ThoughtWorks, Jonathan has consulted internationally, helping others find better ways to work and play together. When not coaching his sons’ hockey teams or cycling to work in the throes of a Canadian winter, Jonathan can be found sharing his experiences with agile delivery methods at his blog, The Agile Warrior. For more information on the Agile Inception Deck check out The Agile Samurai.

Send the author your feedback or discuss the article in the magazine forum.

Hadoop

A Few Good Use Cases

by Jesse Anderson

Solving problems for traveling salesmen and other overburdened folks.

You’ve probably heard of Hadoop and I want to tell you about a few good uses for Hadoop. Mind you, this isn’t the exhaustive list, but it gives you an idea.

If you haven’t heard of Hadoop, there are a few things you need to know about it. First, there is the MapReduce component. This allows you to process the data. Second, there is the HDFS component. This allows you to store massive amounts of data in a cost-efficient way. Third, there is the Hadoop ecosystem. These are projects that help round out some of the use cases that don’t fit into the first two components. If there’s something missing, chances are you’ll find it in the ecosystem.

ETL Workflow

This is the bread-and-butter use case for Hadoop. It’s so common that I use it to show how Hadoop MapReduce works in my screencasts. Using playing cards, I show how an ETL workflow uses MapReduce and scales on the cluster.

ETL or Extract, Transform, and Load, is the process of extracting a data source, transforming it in some fashion, and loading where it will ultimately reside. In this use case, the data can be extracted from a data source like a relational database. The data is then transformed using MapReduce. By using MapReduce, the workload can be spread evenly across the cluster. The transformed data is loaded into HDFS. We’re seeing more companies keep their ETL data in HDFS because it’s more cost effective than other proprietary solutions.

Log Analysis

Processing click-stream data is incredibly difficult due to the sheer amount of interactions a user can have on a website. Many developers opt to place this data in relational databases for easier analysis later on. On a busy site, this quickly becomes unfeasibly slow. Companies are turning to using flat text files to track user interactions. These can be as simple as using the Apache log file. This trades one issue for another, though. How do you correlate user interactions that happen across many files and in random places within the files?

This is an area where MapReduce shines. Using some of the features of the API, you can merge all of the user’s interactions together, in an efficient and scalable fashion. Once all of the data is merged, your algorithm can process the user’s interactions based on their timestamp.

Joining Large Datasets

Few things get you in hot water faster with your DBA than doing large joins and full table scans. Hadoop can handle this with MapReduce in a scalable way to join even massive datasets. Better yet, there are Hadoop Ecosystem projects such as Hive and Impala that make a join as easy as writing an SQL statement. This SQL statement gets translated into a MapReduce program for you. The MapReduce program will join the dataset and save it back to HDFS for any further processing.

Your DBA will be none the wiser and you’ll have joined your dataset. Do your processing and you can bring the summarized results back into your relational database to be served up for web traffic. Everyone is happy and your Hadoop cluster is offloading tasks that are better suited to MapReduce.

Graph Traversal

Here’s a bonus use case: traversing a graph. I’ve written about this use case and have a full code sample for traversing the graph of a Boggle roll. In the case of a Boggle roll, the growth of traversals is factorial and a MapReduce can scale to handle that level. Using the scalability of MapReduce and some other tricks, I’ve run some 100x100 Boggle rolls as opposed to a normal Boggle roll’s 4x4 matrix.

Conclusion

Your use case might mirror the four I’ve written about. Others, like the graph processing algorithm, might require a tweak to an existing algorithm. The problems of handling Big Data are solved using Hadoop and its ecosystem. Whatever your use case, give Hadoop a close look.

Jesse Anderson is a Creative Engineer in Reno with many years of experience in creating products and helping companies improve their software engineering. He works at Cloudera on the Educational Services team as a Curriculum Developer and Instructor. Jesse works on both professional and personal projects. A recent personal project, Million Monkeys Shakespeare project, went viral and gained international notoriety. Jesse’s interviews with national media outlets including the Wall Street Journal and Fox News. He volunteers his time as the President of the Northern Nevada Software Developers Group and he sits on the Technology Advisory Committee at Morrison University.

His blog and website is jesse-anderson.com and his Github account contains other example MapReduce programs.

Send the author your feedback or discuss the article in the magazine forum.

Puzzle

A Sudoku-Anagram Mashup

by Michael Swaine

A short session in mental calisthenics.

Here’s another programming-related sudoku puzzle.

It uses letters rather than numbers, but otherwise it works just the way you expect. Your task is to fill in the empty cells so that every row, column, and each of the 3x3 squares contains one each of the nine letters you’ll find in the cells already filled in.

Oh, one more thing. When you finish the sudoku, rearrange those nine letters to spell the name of a programming language.

S

X

A

F

X

X

X

X

X

X

X

R

Z

X

X

X

X

X

X

X

X

X

X

X

F

N

A

X

X

X

X

I

Z

X

L

X

X

X

X

X

X

A

P

X

X

I

Z

X

X

X

X

X

X

X

N

A

X

X

X

X

L

X

X

X

X

I

X

X

X

X

R

X

X

X

X

L

S

X

N

X

X

And the name is: _ _ _ _ _ _ _ _ _ .

Solution right here next issue. Or on Twitter right now, no doubt.

Solution to Last Month’s Puzzle

R

E

W

U

A

B

N

S

H

H

A

U

S

R

N

B

W

E

N

S

B

W

H

E

A

U

R

S

U

A

W

H

N

B

R

E

R

E

H

U

B

A

N

S

W

B

W

N

R

E

S

H

A

U

E

W

U

A

N

S

H

B

R

A

N

R

E

H

B

W

U

S

S

B

H

U

R

W

E

N

A

And the secret word is: WASHBURNE, the last name of two characters in Firefly.

Calendar

Want to meet one of the Pragmatic Bookshelf authors face-to-face? Here’s where they’ll be in the coming months.

- 2013-08-29, devLink, Chattanooga, TN

How to be a Healthier Programmer: Your health affects your memory and creativity--skills critical to doing your job well. In this talk you'll learn to change habits, boost brainpower, and master exercises that make working at a computer more comfortable.

Joe Kutner (author of Deploying with JRuby and The Healthy Programmer )

- 2013-09-05, Agile on the Beach, Falmouth, UK

Workshop: The Art of Slicing and Dicing User Stories

Rachel Davies (author of Agile Coaching )

- 2013-09-10, 360|iDev - Denver, CO

Dynamic Audio for Apps and Games - In this talk you'll get a brief introduction to the field of sound design and different techniques that can be used to create sound. You'll then be introduced to PureData, an application and platform for working with

Tony Hillerson

- 2013-09-11, BDD Kickstart, Barcelona

Get a flying start with BDD, the collaborative process that's changing the face of software development.

Matt Wynne (author of The Cucumber Book and Cucumber Recipes )

- 2013-09-14, Baruco

TBD

David Chelimsky (author of The RSpec Book )

- 2013-09-14, Barcelona Ruby Conference

Hexagonal rails

Matt Wynne

- 2013-09-17, CukeUp! NYC

A 1-day conference dedicated to Cucumber, Specification by Example and BDD.

Aslak Hellesøy (author of The RSpec Book , The Cucumber Book , and Cucumber Recipes )

- 2013-09-18, Strange Loop, St. Louis, MO

Servo: Designing an Implementing a Parallel Browser

Jack Moffitt (author of Seven Web Frameworks in Seven Weeks )

- 2013-09-19, Hamburg, Germany

Speaking about something fascinating!

Chad Fowler (author of The Passionate Programmer (2nd edition) and Rails Recipes )

- 2013-09-24, Cambridge, UK

Workshop: Slicing and Dicing User Stories and Experience Report: XP at Unruly.

Rachel Davies

- 2013-09-26, CocoaConf Columbus

iPad Productivity APIs workshop

Chris Adamson (author of iOS SDK Development )

- 2013-09-27, CocoaConf Columbus

Intro to iOS Gesture Recognizers

Jonathan Penn (author of Test iOS Apps with UI Automation )

- 2013-09-27, CocoaConf Columbus

Get on the Audiobus

Chris Adamson

- 2013-09-27, Amsterdam, Netherlands

Speaking about something fascinating!

Chad Fowler

- 2013-09-27, CocoaConf Columbus

NSIncrementalStore - Bet you didn't think Core Data could do *that*!

Jonathan Penn

- 2013-09-27, ISVCON 2013 - Reno

I'll be covering how ISVs can use The Cloud to lower costs and improve customer satisfaction.

Jesse Anderson (author of The Cloud and Amazon Web Services and Processing Big Data with MapReduce )

- 2013-09-28, CocoaConf Columbus

Glitch-Free A/V Encoding

Chris Adamson

- 2013-09-30, Aarhus, Denmark

Keynote: The Passionate Programmer

Chad Fowler

- 2013-10-04, NationJS, Washington DC

TBD But It's Sure To Be Amazing

Chris Strom (author of The SPDY Book , Dart for Hipsters , and 3D Game Programming for Kids )

- 2013-10-09, Tel Aviv, Israel

Disposable Components

Chad Fowler

- 2013-10-12, CocoaSlopes (Ogden, Utah)

Testing on iOS With New Hotness

Jonathan Penn

- 2013-10-17, GOTO Berlin

Keynote speech and case study on how we apply eXtreme Programming to product development at Unruly.

Rachel Davies

- 2013-10-18, Southern Fried Agile

Scaled Agile Framework (SAFe) is a popular Agile process for enterprise Agile adoption, but it's easy to misuse. Jared shares his practical experience on how to use it's strengths while avoiding the pitfalls.

Jared Richardson (author of Ship It! , Hands-on Backbone.js , The Cloud and Amazon Web Services , Sublime Text 2 , and Processing Big Data with MapReduce )

- 2013-10-24, CocoaConf Boston

Core Audio All-Day Tutorial

Chris Adamson

- 2013-10-24, CocoaConf Boston

iOS 2D Game Development Workshop

Jonathan Penn

- 2013-10-25, CocoaConf Boston

Get on the Audiobus

Chris Adamson

- 2013-10-25, Rakuten Technology Conference, Tokyo

Talk (and possibly a tutorial)

Dave Thomas (author of Programming Ruby (2nd edition) , Agile Web Development with Rails (3rd edition) , The Ruby Object Model and Metaprogramming , Agile Web Development with Rails 4 , Programming Ruby 1.9 & 2.0 (4th edition) , Agile Web Development with Rails 3.2 , and Programming Elixir )

- 2013-10-25, CocoaConf Boston

NSIncrementalStore - Bet you didn't think Core Data could do *that*!

Jonathan Penn

- 2013-10-26, CocoaConf Boston

Zen And The Intro To iOS Gesture Recognizers

Jonathan Penn

- 2013-10-26, CocoaConf Boston

Glitch-Free A/V Encoding

Chris Adamson

- 2013-10-27, RubyConf China, Beijing

Keynote

Dave Thomas

- 2013-11-01, London, UK

The Emperor's New Clothes: Meaningful Interactions in Stressful Situations

Portia Tung

- 2013-11-08, Software Craftsmanship North America, Chicago

Talk, to be determined

Dave Thomas

- 2013-11-14, CocoaConf Atlanta

iOS 2D Game Development Workshop

Jonathan Penn

- 2013-11-18, BITS - Software Developer Conference, Denmark

I'll be speaking on TDD, and have a hands-on workshop. We'll also offer a 2-day public training class for TDD and embedded C/C++

James Grenning (author of Test Driven Development for Embedded C )

Bookshelf

Pragmatic Bookshelf News

Here’s what’s new and what’s hot from the Pragmatic Bookshelf.

What’s New

Here’s what’s coming next to the Bookshelf:

- Seven Web Frameworks in Seven Weeks [in beta]

- Test iOS Apps with UI Automation [in print]

- Good Math: A Geek's Guide to the Beauty of Numbers, Logic, and Computation [in print]

- 101 Design Ingredients to Solve Big Tech Problems [in print]

What’s Hot

Top-Ten lists are passé—ours goes to 11. These are the top titles that folks are interested in currently, along with their rank from last month. This is based solely on direct sales from our online store.

1^

7

Practical Programming

2^

NEW

Good Math

3v

2

Agile Web Development with Rails 4

4^

NEW

Seven Web Frameworks in Seven Weeks

5^

NEW

Lean from the Trenches

6v

3

Programming Ruby 1.9 & 2.0

7^

NEW

Web Development with Clojure

8v

1

The Healthy Programmer

9v

4

Programming Elixir

10^

NEW

Test iOS Apps with UI Automation

11^

NEW

101 Design Ingredients to Solve Big Tech Problems

What’s Happening

But to really be in the know, you need to subscribe to our weekly newsletter. It’ll keep you in the loop, it’s a fun read, and it’s free. All you need to do is create an account on pragprog.com (email address and password is all it takes) and select the checkbox to receive newsletters.

Guest Column

A Brief, Incomplete, and Mostly Wrong History of Programming Languages

by James Iry

You may have seen this, or a version of it, elsewhere. This is a snapshot of a living document. James updates it as appropriate at its home here. We thought it deserved even more exposure.

1801: Joseph Marie Jacquard uses punch cards to instruct a loom to weave “hello, world” into a tapestry. Redditers of the time are not impressed due to the lack of tail call recursion, concurrency, or proper capitalization.

1842: Ada Lovelace writes the first program. She is hampered in her efforts by the minor inconvenience that she doesn’t have any actual computers to run her code. Enterprise architects will later relearn her techniques in order to program in UML.

1936: Alan Turing invents every programming language that will ever be but is shanghaied by British Intelligence to be 007 before he can patent them.

1936: Alonzo Church also invents every language that will ever be but does it better. His lambda calculus is ignored because it is insufficiently C-like. This criticism occurs in spite of the fact that C has not yet been invented.

1940s: Various “computers” are “programmed” using direct wiring and switches. Engineers do this in order to avoid the tabs vs spaces debate.

1957: John Backus and IBM create FORTRAN. There’s nothing funny about IBM or FORTRAN. It is a syntax error to write FORTRAN while not wearing a blue tie.

1958: John McCarthy and Paul Graham invent LISP. Due to high costs caused by a post-war depletion of the strategic parentheses reserve, LISP never becomes popular[1]. In spite of its lack of popularity, LISP (now “Lisp” or sometimes “Arc”) remains an influential language in “key algorithmic techniques such as recursion and condescension”[2].

1959: After losing a bet with L. Ron Hubbard, Grace Hopper and several other sadists invent the Capitalization Of Boilerplate Oriented Language (COBOL) . Years later, in a misguided and sexist retaliation against Adm. Hopper’s COBOL work, Ruby conferences frequently feature misogynistic material.

1964: John Kemeny and Thomas Kurtz create BASIC, an unstructured programming language for non-computer scientists.

1965: Kemeny and Kurtz go to 1964.

1970: Guy Steele and Gerald Sussman create Scheme. Their work leads to a series of “Lambda the Ultimate” papers culminating in “Lambda the Ultimate Kitchen Utensil.” This paper becomes the basis for a long running, but ultimately unsuccessful run of late night infomercials. Lambdas are relegated to relative obscurity until Java makes them popular by not having them.

1970: Niklaus Wirth creates Pascal, a procedural language. Critics immediately denounce Pascal because it uses x := x + y syntax instead of the more familiar C-like x = x + y. This criticism happens in spite of the fact that C has not yet been invented.

1972: Dennis Ritchie invents a powerful gun that shoots both forward and backward simultaneously. Not satisfied with the number of deaths and permanent maimings from that invention he invents C and Unix.

1972: Alain Colmerauer designs the logic language Prolog. His goal is to create a language with the intelligence of a two year old. He proves he has reached his goal by showing a Prolog session that says “No.” to every query.

1973: Robin Milner creates ML, a language based on the M&M type theory. ML begets SML which has a formally specified semantics. When asked for a formal semantics of the formal semantics Milner’s head explodes. Other well known languages in the ML family include OCaml, F#, and Visual Basic.

1980: Alan Kay creates Smalltalk and invents the term “object oriented.” When asked what that means he replies, “Smalltalk programs are just objects.” When asked what objects are made of he replies, “Objects.” When asked again he says “Look, it’s all objects all the way down. Until you reach turtles.”

1983: In honor of Ada Lovelace’s ability to create programs that never ran, Jean Ichbiah and the US Department of Defense create the Ada programming language. In spite of the lack of evidence that any significant Ada program is ever completed historians believe Ada to be a successful public works project that keeps several thousand roving defense contractors out of gangs.

1983: Bjarne Stroustrup bolts everything he’s ever heard of onto C to create C++. The resulting language is so complex that programs must be sent to the future to be compiled by the Skynet artificial intelligence. Build times suffer. Skynet’s motives for performing the service remain unclear but spokespeople from the future say “There is nothing to be concerned about, baby,” in Austrian accented monotones. There is some speculation that Skynet is nothing more than a pretentious buffer overrun.

1986: Brad Cox and Tom Love create Objective-C, announcing “This language has all the memory safety of C combined with all the blazing speed of Smalltalk.” Modern historians suspect the two were dyslexic.

1987: Larry Wall falls asleep and hits Larry Wall’s forehead on the keyboard. Upon waking Larry Wall decides that the string of characters on Larry Wall’s monitor isn’t random but an example program in a programming language that God wants His prophet, Larry Wall, to design. Perl is born.

1990: A committee formed by Simon Peyton-Jones, Paul Hudak, Philip Wadler, Ashton Kutcher, and People for the Ethical Treatment of Animals creates Haskell, a pure, non-strict, functional language. Haskell gets some resistance due to the complexity of using monads to control side effects. Wadler tries to appease critics by explaining that “A monad is a monoid in the category of endofunctors, what’s the problem?”

1991: Dutch programmer Guido van Rossum travels to Argentina for a mysterious operation. He returns with a large cranial scar, invents Python, is declared Dictator for Life by legions of followers, and announces to the world that “There Is Only One Way to Do It.” Poland becomes nervous.

1995: At a neighborhood Italian restaurant Rasmus Lerdorf realizes that his plate of spaghetti is an excellent model for understanding the World Wide Web and that web applications should mimic their medium. On the back of his napkin he designs Programmable Hyperlinked Pasta (PHP). PHP documentation remains on that napkin to this day.

1995: Yukihiro “Mad Matz” Matsumoto creates Ruby to avert some vaguely unspecified apocalypse that will leave Australia a desert run by mohawked warriors and Tina Turner. The language is later renamed Ruby on Rails by its real inventor, David Heinemeier Hansson. [The bit about Matsumoto inventing a language called Ruby never happened and better be removed in the next revision of this article—DHH].

1995: Brendan Eich reads up on every mistake ever made in designing a programming language, invents a few more, and creates LiveScript. Later, in an effort to cash in on the popularity of Java the language is renamed JavaScript. Later still, in an effort to cash in on the popularity of skin diseases the language is renamed ECMAScript.

1996: James Gosling invents Java. Java is a relatively verbose, garbage collected, class based, statically typed, single dispatch, object oriented language with single implementation inheritance and multiple interface inheritance. Sun loudly heralds Java’s novelty.

2001: Anders Hejlsberg invents C#. C# is a relatively verbose, garbage collected, class based, statically typed, single dispatch, object oriented language with single implementation inheritance and multiple interface inheritance. Microsoft loudly heralds C#’s novelty.

2003: A drunken Martin Odersky sees a Reese’s Peanut Butter Cup ad featuring somebody’s peanut butter getting on somebody else’s chocolate and has an idea. He creates Scala, a language that unifies constructs from both object oriented and functional languages. This pisses off both groups and each promptly declares jihad.

Footnotes

1. Fortunately for computer science the supply of curly braces and angle brackets remains high.

2. Catch as catch can—Verity Stob

James Iry says: “If cars were built like software then... well, I don't know squat about building cars so who knows. It might be kinda cool. But probably not.” Read other things James says here.

Send the author your feedback or discuss the article in the magazine forum.

Rear Window

A Parting Shot

The front and back covers of Ted Nelson’s Computer Lib, the inimitable book that served as the manifesto for a revolution.

“In 1974, before the Altair was announced, before there was anything that could be accurately called a personal computer, Ted Nelson self-published Computer Lib, in which he explained computers to the lay reader in clear and witty prose and laid out the political agenda for the personal computer revolution.

“‘You can and must understand computers NOW,’ the book trumpeted. ‘Computer power to the people! Down with cybercrud!’ The essays, jokes, thoughts, and anecdotes were pasted together in an anarchic layout that made you feel that ideas were jumping off the page.

“The book reached all the right people. It influenced Steve Wozniak. Ed Roberts had it on his desk while MITS was building the Altair. Computer Lib was Thomas Paine’s Common Sense for the computer hobbyist. Lee Felsenstein wrote, ‘Ted succeeded with Computer Lib to rally a rabble of latent crackpots into an anarchistic army which breached the sanctum of Official Computerdom and brought computers to everyone.’”

(From Fire in the Valley: The Making of the Personal Computer, second edition by Paul Freiberger and Michael Swaine. Third edition coming soon from Pragmatic Bookshelf.)

Table of Contents

PragPub 2013-09: Issue #51

On Tap

Choice Bits

Rothman & Lester

Playing with the Play Framework

Concurrent Programming in Clojure

The Automated Tester

Hadoop

Puzzle

Calendar

Bookshelf

Guest Column

Rear Window