PragPub 2011-03: Issue #21

Table of Contents

Features

- Punk Rock Languages
by Chris Adamson
In an era of virtual machines and managed environments, C is the original Punk Rock Language.

- Testing for Web Services
by Noel Rappin
Just because you are using an external web API for your site doesn’t mean that BDD principles need to go out the window.

- Testing for the Cloud
by Adam Goucher
The three big differences cloud computing brings with it are really just modern twists on old practices.

- Software Volatility
by Tim Ottinger, Jeff Langr
The fourth in this four-part series of Big Ideas in software development.

- When Did That Happen?
by Dan Wohlbruck
How a great mathematician solved a classic problem and laid the theoretical foundation for modern computers.

Departments

- Up Front
by Michael Swaine
A Polemic on Programming and Punk Rock

- Choice Bits
A few selected sips from the Twitter stream.

- Way of the Agile Warrior
by Jonathan Rasmusson
If companies simply trusted their people, a lot of the waste on software projects would go away.

- The Quiz
by Michael Swaine
A monthly diversion at least peripherally related to programming.

- Calendar
After a slow-ish winter, things are really heating up this spring.

- Shady Illuminations
by John Shade
John considers why Microsoft jumped onto Nokia’s burning platform.

Except where otherwise indicated, entire contents copyright © 2011 The Pragmatic Programmers.

Feel free to distribute this magazine (in whole, and for free) to anyone you want. However, you may not sell this magazine or its content, nor extract and use more than a paragraph of content in some other publication without our permission.

Published monthly in PDF, mobi, and epub formats by The Pragmatic Programmers, LLC, Dallas, TX, and Raleigh, NC. E-Mail support@pragprog.com, phone +1-800-699-7764. The editor is Michael Swaine (mailto:michael@pragprog.com). Visit us at http://pragprog.com

ISSN: 1948-3562

Up Front

Punk Rock Languages
by Michael Swaine
This month Chris Adamson delivers a sprited polemic on what he calls punk rock languages. You may agree or disagree with his arguments, but you can’t deny that they cut to the heart of how we do our work as programmers. If you find the subject interesting, you can follow further punk rock programming thoughts at @punkrockcode.

Like most software disciplines, testing gets tricky at the boundaries. It’s great when the code is all under your control, but in the modern programming environment of Web services and the cloud, you have to deal with some extra issues. In this issue, Noel Rappin takes on testing for Web services and Adam Goucher looks at what it takes to test for the cloud.

Tim Ottinger and Jeff Langr are back with another in their series on big ideas in software, this time looking at software volatility. Theorizing that the more often a file changes, the more likely it is to result in a code defect, they consider how to reduce and deal with software volatility.

Jonathan Rasmusson’s “Way of the Agile Warrior” deals with a fundamental issue this month: trust, how to earn it, and the freedom and power it buys you.

Dan Wohlbruck’s back with another history article this month, the Quiz returns with a tricky puzzle about relationships, and John Shade ponders train wrecks and burning platforms.

Choice Bits

We thought you might like to know...

Brian Marick is taking over the Twittersphere.

What’s Hot

Top-Ten lists are passé—ours goes to 11. These are the top titles that folks are interested in currently, along with their rank from last month. This is based solely on direct sales from our online store.

1

1

Agile Web Development with Rails

2^

8

Hello, Android

3^

4

The RSpec Book

4^

NEW

Rails Test Prescriptions

5v

2

Crafting Rails Applications

6v

3

HTML5 and CSS3

7v

5

Programming Ruby 1.9

8^

11

The Agile Samurai

9v

7

Seven Languages in Seven Weeks

10v

6

Agile in a Flash

11v

10

Using JRuby

This month, as an experiment, we left out all tweets dealing with weather, travel, the Super Bowl, the Super Bowl halftime show, or what music people were listening to when they tweeted.

Life Is Good...

This morning I am thankful for Rooibos. A steaming mug, an almost-impossible technical task, and the whole day ahead of me. Happy sigh…. — @KentBeck

Heading for home after a week of fire hydrant brain transfer on the coolest project ever. Plus, suitcase is filled with hardware. Awesome! — @jimweirich

Hacking on Cocoa. Watching Boardwalk Empire. Happy. — @alancfrancis

First time skiing in a helmet. Built-in headphones make it all worth while. Sun is shining, #phish is in my head. — @scottdavis99

The secret society continues to treat me well. — @JEG2

“Thank you! Your first invoice will be billed on December 31, 1969.” Sweet! — @ryanirelan

…Until the Pain Sets In

Dry corporate indoctrination right after lunch. Brutal. — @undees

IM’ed to me during a boring conference call: “This reminds me a scene from Attack of the Killer Tomatoes.” — @squarepegsys

Paul’s and my tolerance for lameness is pretty high, but one hour of Lexx 2.0 left us twitching uncontrollably on the floor. — @marick

I’ve always heard that you shouldn’t read slides to your audience. It’s even worse when it’s the first time you’ve read them. — @daveklein

Sadly, The Complete Idiot’s Guide to Supercharged Kama Sutra Illustrated does not have a “Look Inside!” at Amazon. — @marick

“If it wasn’t for disappointment I wouldn’t have any appointments.” — @noradio

Just when my sense of self worth couldn’t get worse, @pandora_radio pegs me as someone who would enjoy that cheesy Just Breathe techno song. — @noradio

Oh crap, I can’t go shopping at Apple. I’m wearing my Apple hoodie from the mothership. — @invalidname

Seriously... why does everything in New Jersey close at 9 pm? — @rit

On days like this I just think, why didn’t I become a doctor like my mom wanted me to? — @jeffcohen

There are times when Beethoven Symphony 9, last movement, just pisses me off. — @marick

Agility Enters Teen Years, Becomes Awkwardness

Agile process in the absence of agility is like wings on a penguin; you can flap but not fly. — @venkat_s

People are attracted to large companies because of their stability. One can see how such people are slow to adopt agility. — @tottinge

Feeling the tension between agile, emergent design and considered, consistent interface design. — @ronhitchens

“In the long run every program becomes rococo—then rubble.”—Alan Perlis (cs.yale.edu/quotes.html) via: @jakedonham — @noradio

Refactoring is a mighty river. Take one step in along the edge which leads you to another and pretty soon you’re swept out to sea. — @KentBeck

Great process is so easy. Just like losing weight or getting in shape. — @jaredrichardson

It’s not the right set of tools that help us succeed, it’s the right set of tools in the hands of the right set of people. — @venkat_s

“Leadership” is a term of art in Agile. We use it to reassure alpha primates that they’ll get to keep their status. — @marick

CSM’s are now walking out of a 2 day course and calling themselves Agile Coaches? AWESOME! — @davidjbland

They Shoot Movies, Don’t They?

“The Road”—Well, that was cheerful. Great sound design. — @marick

Tweets of Wisdom

“The problem is not that there are problems. The problem is expecting otherwise & thinking that having problems is a problem.”—T. Rubin. — @estherderby

Beware certainty: “Doubt grows with knowledge.”—Goethe — @PragmaticAndy

“Reasoning will never make a man correct an ill opinion which by reasoning he never acquired."—Jonathan Swift — @marick

The more I understand about spacetime, the less I understand time. — @mtnygard

If your shirt buttons pucker, you need a bigger shirt. #observationfromlastweek — @adamgoucher

Victory belongs to those in the fight; you can’t win if you don’t play the game. — @bdudney

Just because an ideal can’t be met, is no reason to dismiss the ideal. — @jwgrenning

You’ll never be up to date, but you can be ahead. — @KentBeck

A gentleman is someone who can play the accordion but doesn’t. — @PragmaticAndy

Does Brian Need an Intervention?

Why are there never zombie dogs? Zombie dogs would be cool. — @marick

@marick “Last Man on Earth” with Vincent Price had one. — @tottinge

Apparently the fictional universe is awash with zombie dogs. — @marick

“Zombie stories are inherently Hobbesian.”—the social theories behind “The Walking Dead.” http://bit.ly/hpInzP — @marick

“The legal status of a zombie is an interesting question.” Ain’t the internets grand? http://bit.ly/enloYF — @marick

While I liked “The Walking Dead” this author has a point: it’s not (yet) a zombie show. http://bit.ly/fvhIlj [He overlooked some scenes tho] — @marick

Does Brian have a thing about zombies, or what?

47% of parrots are left-handed, 33% right-handed, and 20% ambidextrous. (It varies by species.) http://bit.ly/hmZ6UE — @marick

That’s better.

Meat-eating furniture: http://n.pr/hQPOq9 — @marick

That’s not.

This Coding Life

West coast is starting to arrive at the office; it’s now email o’clock it seems. — @adamgoucher

Hi, my name is Chris, and I NSLog() too much. — @invalidname

“man ascii” has been my friend for more years than any living human. — @marick

I just discovered droidwords. Now I have to convince my wwf buddies to switch to Android. — @pragdave

Caught up on 2 days worth of tweets. Strangely, I’ve had two productive days. #hmmm — @adamgoucher

OH: Why are all vocal proponents of Software Craftmanship consultants? #speakerconf — @aslak_hellesoy

When is the last time you read something in an ACM or IEEE journal that changed how you program or the tools you use? Ever? — @gvwilson

Is it weird that I download all three PragProg ebook formats to the same device? And use all three? — @noelrap

I wish my IDE had an “I’m Feeling Lucky” button. — @chadfowler

I’m often surprised by what Rubyists think they need a gem for. — @JEG2

Amending previous tweet about how fast FF4 is. It’s fast during the times when it’s not beachballing. — @jeffcohen

Do Swedish Mac users call the command key the “campground key”? #just_wondering — @jeff_lamarche

The Internet is now full—RT @IPv4Countdown: The number of allocatable IPv4s just dropped to zero. #IPv6 — @benrady

Phew. Sure glad there was an IP address left for me to get on the Internet today. That was close. — @bphogan

So... Canada’s ISPs have implemented very low caps on monthly data. So users are now paying to see ads. This changes Google’s model. — @jaredrichardson

Canadians: have things downloaded in America, burned to disk, and shipped to you, avoiding bandwidth caps: http://twitpic.com/3vt1cs — @marick

“@jm: Figuring out really concentrated but non-contrived examples *has* to the hardest part of writing a tech book.” // seconded — @noelrap

I’m kinda glad I don’t have a way to know how much I underbill due to my hatred of time tracking. — @adamgoucher

I’m going to coin the term SPDA: Sales Pitch Driven Architecture. — @wfaler

Untold story of the Egypt crisis: thousands of farms on Facebook are going untended. — @jamesmarcusbach

I wouldn’t be surprised if they wrap the Presidio in a Faraday cage this year after what happened last year. — @jeff_lamarche

Getting the Hang of this “Real Life” Thing

So, listening to the Indigo Girls while driving a big honkin’ Suburban is kinda like a carbon offset, isn't it? #guiltyHippy — @scottdavis99

Anyone else feel like lamb tastes sort of like wool? Need backup on my claim. — @brynary

Have you tried almond milk? Sorry, goats, but the almonds are giving you a run for your money. — @chadfowler

As requested, pictures of deep fried peach pie. twitpic.com/3w9e12 — @rit

21st century problem: strangers who share enough of my genome that we are probably related want to meet. — @neal4d

Apparently “tedia” is not a real word. What's a good one-real-word alternative? #lazytwitter — @dchelimsky

After showing http://youtu.be/H7FEK3_6QaE to the office the general consensus is none of us should have made it out of our 20s. — @adamgoucher

Wernher Magnus Maximilian Freiherr von Braun: As if being a rocket scientist weren’t bad ass enough. — @noradio

Thankfully, the poolside bar opened at noon. — @jeff_lamarche

This month we were captivated by the tweeting chops of Chris Adamson, James Marcus Bach, Kent Beck, David J Bland, David Chelimsky, Jeff Cohen, Scott Davis, Ian Dees, Esther Derby, Bill Dudney, Neal Ford, Chad Fowler, Alan Francis, Adam Goucher, James Edward Gray II, James Grenning, Aslak Hellesøy, Bryan Helmkamp, Ron Hitchens, Brian P. Hogan, Mike Hostetler, Andy Hunt, Ryan Irelan, Dave Klein, Jeff LaMarche, Brendan W. McAdams, Brian Marick, Marcel Molina, Michael Nygard, Tim Ottinger, Ben Rady, Noel Rappin, Jared Richardson, Venkat Subramaniam, Dave Thomas, Jim Weirich, wfaler, and Greg Wilson. Feel free to cast a critical eye on our twitterings. It’s only fair.

Way of the Agile Warrior

The Speed of Trust
by Jonathan Rasmusson
One of the biggest culture shifts in our industry over the last ten years is that, for the first time, we have stopped lying to each other.

The best time to plant a tree is twenty years ago. The second best time is today.–Chinese Proverb

One lubricant that underlies just about everything we do in work, play, and creating great software is trust.

When you’ve got it, you become empowered. Free to take initiative, make your own decisions, and be your own boss.

Without it, life can be tough. You’ll be asked to justify everything you do, your decisions will continuously be scrutinized, and you’ll be kept on a very short leash.

In this article we’re going to look at what a lack of trust costs us on projects, what we can do to earn our customers’ trust back, and what an environment where elevated trust has been earned looks like.

Waste

When you break the big laws, you do not get liberty; you do not even get anarchy. You get the small laws.–G.K. Chesterton

If you work in a typical office environment, you probably don’t have to look too hard to find examples of waste.

It might be waste in the form of unnecessary meetings. It might be waste in the form of documentation that no one will ever read. Or it might be waste in the endless series of sign-offs, hand-offs, plans, documents, templates, signatures, and compliance hoops you have to jump through just to push your software live.

And of course the ultimate form of waste is spending six months writing software that will never see the light of day because the company isn’t able to pull the right people together to release it into production (true story!).

Whatever form it takes, unchecked waste can be demoralizing and soul-sapping. It takes the best out of our people, and drives some to seek employment elsewhere.

But waste isn’t the cause. It’s the symptom. A lot of the waste we have to put up with in our daily lives is a byproduct of a lack of trust.

If companies simply trusted their people, a lot of the waste we see on software projects would go away. We wouldn’t need 10,000 meetings before pushing our software live. We wouldn’t need a six-week regression-testing cycle for two days of bug fixing. And we certainly wouldn’t need those countless checks and balances that ensure that nothing bad happens by ensuring that nothing ever gets done!

But trust is a two-way street. A lot of that waste is there because we haven’t demonstrated enough competency as an industry to show that we don’t need it.

If that’s going to change, it’s up to you.

If you want to be empowered, listened to, respected, and trusted to take initiative and get things done, you’ll need to get good at earning your customers’ trust.

Here are three simple but powerful ways you can go about doing that.

Three Ways to Earn Your Customers’ Trust

“Verifying that my software still works slows me down.” Yes, code can be delivered much faster if having it work is optional–@jasongorman

1. Take away the quality lever.

I was on a project once where a developer told me that he was told explicitly that he wasn’t allowed to test. The project manager said there wasn’t enough time and they simply had to build it and get it out there (bonuses were on the line). The results were about what you’d expect.

One way to nip this kind of behavior in the bud is to take away the quality lever from the customers. Simply don’t give it to them.

Look. Your customers expect you to test their software. It’s not only unprofessional not to test, it’s going to hurt you inside if you don’t. You’ll feel bad about the work you’ve done, and you’ll probably take the heat for it when things blow up.

So don’t give them the quality lever. Just explain that writing untested crappy software that doesn’t work isn’t what we do around here. There is no quality lever.

2. Deliver something of value every week.

This is probably the best way I know to get a customer on your side. Simply break their system down into a whole bunch of small chunks and then, like a metronome, start delivering their most important features as working, tested software every couple of weeks.

Doing this helps you build trust in a number of fundamental ways.

First, it shows that you know what you are doing. There’s still enough dysfunctional software being written that if you show how it’s done and deliver them a stripped-down (but fully tested), working version of their system in a couple of weeks (instead of a couple of months), you’ll earn huge street cred.

Second, they are going to realize (maybe for the first time ever) that they play an important role in the success of the project. Instead of having a project done to them, they enjoy being in the driver’s seat for a change. All they have to do is answer a few questions, tell you what the most important stuff is, and poof! It gets done.

Third, (and most important) it’s hard not to appreciate a team that shows up every week and delivers something of value. Sure, you’d like them to go faster. Yes, you wish they could get more done. But at the end of the day they show up and they deliver. ’Nuff said.

3. Be open and honest about the schedule.

Agile is pretty open and honest about things like schedule. We don’t make commitments until we are relatively sure we can meet them, and we don’t sugar-coat it or hide the state of our projects. We put it out there for all to see.

This is perhaps one of the biggest culture shifts that has taken place in our industry over the last ten years. For the first time, we as adults have stopped lying to each other.

Being open and honest about schedule means not making premature commitments about scope and time. It means leveling with your customer and telling them that you aren’t sure exactly how much you’ll be able to get done by the end of the project because of the sheer number of unknowns (like their level of participation in the project).

Some customers find this kind of candor refreshing—usually because they face the same problems when they deal with their bosses. But by being open and honest about things like schedule we can then focus on delivering value instead of spending valuable shareholder capital thinking about how we are all going to cover our butts when things blow up.

Leveling Up

Few things can help an individual more than to place responsibility on him, and to let him know that you trust him.–Booker T. Washington

Agile projects aren’t perfect. We still do things, set expectations, and plan in ways that wouldn’t be necessary if we had complete trust with our customers.

Here are some examples of how agile teams have morphed their practices once they have earned very high levels of trust with their clients.

1. Questions about estimates go away.

Once you’ve proven you can deliver, most questions about estimates go away.

Your customers know you can deliver 4-6 stories per iteration. They trust you to get the most important stories done every iteration, and that’s good enough for them.

Some agile teams are even able to dispense with the estimation process altogether, by simply sizing their stories roughly the same and delivering the same number each iteration. Nobody cares whether something has 1, 3, or 5 parts—it doesn’t really matter. More time spent adding features, less time estimating and planning.

2. UAT becomes a formality.

Formal User Acceptance Testing (UAT) isn’t an agile practice. It’s a relic from a software delivery process that became entrenched due to a lack of trust (a lack of trust that we as an industry earned).

We still do it simply because we haven’t yet demonstrated that it is no longer necessary.

The good news, though, is that for many teams, formal UAT cycles at the end of a release are becoming a thing of the past.

When you get used to testing and releasing software regularly, a lot of the fear and waste that traditionally comes with releasing software goes away. It becomes breathing. It’s just something you regularly do.

I don’t recommend teams new to agile start here. You’ve got to earn the right to lighten up on UAT before you consider dropping it. But there are a lot of companies for whom UAT is quickly becoming a thing of the past and they are able to redirect that energy in more useful ways.

3. Mistakes are easily forgiven.

You’re going to make mistakes. It comes with the game, and it comes from continuously trying to improve. But when you do make a mistake, customers are way more forgiving when you’ve demonstrated that you have their best interests at heart and are busting your hump for them each and every iteration.

I remember once doing a print story for a construction project I was working on. The story was pretty simple (print the work permit for the construction site) but my first attempt was less than stellar (and I knew it as soon as I demoed it to my customer, the director of construction).

But because I had just spent the last five iterations delivering really valuable stuff, I had banked enough good will that it wasn’t a big deal. He just said try again and we’ll take a look at it again next week. I didn’t let him down.

Summary

Trust is speed. If you want to go faster and have more fun on your project, you are going to need to earn the right to work in a high-trust environment. Three things you can do to get there are:

1. Take away the quality lever.

2. Deliver something of value every week.

3. Be open and honest about the schedule.

It won’t happen overnight. And you can easily lose it if you are not careful. But the dividends it pays justify the investment. You will be happier, and instead of dreading the thought of coming in to work and being told what to do, you’ll be free to take the initiative and empowered to direct yourself and your team in how you’d like to work.

To learn more about the impact of trust and how it affects your life, I highly recommend reading Steven Covey’s excellent book, The Speed of Trust.

Jonathan Rasmusson is the author of The Agile Samurai. As an experienced entrepreneur and former agile coach for ThoughtWorks, Jonathan has consulted internationally, helping others find better ways to work and play together. When not coaching his sons’ hockey teams or cycling to work in the throes of a Canadian winter, Jonathan can be found sharing his experiences with agile delivery methods at his blog, The Agile Warrior. For more information on the Agile Inception Deck check out The Agile Samurai. Send the author your feedback or discuss the article in the magazine forum.

Punk Rock Languages

A Polemic
by Chris Adamson
That C has won the end-user practicality battle is obvious to everyone except developers.

The year is 1978, and the first wave of punk rock is reaching its nihilistic peak with infamous U.K. band the Sex Pistols touring the United States and promptly breaking up by the time they reach the West Coast. Elsewhere, Brian Kernighan and Dennis Ritchie are putting the finishing touches on their book The C Programming Language, which will become the de facto standardization of the language for years. While totally unrelated, these two events share a common bond: the ethos of both punk rock and C have lasted for decades, longer than anyone in 1978 could possibly have imagined.

And in many important ways, C is the programmer’s punk rock: it’s fast, messy, dangerous, and perfectly willing to kick your ass, but it’s also an ideal antidote to the pretensions and vanities that plague so many new programming languages. In an era of virtual machines and managed environments, C is the original Punk Rock Language.

Lightning Strikes (Not Once But Twice)

This is a chord, this is another, this is a third. Now form a band.–Cartoon in Sideburns magazine

C has all the power of assembly language combined with all the elegance and poetry of... assembly language.–Erica Sadun

Punk rock emerged as a reaction to the increasingly self-indulgent and misguided musical trends of its time, with progressive rock insisting on merging in influences from jazz and classical music (typified by Emerson, Lake and Palmer performing Mussgorsky’s “Pictures At An Exhibition” as a rock album), and second-rate guitarists noodling away in awe of Hendrix and Clapton with what Joey Ramone called “endless solos that went nowhere.” Punk stripped away the nonsense of the times by focusing on faster, shorter, simpler songs. Preparing America for the Sex Pistols, Time magazine wrote “in the U.S. the movement is more purely musical: groups like the Ramones, Talking Heads, Television and Richard Hell and the Voidoids have rejected the rococo sophistication of much 1970s rock and turned back to basic buzz and blast.”

C’s development was, if anything, the opposite: it was meant as an alternative to programming the PDP-7 in assembly. In Dennis Ritchie’s history of the language, he describes it as the result of several attempts to provide higher-level languages for the platform, a successor to BCPL and B (which Ritchie describes as “BCPL squeezed into 8K bytes of memory and filtered through [Ken] Thompson’s brain”). But this development wasn’t driven by academic niceties or intellectual noodling. It was about getting stuff done. As Ritchie recalls:

“By 1971, our miniature computer center was beginning to have users. We all wanted to create interesting software more easily. Using assembler was dreary enough that B, despite its performance problems, had been supplemented by a small library of useful service routines and was being used for more and more new programs.”

Punk is famous for needing little more than three chords. C basically has three types—int, float, and char—embellished only by being able to increase their bit-length as longs and doubles. It’s gallingly simple to modern eyes, but hones quite closely to what’s actually on the CPU. Look through the x86 or ARM7 instruction set and you’ll find no references to Unicode strings or “objects” of any kind; what you get is simple arithmetic and logical operations that work on operands of 8, 16, and (maybe) 32 bits. C is simple because, at their core, computers are simple.

Of course, you can build any amount of complexity you like atop this simple foundation. The American National Standards Institute (ANSI), spent most of the 80s codifying the C Standard Library, the collection of always-available libraries to perform essential operations like memory management, string manipulation, and networking. And you can then build your own libraries atop these. But as long as you’re still working in C, it’s fairly easy to keep your code running close to the metal, working with simple structures and function calls. Ritchie again:

“Despite some aspects mysterious to the beginner and occasionally even to the adept, C remains a simple and small language, translatable with simple and small compilers. Its types and operations are well-grounded in those provided by real machines, and for people used to how computers work, learning the idioms for generating time- and space-efficient programs is not difficult. At the same time the language is sufficiently abstracted from machine details that program portability can be achieved.”

With the rise of object-oriented programming, new languages co-opted C to provide OO features: C++ has proven the most popular over time, while the Smalltalk-flavored Objective-C might well have remained a footnote had it not been adopted by NeXT, which led to its use in Apple’s popular products. Even so, within these languages, you still mix procedural C calls with the object-oriented additions.

Police On My Back

Ever get the feeling you’ve been cheated?–Johnny Rotten, January 14, 1978

Segmentation fault–Command-line output of a C program with a broken pointer.

The largely unmanaged freedoms of C, C++, or Objective-C provide the opportunity to cause extraordinary havoc with your code, intentionally or not. What’s most distinctive about C compared to other popular languages today is the exposure of memory pointers, and the disasters that come with their misuse. But then again, this is an intrinsic part of how computers work: you have a big block of memory (real or virtual), containing the system, your program, and other programs. If you can address any point in memory, and you read or write to an address that your program doesn’t own, what should happen? One option is to allow programs to gallivant through each other’s memory space. As entertainment, this is “Core Wars”; as a coding error, it’s a security nightmare. In the real world, such mistakes are prohibited by operating systems, and the offending program is terminated.

A lot of people find this price of failure too high, even though the fix is to just write better code, and instead opt for an even more draconian option: taking away the keys. Most modern languages provide their own memory management paradigms, and never allow the developer to see a pointer. Take a look at the monthly TIOBE Programming Community Index, and you’ll see three kinds of languages in the top 10:

- C and its OO derivatives: C, C++, Objective-C

- Interpreted languages: Python, PHP, Visual Basic, Perl

- Virtual Machine languages: Java, C#

The latter two groupings add profound new layers to the programming model: interpreters to parse and interpret program code, or virtual machines that create an entirely new execution environment for bytecode. Importantly, in both of these cases, the programmer’s code is not the executable: run ps and you’ll see that the interpreter or VM is what’s running, not your code per se.

As we build abstraction upon abstraction, we get further and further away from what’s really going on in the CPU and memory. In 2011, we now have popular scripting languages like Scala and Groovy whose interpreters run in the Java Virtual Machine. We’ve gone from working with the same data types that the CPU processes to an elaborate stack of layer after layer of artifice and illusion. If C is a punk band, blaring out the two or three basic types that the CPU understands, this new style of programming is Rick Wakeman hiring the London Symphony Orchestra to play backup for his synthesizer.

Now we have an Anti-IF Campaign, which vows we will get more agile code by casting off if statements, replacing them (or at least moving the problem) through the use of elaborate subclassing and other language novelties.

There’s a price to pay for all this, the “abstraction cost” of creating general-purpose interpreters and virtual machines. Consider “Hello World” in C, Ruby, and Java, with an added sleep so the program doesn’t immediately terminate:

#import "stdio.h"

int main(int argc, char *argv[]) {

printf ("hello, world!\n");

sleep (30);

}

puts ’Hello world’

sleep (30)

public class HelloWorld {

public static void main (String[] args) {

System.out.println ("Hello World!");

try {

Thread.sleep (30000);

} catch (InterruptedException ie) {}

}

}

Run all three of these and take a look at the resources they demand for this trivial program. As the figure illustrates, on Mac OS X the C program takes 356 KB, the Ruby interpreter 1.6 MB, and the Java Virtual Machine 25 MB. Java also starts up 24 threads for this trivial program.

Nobody Likes You

As someone remarked: There are only two kinds of programming languages: those people always bitch about and those nobody uses.–Bjarne Stroustrup

We are the cries of the class of ’13 / Born in the year of humility / We are the desperate in the decline / Raised by the bastards of 1969—Green Day, “21st Century Breakdown”

Fans defend scripting and VM languages with arguments based around the idea of “programmer productivity”: that faster CPUs and cheaper memory make it more economical to accept these abstraction costs, in the interest of getting working code out the door faster. Java has long been accused of tolerating bad programmers gladly, but that knock against it applies equally well to all the scripting and VM languages.

Furthermore, look who’s making these languages popular: it’s the web developers. All of the scripting and VM languages in the TIOBE top 10 are primarily associated with developing web applications. Yet when we cast our gaze over to the desktop and the mobile device, the scripting and VM languages largely disappear. PHP is clearly meant for web use, but Ruby and Python are general-purpose enough that they could be lashed to a UI framework to power desktop apps. In fact, Apple did just this in Mac OS X 10.5, making Ruby and Python first-class languages for Cocoa development. But in 10.6, the Ruby and Python templates have disappeared from Xcode. It appears that few developers took the bait, just as Cocoa-Java found few takers in earlier versions of Mac OS X and was retired.

(For those still willing to give scripting for OS X a go, Brian Marick’s Programming Cocoa with Ruby is a fine introduction to this, wisely presenting Cocoa to the Ruby developer, instead of Ruby to the Cocoa developer.)

The VM languages have had some success on the desktop, most obviously C# in the Windows realm, where the go-to language was once Visual Basic, whose compiled P-code was also typically run by a VM. But Desktop Java’s long miserable history of underachievement tarnishes the viability of desktop VMs, and with one exception (highlighted later in this article), there’s little apparent real-world use of scripting languages on the desktop. Even on Linux, where scripting languages thrive on the server, GNOME, GIMP, vi, and emacs are written in C, while KDE, Firefox, Thunderbird, VLC, and OpenOffice.org are in C++.

Meanwhile on smartphones, the C languages are the only real option for Apple’s market-leading iOS devices. And while Android uses the not-really-Java-nudge-wink “Dalvik” VM, it offers the Java Native Interface as an escape slide to C for performance-hungry applications like games.

The takeaway is that when your code is running in the immediate presence of the user—on his or her desktop, phone, game console, or tablet—the advantages touted by interpreted and VM languages suddenly diminish. Without network latency to hide behind, and with resources at a premium (particularly on the mobile device), the extravagances of the webapp stack suddenly becomes a lot less desirable. Users see slow, memory-intensive applications on their systems and say, as the last Clash album did, “Cut The Crap.”

That C has won the end-user practicality battle is obvious to everyone except developers. Paul Graham’s essay “The Hundred Year Language” manages to not mention C, despite the fact that it’s already got 40 years under its belt, and has never ranked lower than #2 on the TIOBE list.

Come Out and Play

Well, you know, for me punk has always been about doing things your own way. What it represents for me is an ultimate freedom and sense of individuality. Which basically becomes a metaphor for life and the way you want to live it.–Billie Joe Armstrong

C is quirky, flawed, and an enormous success. While accidents of history surely helped, it evidently satisfied a need for a system implementation language efficient enough to displace assembly language, yet sufficiently abstract and fluent to describe algorithms and interactions in a wide variety of environments.–Dennis Ritchie

One of the defining traits of punk is the do-it-yourself (DIY) ethic, a rejection of the need to buy products or use existing systems, and instead to attend to your own needs. This attitude clearly suits C programming as well. Indeed, it defines the language’s history, with the messy incompatibilities of early versions of the language on different hardware, and the development of wide-ranging libraries that don’t always use like-minded idioms or patterns. As the decades have rolled on, a motley collection of C code has been developed by programmers around the world and across the decades, providing everything from XML parsing to 3D graphics, even if a libxml call looks nothing like an OpenGL one.

Some of these libraries are open-source and part of GNU. Others are commercial and liberally licensed. And there’s an incalculable amount of proprietary C code hidden away in businesses around the world. It doesn’t matter: C has no licensing concerns of any kind. C isn’t owned by a company (like Java, Objective-C, and C# effectively are), but neither is it a slave to the FSF or any other political movement. Any programmer can pick it up and use it for any cause, good or evil, that he or she sees fit. No $99 developer program memberships, no license agreements, no genuflecting to Richard Stallman... just code, compile, and run.

The Kids Aren’t Alright

Punk is musical freedom. It’s saying, doing and playing what you want. In Webster’s terms, ‘nirvana’ means freedom from pain, suffering and the external world, and that’s pretty close to my definition of Punk Rock.–Kurt Cobain

Having made the case for C as a sort of “punk rock language,” it’s worth asking what other languages share C’s traits of ruthless usefulness. Let’s continue the musical analogy: once the original wave of punk spent itself, its values were adopted by a number of successor genres, like post-punk, power pop, and new wave. Bands like The Clash and Television combined punk’s immediacy with better chops and more outside influences. The Offspring and Green Day revived punk in the 90s with catchy melodies, launching an era of pop punk that persists to this day. In the early 90s, punk contributed to grunge, which helped end the era of glammy hair metal (sorry, Jon Bon Jovi and David Lee Roth). As long as there’s a musical trend that’s too full of itself, punk or one of its spin-offs will be waiting to tear it down.

What’s the least pretentious language in widespread use today? Surely it has to be the much-derided JavaScript. Though it’s one of many languages that inherits much of its syntax from C, it is an interpreted language, object-oriented and packed with novel features like dynamic typing, first-class functions, and closures.

It is possible to write elegant, clever code with JavaScript. In true punk rock fashion, almost nobody does.

JavaScript in anger is a brutally efficient language. Used primarily in the context of a browser, it is often used to slash out quick-and-dirty onClick() implementations. Despite being burdened with the confusing “Java” name, no class declarations, package and visibility statements, or other niceties are required; often enough, JavaScript code is rammed into the attribute of an HTML element.

Older developers bemoan the lost days when computers shipped with BASIC built in, or of learning scripting with HyperCard. “Where,” they ask, “are today’s kids learning how to program?” missing the browser that’s staring them in the face. The web browser offers a JavaScript interpreter and enough toys in the DOM to keep the young programmer occupied for years. Anything that could be written in AppleSoft BASIC can just as easily be created with JavaScript and the <canvas> tag, and the presence of the tree-structured DOM and JavaScript’s support for regular expressions give the young programmer access to more significant and interesting data structures than us old folks were ever going to develop on a middle school VIC-20. And thanks to the arms race between browser developers, JavaScript performance continues to improve by leaps and bounds.

Shouldn’t beginning programmers learn from a more cleanly structured language? That’s always been the complaint, but the slop of BASIC didn’t harm my generation of developers (and mandatory Pascal didn’t clean us up), so why should we fear that JavaScript will pollute the next generation? If anything, JavaScript’s freedom to be as messy and unstructured as you want is a gift: once the mess gets out of hand, the young developers will have to figure out how to bring in classes, structure, OO, and the other tools that JavaScript provides to set things right. But it’s DIY: JavaScript lets you be a good programmer, but doesn’t force you to. How you structure your code and how you deal with problems is up to you.

Institutionalized

Don’t do anything by half. If you love someone, love them with all your soul. When you go to work, work your ass off. When you hate someone, hate them until it hurts.–Henry Rollins

I’ve probably written an ungodly amount of C code, but I hate C with a passion. I’ve been writing a lot of Objective-C code lately... which just fuels my hatred for C. .h files are absolutely the work of the devil.–James Gosling

Not every language can be “punk.” Not every language should—sometimes you do want the protections and comforts of the Web scripting languages, just as sometimes you’d rather listen to The Beatles than Black Flag. So, can we decide where to draw the line? Since C and JavaScript seem to have become punk rock languages through their evolution and use, not from their design, it might be more useful to list the traits that we associate with punk rock languages.

Here are some signs that you’re working with a punk rock language:

- Owned by nobody—As pointed out before, C isn’t owned by any company, nor is it controlled by the FSF or any other part of the free software movement (though it was clearly instrumental in the development of Unix, Linux, and the GNU software stack). Similarly, JavaScript is nobody’s property. It’s OK for these languages to have formal specifications, and to accept contributions from all sources, but no one body controls them, and that’s a great thing.

- Allows the user to apply as much or as little structure as he or she chooses—It’s not punk to keep the training wheels on all the time.

- Is used in real-world systems or applications—Perhaps we can create a category of “garage band languages” for those that aren’t ready to play real gigs yet. But until it’s useful to someone, a language with no users is always at risk of turning out to be a poseur.

- The natural appeal of the language is to write software with it, not to mess with the language itself—Solve your users’ problems rather than indulging your own programming fetishes.

- Inspires disdain, disgust, or outright hatred from people smart enough to know—Any idiot with a Slashdot handle can talk crap about anything. It’s when you piss off the smart developers that you know you’re working with something interesting.

Feel free to come up with your own definitions and categorizations. After all, if there are too many rules, it’s not punk anymore.

Good Riddance (Time of Your Life)

So, you have a choice. You can keep adopting the flavor-of-the-month language, each one giving you some beautiful new way to iterate over collections, so your method to collate all the user’s previous transactions into a table can be done in 10 lines of code instead of 15. Or maybe you can use so much duck-typing that you get to add the string “4” to the number 2 (and be able to accurately predict whether you’ll get the string “42” or the number 6). Or maybe the next language will do away with all the traditional branch and loop statements (if, for, while, etc.), in favor of some new exotic side-effect-driven style of programming that will be awesome as soon as you and everyone else “get it.”

Or, maybe you can drop the pretensions, dismiss the illusions, and tear down the fake world of intents, monads, and whatever crazy new thing they come up with next.

What will be left is you, your code, and a CPU waiting to do stuff.

Kick... ass.

Chris Adamson is a writer, editor, developer, and consultant specializing in media software development. He is the co-author, with Bill Dudney, of iPhone SDK Development. He has served as Editor for the developer websites ONJava and java.net. He maintains a corporate identity as Subsequently & Furthermore, Inc. and writes the [Time code]; blog. He listened to The Clash, The Offspring, Green Day, and Fear while writing this article. Send the author your feedback or discuss the article in the magazine forum.

Testing for Web Services

Keeping It Real in the Wilds of the Web
by Noel Rappin
Just because you are using an external web API for your site doesn’t mean that BDD principles need to go out the window.

Chances are that the web site you’re developing will be dependent on an external web service for some of its functionality—since a third-party API is often part of the solution for unimportant features like, you know, getting paid. It can be challenging to write your integration with this external service using a clean Behavior-Driven Design (BDD) process. But by letting a few basic ideas guide the design of your code and your tests, you can craft your critical web service interaction with confidence.

Start With an Integration Test

For the sake of argument, we’re going to assume a situation where the third-party Internet service also has an existing Ruby gem that interacts with it. This is informed by my experiences using the Braintree gem for web payment, and the whois gem for, well, Whois data. We’ll use the Whois case for most of the code because it makes for a more straightforward example.

Our testing solution for this third-party tool is going to depend on a fair amount of mocking and stubbing. There’s no good way around using doubles of some kind to test an external tool. Having your actual tests be dependent on the existence of a remote web server is a recipe for slow and unreliable tests.

Because the mocking we’re going to do essentially isolates various parts of the system and assumes that the test of the system is working, it’s very important that there be an integration test that exercises all the pieces in concert. We’ll write the integration test, and where that fails, we’ll drop down to unit tests and write the code from there. In the whois example, the whois call is being made implicitly to match the request ID with the user’s provider, for the purpose of populating information about the user’s Internet service.

Background:

Given I am a registered and confirmed user

Scenario: Whois lookup for an existing provider

When I make a request from an IP for a known provider

Then my provider is the default choice on the comment form

The point of interaction with the third-party library is in the first step, When I make a request from an IP for a known provider. The results of that interaction show up in the second step. Note the explicit mention of a “known” provider. Among other things, this sets up the possibility of testing error conditions with a different step definition.

We’re not going to get the Cucumber step to pass without writing code but we can at least set up the step definition and get an actual failure that we can use to drive our RSpec tests and code. For the first time, we come up against the dilemma of testing a third-party library and having to avoid actually calling the library during testing.

The way out of the dilemma from a Cucumber test is to realize that while a Cucumber test is meant to treat your application as a black box, you don’t need to treat the entire Internet the same way. For the Whois example, I probably should have dug into the whois gem to determine how the gem makes the call to whois, and either mocked that actual call, or mocked the internals of the gem. Instead, I created a mock at the surface of the gem. (For the payment gateway or a service that uses HTTP, it’s easy to fake the expected HTTP call using Webmock or Fakeweb.)

def mock_whois_client(expected_result)

mock_answer = mock(:technical => (mock(:name => expected_result)))

mock_client = mock

mock_client.should_receive(:query).and_return(mock_answer)

Whois::Client.should_receive(:new).and_return(mock_client)

end

When /^I make a request from an IP for a known provider$/ do

@my_provider = "Xfinity"

mock_whois_client(@my_provider)

visit(new_comment_path)

end

This is sort of cheating, in the sense that I’m faking a call that I’m not confident in. As we’ll see in a bit, that will come back to bite me.

If you are doing the HTTP mocking, then for the initial test, I’d recommend you gather the sample data by using a web browser to make the actual call and save the text in a file in your app. At the end, we’ll talk about keeping this file up to date.

Move to Unit Tests

With the Cucumber test failing because there’s no code in place actually making the third-party call, it’s time to write that code. The most important thing you can do here is to wrap the external API in a module or class that fully belongs to you. This is true even if, as with Whois or Braintree, the third-party service already has a Ruby gem that wraps the actual Internet call.

The standard Object-Oriented rationale for doing something like this is that it allows you to have your own API in your own domain terms that are more meaningful and possibly simpler than the everything-to-everybody third-party tool. I’m not going to be contrarian about it. Even where you aren’t doing much beyond just calling the external API, you’ll have integration points with your own models, which are easiest to manage in a single place. In addition, having a wrapper makes it easy to consolidate common logic that you might need before or after your external call.

It’s very useful to have the rest of your program not care about the details of the actual library. This gives you a single, small, tightly controlled place where the interaction with the third party lives—which makes the actual interaction easier to test, and also makes the rest of your application less dependent on the specific third-party library you are using—which is great if you suddenly have to switch.

When testing, once you have the wrapper, other tests can mock or stub at the wrapper level, not at the third-party API level. This is helpful because usually your API will be easier to stub than the actual web service (which might involve, say, an annoying set-up call to something like Webmock).

All that said, you need to be 100% sure that the wrapper is golden, because none of your other tests are going to touch it.

The wrapper itself should have methods that are meaningful in your application domain, and should return real, usable data to the rest of your application. Meaning if the third party is giving data that needs to be parsed before use, that should be the wrapper’s responsibility. Ideally, the wrapper is returning model objects, but depending on the particulars of your system, returning a Hash might be better. As far as the rest of your application is concerned, this data is already suitable for being used by the actual business logic of your model.

A strict BDD approach, then, would have you testing this code in two stages. Stage one is in the logic code of the application, and establishes that the model calls our wrapper. In that test, we can mock, but at the wrapper level, not at the third-party level. In the Whois code, the logic that calls the whois wrapper is a class method of my ServiceProvider class, so that’s where my main logic tests go.

describe ".find_or_create_from_whois_lookup" do

let!(:provider) { Factory(:service_provider, :name => "Xfinity") }

it "finds existing by whois lookup" do

WhoisWrapper.should_receive(:new).and_return(

mock(:name_lookup => "Xfinity"))

result = ServiceProvider.find_or_create_from_whois_lookup("xf.com")

result.name.should == provider

BroadbandServiceProvider.all.map(&:name).should == ["Xfinity"]

end

it "creates non-existing by whois lookup" do

WhoisWrapper.should_receive(:new).and_return(

mock(:name_lookup => "Verizon"))

result = ServiceProvider.find_or_create_by_whois_lookup("xf.com")

result.name.should == "Verizon"

ServiceProvider.all.map(&:name).should =~ ["Xfinity", "Verizon"]

end

it "does not create if nil" do

WhoisWrapper.should_receive(:new).and_return(

mock(:name_lookup => nil))

result = ServiceProvider.find_or_create_by_whois_lookup("1.2.3.4")

result.should be_nil

ServiceProvider.all.map(&:name).should == ["Xfinity"]

end

end

Testing the application model before you create the wrapper has two design advantages—it allows you to design the wrapper API based on actual use cases, and it ensures that you only add features to the wrapper that are actually driven by an application need. Building this kind of wrapper is definitely a case where it’s easy to get caught up in the wrapper ahead of any actual usage. I mean, so I hear.

The following code should pass those tests:

def self.find_or_create_by_whois_lookup(ip)

name = WhoisWrapper.new.name_lookup(ip)

return nil if name.blank?

provider = find_by_name(name)

if provider.nil?

provider = create(:name => name)

end

provider

end

As for the wrapper itself, the most basic case is to have it be a class whose methods call methods of the original API. As a matter of personal preference, I don’t like for the wrapper to be a singleton, and I don’t like having all the wrapper methods be class methods. I’d rather actually create a new instance every time I make a call, as in WhoisWrapper.new.name_lookup. In part this is because when the wrapper is a normal instance, it’s easier to add further transaction logic later on.

That’s the basic plan, but you can make the wrapper more complicated. The last time I worked with a payment gateway, our application had a lot of custom subscription logic that needed to be tracked on our side. We wound up implementing the basic wrapper as a class and the subscription logic in a Subscribable module that was mixed in to various entities that could own subscriptions, so that module was the only point of contact with the wrapper.

Writing a Wrapper

We’re building kind of a chain of test responsibility here. The model test assumes that the wrapper works, passing responsibility to the wrapper test, which assumes that the third-party API works as specified. Which prompts the question of how to test the wrapper to really nail down the wrapper functionality.

There are two basic solutions. One is the same as we used for the integration tests, namely stub the actual external requests that the wrapper will make. This approach is consistent with the general principle that you shouldn’t mock a library you don’t control. However, managing the Webmock calls for all the different kinds of API requests can be a pain. In general, you’ll need to make a larger number of API requests from the unit tests than from the cucumber tests.

If the third-party library exists as a gem, then another option is to break the general principle and mock the gem’s methods, like so:

<code language="ruby">

describe "WhoisWrapper" do

it "#name_lookup" do

mock_answer = mock(:technical => (mock(:name => expected_result)))

mock_client = mock

mock_client.should_receive(:query).and_return(mock_answer)

Whois::Client.should_receive(:new).and_return(mock_client)

WhoisWrapper.new.name_lookup("1.2.3.4").should == "Xfinity"

end

it "#returns nil on timeout" do

mock_client = mock

mock_client.should_receive(:query).and_raise(Timeout::Error)

Whois::Client.should_receive(:new).and_return(mock_client)

WhoisWrapper.new.name_lookup("1.2.3.4").should be_nil

end

end

</code>

The main downside to this plan is that if your test is mocking the wrong API method, you’ll never know. (To be fair, that’s a potential problem in the Webmock strategy as well, but actually exercising the API to generate the fake data helps find that kind of error.) I’d still lean toward this approach in cases where the wrapper is very simple, especially for the conversion of the output—simple JSON, for example. If the wrapper is doing more elaborate parsing on the output, I’d tend to want more realistic fake output and would be more likely to use the Webmock option.

Here’s the wrapper, at a first pass:

class WhoisWrapper

attr_accessor :whois

def initialize

@whois = Whois::Client.new

end

def name_lookup(ip_addr)

answer = whois.query(ip_addr)

answer.technical.name

rescue Exception

nil

end

end

A funny thing happened to me when I tested the Whois code in the browser. I was confident—the Braintree code had been much more complex and had gone pretty well when I went to manually test it. So I was surprised when the Whois code failed because the answer object above had no technical field. (I was basing the API on the online docs.)

Like I said, I took a shortcut in my integration tests, and now it was time to pay. I assumed the Whois output would be correctly parsed by the gem and, for the specific queries I was making, the gem didn’t parse the text. Luckily, my parsing needs were few, and because I had the wrapper already, I was able to get things going with only a few minor adjustments. I changed all my mocked Whois output to something like this:

mock_answer = mock(

:content => "Something\nOrgName: Xfinity\nSomething else")

That’s not perfect, it’s still a lightly processed version of the output, but it’s better, because it assumes less processing from the gem (the gem always puts raw content in the content attribute). Ideally, I’d go back and mock the whois gem’s actual external call—kind of complicated in this case because of the way the whois gem works. In any case, the wrapper method now looks more like this—the actual method is more complex to handle other cases, but this gives you the basic idea:

def name_lookup(ip_addr)

answer = whois.query(ip_addr)

org_match = answer.content.match(/OrgName:\s*(.*?)$/m)

if org_match then org_match[1] else nil end

rescue Exception

nil

end

Keeping it Fresh

At this point, we have, at least in theory, a well-tested wrapper that contacts the API, a well-tested model that interacts with the wrapper, and an integration test that merges the two. What’s missing? Protection against changes in the external service.

There are two related ideas that are handy for keeping the code in sync with the Internet service. You could do a smoke test, or you could do an active fixture refresh. Or both; they aren’t mutually exclusive.

A smoke test runs some or all of your cucumber tests that touch the third-party library, but without the Internet call being mocked. In other words, it’s kind of a super-integration test. You don’t want to run this all the time, of course, because it’s slow and unreliable. But if you run it daily, or every couple of days, you’ll have some warning if the web API changes or is otherwise flaky. (This is also a good thing to have your Continuous Integration tool do.)

For the logistics of the smoke test, mark any such test with a Cucumber tag—@smoke springs to mind. Then, head into your cucumber.yml file and add --tags ~@smoke to the std_opts line—you need to have a separate tags option so that the “don’t run smoke” tag is logically ANDed with the “don’t run wip” tag that should already be there.

That way, the regular cucumber task will skip smoke tests the same way it skips tests marked @wip. You’ll also want to add a rake task for cucumber:smoke.

smoke: --tags @smoke features

Then the task can be added to the lib/tasks/cucumber.yml file, something like this:

Cucumber::Rake::Task.new({:smoke => ’db:test:prepare’},

’Run smoke tests’) do |t|

t.binary = vendored_cucumber_bin

t.fork = true # You may get faster startup if you set this to false

t.profile = ’smoke’

end

An active fixture refresh is simpler. All that entails is automating the manual process we used for getting sample data for our mocks. You can do this with a Rake task that goes to the shell and uses wget to access the remote page, or in the case of the Whois gem, just calls whois from the shell.

Again, this isn’t something you’d want to run all the time, but every few days or so will keep your confidence in the tests high.

And in the End

Just because you are using an external web API for your site doesn’t mean that BDD principles need to go out the window. Keep working in BDD and remember this:

- Always start with an integration test.

- All interactions with the API library should go through a wrapper that is part of your application. Tests should mock at the level of your wrapper.

- Only use a mock for the actual third-party library if the only thing you are testing is your call to that library, and even then consider placing the mocks at the level below the third-party call.

- Use a smoke test or an active fixture refresh to keep your tests in line with the API.

Armed with these tips, you can work with what might be the most important part of your application with confidence.

Noel Rappin is a Senior Consultant at Obtiva. A Rails developer for six years, Noel has spoken at RailsConf and Windy City Rails, and is the author of Rails Test Prescriptions. A blog relating to this book can be found at www.railsrx.com. Send the author your feedback or discuss the article in the magazine forum.

Testing for the Cloud

Playing It Safe in Cloud Space
by Adam Goucher
The three big differences cloud computing brings with it are really just modern twists on old practices.

I don’t understand the hype around “testing cloud-based applications.”

Making the rounds of test-related conferences this past year, I’ve noticed that—because our industry isn’t immune to hype—cloud computing and how to test it has started to be a talking point. One conference organizer came right out and said they wanted a couple of talks on the subject for their next event. And if it is a looked-for conference topic then you just know that mainstream testers are going to think they need to pay attention.

But they really don’t need to work themselves into a tizzy over it. Cloud or not, it’s still just an application that needs to be tested. All the same tricks and techniques that you would use if the application resided nice and safe and sound behind your corporate firewall still apply. In fact, after working on an application for 18 months that was entirely in the cloud, I can say that testing a cloud application is almost exactly the same as testing an earthbound application, except that you need to pay extra attention to three things: location, fault tolerance, and elasticity.

Location

Being outside of the firewall is an important difference. Security is the first thing people think of when their application gets deployed into the cloud, and having tested a security product for almost six years I think that’s an appropriate focus. But security is only a small part of the ramifications of being on the other side of the firewall. Here is the beginning of a list of location-driven questions that will need to be answered—and those answers tested:

- Can your internal systems reach it? Outgoing holes in the firewall will need to be made.

- Can it reach your internal systems? Incoming holes in the firewall will need to be made, too.

- What about communication to partner or other third-party systems? Does it need to be in a different form if you allow those communications from the cloud?

- Is the machine stripped of unnecessary applications and locked down? You can’t rely on the firewall any longer to catch the easy things. They need to be addressed both through the cloud management tooling and the instance itself in the cloud, not just one or the other.

- What about intrusion detection?

- What is the patch/update plan?

- How do people authenticate onto the machine to do all this maintenance?

- What is, and more importantly, isn’t encrypted when entering or leaving the cloud? And there better be a good reason for everything that isn’t.

- Where are the backups of logs and databases going? And how are they getting there?

- What are the costs for actions and events inside the cloud? Amazon’s AWS cloud, for instance, has different pricing internally and externally. Applications will need to be modified to maximize their use of the lowest-cost transport and storage options.

- Can mail from your cloud instance reach its intended destination?

But if you have been around classically deployed applications for a while, the odds are these questions are just variations of ones you would have had to ask anyway. So, initially, location is no big deal.

Fault Tolerance

The next point of distinction is fault tolerance and how you react to it. In classic situations you can have all sorts of hardware-level heartbeats and monitors in place for things like I/O rate, bad sectors, temperature, etc. In the cloud you don’t have any of that. What you get instead is an email that reads something like this:

The host needs to undergo maintenance and will be

taken down at 12:00 GMT on 2009-01-13.

Your instances will be terminated at this point.

This was directly taken from the notice Amazon sends to customers when the physical hardware hosting their instance is having problems. And it is scary if you’re not expecting it. (Especially if the instance in question is your database server, no one in the company has the root password, and you’ve managed to corrupt the sudoers file. Don’t ask...).

It is important to think of your cloud resources as something that can go away at any time. And if my experience at a cloud-based company is any indication, they are indeed likely to go away more often than if you own the actual hardware. Thinking this way leads to three critical questions:

- If an instance goes away unexpectedly, how does that affect the overall system?

- Given advanced warning, is there a plan for replacing an instance?

- How much time do you need, at a minimum?

The answers, ideally, should be “negligible,” “yes,” “about the time it takes to run a script.”

The DevOps community has grown significantly over the last year and has developed techniques to solve these problems. At the time I left, our system could (pretty) gracefully recover from any part of it going down through a combination of round-robining through a cluster of app servers and database hot-backups. The weakest link in the system was the load balancer, but even it could be re-created through a combination of a custom image and cfengine—meaning that it could be rebuilt and running in about five minutes. Regular instance creation and migration became almost routine with the right tooling and processes.

Elasticity

Of course, getting that system in place took a lot of work. And it took even more testing, because it really did need to work. Being able to auto-provision a server is not just a goal for cloud applications, though. The same steps and techniques apply to provisioning a physical piece of hardware, it’s just that in the cloud there’s an additional level of scale.

Scaling, or more correctly elasticity, is the only thing that is really different when testing cloud-based applications. Much of the early marketing for cloud-based computing touted being able to scale up and (more importantly) scale down your computing resources as load demanded, which the cloud enables since it’s all just software. Hardware elasticity is in theory also possible, but the costs in terms of hardware and human capital can be prohibitive.

From an implementation perspective, this is really just the fault-tolerance problem wrapped in a fully automated skin.

The testing around elasticity is designed to ensure that these fully-automated scripts function correctly—around the triggering conditions for these scripts, and for scaling up and down. These triggering conditions used to be pretty simple things to monitor—like CPU, memory, and number of web processes. Now they are just as likely to include some aspect of business metrics. Improperly reacting to any of these metrics can cost the company money, either directly (another instance running) or indirectly (slow site turns away customers), so their collection and interpretation absolutely need to be tested.

Unlike location and fault tolerance, though, elasticity testing is unique to testing for the cloud. But even so, its uniqueness is based on it being a software procurement event rather than hardware.

Cloud computing is certainly hype-worthy. AWS and similar services have dramatically lowered the cost of creating a start-up and have reshaped how organizations think about their computing resources. When it comes to testing, though, I don’t think the hype it is starting to receive is warranted. The three big differences cloud computing brings with it—location, fault tolerance, and even elasticity—are really just modern twists on old practices: not completely new things we don’t understand, just modifications to how we have always tested.

Adam Goucher has been testing professionally for over 12 years at a range of organizations from national banks to startups. A large part of that time has been spent augmenting his exploratory testing with automation. He is the maintainer of Selenium IDE and consults on automation through his company, Element 34. Send the author your feedback or discuss the article in the magazine forum.

Software Volatility

Do Most Changes to Your Code Base Occur in Just a Few Files?
by Tim Ottinger, Jeff Langr
In the fourth in their four-part series of Big Ideas in software development, Tim and Jeff tackle software volatility.

What is Volatility?

As the final of our “big ideas in software development,” we look at volatility. We provide a very simple definition of volatility for purposes of this discussion: Volatility is a measure of how often a file actually changes—never mind the extent or reason for the change. Using this definition, we theorize that volatility directly correlates to the risk of making a change: the more volatile a file is, the more likely it is that changes to that file will result in a code defect.

Measuring Volatility

Michael Feathers recently posted a graph showing that commits across a code base are not evenly distributed. Some files change frequently, even continually, while the majority seem never to be touched at all. His blog report of these findings was widely tweeted and referenced.

Noticing the same trend, Tim presented the idea of a Heat Map last year, in a blog post in which he counted Jira tickets associated with source files in the source control system for a sizable production web application. Some changes were due to feature enhancements, whereas others were due to bug fixes and others were merely side-effects of a refactoring (usually Rename or Move Method).

In Tim’s analysis, a small number of files were shown to be involved in the vast majority of changes made in the system. When the five most volatile files were identified, they were shown to be poorly covered with tests, if at all, and less than optimally structured. (Why does this not surprise us?) Developers were hesitant to touch these files because of their high implementation complexity, yet they were still visited frequently.

Michael Feathers suggested that counting commits was a reasonable approximation for the volatility of a code module, and we agree. Here are a few projects you can check for yourselves:

1. In the Gwibber project, the most changed files are client with 302, gwui with 149, and dispatcher with 95 commits. All 247 other files have less than 70 updates and the vast majority of them have less than 30.

2. In the Mongo project, most files have fewer than 100 modifications, but the most-modified file has 663 (caveat: an Scons file). The top nine have from over 300 to over 500 changes. This is another data point suggesting that changes clump, rather than spreading over the code base.

3. Source files in the rails project tend to have fewer than 150 changes per file, the vast majority having fewer than 20 changes. However, the top five have 953, 568, 481, 309, and 307 changes respectively.

Files can change frequently because:

1. they are “god classes” (poor cohesion),

2. they have poor dependency structure (poor coupling),

3. they depend on many implementation details (poor abstraction), or

4. the code is poorly written and has many flaws.

Sometimes all of the above are true!

Circumstances do exist where heavy change is not due to poor design—files can contain key abstractions in the solution domain or the problem domain. New features in a banking system might touch upon Account or User. In a retail app, it might be Sale or Receipt or Product.

No matter the reason, we can confirm that certain source files change much more often than others, and then we can form strategies to deal constructively with code that shows high volatility.

Risk of Breakage

Programming is more an art of continual tailoring than invention. Systems gather changes over time: functional alterations, adjustments to user experience, performance and scalability changes, upgrades to the supporting platform or platforms, integration with other systems, and adjustments or corrections to internals (both functional and structural). It has been said that a system spends 80% of its life in maintenance. A successful system should spend far more years in extension and improvement than it did in original authoring.

A sad fact of software development is that every change to existing code has a non-zero chance of causing breakage. Some changes are simple errors in function name or type, some are simple miscalculations, some are naive changes that damage scalability or performance, and some are subtle effects that only show up when seemingly unrelated parts of the code base fail. Many defects are coded due to a simple misunderstanding of how the code currently operates.

In our previous article on coupling, we mentioned that errors tend to fan out along dependency lines, including implicit dependencies. In our articles on cohesion and abstraction, we mentioned how duplication makes maintenance a “sometime thing,” since a programmer is unlikely to find all the occurrences of a duplicated passage of code and correct them all. A bug may return several times—there exists potentially one “emergency bug fix” for every point of duplication in the code base.

In general, the sooner a program error is detected, the less damage it does to the project’s schedule and to the product’s reputation. If the programmer realizes an error while typing, he may backspace and rewrite the code immediately with no ill effect at all. If the error escapes the programmer, it may be spotted by another programmer in the same development cycle or by a QA team member.

An otherwise undetected error escapes “into the wild,” the real world where customers call technical support and programmers are pulled from feature work to diagnose and fix the code. The defect may bring servers to their knees, denying users access. It may cause data to be corrupt, requiring painstaking data repair. It may have consequences well beyond those reported by the users. Of course, the bug could also be cosmetic, or a trivial problem with an easy work-around.

Simple odds suggest that the likelihood of breakage is greater in more volatile modules. We also have observed that messy, ill-structured, badly written modules are easy to misunderstand and hard to repair. Where there is duplication, a lack of abstraction, and needless couplings, the breakage will be most likely to fan out across the code base.

To re-emphasize: Where we have messy, volatile code we are at the greatest risk of failure. What are we going to do about it?

The Review Alternative

“Given enough eyeballs,” says Linus’ Law, “all bugs are shallow.” Various forms of review—inspection, code walk-through, hierarchical review, and pair programming—exist to provide structure for the eyeballs.

Many companies and open source projects use hierarchical review. Someone submits a patch, which is reviewed at the first tier. If the patch has no obvious defect it is passed up to the next layer, otherwise it is returned or rejected. This is an effective strategy, but many companies do not or cannot adopt the strategy because it requires layers of skilled programmers to do little but monitor changes. It can be hard to keep up with those writing the code. The reviewer/acceptors need to be among the most skilled developers and those most familiar with the code base, and while they are spending time reviewing, their skills are not being put to use producing product.

Other companies, especially those employing agile techniques, perform constant review via pair programming. They switch partners frequently, always having two heads involved in the decision-making, and having multiple sets of eyes on the code. All of the programmers are actively involved in code-writing most of the time.

Our most volatile code demands the most eyeballs. Even if your team has reviews for no other code and won’t consider pair programming, consider either when entering into its risky, volatile areas.

The Testing Alternative

Breakage often occurs somewhere else in the code base, far from a change. “Never thought that would break!”

It is foolhardy to release a product on faith alone. Most companies supplement faith with a QA department full of people knowledgeable about the product and skilled at operating it. They may have little knowledge of the code that comprises the product, but are brilliant at sniffing out defects and their causes. These QA folks fulfill the innately human needs in testing—testing scenarios that require reasoning about the software, predicting usage errors, noting inconsistencies in data handling or presentation, and so on. Such exploratory testing is labor-intensive but invaluable.

On the other hand, often a testing department involves a small army of testers who run explicit test scripts more or less by rote. They repeat the same steps month after month and release after release. Sometimes they find errors, but ideally and usually they run all of their tests without seeing any errors. Their job is to provide assurance that important procedures still work just as they always did. Maintaining and running manual regression scripts is expensive and slow, but it is the last chance to spot defects before releasing code to customers.

If an error was detected immediately after it was created, it would be corrected without delay. It makes sense that developers test as they work, but if a developer manually ran all of the test scripts after every line of code, productivity would be dismal. We might see a feature developed in the course of a decade.

An undetected error in a very volatile source file might have many layers of additional change heaped on it by the time it reaches release, so early detection is key. Errors indirectly related to a change in volatile code must have an automated test produced to reproduce it.

In order to develop code quickly and well, our most volatile code requires the most extensive automated testing. The tests need to run frequently and quickly so that developers can run them many times per hour. This makes solving the problem easier, but also protects against future breakage being undetected or detected late.

Agile teams use extensive testing at the requirement level (acceptance tests), at the level of individual effects of functions or classes (unit testing), and often at the UI level as well (though UI tests tend to be too slow to be run by human beings and are offloaded to a continual testing environment). Many agile teams tackle test-first programming (aka test-driven development, or TDD) as well, to both build up a significant body of tests and guide an improved design.

On the surface, it may appear that the investment in automating such test scripts is too high, since they seem to report only rarely on defects in the system. In reality, the tests prevent gobs of bugs—since these tests are written first, they gate the defects from ever making their way past the programmer and into the code base (in the case of TDD) or past the QA team and into the release.

Automated tests protect you against the exploding minefield that is volatile code.

The Wrap-up

This trip through Cohesion, Coupling, Abstraction, and Volatility—the four most important ideas in software development—brings us to the premise of the agile development process. Software development should be fluid and productive, but problems that develop in the code base can slow releases and frustrate the authoring of new features. For a project to be successful, it must have constant attention to quality, especially in the code that is touched most often.

Hardcore agile teams use constant testing, constant teamwork, continual integration, and constant refactoring to manage the code base. The testing practices actively shorten the period between defect creation and detection. The refactoring practices improve cohesion, reduce coupling, and add abstraction in order to keep the code workable. Keeping the code workable leads to more productive teams and higher quality month over month.

Even if your team does not wish to identify as an agile team, we suggest that it might be prudent to at least adopt these practices when it comes to the most volatile (and therefore most risky) code in your code base.

If you are working in an agile team, the proof of your growing agility is in examining the quality and test coverage of your most volatile code. If it is being touched frequently, it should be continually improving in structure, readability, and code coverage. If it is not improving, then you have some technical practices to work on, and some debt to pay down.

Jeff Langr has been happily building software for three decades. In addition to co-authoring Agile in a Flash with Tim, he’s written another couple books, Agile Java and Essential Java Style, contributed to Uncle Bob’s Clean Code, and written over 90 articles on software development. Jeff runs the consulting and training company Langr Software Solutions from Colorado Springs.

Tim Ottinger is the other author of Agile in a Flash, another contributor to Clean Code, a 30-year (plus) software developer, agile coach, trainer, consultant, incessant blogger, and incorrigible punster. He writes code. He likes it. Send the authors your feedback or discuss the article in the magazine forum.

When Did That Happen?

Alan Turing and the Invention of Computing
by Dan Wohlbruck
How a great mathematician solved a classic problem and laid the theoretical foundation for modern computers.

In March 1935, at the age of 22, Alan Turing was elected a fellow of King’s College at Cambridge University. In May of the next year, he submitted a paper with the opaque title of “On Computable Numbers, with an Application to the Entscheidungsproblem.”

It was shortly after Turing received his fellowship that he was introduced to the challenge of this Entscheidungsproblem. Posed by David Hilbert in 1928, the Entscheidungsproblem asks whether a definite procedure can be devised by which any mathematical assertion can be analyzed to determine if that assertion can be proven.

Hilbert had set out a grand challenge: he intended to put mathematics on a solid footing, and to do so, he explained that you had to show three things. You had to show that mathematics was complete, meaning that all true statements could be proven and all other statements would be provably false; that it was consistent, meaning that no contradictions could be arrived at within the system; and that it was decidable, meaning that for any assertion there was a finite mechanical process that would decide its truth or falsity. He set these challenges for the mathematicians and logicians of his day, and the third of these challenges was the Entscheidungsproblem.

In English, “Entscheidungsproblem” translates to “decision problem,” and three quarters of a century later, every computer that has ever existed can trace its origins to the way that Alan Turing went about meeting the challenge that the decision problem presented.

A Lonely Genius

From early childhood, Alan Mathison Turing stood out from the crowd. He taught himself to read in three weeks, but he refused to write, almost until he entered University. He objected to the conformity of public school but didn’t rebel against it so much as withdraw from it. On his own, he performed chemical experiments and perfected mathematical equations. He became a long-distance runner of considerable ability. And when, against the judgment of his teachers, he sat for the examination to enter King’s College, he was accepted.

Turing became a fellow of King’s largely as the result of a dissertation in which he independently proved the central limit theorem of probability. He is most often thought of as a mathematician. But his interests were broader than this: in his early years at King’s College, he became acquainted with Einstein’s theories and Russell’s symbolic logic. Turing particularly liked the fact that Einstein was more concerned with the outcome of his theory of relativity than with the rigor of its proof.

Given his background, Alan Turing would have begun his assault on the Entscheidungsproblem by restating it in his own words and then reducing the problem to symbolic logic. In his own words, Turing might have said that he was looking for an algorithm that can decide whether any other algorithm is computable. A mathematical expression is computable if there is an algorithm that allows its values to be calculated. Turing conjectured that no algorithm existed that could prove the computability of every expression.

In order to reduce his conjecture to symbolic logic, he looked at how people solve problems. He reflected in great detail on the individual steps that a human takes when solving a problem. In his own words, what was someone thinking about as they go about multiplying the two numbers 4,231 and 77? Turing noticed that the human computer began with the units column and multiplied 7 and 1. The human computer must then change his or her state of mind (Turing’s expression) in order to write the product on a piece of paper. After storing the result on paper, the human computer proceeds to multiply the numbers in the tens column, and repeats similar steps and similar changes in his or her state of mind. At the conclusion of the operation, the product—325,787—is known. Through this detailed examination of the calculating process, Turing determined that at any single instant in time, the human computer was concentrating on a single operation.

And it was here, at this moment in his deliberations, that Turing’s genius was manifest.

The Invention of Computing

While on a run in the English countryside, he thought about the human computers and their calculations. As one stride followed another, he dissected each logical step. If he could devise an algorithm for the steps, he thought, then the process of calculation could be mechanized. As the miles went by, the rhythm of the run gave Turing the inspiration that he needed. When he returned to his room at King’s College, the world had changed. For the first time in the history of the human race, one of its members had thought like a modern computer—Alan Turing had invented computing.

He reasoned that a machine, using an infinitely long strip of paper tape segmented in squares, could reproduce the steps of the human calculator. When the human wrote an interim result on paper, the machine would record a result on the paper tape. When the human changed his state of mind, the machine would, too. People have memory, and use it in calculating. If the paper tape could move backward and forward, the machine could have some short term memory, too. And, most remarkably, the machine would use symbols to identify the operation and the result. All of the steps from each completed computation, like the product of 77 and 4,231 or the solution to 2x equals 3 plus 5, would be kept, and could be rerun to duplicate the algorithm.

Now came a great mental leap. Turing imagined another machine, a universal machine, that would take as input the rolls of paper tape from the first machine’s completed computations and detect if that specific algorithm had a solution.

In his paper “On Computable Numbers,” Turing described just such a machine. It would be able to execute instructions that would enable the machine to read, print, move the paper tape, and set the machine’s state of mind. After providing several examples of how his machines would work, Turing concluded: “We are now in a position to show that the Entseheidungsproblem cannot be solved.” With that, Turing the mathematician solved the decision problem and Turing the genius laid the theoretical foundation for modern computers.

Alan Turing died in 1954 but his legend, like his machines, lives on.

Dan Wohlbruck has over 30 years of experience with computers, with over 25 years of business and project management experience in the life and health insurance industry. He has written articles for a variety of trade magazines and websites. He is currently hard at work on a book on the history of data processing. Send the author your feedback or discuss the article in the magazine forum.

The Quiz

With Friends Like These
by Michael Swaine
This puzzle can’t be solved by a computer program. No algorithm will get you the answer. But if you’re really into algorithms, there’s a good chance that you’ve seen the solution recently, although you may not recognize this puzzle as corresponding to that solution. And if you haven’t, there are enough clues here that you should be able to figure it out. Oh, and while the puzzle can’t be solved by a computer, you’ll probably need Wikipedia to help you work out the details.

A certain club has just over four dozen members. It’s not what you’d call a social club. Most of the members are friends with only a few others.

For example, Richard, Lincoln, Christine, Jack M., and Nikki are particularly lonely, as each has only two friends. And Paul is almost as lonely as the utterly friendless Sean and Neil, since he has just one friend, John L.

Friendship, I should mention, is a reciprocal relation. If I’m your friend, you’re my friend. Substitute any other reciprocal relation if it makes the problem clearer to you. I won’t mind. We're friends, after all.

Nobody else in the club has as many friends as Jay or Bill, who have eight each. Perhaps we should not be surprised that these two relatively gregarious members are friends with each other. And with Steve, who has seven friends.

You could say that Bob has six friends and Martin has five, but only if you count Vincent, and Vincent isn’t a member. Forget I mentioned Vincent. Vincent is right out.

It’s a cliquish club, this one. Some members are off in their own chilly little corner. If it weren’t for Andrew, all six of Peter, Paul, Lincoln , John L., Deval, and Dan would only have each other for friends. He’s their only link to all the rest. Well, all except Sean and Neil.

There are also what you might call strings of friends. Jerry and Jack M. are remotely connected, because Jerry is friends with Young Brian, who is friends with Gary, who is friends with John H., who is friends with Sam, who is friends with Jay, who is friends with Pat, who is friends with Mitch, who is friends with John K., who is friends with Tom, who is friends with Chris, who is friends with Jack M.—still with me?

A similar string connects Jan with Richard, running from Jan to Susana to Mary to James Richard to Mike to Bobby to Haley to Robert to Nathan and ultimately to Nathan’s friend Richard.

Friendships in the club are fairly permanent, but thin. No four people are all friends with one another, but there are many mutually-friendly threesomes, like Beverly, Nikki, and Nathan.

Some of these friendship triangles do overlap. The three-friend triangle of Butch, Old Brian, and Matt, for example, overlaps with the Old Brian-Matt-Dennis triangle, which shares Old Brian and Matt and which also overlaps with the Dennis-Jack D.-Mark triangle, which overlaps with the Mark-Scott-Terry triangle, which in turn overlaps with the Terry-Dave-Dennis triangle, which overlaps with…

Well, this is getting a little complicated, with these friendship triangles that one can be a member of many of, if you follow me. I mean, Earl is a member of five such triangles and Doctor John is a member of three, although Rick is a member of only one. But I think you have enough information now.

So if I tell you that Christine has exactly two friends, can you name them?

Here’s a bonus question: Where can you find a map of the club’s friendship structure? I’ll give you a hint: it’s in Figure 17 on page 15 of a book.

Calendar

Author sightings, partner events, and other notable happenings.

Author Appearances

- Mar 4–5, Codepalousa, Louisville, KY
SpeakingJeff Cohen, co-author of Rails for .NET Developers

- Mar 4–5, Twin Cities Software Symposium, Minneapolis, MN
Talks on JVM languages, concurrency, and testingVenkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

- Mar 4–6, Twin Cities Software Symposium, Minneapolis, MN
“Rediscovering Apprenticeship in the 21st Century” and “Grails: Bringing Radical Productivity to the JVM”Dave Klein, author of Grails: A Quick-Start Guide

- Mar 7–10, Pragmatic Studios, Santa Clara, CA
Pragmatic Rails II StudioDave Thomas of Pragmatic Programmers and Chad Fowler, author of The Passionate Programmer

- Mar 7–10, Pragmatic Studios, Santa Clara, CA
“iPhone/iPad Programming” with Matt DranceDaniel Steinberg, co-author of iPad Programming: A Quick-Start Guide for iPhone Developers and author of Cocoa Programming: A Quick-Start Guide for Developers

- Mar 7–11, QCon London, London, England
Running my Secrets of Agile Architecture tutorial on March 7 and speaking on the Enterprise Agile Transformation track (Topic TBD).Dan North, co-author of The RSpec Book

- Mar 10, Crisp Rocket Day, Stockholm, Sweden
“Regex—the Future Programmer’s Best Friend”Staffan Nøteberg, author of Pomodoro Technique Illustrated

- Mar 10, DevLin 2011, Linköping, Sweden
“Pomodoro Technique”Staffan Nøteberg, author of Pomodoro Technique Illustrated

- Mar 11–12, No Fluff Just Stuff, Boston, MA
Keynote plus talks on JVM languages, concurrency, and testingVenkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

- Mar 16, MountainWest RubyConf 2011, Salt Lake City, UT
“Improve Your Ruby Skills” and “Get Started With TDD in a Legacy Environment”Noel Rappin, author of Rails Test Prescriptions

- Mar 16, Turku Agile Day 011, Turku, Finland
“Regex—the Future Programmer’s Best Friend”Staffan Nøteberg, author of Pomodoro Technique Illustrated

- Mar 17, Turku Agile Day 011, Turku, Finland
“Pomodoro Technique”Staffan Nøteberg, author of Pomodoro Technique Illustrated

- Mar 21–22, DevNexus, Atlanta, GA
Keynote and talk on Programming ScalaVenkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

- Mar 21–22, SDC, Wellington, NZ
“Keynote: Creating an Adaptable Life” and “Keynote: Who’s On Your Team?” Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

- Mar 24–25, SDC, Sydney, Australia
“Keynote: Creating and Adaptable Life” and “Keynote: Who’s On Your Team?”Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

- Mar 28–29, SDC, Sydney, Australia
“Working Effectively With Distributed Agile Teams”Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

- Mar 29–31, Pragmatic Studios, Reston, VA
Pragmatic Rails StudioDave Thomas of Pragmatic Programmers and Chad Fowler, author of The Passionate Programmer

- Apr 1–2, Ruby Nation, Reston, VA
KeynoteDave Thomas of Pragmatic Programmers

- Apr 1–2, Ruby Nation, Reston, VA
“So You Think You Need a Rewrite?”Chad Fowler, author of The Passionate Programmer

- Apr 1–2, No Fluff Just Stuff, Orlando, FL
Talks on JVM languages, concurrency, and testingVenkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

- Apr 1–3, Greater Florida Software Symposium, Orlando, FL
“Hello! Groovy,” “Rediscovering Apprenticeship in the 21st Century,” and “Grails: Bringing Radical Productivity to the JVM”Dave Klein, author of Grails: A Quick-Start Guide

- Apr 4, SDC 2011, Gothenburg, Sweden
“Regex—the Future Programmer’s Best Friend”Staffan Nøteberg, author of Pomodoro Technique Illustrated

- Apr 4–6, iPhone/iPad DevCon East, Boston, MA
“Coming Up to Speed on Objective-C and Cocoa Fundamentals,” also technical sessions on blocks and Xcode4Daniel Steinberg, co-author of iPad Programming: A Quick-Start Guide for iPhone Developers and author of Cocoa Programming: A Quick-Start Guide for Developers

- Apr 4–6, iPhone/iPad DevCon East, Boston, MA
“Using Core Animation to Build Complex and Attractive Interfaces,” “iOS 4 Event Handling Using UIGestureRecognizer,” and “Practical UIKit: Creating Custom Views and Components”Nathan Eror, co-author of Core Animation for Mac OS X and the iPhone

- Apr 5–7, QAI, QUEST 2011, Boston, MA
Tutorial: “Coaching as a Two-Way Relationship,” and talk: “It’s the People; It’s Always the People”Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

- Apr 6–8, 33degree , Krakow, Poland
Keynote plus talks on JVM languagesVenkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

- Apr 6–8, Scottish Ruby Conference, Edinburgh, Scotland
SpeakingChad Fowler, author of The Passionate Programmer

- Apr 7, Scottish Ruby Conference, Edinburgh, Scotland
Charity tutorialChad Fowler, author of The Passionate Programmer

- Apr 9–10, Voices That Matter: iPhone Developers Conference, Seattle, WA
“Advanced Media Manipulation with AV Foundation”Chris Adamson, author of iPhone SDK Development

- Apr 9–10, CodeConf, San Francisco, CA
“Project and Community Management with Github”Andy Lester, author of Land the Tech Job You Love

- Apr 11–13, JAX London, London, England
Keynote (Tuesday 12) on “Deliberate Discovery and Programming Patterns”Dan North, co-author of The RSpec Book

- Apr 15, GLSEC Conference, Grand Rapids, MI
Keynote: “Thriving in a Competitive Marketplace”Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

- Apr 15–16, Southern California Software Symposium, Los Angeles, CA
“Rediscovering Apprenticeship in the 21st Century” and “Grails: Bringing Radical Productivity to the JVM”Dave Klein, author of Grails: A Quick-Start Guide

- Apr 16, ACCU conference, Oxford, UK
“Overcoming Resistance to New Ideas/Breaking through the Brick Wall”Rachel Davies, co-author of Agile Coaching

- Apr 17, Chennai Java User Group, Chennai, India
Talk on JVM LanguagesVenkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

- Apr 18–21, Pragmatic Studios, Denver, CO
“iPhone/iPad Programming” with Mike ClarkDaniel Steinberg, co-author of iPad Programming: A Quick-Start Guide for iPhone Developers and author of Cocoa Programming: A Quick-Start Guide for Developers

- Apr 19–22, Great Indian Developer Summit, Bangalore, India
Keynote plus talks on Java, RichWeb and .NET related topicsVenkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

- Apr 21, Red Dirt RubyConf, Oklahoma City, OK
“Using BDD in JavaScript from Ruby”Noel Rappin, author of Rails Test Prescriptions

- Apr 22–23, RedDot RubyConf, Singapore
SpeakingChad Fowler, author of The Passionate Programmer

- Apr 22–23, RedDot Ruby Conference, Singapore
SpeakingDave Thomas, Pragmatic Programmer

- Apr 26, Mongo Philly, Philadelphia, PA
Speaking on MongoDB Administration and possibly one or two other talks.Brendan McAdams, author of the forthcoming MongoDB: A Quick Start Guide

- Apr 27–28, Emerging Tech Conference, Philadelphia, PA
“Say Yes—or Say No? What to Do When Faced with the Impossible”Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

- Apr 30, No Fluff Just Stuff, Reston, VA
Talks on JVM languages, concurrency, and testingVenkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

- May 10–13, XP2011, Madrid, Spain
KeynoteBrian Marick, author of Programming Cocoa with Ruby and Everyday Scripting with Ruby

- May 12, XP2011, Madrid, Spain
Discussion on Agile CoachingRachel Davies, co-author of Agile Coaching

- May 13, GeeCON 2011, Krakow, Poland
“Regex—the Future Programmer’s Best Friend”Staffan Nøteberg, author of Pomodoro Technique Illustrated

- May 16, RailsConf Baltimore, Baltimore, MD
“Building Bulletproof Views”John M. Athayde and Bruce Williams, who are writing The Rails View for the Pragmatic Bookshelf

- May 16–19, RailsConf, Baltimore, MD
TBDChad Fowler, author of The Passionate Programmer

- May 18, DevSum 2011, Stockholm, Sweden
“Regex—the Future Programmer’s Best Friend”Staffan Nøteberg, author of Pomodoro Technique Illustrated

- May 28–29, RubyConf India, Bangalore, India
TBDChad Fowler, author of The Passionate Programmer

O’Reilly Events

- Mar 28–31, Web 2.0 Expo, San Francisco, CA
O’Reilly Web 2.0 Expo SF: “The program will spotlight experts, leaders, and under-the-radar innovations, and in the spirit of Web 2.0, there will be ample opportunity for attendees to connect, contribute, and collaborate. Web 2.0 Expo will be a place for creativity, engineering, and innovation.”

- Apr 11–14, MySQL Conference & Expo, Santa Clara, CA
MySQL Conference & Expo: “The O’Reilly MySQL Conference & Expo is committed to spreading the latest and best knowledge on MySQL and related technologies to the global community of DBAs, developers, and managers. You’ll find answers to your questions about MySQL in an independent and inclusive environment. You’ll also find practical guidance to help you add value and manage costs to your project or organization.”

- Apr 19–21, Where 2.0, Santa Clara, CA
O’Reilly Where 2.0 Conference: “Where 2.0 brings together the people, projects, and issues building the new technological foundations and creating value in the location ecosystem. Join with developers, technologists, CTOs, researchers, geographers, academics, business developers, and entrepreneurs to debate what’s critical now, and what’s pushing the boundaries of the location frontier.”

- May 16–19, RailsConf, Baltimore, MD
RailsConf: “Co-produced by Ruby Central and O'Reilly Media, RailsConf is the official and largest gathering of the Ruby on Rails community. RailsConf incorporates keynotes, sessions, and tutorials to arm participants with the knowledge and skills they need to capitalize on the wealth of services and opportunities the Rails framework has to offer.”

- May 21–22, Maker Faire 2011, San Mateo, CA
Maker Faire 2011: “A two-day, family-friendly event that celebrates arts, crafts, engineering, and science projects and the Do-It-Yourself (DIY) mindset. It's for creative, resourceful people who like to tinker and love to make things. We call them Makers.”

USENIX Events

What’s coming from our USENIX friends.

- Mar 29, Hot-ICE ’11, Boston, MA
Workshop on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services: “The first Hot-ICE workshop seeks to bring together researchers and practitioners working on network and service management in the Internet, cloud, and enterprise domains. The scope of Hot-ICE includes all aspects of network and service management.”

- Mar 29, LEET ’11, Boston, MA
4th USENIX Workshop on Large-Scale Exploits and Emergent Threats: “LEET aims to be a true workshop, with the twin goals of fostering the development of preliminary work and helping to unify the broad community of researchers and practitioners who focus on worms, bots, spam, spyware, phishing, DDoS, and the ever-increasing palette of large-scale Internet-based threats.”

- Mar 30—Apr 1, NSDI ’11, Boston, MA
8th USENIX Symposium on Networked Systems Design and Implementation: “NSDI ’11 will focus on the design principles, implementation, and practical evaluation of large-scale networked and distributed systems.”

- Apr 10–13, EuroSys11, Salzburg, Austria
European Conference on Computer Systems: “The EuroSys conference series brings together professionals from academia and industry. It has a strong focus on systems research and development: operating systems, data base systems, real-time systems and middleware for networked, distributed, parallel, or embedded computing systems. EuroSys has become a premier forum for discussing various issues of systems software research and development, including implications related to hardware and applications.”

- May 8–10, HotOS XIII, Napa, CA
HotOS XIII: “Three days of active interaction among innovative practitioners and researchers in computing systems, broadly construed. Continuing the HotOS tradition, participants will present and discuss new ideas about computer systems research and how technological advances and new applications are shaping our computational infrastructure.”

- May 26–27, HotPar '11, Berkeley, CA
3rd USENIX Workshop on Hot Topics in Parallelism: “HotPar '11 will bring together researchers and practitioners doing innovative work in the area of parallel computing. HotPar recognizes the broad impact of multicore computing and seeks relevant contributions in all fields, including application design, languages and compilers, systems, and architecture.”

Other Happenings

- March
Maybe you knew that Edsger Dijkstra’s famous letter “Go To Statement Considered Harmful,” appeared in Communications of the ACM in March of 1968. But maybe you didn’t know that the phrase was not in fact Dijkstra’s, but was attached to the letter by the publication’s editor, Niklaus Wirth.

- Mar 3
GNOME first came out of its burrow on this date in 1999.

- Mar 9
Jef Raskin would have turned 68 today.

- Mar 11
J.C.R. Licklider would have turned 96 today.

- Mar 13
On this date in 1986, Microsoft went public, creating four billionaires.

- Mar 15
A-life pioneer Craig Reynolds, who worked on the first Tron, is 58.

- Mar 15
It’s the Ides of March. Beware.

- Mar 16
Richard Stallman is 58 and MINIX author Andy Tanenbaum is 67.

- Mar 24
It’s Ada Lovelace Day, honoring women in technology and science.

- Mar 24
Mac OS X is X today.

- Mar 25
On this day in 1995, Wiki went worldwide as Ward Cunningham installed WikiWikiWeb on the Web.

- Mar 26
Larry Page is 38.

- Mar 31
Evan Williams, co-founder of Twitter, is 39.

- Mar 31
It’s the second anniversary of Steve Wozniak’s last appearance on “Dancing with the Stars.”

- Apr 4
Larry Page takes over as Google CEO.

- Apr 5
SATAN’s father, Dan Farmer, is 49.

- Apr 5
Inventor Dean Kamen is 60.

- Apr 6
MacOS co-creator Andy Hertzfeld is 58.

- Apr 9
ENIAC co-inventor J. Presper Eckert was born on this date in 1919.

- Apr 9
SQLite architect D. Richard Hipp is 50.

- Apr 14
Ruby designer Yukihiro Matsumoto (Matz) is 46.

- Apr 18
PUGS project initiator Audrey Tang is 30.

- Apr 20
Chief Yahoo David Filo is 45.

- Apr 22
Psychedlic ruminant-fancying game designer Jeff Minter (Yak) is 49.

- Apr 24
Champion of modelessness Larry Tesler is 66.

- Apr 28
Mathematician Kurt Gödel was born on this date in 1906.

- Apr 30
Information theory pioneer Claude Shannon was born on this date in 1916.

- Apr 30
Colossal Cave Adventure creator Don Woods is 57.

- Apr 30
The World Wide Web is 18.

- May
In May, 1980, Erno Rubik’s puzzle started selling internationally under the name “Rubik’s Cube.”

- May 1
2006 MacArthur Fellow Jim Fruchterman is 52.

- May 2
Protoblogger Dave Winer is 56.

- May 3
Simon Tatham, known for playing with PuTTY, is 34.

- May 8
jQuery creator John Resig got to age 27 one day ahead of Mark Zuckerberg.

- May 9
Red Hat founder Marc Ewing is 42.

- May 11
Edsger Dijkstra’s 81st birthday.

- May 14
Zillionaire entrepreneur Mark Zuckerberg is 27.

- May 17
Youthful Alan Kay is 71(!)

- May 19
Today is the tenth anniversary of the opening of the first Apple retail stores. Also, Java daddy James Gosling is 56 today; Theo de Raadt, founding member of NetBSD and founder of OpenBSD and OpenSSH, is 43; SNOBOL creator Ralph Griswold would have turned 77 today; and CP/M inventor Gary Kildall would have turned 69.

- May 26
Ward Cunningham is 62.

- May 27
PEZ was trademarked on this date in 1952.

- May 28
Networking hotshot Adam Dunkels is 33.

- May 29
62nd birthday of shareware co-creator Bob Wallace.

- May 31
John Kemeny was born on this day in 1926.

Shady Illuminations

Train Wreck
by John Shade
In the good old days, it only took Microsoft 3.1 versions to get it right, but these days they have to put at least a half-dozen locomotives in the ditch in their effort to find themselves.

We all love a train wreck.

I think I speak for everyone when I say there’s nothing we enjoy more than watching stuff blow up—or implode or somehow get wrecked. There’s the wholesale carnage of the classic disaster movie, from Towering Inferno to 2012, but the more narrowly targeted wreck motifs are sometimes even more satisfying. Every action feature must have a scene where the hero buddies leap from the exploding something just ahead of the ball of fire and shock wave. What we call car chases, a stock element of movies, are really usually car pileups, especially in light-hearted comedies. And of course nothing’s funnier than watching pedestrians scatter as the good guys smash some foreign entrepreneur’s fruit stand. But train wrecks are the gold standard. The slow inexorability of the chain-reaction disaster takes the simple pleasure of watching something blow up and smears it out over time. No wonder it’s the metaphor of choice for a plan that is so clearly going so very very wrong. Like the Microsoft-Nokia deal. Everybody can see that those tracks are torn up and this train is careening headlong for a spectacular cinematic derailment denoument. Beautiful.

Of course if you work for Nokia you might have a hard time seeing the fun in the impending disaster. Being too close to the action can distort your perspective.

Bill and Coup

It can’t be much fun to hear the deal described as a coup, a major blunder that makes Google look good, or as Microsoft buying Nokia for $0. You can’t enjoy seeing your new management described as a puppet government, or reading that your company is toast or that the announcement was effectively a “suicide note.”

If it were just journalists dumping on you it wouldn’t be so bad. But Standard & Poor’s and Moody’s are considering downgrading your rating, your stock tanked, and your fellow employees sat out in protest (although with the generous flex time in Finland, it was hard to tell). Not to mention that your own CEO said you were on a burning platform, pouring gasoline on the fire. He went on to say that you weren’t even fighting with the right weapons. You brought devices to an ecosystem fight.

Still, Microsoft has more cash than the Koch brothers, and that has to count for something, assuming Microsoft ultimately figures out the market. In the good old days, it only took Microsoft 3.1 versions to get it right, but these days they have to put at least a half-dozen locomotives in the ditch in their effort to find themselves. In a post filed under “nostalgia,” Horace Dediu recalled former mobile/telecom partners of Microsoft and where they ended up. LG: moved to Android, Motorola: now droid-only, Palm: sold self to HP, Nortel: bankrupt, Verizon: droid, Sendo: bankrupt.

Ed Burnette, hardly a knee-jerk Microsoft basher, translates the Microsoft-Nokia announcement as “When we grow up, we want to have fanbois like those Apple and Google guys,” and calls the deal all vaporware and press releases.

Robert Scoble doesn’t think the deal is a train wreck, but you can’t take much consolation in that. He thinks your current operating system sucks, you have no apps, and you’re stupid. He says you could have gone with Android, but your executives weren’t smart enough to make it work. “You add that up as a salad,” he suggests inexplicably, just before calling you all nuts.

Your enemies are merciless: Google VP Vic Gundotra says of the deal, “two turkeys don’t make an eagle.” One writer derided Microsoft Phone 7’s “complete dearth of features.” And the Register points out a familiar feature of the deal: no mention of exclusivity on Microsoft’s part. That’s the standard Microsoft pre-nup: we get to cheat.

Always Look on the Bright Side of Life

But just because everybody thinks you’re headed for a train wreck is no reason why you should wallow in despair. It could work out. I’m pretty sure I saw this tweeted somewhere:

Many at Nokia are optimistic about the deal.–#kentbrockman

John Shade was born under a cloud in Montreux, Switzerland, in 1962. His parents were a Russian novelist and a line from Timon of Athens. Subsequent internment in a series of obscure institutions of ostensibly higher learning did nothing to brighten his outlook. Send the author your feedback or discuss the article in the magazine forum.