PragPub 2009-09: Issue #3

Table of Contents

Features

Responsive Design

by Kent Beck

Kent reflects on his years designing software and concludes that we need a new approach, one that embraces ambiguity and tension.

Clone Yourself

by Chad Fowler

Thinking of your career as a system leads to some surprising conclusions.

...And Your Bugs Can Sing

by David Koelle, Brian Tarbox

What if your log files sang to you?

Beauty in Code

by Scott Davis

Scott talks about striving for beauty in the code as it runs and as it is written.

Departments

Up Front

by Michael Swaine

We are three.

Choice Bits

Gems from our forum and other sources.

How Do We...?

by Dave Thomas

People often ask how we do what we do. This series explains…

Get a Life

by Daniel Steinberg

Cut yourself some slack. You deserve it, and your work may benefit from it.

Swaine’s World

by Michael Swaine

Mike gets a surprise visitor and surveys computer calisthenics of the past, present, and future.

Shady Illuminations

by John Shade

Open source needs women.

The Quiz

by Michael Swaine

A monthly diversion at least peripherally related to programming.

Calendar

Author sightings and other notable events.

Except where otherwise indicated, entire contents copyright © 2009 The Pragmatic Programmers.

Feel free to distribute this magazine (in whole, and for free) to anyone you want. However, you may not sell this magazine or its content, nor extract and use more than a paragraph of content in some other publication without our permission.

Published monthly in PDF, mobi, and epub formats by The Pragmatic Programmers, LLC, Dallas, TX, and Raleigh, NC. E-Mail support@pragprog.com, phone +1-800-699-7764. The editor is Michael Swaine (mailto:michael@pragprog.com). Visit us at http://pragprog.com

ISSN: 1948-3562

Up Front

On Software Design

by Michael Swaine

In our third issue, we welcome some new writers and ask more questions than we answer.

If this issue has a theme, it’s software design.

There is no shortage of confident advice on how to design software, but does it all beg the fundamental question: Is software design even a rational process? Is software design really design, in the sense that automobile bodies and dresses are designed? Can code be beautiful, and if it can, is that an important goal to pursue, or just a frill?

These are some of the questions teased at by our authors in this issue. But they do more than tease.

To our great good fortune, Kent Beck (Mr. Extreme Programming himself) shares with us some of the insights he’s acquired over the years. But his reflections are altogether forward-looking as he begins a new project related to software design and lets us look over his shoulder as he maps it out.

Scott Davis (Groovy Recipes: Greasing the Wheels of Java) draws some thoughtful connections between music and code in an artful essay. Then David Koelle and Brian Tarbox go him one better and literally turn code into music. The ear’s answer to the eye’s way of understanding software, application visualization, is called application sonification. David and Brian use it to show you how you can listen to your code running for diagnostic purposes like an auto mechanic listening to an engine.

Chad Fowler applies system thinking to that most personal of jobs, the job of managing yourself, and comes up with some surprising conclusions.

In our regular departments and columns, Dave shares more inside info on how we do the things we do, Daniel celebrates the benefits of slacking off, John Shade makes a few more enemies, and I reminisce about programming challenges of the past, present, and future.

You may have detected in this list of writers a different balance between outside writers and authors you know from the Pragmatic Bookshelf. This is deliberate: we are widening our net. If you read into this an invitation to contribute, you are reading it right.

Choice Bits

Overheard or Understood

This month we’re sharing some feedback we received in the forum, along with other Choice Bits.

What’s Hot

Top-Ten lists are passé—ours goes to 11. These are the top titles that folks are interested in currently, along with their rank from last month. This is based solely on direct sales from our online store.

1^ | 5 | Agile Web Development with Rails, Third Edition
2 | 2 | iPhone SDK Development
3 | 3 | The RSpec Book
4v | 1 | Metaprogramming Ruby
5^ | 6 | Programming Ruby 1.9
6^ | NEW | Agile Coaching
7 | 7 | Programming Clojure
8^ | 11 | Core Data
9 | 9 | Programming Scala
10v | 8 | Language Design Patterns
11v | 10 | Writing Your First iPhone Application

And Here’s What Folks Are Saying…

A question to Michael on “Happy Campers” [about employee retreats, writers’ workshops, and code camps]: we’ve thought about something similar, but wasn’t sure whether people should bring along partners and little people. Any thoughts or experiences? — Ben Still

Ben, our experience with retreats has been that encouraging employees to bring partners and little people really enhances the experience. We all get to see each other as more rounded individuals. But it depends on what you want to get out of the retreat and to some extent on the makeup of your staff. If you hope to get any work done or talk a lot about work issues, families may feel left out unless you plan separate activities for them. With a small staff you might want to play out the dynamics in your head: will John’s kids be the only little ones there, or if two employees have little kids, would they enjoy playing together? And if you do include families, the venue has to have activities/attractions for everyone who’ll be there, enough to keep them occupied all the time. — Michael Swaine

What a great article from John Shade. Just great. I am very excited about the next issue, what Shade will bring next. But I was confused this time, because I don’t get the idea behind the article “Get a life.” Thanks for your effort to make this great magazine. My only complaint about this magazine is that it has a lot of information about the Mac Platform but I am a PC user. Please more about this system would be great. — Tony Muraty

Tony, thanks for the very helpful feedback. The Mac vs. PC thing is part of a bigger challenge we will always face: striking the right balance between platforms, languages, etc. We’ll keep working on it. “Get a Life” is a little outside the frame, but that’s its mission. In upcoming issues it will focus on music and slacking off. Sometimes, I hope, it’ll be outside the frame in a way that works just right for you. I’ll pass your comments on to John Shade. — Michael Swaine

If I'm not saying “HELL YEAH!” about something, then say no. Meaning: When deciding whether to commit to something, if I feel anything less than, “Wow! That would be amazing! Absolutely! Hell yeah!”—then my answer is no. When you say no to most things, you leave room in your life to really throw yourself completely into that rare thing that makes you say “HELL YEAH!” — Derek Sivers

Reason #99 why I don’t bother with Wolfram Alpha: “Wolfram|Alpha isn’t sure what to do with your input” — Scott Hanselman

Can you imagine a worse show to go to than the currently touring Creed with Hoobastank? I challenge the Twitterverse! — Andy Lester

No laughing matter: Nitrous Oxide (a.k.a. laughing gas) Fingered as Monster Ozone Slayer. — John Makower

Lego Sudoko Solver: http://bit.ly/1yukHs Machine learning, robotics, awesome. — Tim O’Reilly

How Do We...?

How Gerbils Make Sausage

by Dave Thomas

People often ask how we do what we do. This series explains…

This month's question: How do you create book-specific markup?

We create our books using a custom XML-based markup language called PML (Pragmatic Markup Language, natch). We then process this to create the printed, PDF, epub, and mobi versions of our books. (When we remember, we also use the same source to update the book’s table of contents in our online store.)

Each book project starts out with the standard set of markup tags. We try hard to use semantic markup, rather than layout markup, so authors use tags such as <methodname> and <keyword> rather than <code> and <b>. And we encourage authors to try to work with that markup: over the years, it has grown to encompass most of the things we need to create our books. But every now and then an author needs something special—something not covered by the standard markup. For example, when writing Rails Recipes, Chad Fowler wanted to be able to write Arabic text for an example that illustrated internationalization.

So, the question is: how do we support custom markup for each individual book, while still keeping the whole build system standardized? It all comes down to some careful management of search paths, combined with some cool XSLT hacks.

Every book starts out as a clone of a single master book project in our repository. When authors check their books out, they find two main top-level directories. One, called Book, is where they do their work. The other, called PPStuff, is a read-only version of our toolset. The PPStuff files are never changed for a particular book—all the per-book customization occurs below the Book directory.

Let’s start with the DTD, as this defines the markup that’s supported for a particular book. Each book defaults to using the standard entities and schema that we provide, but they also get to override them and add their own. We do that using a trick that effectively lets us subclass the DTD for each book.

A couple of levels deep in the Book tree you’ll find the DTD that our tools use. It looks something like this:

| <!-- add extra entities here -->
| <!ENTITY % PPBookDTD SYSTEM "../../../PPStuff/util/xml/ppbook.dtd">
| %PPBookDTD;
| <!-- add extra elements here -->

The two lines in the middle load up the global DTD from the shared PPStuff tree. That’s where the standard markup is defined. But, if the DTD in the Book tree defines entities before it includes the master DTD, those entities will take precedence—the local DTD can override our normal entities. Similarly, any elements defined after the master DTD is included will take precedence over elements in the master DTD.

You can use these two facts together to allow the document structure to be redefined. For example, the master DTD defines the structure of tags that can appear in normal text flow using something like this:

| <!ENTITY % local.char.flow "">
| <!ENTITY % char.flow
| " #PCDATA
| | acronym
| | author
| …
| | xmltag
| | xmltagpair
| | xref
| %local.char.flow;
| ">

A tag that includes a character flow (like the <p> tag, which formats a paragraph), is defined in terms of this character flow entity:

<!ELEMENT p ( %char.flow; )*>

The master DTD defines the local.char.flow entity to be empty. When it is included in the char.flow entity, it has no effect. But Chad’s book needs the ability to mark up Arabic text inside a paragraph. So in its book-local DTD, you’ll find:

| <!ENTITY % local.char.flow
| "| arabic
| | idxmethod
| ">
| <!-- load standard PPBook -->
| <!ENTITY % PPBookDTD SYSTEM "../../../PPStuff/util/xml/ppbook.dtd">
| %PPBookDTD;
| <!ELEMENT arabic ( %char.flow; )* >

By defining local.char.flow before the definition in the master DTD, Chad’s DTD inserts <arabic> as a valid tag inside paragraphs. It then defines the content of the tag itself after including the master DTD.

Although this sounds complex, it actually gives us a lot of flexibility. The master toolset is never changed for a particular book, which means we can share it across all of the titles under production. If we want to make a global change, we change it in one place, and all books get the update. At the same time, each individual book has hooks into the toolchain to allow it to add customizations.

Next month we’ll see how those customizations work when it comes to formatting the book itself.

Dave Thomas is one of the Pragmatic Programmers.

Get a Life

Slacking Off

by Daniel Steinberg

When a reporter asked Spencer Tracy what he looked for in a script, Tracy quipped, “Days off.” But what if his days off actually made him a more rounded actor?

I love the rhythm of the academic year. There’s a regularity to the day that feels refreshing after the anarchy of summer. A month ago the streets were empty this early in the morning. Now there’s a parade of kids each carrying an overloaded book bag and a musical instrument. The high school kids walk one way down our street and the elementary kids head the other way a half hour later.

What makes this all seem fresh and fun when it begins again in September is the summer break. The kids have had a chance to get away from the routine of school. They’ve gone their own ways and done their own things. When they come back in the fall they are in a new grade, doing something different than they did last year. Eighth grade uses the skills they acquired in seventh grade, but the challenges and tasks eighth graders face are new.

Why can’t we have that experience as adults?

I was spoiled as an academic. In my last semester of college I asked my friends what they would be doing that summer. Other than those going on to graduate school, they looked at me as if I needed to grow up. We aren’t kids any more. We don’t take summers off and then come back to work. We start our jobs and take the vacation days we earn—and perhaps when the company tells us we can. At least that’s how it is in the US.

According to Infoplease.com, the average US employee gets thirteen vacation days per year. That’s about two and a half weeks. The next closest on the list were Japan, Korea, and Canada, where employees averaged twice as much. In the UK employees averaged nearly six weeks and in Brazil, Germany, and France employees averaged nearly seven weeks per year. Topping the list was Italy where workers average seven weeks of paid vacation per year. That’s nearly a summer’s worth.

Imagine getting a summer break every year like you did when you were a kid. Now reflect on the fact that this is the reality for Italian workers. Ah, but the US is so much more productive than Italy, right? Whether or not that’s the case, there is a dangerous delusion lurking here.

That’s the phrase Tom DeMarco uses in his book Slack. He begins the book with the warning of “a dangerous corporate delusion: the idea that organizations are effective only to the extent that all their workers are totally and eternally busy.” He talks about the companies that pared their staff to the bone until all they were left with was people who were working too hard to have the ability to really be effective. He says, “What got cut out of the most aggressively purged organizations is the capacity to change.” Companies traded their futures to meet short-term goals.

DeMarco’s book is a look back at the legacy of the nineties, but now that we are seeing record-level layoffs it is worth revisiting, as its lessons also apply to today’s world.

It goes against the grain to encourage slack in an organization, even though there are cases that show how an organization can benefit from slack. Google’s well-known 20-percent time, where engineers spend one day a week working on something they are passionate about, pays off for them in many ways. Understand, the engineers aren’t goofing off; they are working on something of their own choosing, something other than their day-to-day jobs.

Google publicizes the projects that have resulted from this 20-percent time but benefits would be realized even if none of those projects ever shipped (or more likely because it’s Google, went Beta). It’s similar to taking a summer vacation, but the time off the job is woven it into the work week in a way that impacts the organization less.

There’s a tremendous benefit to the individual in taking time off from work and doing something completely different. In the last issue of PragPub, editor Michael Swaine wrote about the benefits from the retreat that his partner’s staff goes on together. They see each other in a different setting and bring back this shared experience when they return to work.

But what practical use is this insight to those of us who are stuck in a country and a corporation that isn’t buying it? What do you do if your company isn’t supportive of these shorter or longer breaks?

I had a friend who was determined to take his summers off. He worked at a very large company in the early 80’s and took every summer off without pay to refresh and re-energize. One summer they told him that they just couldn’t give him summers off any more. He told them he saw their point, but that he was taking the summer off anyway, and they could decide in September whether or not they wanted him back. When the end of summer came, sure enough, the big company didn’t take him back. He shrugged his shoulders, smiled, and said goodbye.

“Well, that’s no good,” you say, “I could never do that. Not in this economy.”

OK, maybe not. But that’s just one example. Here’s another example of someone who thought differently about time off. A year ago, Dave, Andy, and I met someone with a radical approach to how much he should work each year. Before the year began, he figured out two numbers. The first number was the amount of money he needed to support his family and his obligations and provide a responsible amount of cushion. This was the minimum amount he needed to make each year. He would ensure that he booked enough business to make at least this much money.

He also had a second number that represented the amount of money that he wouldn’t feel comfortable making more than. He felt that there is a limit to how much you can earn in a year without it being excessive and a bit selfish.

So each year he set about working enough to earn more than his first number. Once he reached that threshold he would slow down and only take on other projects that interest him. When he hits the top number he tries to eliminate all of his billable work. He might work on a new software technique or learn a new language or, as he had just done, spend time training to swim competitively.

Then there are sabbaticals, where you take an entire year off every so many years. Some companies actually provide sabbaticals, but it’s something you could create for yourself, too. I just read Daniel Pink’s blog post about Stefan Sagmeister. Pink reported that at the recent TED Global conference Sagmeister reasoned that if you look at your life, “The first 25 or so years are devoted to learning, the next 40 or so to working, and the final 25 to retirement.”

So what if you made a simple change. Take just five of those years from your retirement and slot them into your working years. This is like a traditional academic sabbatical. Every seven years Sagmeister takes a year off. Pink writes that  “Sagmeister closes his design shop [and] tells his clients he won’t be back for a year [...] It sounds costly, I know. But he says the ideas he comes up with during the year ‘off’ are often what provide the income for the next seven years.”

Back when I was in school I looked forward to the school year when I would learn new things. As much as I enjoy learning while I’m at work, I really look forward to the time when I can step away from my work a bit and  learn new things with no clear benefit.

So imagine for a moment that you could take time off to work on something entirely different. How much time? Maybe it’s a day per week, maybe it’s a summer break, maybe it’s the end of the year after you’ve earned a set amount of money. Or maybe it’s a sabbatical. Whatever the structure of your time off, what would you do with it?

Daniel is the editor for the new Pragmatic Life series and the series of Mac Developer titles for the Pragmatic Programmers. He writes feature articles for Apple’s ADC web site and is a regular contributor to Mac Devcenter. He has presented at Apple’s Worldwide Developer Conference, MacWorld, MacHack, and other Mac developer conferences. Daniel has produced podcasts for Apple featuring the work of developers and scientists working on the platform. He has coauthored books on Apple’s Bonjour technology as well as on Java Programming and using Extreme Programming in Software Engineering classes.

Swaine’s World

Your Hands Are Like Dogs

by Michael Swaine

“Your hands are like dogs, going to the same places they’ve been. You have to be careful when playing is no longer in the mind but in the fingers, going to happy places. You have to break them of their habits or you don’t explore; you only play what is confident and pleasing. I’m learning to break those habits by playing instruments I know absolutely nothing about, like a bassoon or a waterphone.”—Tom Waits

We were dining with friends when Tom Waits walked in.

It had been a day for surprises at Summer Jo’s, the Oregon restaurant that my partner Nancy and I own and she manages. Old friends from out of state had been popping in all day long—a cousin from Phoenix, a friend from Wisconsin, former colleagues from the Bay Area. But I didn’t do a double-take until the little guy in the porkpie hat sauntered past our table.

I don’t usually get all giddy over celebrities, but Tom Waits is one of my heroes. Whether or not you dig his whiskey voice and dive-patron persona, you have to admire an established artist who still pushes himself to pick up instruments he knows nothing about in order to break his fingers of their old habits. Admire, and maybe emulate.

While googling Waits lyrics, I came across the quote you’ll see at the top of this column. It started me thinking about that experience of picking up a truly unfamiliar (programming) instrument, and how it can stretch your mind.

Yesterday Is Here

The first time I had that experience as a programmer, it was forced on me.

In grad school, as a teaching assistant to Dan Friedman, I was tasked with running an undergraduate lab class in Basic. When Dan handed me a peculiar little book he’d written and told me to “learn it,” I was unimpressed. The Little LISPer didn’t look like it had anything to do with Basic.

In fact, the book didn’t have much of anything to do with anything I’d ever encountered—in subject matter, pedagogy, or purpose. It consisted of nothing but questions and answers, and most of the questions were unanswerable on the basis of the knowledge you could be expected to have when encountering them. Years later, when I was trying to develop software for Apple’s newly released Macintosh computer and struggling through Apple’s developer documentation, Inside Macintosh, the joke was that the prerequisite for reading Inside Macintosh was a thorough knowledge of Inside Macintosh. But it seemed literally true that you couldn’t begin to understand The Little LISPer until you had mastered The Little LISPer.

Somehow it worked its magic, though: by the time I’d puzzled through that strange little book I really got LISP. More than that, more than adding to my then-meager store of programming knowledge, I’d learned a new way of thinking about programming. My old strategies for decomposing a problem into parts no longer fit this new way. Wherever I’d employed the hammer of iteration I now tried the lever of recursion. My ideas about how big a program should be and in what order I should tackle the subproblems of a programming task changed drastically. I’d put down the guitar and picked up the bassoon.

I never did figure out how to use it in teaching Basic to undergraduates, but I don’t think Dan expected me to.

Although I moved on to more sophisticated LISP books and projects, for practical reasons, I’ve done no serious programming in LISP since grad school. Nevertheless, I think that learning LISP did me more good as a programmer than learning C or Java or Ruby. It’s like the Latin I took in high school: I don’t read or write it today, but the Latin I learned lets me see inside words in a way that I couldn’t without that experience. And knowing LISP lets me see inside problems in different and helpful ways.

Or so it seems to me. Of this I’m sure: Learning this new instrument of LISP was excellent mental finger exercise.

Trouble’s Braids

Musing on that Waits quote I flashed forward to 2009, and asked myself what a really challenging and useful finger exercise for programmers would be today. Hmm, how about parallel programming?

It’s odd that parallel programming would feel unnatural. The universe is parallel. Your brain is parallel. If you play the piano, your hands play in parallel, and not SIMD parallel, either. The Web is parallel.

And now, belatedly, the computer on your desk is parallel. Problems in heat dissipation, power consumption, current leakage, and the speed of light have spelled the end of single-core growth. “Like it or not, folks,” Ted Neward sums it up, “the path forward isn’t one that you get to choose.” Intel and the other chip makers have already chosen it for you. Multicore processors are here and waiting to be exploited.

It’s not just multicore, of course. There are many kinds of parallel programming. And the problems are not really new. We’ve been dealing with concurrency in operating systems for decades. But when it comes to developing software for richly parallel hardware, we’re in the era of rabbit ears, says James Reinders, who evangelizes all things parallel for Intel and is the author of a new book called Intel Threading Building Blocks.

In the pioneering days of television, the early adopters had to be willing to diagnose and replace vacuum tubes, fuss with vertical and horizontal hold, and experiment with the positioning of the rabbit-ears antenna on top of the set to get the best picture. In Reinders’ analogy, you are the poor sap balancing the rabbit ears on his head and twiddling the knobs, trying to get a decent picture. The tools to help you do the job are, Intel implicitly admits, somewhat primitive or lacking.

Which may explain why Intel identifies as the top challenge for parallel computing—er, you. Or as they put it, “Finding concurrency in a program—how to help programmers ‘think parallel.’” See, if only programmers could think parallel, we’d be rocketing ahead in the exploitation of the power that Intel is so generously building into modern processors. It’s your fault.

OK, maybe they don’t mean it in quite that way. But the fact is, thinking in parallel is challenging. Despite all the parallelism going on around us and inside us all the time, thinking of problems in parallel terms doesn’t seem to come naturally to us.

Reinders would have parallel programming taught as a fundamental component of any computer science curriculum, starting in high school. “We’ve progressed from the introduction of multicore processors in some computers to nearly all new systems having multicore processors,” he says. Yet “parallel programming remains an advanced topic in graduate school studies and notably absent from too many university undergraduate courses.... [W]e can do more.”

“We can do more” is a good summary of the state of parallel programming today. To master parallel programming, you need to master new tools (if you can find them), new rules, new languages (there are starting to be some good ones), discover or adopt new algorithms, patterns, design strategies, rethink things you take for granted, and come to programming with fresh eyes, willing to see anew. Learning a new language is only part of it; we’ve really entered a new generation of programming and those who actually learn to think in parallel will have an edge. The positive spin on this daunting state of affairs is that there is much to engage the willing learner.

In other words, good finger exercises. Challenging in the best sense. In parallel programming, there are fundamental algorithms yet to be discovered, breakthroughs to be made by someone. Might as well be you.

We are in the rabbit-ears era of parallel programming. But that’s a good thing.

Innocent When You Dream

But what about something that really takes your coding fingers out of their happy places?

Sometimes the best mental exercises are the ones that have no immediate chance of being applied to any practical problem. Pure mind play. That’s what Jack Woehr has been up to recently, teaching himself quantum computing in public in his Dr. Dobb’s Journal blog. He’s been interviewing experts, chatting with readers, thinking out loud.

“Over the years,” he writes, “I have slowly imbibed a tiny sip of quantum mechanics in pursuit of quantum computing. It’s cumulative and it’s starting to make sense. I’m now going back and reading early classic papers on the subject by the originators of QM.” Jack entreats the experts he interviews to “talk to me like an engineer.” Researching quantum computing (QC) is definitely an academic pursuit, but what Jack is after is a practical, engineer’s understanding of the subject.

In 1992, David Deutsch and Richard Josra demonstrated that a quantum computer would be able to solve exponential problems in polynomial time: to make the intractable tractable. A quantum computer, if such a thing existed, could solve problems that were heretofore not merely unsolved, but practically unsolvable. The payoff for creating a quantum computer would be enormous.

And there has been progress, some of it quite recent. On August 6, the National Institute of Standards and Technology announced that NIST researchers had “demonstrated on a small scale all the generally recognized requirements for a large-scale ion-based quantum processor. Previously they could perform all of the... processes a few at a time, but now they can perform all of them together and repeatedly....”

The processes include initializing qubits (the quantum analog to bits) to the desired binary value, storing qubit data in ions, performing logic operations on qubits, transferring information between different locations in the processor, and reading out qubit results. Obviously, this is a long way from having a working quantum computer; equally obviously, if you can do these things you should be able to build one.

The computer you can build, though, will be very different from any computer we’ve ever encountered. Quantum mechanics, as one of the experts Jack interviews points out, isn’t binary logic, and qubits are not bits. Which is why QC will lead to entirely new ways of computing things, and why QC will require—and inspire—new algorithms. Jack interviewed Jonathan Home, one of the NIST researchers, who said, “[I]t’s an open research problem, what problems will be better-solvable by quantum computing.” This is the wild west of computing, an untamed frontier.

And out on the frontier you don’t have all the conveniences of home. As Jack says, “QC today is not what ‘us programmers’ usually call programming. This is gate design and basic logic.” Languages? It’s a little early for such frills. “When we see what are the basic operations supported by the machine we’ll weave languages.”

That doesn’t mean people aren’t developing QC algorithms, though. Shor’s algorithm for factoring large numbers rapidly is all by itself reason to try to develop a quantum computer—and to worry about how the world will change when affordable quantum computers running Shor’s algorithm make today’s encryption solutions obsolete. Robert Tucci, who has weighed in on Jack’s blog and who writes about quantum computing on his own blog, has developed a Java application that generates the qubit circuit necessary for doing simulated annealing. You can do quantum computing today, it’s just that you will find it a strange exercise.

Viewed as an instrument, quantum computing would be an odd one—maybe a kazoo. Viewed as a mental exercise... “Quantum computing,” Jack says, “depends on quantum mechanics and quantum mechanics makes my head hurt.” Indeed. As mental exercise, studying quantum algorithms is extreme sport. But Jack’s glad to be doing the exercise. He says, “It’s revitalizing my technical chops.”

Thinking about all these mental exercises tired me out. Over dinner, I told Nancy that I knew just what I needed to take me out of my comfort zone and get me thinking differently. That’s right, I said, I need to get me a porkpie hat.

Michael Swaine is the editor of PragPub. He’s been looking for a good hat since he first felt a draft on the top of his head.

Responsive Design

Coming to Grips with the Chaos That Is Software Design

by Kent Beck

Software design isn’t the rational process we think it is, Kent has decided, and he’s embarked on a project to discover just what it really is.

In the Beginning

The second program I ever tried to write was a copy of the Star Trek simulator I had seen at the Lawrence Hall of Science. I had gotten through the BASIC language workbooks in a weekend. I had written a Roman numeral printer with no problems. How hard could Star Trek be?

I can remember my feelings sitting down at the teletype: anticipation, excitement, puzzlement, frustration, despair. I had no idea what to type. I knew the statements of the language, no problem. What I didn’t understand was design. I had no idea what elements there could be above the level of statements or how those elements could be related to produce the behavior I was imagining.

That day I just gave up and walked away. Over the next decade I slowly got a sense of what elements there could be in a program and how they could relate to each other. I acquired an aesthetic sense. I could take pleasure in a good design and feel discomfort in the presence of a poor one. Eventually I could take a problem and design software to address it.

But something was still missing. I couldn’t explain what I did when I designed. Little pieces like refactorings I learned, with a lot of collaboration, to articulate, but I couldn’t escape the nagging sense that design had hidden structure, like the plumbing and wires in the walls, structure that was critical to really understanding the process and product of design.

Revealing that hidden world of design is the goal of the Responsive Design Project. I want to understand and articulate the deep structure of software design. While the project has only just begun, I have some practical results I’d like to take you through briefly in this introductory article. And I’ll share with you the result that most disturbs and excites me:

Our illusion of control over software design is a dangerous conceit best abandoned.

In the Slightly Later Beginning

In 2005 I was invited to sit on a panel celebrating the 25th anniversary of the publication of Ed Yourdon and Larry Constantine’s Structured Design, the book that introduced the terms coupling and cohesion. I’d had a copy in my library since I’d used it as a college textbook, but, since I had used it as a college textbook, I hadn’t really read it. As I studied it in preparation for the panel, I realized that many of the questions I had about software design had been answered two decades before. This motivated me to make my informal study of design more structured and rigorous, and to communicate the results to a new generation of programmers. This was the official genesis of the Responsive Design Project.

Design is good. Design is central to effective software development. Programmers can add features steadily to well-designed software. Programmers can easily test well-designed software. Well-designed software is easy to tune for better performance. Most of the hard problems in programming turn out to be design problems.

But design has a dark side. While there isn’t a single best design for any system, there are many poor designs for that same system. The more experience a programmer has, the more design ideas he knows. The temptation is to put these design ideas in the system now because you just know you’ll need them eventually. Over-designing early leads to delaying feedback from real usage of the system, makes adding features more complicated, and makes adapting the design more difficult. By the same token, under-designing makes adding features more complicated, increases defects, and makes adapting the design more difficult.

The word “responsive” in the name of the project reflects this need for balance. Some design needs to be done in advance of coding, but over the life of the project most design will be done in response to the changing needs of the system and the changing and growing understanding of the developers.

The project is based on three parallel tracks of inquiry:

Quantitative. Measure real designs and see how they behave “by the numbers.”

Introspective. Carefully watch my own practice of design and draw lessons from the patterns I find.

Pedagogical. Teach software design and draw lessons from the students and their subsequent practice.

The Tour

Here is a quick overview of the lessons I have learned so far. Each topic deserves an essay of its own, or even a book chapter (say, you don’t know anyone who publishes technical books, do you?).

Beneficially Relating Elements. Designs are made up of elements that relate to each other in beneficial ways. Looked at actively, designing is creating and deleting elements, creating and deleting relationships, and increasing the benefit of existing relationships. Elements contains elements and so on down (and up).

Design is Fractal. There is no fundamental difference between implementation, design, and architecture. They are all a matter of beneficially relating elements. Learn to bounce between levels of abstraction. Most design takes place at a single level of abstraction but big advances often come as a result of rearranging the levels.

Safe Steps. Resolve the tension between efficiency and safety in favor of safety, then learn to take small, safe steps fast enough that from the outside it looks like you are flying.

Isolate Changes. Before making a change, isolate the area to be changed from the rest of the system so you can change an entire element at a time. For example, before changing a part of a procedure, extract the area to be changed into its own procedure. Make the change, then inline the changed sub-procedure if appropriate.

Embrace Ambiguity. Half done with a change and not sure what to do next? Stop and deploy (you can do this if you are working in safe steps). Complete the change when you know what to do.

Cultivate Confidence. Master your tools. Your feeling of mastery will improve your cognition.

Cultivate Humility. Try tools or techniques you aren’t comfortable with. Being aware of your limitations will improve your effectiveness.

Exploit Symmetries. Divide similar elements into identical parts and different parts.

Trust Succession. Design simply today, knowing you can add complexity needed tomorrow.

Design Is a Team Sport. The social side of design is as important as the technical side. Learn the skills needed to communicate and understand designs.

Play with Words. Recasting a design using a different metaphor can revolutionize your understanding.

Play with Pictures. Explain a design using hand-drawn pictures. Ask others to do the same. Design benefits from your whole mind: symbolic, verbal, visual, kinesthetic, conscious and unconscious.

Clear Strategies. All design changes use one of four strategies:

Leap. Make the new design and use it immediately.

Parallel. Make the new design and run both new and old simultaneously.

Stepping Stone. Create a platform to bring the desired new design within reach.

Simplification. Only implement part of the design now and add complexity bit-by-bit later.

Inside or Outside. Change the interface or the implementation but not both at the same time.

Toss It. Checkpoint often. If you get confused, learn what you can and revert. Now. Beware the sunk cost fallacy.

Keep It. If the system is working and creating value, keep it and improve it.

Start over. Sometimes the constraints of the problem plus the constraints of the existing system are just too much to handle at one time. I prefer incremental improvement, but sometimes it's best to just wipe the slate clean and start over.

Both. Faced with design alternatives without a clear winner, do it every way. Just coding each alternative for an hour is more productive than arguing for days about which is better in theory, and a lot more satisfying.

Suit Design to Needs. A design strategy intended to maximize feedback from rapid changes looks very different from a design strategy intended to increase value by reducing cost. Both are appropriate at different phases of a system’s lifecycle.

Use Tension. There will always be tension between designing better and delivering sooner. Learn to fuel your creativity from this tension.

Conclusion

Many metrics in design are power-law distributed. For example, if you graph a histogram of the cyclomatic complexity of the methods in a large system on a log-log graph you get a straight line. In other words, there are many very simple methods and a few very complicated ones.

This same distribution is found in many measurements from nature—the intensity of avalanches, for example. This leads me to the most exciting and disturbing lesson I have learned about design so far:

Design is not a rational process.

Much as we like to think we are making design decisions, much of the structure of our systems emerges from the nature of the problem and the nature of the tools we use. Our designs shape us as much as we shape our designs. Insisting that we as designers are “in control” is counter-productive, leaving us wasting energy trying to act against nature. The challenge is to embrace responsibility and at the same time take an appropriate role in designing. Learning to achieve this balance is the next goal of the Responsive Design Project.

Kent Beck is the founder and director of Three Rivers Institute (TRI). His career has combined the practice of software development with reflection, innovation, and communication. His contributions to software development include patterns for software, the rediscovery of test-first programming, the xUnit family of developer testing tools, and Extreme Programming. He currently divides his time between writing, programming, and coaching. Beck is the author/co-author of Implementation Patterns, Extreme Programming Explained: Embrace Change 2nd Edition, Contributing to Eclipse, Test-Driven Development: By Example, Planning Extreme Programming, The Smalltalk Best Practice Patterns, and the JUnit Pocket Guide. He received his B.S. and M.S. in Computer Science from the University of Oregon.

Kent is currently researching effective software design. If your company is interested in sponsoring this research in return for early access to his results, please contact him directly. His other business activities include contract programming using Java/Eclipse, writing, consulting (mostly remote), and presenting workshops with his partner Cynthia Andres.

Kent would like to thank Dave Ungar and Mike Swaine for timely, thoughtful feedback on an earlier draft.

Clone Yourself

Destroy Your Job Through Automation and Outsourcing

by Chad Fowler

Chad explains why you should hire the least-skilled person possible for any job—even if it’s yourself.

Since the release of The Passionate Programmer, I’ve been thinking a lot about goals. The book is about how to build a remarkable career in software development. But, I ask, what does that lead to? Having a remarkable career sounds like a pretty good goal, but is it?

In a job I had seemingly several careers ago, I was a Six Sigma Blackbelt. That job was remarkable in that I had arguably the lamest title possible in any industry other than Master Blackbelt, which was the “promotion” from Blackbelt. Six Sigma is one of those methodologies that starts life as a good idea and, by the time it reaches anyone who has to actually do anything, is indistinguishable from self-satire. But, despite its amazingly bad implementation, I still managed to learn some pretty important lessons from this data-driven quality management approach. One of the simplest of such lessons is that we humans often confuse requirements with design. By that I mean that when we think we’re stating requirements for a system, we’re actually designing the system and stating our design assumptions.

A programmer acting in the role of a customer, for example, might say that a web system needs to support distributed session replication. That way if an application server dies, the user’s session isn’t lost. But that’s not a requirement. It’s an implementation approach. Other approaches to the real requirement (sessions don’t die when servers do) might include storing sessions in browser cookies or in a database. Distributed session replication is, in fact, probably the most complex and expensive approach we could design for this requirement.

When you’re doing something important, you can’t just take requirements at face value. You have to determine whether they’re actually requirements at all.

So, remarkable career... is that a requirement or a design assumption? I think it might be a little of both, depending on the audience.

What We Want

If having a remarkable career is not the real goal, then what is? What goal do all of us programmers seek? Money? I don’t think so. We’re creative people. We devote too much of ourselves to this craft for it all to boil down to cash. Money is important but, again, I think it’s part of the implementation of the real goal.

Recognition? Again, that’s a kind of currency. On a small scale, it buys you respect among your co-workers and peers. At a large scale it does the same industry-wide (or beyond).

So if cash and recognition are both types of currency, what do they buy us?

I’ve come to believe that what we all want is freedom.

Freedom from What?

You might say freedom is also a currency like money and recognition. But it’s hard to break down any further.

So what do we want freedom from or for? As creative people, I believe that most of us are ultimately looking for freedom to work on what we believe is important and to do it in a way that we believe to be right. In short, we’re looking for the freedom to be creative.

Early in our careers, we move from low-level jobs toward higher-level jobs partially for the money and recognition but largely for the ability to make decisions.

Let’s not confuse the issue—decision-making power and freedom are not the same thing. That’s where many of us go awry. Management jobs are effectively decision-making power wrapped up in a seemingly neat little package and hung in front of us like piñatas waiting to be shaken until the candy comes out. But this is the kind of candy that tastes OK but might make you really sick depending on how your immune system responds. It’s definitely not for everyone.

And it definitely doesn’t amount to freedom. By taking on a management job you’re assuming a greater amount of responsibility but relying on a bunch of other humans for your success. As an outside observer, it might look like an easy job, but experience has shown me that any time I have to rely on a large number of people to do even something trivial, it can feel like more work coordinating, motivating, and steering than just doing all of their jobs combined. (I did say “feel”—it’s not really more work. You get the idea.)

Of course, there are ways to manage teams that don’t eat up your every waking hour. And the manager’s solution to this problem leads us down the same path we must traverse to find freedom in our careers generally.

It’s Not about the People, Stupid!

Conventional wisdom says the following about people and teams: hire the best, get out of their way and let them do their jobs and you’ll achieve the greatest success. The more skilled and talented your team, the better chance you have of exceeding expectations. Hire inexperienced people and pay them low salaries, and it’ll be an uphill battle.

That’s what they say anyway.

As much as I’d love to believe this (it makes us feel good to believe it, doesn’t it?), my experience as a manager says otherwise. I’ve taken a chance on people I thought had potential but not enough experience or depth and been blown away. I’ve hired the smartest, brightest people and been completely underwhelmed. In fact, in my experience, this occurs almost frequently enough to call it typical.

I was recently reading Michael Gerber’s The E-Myth Revisited and came across a piece of wisdom which shone a light on this paradox. The E-Myth Revisited is a guide for entrepreneurs with the goal of leading them away from the classic, constantly repeated problems that cause the majority of new businesses to fail. In the book, Gerber lays out a methodology for designing a business as a system as opposed to just a new job for the business owner.

When you look at a business as a system, it significantly changes your approach to managing that business. Systems are consistent and are comprised of repeatable processes. Systems are measurable.

These aren’t surprising ideas. What was surprising to me was that Gerber recommends hiring the least skilled person possible for each job in your business. When I read this I did a double-take. I was sure I had misread it. But no, I hadn’t. And the reasoning is brilliant.

You don’t hire the least skilled people simply to save money. Sure, the tactical advantage of saving money is great, but you hire the least skilled person to fill each job so that it forces you to perfect your system and to not rely on the flakiness of human whim or mood for the success of your business.

If you hire low-skilled labor for each job in a business, the only way to survive is to create repeatable, measurable processes for each position so the low-skilled laborers can follow your directions, approximating the type of job you would do if you were doing it yourself.

The Low-Skilled You

The Passionate Programmer advocates planning and executing your career as a software developer as if you were an entrepreneur running your own business with you as the product. The parallels are strong enough that it’s more than a metaphor—your career literally is a business. Most of us just don’t think of it that way.

As I reflected on E-Myth, I naturally started to wonder how to apply its lessons to the business of one’s career. What would it mean to hire low-skilled replacements for me, the business owner (of my career)? Does this idea apply at all?

My conclusion is that it does—and very well.

Dissecting the Company that Is Yourself

As you begin to think of your career as a business, there are two ways you can deconstruct the task. The first is to think of it as a process, as I lay out in The Passionate Programmer. The process might look something like this:

Choose Your Market. Choose both the technology and the domain you are going to invest in. Don’t forget that there’s more to professional life than just which programming language or framework you are going to learn. Working in the medical domain is vastly different from working on a personal productivity suite, for example, and provides different challenges and learning opportunities.

Invest In Your Product. Now that you know what you’re going to do, make the investment. Develop deep expertise in your technologies of choice, master your chosen domain, and learn the business models of that domain.

Execute. Perform like a star, optimize your workflow, minimize your defects, anticipate your customers’ needs.

Market. If you have a great product and don’t tell anyone about it, you’re robbing them of the opportunity to benefit from it. You are that product. You have invested and executed and made yourself remarkable. Your employers and customers (or potential employers and customers) need to hear about you.

Refresh. Maintain your edge. Repeat the process. Constantly evaluate yourself and your position in the market as well as your long-term goals. Are you on the right path to realize your life’s goals?

While this is a coarse-grained view of what the career management process might look like, at this stage it is helpful to break it down in a different way: as a set of different jobs. If you were running a multi-person company, you might structure this as an organization chart.

Organization charts serve two purposes. The first and most obvious is to show how an organization is structured. This can be helpful in a big organization but it obscures the more important purpose of an organization chart: to catalog and define the jobs required to make an organization run effectively.

Breaking down your own different sub-jobs into an organization chart can help you crystallize the various tasks that go into managing your career.  Here’s an example that I might brainstorm for myself:

Research & Development

Frameworks

Languages

Domains

Operations

Admin (email, expense reports, status reports, etc.)

Programming

Testing (non-TDD)

Support (production issues, bug fixing, etc.)

Finance/Budgeting (money, time, passion as currencies)

Marketing

Customer Relations

Identity and Branding

Communications

Create a System

Now imagine your salary is magically ten times what it is today. You’re accountable for the same results from your work. You just get paid ten times more to get the work done. And on top of that you’re given total autonomy to get your work done as you see fit.

Obviously one option is to keep slugging away, putting in your 40 to 60 hours per week but getting paid ten times more for it. Another option, however, would be to see if you could use some of that money to buy yourself some freedom. What if you could hire a few part time employees to do some of the jobs we identified in the previous list? What if you could pay someone else to do all the stuff you hate about your job, leaving you to focus on the work you love?

And what if the people doing each of these jobs was more focused on them than you ever are? Might they even be able to do a better job than you usually do?

Here’s where the magic starts. Take any one of the jobs from the previous list and start listing its responsibilities. This is a job description you’re creating so you can hire the replacement for this part of your professional life. What would the ideal candidate get done? What would his or her typical work day consist of?

If you had absolute freedom to devote yourself to this one aspect of your career, what would you do every day with it to make it excellent?

What starts as a job description can quite easily become a task list and ultimately a how-to manual for the job. Sure, for a creative person like yourself a lot of the overall job is informed by split decision making and your on-the-spot creativity. But as you dissect your career management system into a set of clearly defined sub-jobs, it is increasingly possible to codify just about every aspect of what needs to be done into a neat daily, weekly, monthly, and annual task list.

There may still be parts of each sub-job that need to be left to the creativity of the individual doing them. Your goal is to eliminate as many of these as possible. The rest, you can handle as CEO of your career.

Do this with all of the sub-jobs and you have created a career management system. A clearly implementable strategy. There’s just one thing missing.

Make It Measurable

In software projects, we have learned over time that the best way to know when you’re done with some code is to put a test in place for it. If the test passes, it’s done. If the test doesn’t pass, it obviously needs more work. Each complex requirement is implemented in terms of a set of tests. If all of the tests pass, the requirement as specified “works.”

Many of us in the software industry use a technique commonly called Test-Driven Development to manage this process. Test-Driven Development is iterative and starts with the tests. You write a failing test; then you make it pass.

Tests (or, to be exact, their assertions) are discrete measurements. We take a reading and if the result is correct we register a successful assertion. Each test case is a collection of the measurements recorded for each assertion, and each test suite is a collection of those test cases’ aggregate readings. So as software developers, we’re very much used to using measurement systems to drive complex processes and products.

Each task for each sub-job in your organization chart should be measurable in this same way. What makes your customer relations “job” successful? What are the big measurable outcomes of the job? An example might be performance or survey review ratings from your customers. What are your goals for those measurements?

As the world’s leading expert in managing your career, what do you believe are the measurable components that would contribute to that big measurable outcome? Are these components atomic and easily measurable? If not, all you have to do is break them down further and recurse until you hit the truly measurable part.

As a Six Sigma Blackbelt, I used a technique known as “Quality Function Deployment” (QFD, for short) to formalize this hierarchical decomposition of measurement. I recommend learning about QFD. It’s an excellent planning tool. Entire books have been written about it, so I’ll avoid going into any more detail here.

Automate and/or Outsource (to Yourself?)

How do we prevent our team mates (and ourselves) from checking in code which breaks the build? Continuous integration.

How do we ensure that those performance enhancements we made to our Web application continue to be effective and don’t regress into an unusable state? Web application monitoring and alerting.

How do we make sure we have decent test coverage on our production code? Code coverage analysis tools.

We as technologists know that when something is important, it needs to be automated. Automation saves you time, but that’s not the point. Automation also makes things... automatic. If something must happen in a system, you should never rely on your memory. Instead, you should automate either that task or a reminder and a check for the task to ensure that either the system or a person takes care of it.

In Six Sigma parlance, this is known as the “Control phase” of a project. The Six Sigma process includes a host of tools and techniques to define problems, make them measurable, analyze them, and make improvements or designs. But the finish of every Six Sigma project is to put a control plan in place to ensure that the measurable outputs of a process stay within an acceptable range of performance.

Running a business is no different. If you were in the business of nuclear power generation, you’d damn sure have a system in place to measure your power plant’s outputs. If you were in the business of flying airplanes, you would certainly learn how to read the relevant measurements from the airplane both in real time and in after-the-fact analysis. And you would put a process in place to make sure you read, interpreted, and reacted to those measurements.

This brings us to the control phase of our career management system. How do we actually put all of these jobs into motion, and just as important, how do we measure their output? Automate what can be automated and outsource what can be outsourced.

Many of the tasks which eat up your time can be automated. When looking at your daily work-life as an amorphous blob of responsibility it’s hard to see them. But as granular, measurable tasks you’ll find that tools can be put in place to automate them. Status reporting, customer communication—even some programming tasks can be automated when you can see them for their sometimes-robotic nature.

And, even if you haven’t magically gotten a 1000% salary increase, some of the jobs or tasks you have defined can be outsourced. One option is low-cost programmers. Do you work with junior programmers, interns, or outsourced developers from lower-cost countries? If so, chances are you arrogantly don’t have much faith in their ability to perform any of your job at your performance level. And as a whole, you’re probably right.

But give a passionate intern a clearly defined task with a measurable outcome (and tell them what that measurable outcome is), and you will often be very pleasantly surprised at the result. The same goes for junior programmers, offshore outsourcing partners, and others you may be working with even today.

You might also actually experiment with outsourcing your daily work. I’ve recently experimented successfully with two companies which provide offshore virtual assistance for low prices.  After a few bumps in the road, I’ve had them do everything from scheduling doctor’s appointments to detailed research for keynote presentations and a writing projects. Six dollars per hour and some creative delegation has saved me collectively a great deal of time, money, and most important, energy.

If you can’t automate a task and you don’t have the resources to outsource it, your last line of defense is to outsource it to yourself. This may sound pointless, but it’s not. You have now clearly defined each job in your enterprise and have decomposed it into a set of measurable tasks. Even if you have to do the tasks yourself, if you have an automated measurable system in place to make sure you do them coupled with concrete deliverables, you’ll find that you do the tasks better and faster than you ever did before. You are, in effect, not the same worker completing the work.

Freedom

Test-Driven Development frees us to change our software without fear. Continuous automation frees us to deploy our software without wondering if it’s horribly broken.

Business management systems free business owners to focus on the parts of the business they love.

Hopefully this little thought experiment will bring you closer to that same kind of freedom. And hopefully that freedom will make you happier.

Besides being a Pragmatic Bookshelf author, Chad Fowler has been a software developer and manager for some of the world’s largest corporations. He recently lived and worked in India, setting up and leading an offshore software development center. He is co-founder of Ruby Central, Inc., a non-profit corporation responsible for the annual International Ruby Conference and The International Rails Conference, and is a leading contributor in the Ruby community.

...And Your Bugs Can Sing

I can name that bug in five notes.

by David Koelle, Brian Tarbox

With application sonification, you can listen to your code and tell when it’s hurting.

JFugue is an open-source API for programming music in Java. It redresses the shortcomings of Java’s Sound package for working with Musical Instrument Digital Interface (MIDI) data. For example, the following segment of JFugue code plays a scale:

| Player player = new Player();
| player.play("C D E F G A B");

When music programming gets this simple you can build more advanced musical capabilities on top of it without adding significant complexity to the API. For example, you can place musical segments like the above into Pattern objects, and recombine or alter Patterns in a variety of ways (reversed, inverted, pitches or durations changed, etc.) to experiment with musical phrases. JFugue provides an API for creating rhythmic patterns and for microtonal music as well as an API for sending music to, and receiving music from, external MIDI devices.

In this article, we describe an application of JFugue to—of all things—log file analysis.

Debugging on Your Hands and Knees

The enterprise space is characterized by large, complex systems; multi-threading and multi-vendor configurations are the norm. Debugging in this space is painful. Given the importance of these systems it is generally not an option to debug the live production system, and given the scale and cost there is often no realistic lab system available either. And the complexity of the systems means that even the author can’t predict the behavior of each component.

So you end up scrutinizing huge log files trying to understand why the system fell apart. Most of us are all too familiar with these log files. They are a case study of bad signal-to-noise ratio. Typically very large, they contain only fragments of the information we actually need. One of my first experiences as a developer was trying to track down a memory leak while working for Digital Equipment Corporation. I was toiling at a facility known as The Mill, a converted textile mill with enormously long hallways. To go through a log file, I printed off a 50-foot (yes, fifty!) log file printout and spread it out over the floor. For three hours, I crawled on my hands and knees highlighting interesting parts of the log.

You would think that, 20 years later, our technology would have dramatically improved. Yet today it still seems that grep and awk are our frontline tools for log file analysis.

Application Sonification: Use More of Your Senses

One powerful technique for trying to make apparent the behavior of a system is application visualization, in which we create a visual representation of some aspect of the system, using charts, graphs, animated diagrams, or other techniques. We are visual creatures. Our brains are wired to find patterns and anomalies in the information processed by our visual cortex. So it is natural and fruitful to convert software behavior to the visual realm for analysis. But maybe it’s worthwhile to consider our other senses. After all, in the agile and design communities, a common way to describe poor quality is that is has a bad smell.

One drawback of application visualization is that it requires one’s attention—you have to actively look at the visualization to understand it. This prevents you from focusing your attention elsewhere, and it does not facilitate a general ambience of information. We do experience ambient information when we drive a car: our visual senses are focused on the road, but we will know instantly if the car starts to sound different, or if the engine is revving too high.

Enter application sonification. This is a technique for converting a program’s behavior into an audible pattern. In our application Log4JFugue, we convert a log4j output stream into a JFugue music stream and listen to the program’s behavior.

Log file analysis is time-consuming and boring. This is especially true if you are monitoring a system rather than actively diagnosing an existing problem. Monitoring a system visually requires your full attention, whereas listening to changes can be done without attention. Log4JFugue allows you to listen to a rhythmic music stream in the background, hearing the shifts in the beats and instruments without paying active attention. In the same way that an auto mechanic can listen to your car engine and hear the problem, Log4JFugue lets you experience your application’s changing behavior in a new, audible way.

The primary application where Log4JFugue has been used is a Video On Demand (VOD) system. This system meets all of our enterprise criteria in that it is multi-threaded, multi-vendor, distributed, and mission-critical. If this system fails, users can not watch movies, and providers cannot make money. This system also has the characteristic of failing on nights and weekends because that is when people watch the most movies (and it is hopefully a time when developers are asleep or at least away from the office).

How Log4JFugue Works

Log4JFugue creates a mapping from the system verbs to musical instruments. For example, in the VOD space the major verbs are stream create (stream is the term used to describe a video content being played), stream complete, stream failure, and so on.

Log4JFugue contains a Spring configuration file that maps these various verbs to instruments such as:

| stream create - Bass Drum
| stream complete - Snare Drum
| stream failure - Cymbal Crash

Whenever we see a log message containing the string for a particular verb, we increment the count of the associated instrument. More stream creates means more Bass Drum beats, more stream failures means more Cymbal Crashes.

Log4JFugue breaks time into one-second intervals. In each second, it is gathering the counts of the various instruments and log messages for the current second while playing the music stream that was created for the previous second. We use a standard double buffering system, accumulating in one buffer, playing from the other buffer, then swapping buffers. Each second consists of a 16-beat measure. Each instrument comprises one layer of the rhythm.

Count-based music (i.e., rhythm) lends itself nicely to this approach. If one stream create event arrives in a second, we play a Bass Drum whole note. If four stream create events arrive in the next second, we play four Bass Drum quarter notes. JFugue allows the creation of arbitrary notes, so we can for example play seven seventh notes.

The actual code to accomplish this is surprisingly simple, thanks to the JFugue API design. The following code creates a layer that will play one instrument note during the next second:

rhythm.setLayer(1, ".......O........");

This code creates another layer that will play another instrument four beats during the next second:

rhythm.setLayer(2, "..X...X...X...X.");

JFugue’s Rhythm API combines these layers into a single piece of music.

Putting It All Together

When we demo Log4JFugue we first show people a real log file, gathered on a real system. We feed this 100,000 line file to the screen and ask people to raise their hands when the ratio of creates to failures falls below 80%. Of course, no one can read a log file printed at full speed but it does tend to get a sympathetic laugh. We then run Log4JFugue against the same log file and ask the audience the same question. At first they hear a few Bass Drum beats, followed by a few Snare Drums. Then the pace of the beat increases with more Bass and Snare beats. People can literally hear the system increasing in speed. After about 15 seconds of our demo we hear a Cymbal crash and everyone puts up their hands. They heard a change in the behavior.

But why just rhythm? Originally, Log4JFugue used tonal music, not rhythms, and it sounded awful!

The problem was that Log4JFugue creates a single dependent variable and tonal music requires at least two variables: pitch and duration. We tried creating music using tonal instruments such as pianos, flutes, and guitars all playing the same pitch such as Middle-C, and the results were jarring. We tried varying the pitch based on the count (higher counts per second gave higher pitches), and that sounded bad, too. While not closing the door on further experiments, we decided to switch to rhythm-based instruments where we could ignore pitch.

Performance Considerations

While Log4JFugue provides benefit, it also incurs some cost in terms of system resources. Each log message that your program outputs must be analyzed by the system, and each message must be compared against the system verb strings you have configured. This results in processor overhead, which can be mitigated in several ways.

First, you can use Log4JFugue on an existing log file, presumably on a non-production machine. By simply changing a Spring configuration file, you can change from live analysis to post-hoc analysis. You can even change the replay speed if desired.

Second, even when doing live analysis you can use the log4j SocketAppender and send the messages off-box for processing. Log4JFugue can run on any machine with connectivity to your production system.

Third, the number of system verbs that you will configure for analysis is limited by the music string you are willing to listen to. Most people lose the ability to distinguish between instruments after about a dozen.

Finally, the string comparison that Log4JFugue uses is not the stock strcmp that comes with the JDK. Instead, we use a highly optimized precompiling pattern matching system from MIT. So, even though Log4JFugue incurs some overhead, there are a number of ways to successfully mitigate that overhead.

Spring-Based, Loosely Coupled

Log4JFugue is a Spring-based system, which means it is a loosely organized collection of Plain Old Java Objects (POJOs). It ships with a number of alternate classes that can be selected via the Spring configuration file. (See figure for system architecture). It is designed around four primary classes, each of which has a number of derived alternative.

The DataGetter retrieves the raw log messages from a socket or a file.

The MessageReceiver processes the messages and increments the instrument counts.

The SoundBuilder converts the instrument counts into a music string (tonal or atonal).

The Player sends the music string to JFugue or just to a file (for later playing).

Setting up Log4JFugue and Your Application

You can download Log4JFugue from www.jfugue.org.

Using Log4JFugue requires configuring both your application and log4j. The only change to your application is to create a SocketAppender in your log4j configuration file. You should not even need to restart your application, as log4j reprocesses it configuration file upon change.

| log4j.appender.SOCKET=org.apache.log4j.net.SocketAppender
| log4j.appender.SOCKET.Port=4445
| log4j.appender.SOCKET.RemoteHost=192.168.1.3

To configure Log4JFugue, set the dataGetter bean to be the SocketDataGetter.

| <bean id="dataGetter" class="org.log4jfugue.SocketDataGetter">
| </bean>

You need to decide on the key verbs of your system and identify distinctive log strings to look for in the message. You then associate those strings with particular instruments.

| <prop key="OPEN_HI_HAT">sendTrap</prop>
| <prop key="MARACAS">stream create</prop>
| <prop key="HI_WOOD_BLOCK">destroyImpl</prop>

Conclusion

JFugue is a wonderfully simple API that lets developers create and experiment with music. By lowering the barrier to entry for music programming, JFugue opens the door for many innovative new applications. Log4JFugue is one such application that uses music in a new and productive way, converting the repetitive and boring task of log file analysis into a musical venture. At one level, Log4JFugue is a thought experiment that presents an alternate way to perform a mundane task. If music can be applied here, where else can it be applied? At another level, Log4JFugue stands on its own as a useful solution to a real problem. More information about JFugue can be found at www.jfugue.org and www.jfugue.org/press and Dave has written a JFugue book.

  David Koelle is the author of JFugue. Separately, David is a Senior Software Engineer at Charles River Analytics Inc., where he develops user-centered tools that apply intelligent systems techniques to solve real-world problems.

Brian Tarbox is the author of Log4JFugue.  He is also a Principal Staff Engineer at Motorola where he designs server side solutions in the Video On Demand Space.  He writes a blog on the intersection of software design, cognition and Tai Chi at briantarbox.blogspot.com.

Beauty in Code

Songs in Code and Code that Sings

by Scott Davis

What if Dylan wrote in Dylan?

If you have a Twitter account, you probably noticed the recent avalanche of messages tagged with #songsincode. If you missed it, you can follow the trend here. The premise is simple: rewrite the lyrics of a song in a programming language. For example, I’ll start you with an easy one:

| 4.times{
| !woman == !cry
| }

This, of course, is the opening of Bob Marley’s “No Woman, No Cry” written in Groovy. Here’s another one:

| if(TheOneYouLove == null){
| TheOneYouAreWith.love()
| }

If you guessed “(If you can’t be with the one you love) Love the One You’re With” by Crosby, Stills, and Nash, I can only give you partial credit—technically, it was a solo release by Stephen Stills. Here’s one more:

| ["I", "They"].each{
| assert it instanceof Eggman
| }
| assert "I" instanceof Walrus
| println "Goo Goo Ga Joob"

If you didn’t recognize “I am the Walrus” by the Beatles, go back and check out the lyrics on the original dust jacket of “The Magical Mystery Tour.” Since Groovy hadn’t been invented yet, John Lennon was forced to write the lyrics in BASIC:

| 10 PRINT "I am the Eggman"
| 20 PRINT "They are the Eggmen"
| 30 PRINT "I am the Walrus"
| 40 GOTO 10

The real fun of this Twitter meme is not only looking at people’s choice of programming language, but looking at their taste in music as well. (“Karma Chameleon” in Ruby? Really? REALLY?!?)

But as I read one nearly indecipherable post after the next, a discouraging realization hit me—this is what we do for a living. We translate user stories into source code. How much of it survives the operation? More importantly, how much of it gets lost in the translation? If #songsincode is in any way representative of the best that we can do, we’re in big trouble.

Moving beyond music for a moment, take the case of Google Maps. There is something deeply beautiful about the way you simply drag the map around to navigate. But is that beauty evident in the implementation?

| function startDrag(event) {
| iAmDragging = true
| //capture the initial point where dragging began
| dragStartX = Event.pointerX(event)
| dragStartY = Event.pointerY(event)
| //capture the current top,left of wholeMap
| var wholeMap = $('wholeMap')
| wholeMapTop = stripPx(wholeMap.style.top)
| wholeMapLeft = stripPx(wholeMap.style.left)
| //change the mouse cursor
| wholeMap.style.cursor = '-moz-grab'
| Event.stop(event)
| }

For the record, that code is mine. If I showed you Google’s obfuscated/minified JavaScript, the unreadable source code would stand in even starker contrast to the joy of the user experience.

But in the depth of my existential angst I had another, more Zen-like realization—the solution is both in the code and not in the code. Put another way, the code is simultaneously the means and the end. If the end user says, “I want to be able to drag the map around in my browser like I’m spinning a globe,” I’m not going to hand them a printout of the JavaScript for this. I’m going to pull up the web page that contains the JavaScript and let them drag the map around to their heart’s content.

The beauty of this:

| <blink>
| <span style="font-size:-1em;">Star</span>
| </blink>

...is the Twinkling Little Star that gets rendered by the browser, not the HTML markup itself.

This example also demonstrates that we, as programmers, are more transliterators than translators. Transliteration is “using the closest corresponding letters of a different alphabet.” Since HTML offers neither “twinkle” nor “little,” “<blink>” and “<span style>” will have to suffice.

And that is the joy of programming in a nutshell. How well do you transliterate the user story into code, bridging the gap between the technical (expressive?) limitations of the programming language and the English prose of the request? The joy is in the puzzle solving.

Musicians do it all the time. Ask Bob Dylan to explain life... in eight lines... that rhyme. He’ll reply (in “Subterranean Homesick Blues”):

| Get born, keep warm
| Short pants, romance, learn to dance
| Get dressed, get blessed
| Try to be a success
| Please her, please him, buy gifts
| Don’t steal, don’t lift
| Twenty years of schoolin’
| And they put you on the day shift

Now ask yourself, “How many Bob Dylan albums do I have in my iTunes library?” (Please feel free to transliterate “Bob Dylan” and “iTunes” into the artist and software of your choice.) Here’s my response, in Groovy:

| def library = new File("/Users/sdavis/Music/iTunes/iTunes Music")
| def artists = [] //this is a list
| def albums = [:] //this is a map
| library.eachDir{artist->
| if(artist.isDirectory()){
| artists << artist.name
| artist.eachDir{album->
| albums.put(album.name, artist.name)
| }
| }
| }
| println "Number of artists: ${artists.size()}"
| println "Number of albums: ${albums.size()}"
| def results = albums.findAll{album, artist->
| artist == "Bob Dylan"
| }
| println "I have ${results.size()} Bob Dylan albums in my library:"
| results.each{album, artist->
| println album
| }

And for the record, here are the results:

| $ groovy artists.groovy
| Number of artists: 226
| Number of albums: 444
| I have 11 Bob Dylan albums in my library:
| Another Side of Bob Dylan
| Blonde on Blonde
| Blood on the Tracks
| Bob Dylan
| Bringing It All Back Home
| Highway 61 Revisited
| John Wesley Harding
| Nashville Skyline
| New Morning
| The Freewheelin’ Bob Dylan
| Time Out of Mind

As soon as I finished writing the code, I started asking myself questions:

Should I replace “Bob Dylan” with “args[0]” so that I can pass in any artist from the command line?

Is there a simple way to return the list of albums chronologically instead of alphabetically?

Where did Bob Dylan’s “Greatest Hits” album go? D’oh! I lost all of the albums named “Greatest Hits” except for the last one since key values in HashMaps must be unique. Looks like I need a unit test and a new way to store albums.

You were most likely asking yourself a different set of questions as you read my solution—“Why did he choose Dylan instead of Metallica (or Snoop Dogg) to explain life?” and “Why did he choose Groovy instead of [my language of choice] to walk the iTunes directory?” I’m sure that you could find more beautiful answers to each question. In both music and programming languages, there’s no accounting for taste.

But at least in the case of programming languages, you can frame the question in terms of appropriateness for the task at hand.

Stuart Halloway (author of Programming Clojure) wrote a series of articles called “Java.next.” The first three articles compared solutions written in Groovy, JRuby, Scala, and Clojure. Each showed dramatic improvements over the baseline Java example—not just in a reduction in gross lines of code, but in clarity as well. But then in the fourth article, Stuart tackled immutability. This time, he narrowed the field to just examples written in Scala and Clojure. Here is how he explained his decision:

“Could you write the example above in Groovy, JRuby, or even Java? Yes, but it would be non-idiomatic, even ugly.... Languages are not about what they make possible, but about what they make beautiful.”

So perhaps the lack of beauty in the #songsincode transliterations was due to using the wrong language for the job. Said another way, clearly the humor in each #songsincode post rests squarely in the awkward transliteration. I doubt that Dylan (the singer) could sound any better, even written in Dylan (the programming language).

But let me leave you with one final thought: the end user only gets to see one dimension of beauty—the result of the running code. If you do it right, the beauty you see is multifaceted. If the code (as it runs) is as beautiful as the code (as it is written), then you are singing it just right.

Scott Davis is an internationally recognized author, speaker, and software developer. His books include Groovy Recipes: Greasing the Wheels of Java, GIS for Web Developers: Adding Where to Your Application, The Google Maps API, and JBoss At Work (co-author). Keep up with Scott at thirstyhead.com.

Shady Illuminations

The Boys’ Club

by John Shade

John takes on gender attitudes in the open source community using comic book references.

“This new creature with the long hair is a good deal in the way. It is always hanging around and following me about. I don’t like this; I am not used to company.”—Mark Twain, “Extracts from Adam’s Diary”

A presenter at a Rails conference builds his slideshow around images of naked women in degrading roles, then attacks his critics when called on it. A presenter at a Flash conference mimes humping the podium and subjects a female audience member to lewd comments. Mark Zuckerberg gets dumped by a woman and in a bitter rage invents Facebook to get back at her. OK, I don’t get that last one, either. It sounds like the origin story for a Batman villain. Everything I know about Zuckerberg I got from tech journalist Megan Angelo (by way of Ben Mezrich's best-selling allegations).

Do we have a problem here?

We Have a Problem Here

Megan Angelo says “prehistoric woman-bashing is making a comeback” in Silicon Valley. Ted Neward says that we in the open source community sometimes sound like “a bunch of 13-year-olds giggling over the fact that somebody said ‘Boobies’.” Programmers with experience working in long-established companies report being shocked at the attitudes and actions that are prevalent in open source. Long-hairs who pioneered the personal computer industry lament that a frat boys mentality has infected the field. A Rails spokesman resigns because he feels that the leadership of the Rails community actively supports sexist behavior.

And after each incident, heated discussions break out online, mostly wrangling over the definition of sexism.

Because, I mean, how could we in the open source community be sexist? We’re like all egalitarian and everything and we judge people solely on their ability. Right?

Suuure. And how does that work out in practice? Here’s how: when a woman finds herself on a open source project working with a bunch of guys, she has to fit into one of three roles. If she isn’t better than ninety percent of the team, she’s an outsider, a suit, a zero, a non-geek. If she’s really good, she can either be The Freak (a woman who can hold her own against us rock star programmers) or Invisible Girl (aka Honorary Male: we can’t reconcile your ability with your gender, so we’ll refuse to see your gender).

This is not a column on sexism. I won’t get sucked into the argument about sexism in the open source community, because it’s just a smokescreen covering the real issue. Which is misogyny in the open source community. About which there is no argument. It exists, and it’s as ugly as a Batman villain.

Guys 101

Unlike sexism, misogyny has just one definition: hatred of women. Why, you may ask, would a guy hate women? Here’s where I need to explain guys to any reader of this column who is not of the male persuasion. But who am I, you may ask, to explain guys? Am I a trained expert in psychology? Or guychology?

You sure ask a lot of questions. What I am is an expert on rejection. I trained in the School of Getting Dumped. I have a Master’s Degree in Rejection. So take my word for it: there are two kinds of guys, which we may call, for convenience, by which I mean my convenience, drips and dweebs. They are differentiated by their reaction to rejection.

At some point every boy gets rejected. This point in their relations with the opposite sex is their one moment of free will. The rest of the time they are completely driven by hormones. But then they have a choice in how to react to rejection, and what they decide will condition their attitude toward women for the rest of their lives. That’s my theory, and I am the Renaissance Man of Rejection, so you should trust me.

Half the guys you will encounter met that initial critical-period rejection by writing the girl’s name on the bathroom wall, slandering her to their friends, and calling her the B word (rhymes with Barbara Bush). In extreme cases, they founded Facebook. These guys are the drips.

The other half curled up in a foetal position and whimpered. They are the dweebs.

These are the only kinds of guys there are. These are the choices. I know. I’m sorry, gals. But while their basic nature can’t be changed, their behavior can.

No Girls Allowed

So that explains Mark Zuckerberg, but it doesn’t explain the unusually high amount of misogyny in open source. But I can explain that, too.

It’s the boys’ club phenomenon. Little boys spend idyllic days frolicking in a kind of innocent Eden until the sex thing comes along and complicates their lives. Then come the painful social encounters, the emotional roller-coaster rides, the self-worth issues in the locker room, the not wanting to be around when Mom’s washing the sheets. Naturally, the boys want to keep the girls out of the clubhouse. Girls are weird and will change everything.

In the boys’ clubhouse that is open source, it’s worked extremely well. Researchers at Cambridge University undertook a study of gender in F/LOSS communities in 2006. They were inspired by an NSF report that the percentage of women in commercial software, while low, was roughly twenty times as high as in the F/LOSS community. At that time women made up less than two percent of F/LOSS programmers. Even today, there are remarkably few girls in the clubhouse.

It’s unfamiliarity, not familiarity, that breeds contempt. If you don’t encounter a particular class of people in significant numbers in your community, it’s easy to develop attitudes toward them that would be hard to maintain if you were working with them every day in that environment. This is useful if you’re a military operation trying to desensitize a bunch of teen-agers to kill strangers on order, but one likes to think that the open source community has other goals in mind.

So the boys’ club thinking is correct: if you let girls into the clubhouse, they will change everything. In particular, they will change boys’ minds.

The Solution Is Simple

That Cambridge study recommended just that: get more women into F/LOSS, and minds (and behavior) will change. What could be simpler? I imagine they were inspired by the X-Men.

Chris Claremont, the writer most associated with the X-Men, had a rule. When developing a new superhero, he always stopped at a certain point and asked himself, “is there any reason this character couldn’t be a woman?” If not, he defaulted to female, thus helping to redress the lamentable gender imbalance among comic book mutant superheroes. Not quite up there with Little Lulu outsmarting Tubby and his clubhouse gang, but still a forward step for women in comics. Of such small victories is progress’s path paved.

There’s another, more difficult, way to remove that contempt-breeding unfamiliarity, too: it’s for the boys to get the heck out of the clubhouse now and then and spend more time around women. And strip clubs don’t count. I know, it’s a big scary world out there. But ultimately they’ll find it’s more rewarding outside the clubhouse than inside.

“After all these years, I see that I was mistaken about Eve in the beginning; it is better to live outside the Garden with her than inside it without her.” -Mark Twain, “Extracts from Adam’s Diary”

John Shade was born in Montreux, Switzerland in 1962. Subsequent internment in a series of obscure institutions of ostensibly higher learning only confirmed his dark view of his fellow man, not excluding himself. He aspires no higher than to serve as an example that one can be a misanthrope without being a misogynist.

The Quiz

Chickens and Eggs

by Michael Swaine

Below are a series of pairs of events. Your challenge is simply to say which of each pair happened first. Try to answer as many as possible without resorting to The Google.

ARPAnet or Ethernet?

“Go To Statement Considered Harmful” (Dijkstra) or the IBM System/360?

The first instance of the same program being run on different computers from different vendors and producing the same result or Etch A Sketch?

Gary Kildall creates the BIOS for CP/M or Cray delivers the first Cray-1 supercomputer?

Tron (the original movie) or C++?

Microsoft releases Windows or Apple “releases” Steve Jobs?

The C Programming Language (K&R) or VisiCalc?

Buck Rogers or Flash Gordon?

The patenting of the integrated circuit or the first legal article on computer law?

The Mosaic browser or Apple’s HyperCard?

The birth of Fortran or the birth of William Henry Gates III?

PHP or Ruby?

The UNIVAC 1 or The Day the Earth Stood Still?

The Soul of a New Machine (Tracy Kidder) or Thinking Machines Corporation (Danny Hillis)?

Forth or The Art of Computer Programming Volume I: Fundamental Algorithms (Knuth)?

Extreme Programming or Extreme Sports’ X Games?

Tinkertoys or the use of magnetic tape for computer storage?

The Shell Sort or color television?

Yahoo or XML?

Answer to Last Month’s Quiz

Last month we presented a Sudoku puzzle with a twist—it used letters instead of digits and hidden within it was the name of a well-known person in personal computer history. The hidden name is Jef Raskin, in row 3:

I
A
N

R
K
S

J
E
F

F
K
E

J
N
I

R
A
S

R
S
J

A
F
E

K
I
N

F
N
E

S
I
K

A
R
J

K
I
R

A
E
J

S
F
N

S
J
A

F
N
R

E
K
I

N
F
I

E
J
A

K
S
R

E
R
K

I
S
F

N
J
A

J
A
S

N
R
K

I
E
F

Calendar

Sept 2, PMI Silicon Valley

“Manage the Project Portfolio”

Johanna Rothman, author of Manage It! and Behind Closed Doors: Secrets of Great Management

Sept 2-5, Famagusta, North Cyprus

ICSCCW (20009 Fifth International Conference on Soft Computing, Computing with Words and Perceptions in System Analysis, Decision and Control)

Sept 3, Edinburgh, Scotland

Haskell Symposium

Sept 3-6, Minneapolis, MN

EMBC (2009 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society)

Sept 4

John McCarthy’s birthday

Sept 4-7, Linz, Austria

2009 13th IEEE International Symposium on Wearable Computers (ISWC)

Sept 7

David Packard’s birthday

Sept 8, Seattle, WA

Study of Nanoscience and Emerging Technologies Nanoethics Symposium 2009

Sept 9

Dennis Ritchie’s birthday

Sept 9

64 years ago today Grace Hopper got the Harvard Mark II running again after extracting a moth from a relay, the first computer bug.

Sep 9, Trinug, Raleigh NC

“Refactor Your Wetware”

Andy Hunt

Sept 9-10, Washington, DC

Gov 2.0 Summit

Sept 9-11, Seattle, WA

2009 IEEE Ninth International Conference on Peer-to-Peer Computing (P2P)

Sep 10-11, Agile Open Holland, Baarn, The Netherlands

Topic TBD

Rachel Davies, author of Agile Coaching

Sept 10-12, Amsterdam, The Netherlands

ACII 2009 (2009 3rd International Conference on Affective Computing and Intelligent Interaction)

Sep 11-12, Agile China, Beijing

Topic TBD

Dave Thomas

Sep 11-13, New England Software Symposium, Boston, MA

“Third time’s a charm: What’s new in Spring 3.0,” “WAR and Pieces: Breaking down monolithic web applications with Spring-DM and OSGi,” “That old Spring magic has me in its SpEL: DI Wizardy with the Spring Expression Language,” “Beyond JUnit: Powertools for Test-Driven Development”

Craig Walls, author of Modular Java

Sept 12, WindyCityRails, Chicago IL

“Behaviour Driven Rails with RSpec and Cucumber”

David Chelimsky, author of The RSpec Book

Sept 12-16, Raleigh, NC

PACT (2009 International Conference on Parallel Architectures and Compilation Techniques)

Sept 14-16, North Cyprus

ISCIS (2009 24th International Symposium on Computer and Information Sciences)

Sept 14-16, Santa Clara, CA

2009 IEEE Third International Conference on Semantic Computing (ICSC)

Sept 15-18, Chicago, IL

4G World

Sep 19, South Jordan, UT

Utah Code Camp

Sep 19, Raleigh, NC

Raleigh Code Camp

Sept 20-23, Anchorage, AK

2009 IEEE Vehicular Technology Conference (VTC 2009-Fall)

Sept 20-24, Corvallis, OR

2009 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

Sept 20-25, Beijing, China

MobiCom 2009

Sept 20-26, Edmonton, Canada

SCAM 2009 (2009 Ninth IEEE International Working Conference on Source Code Analysis and Manipulation)

Sept 21-24, Boston, MA

ESC

Sep 22, CareerCamp in suburban Chicago (Wheaton, IL)

“Effective Job Interviewing from Both Sides of the Desk”

Andy Lester, author of Land the Tech Job You Love

Sept 22-24, Cambridge, MA

EmTech09

Sept 22-24, San Francisco, CA

Intel Developer Forum

Sept 23, Agile Boston User Group, Boston MA

Topic TBD

Johanna Rothman

Sep 23-27, Code Workz, San Francisco(23rd), Los Angeles (25th), Dallas (27th)

“Getting Git” and “Micro-Frameworks in PHP”

Travis Swicegood, author of Pragmatic Version Control Using Git

Sep 25-26, Western Canada Software Symposium, Calgary CA

Topic TBD

Jared Richardson

Sep 26, Houston Tech Fest, Houston TX

“Introduction to Test Driven Development” and “Stuck in Agility?”

Ben Rady

Sep 27, 360 iDev Conference, Denver CO

Topic TBD

Marcus S. Zarra, author of Core Data

Sept 28-30, Valencia, Spain

2009 IEEE-PES/IAS Conference on Sustainable Alternative Energy (SAE)

Sept 28-30, Berlin

NanoTech Europe

Sept 29-30, San Diego, CA

Software Business 2009

Sept 30-Oct 3, Orlando, FL

11th International Conference on Ubiquitous Computing

Oct 2-4, Twin Cities Software Symposium, Minneapolis, MN

“Third time’s a charm: What’s new in Spring 3.0” “WAR and Pieces: Breaking down monolithic web applications with Spring-DM and OSGi” “That old Spring magic has me in its SpEL: DI Wizardy with the Spring Expression Language” “Beyond JUnit: Powertools for Test-Driven Development”

Craig Walls

Oct 3-4, Los Altos Hills, CA

Silicon Valley Code Camp

Oct 4

John Atanasoff’s birthday

Oct 4-6, JAOO, Denmark

Behaviour-driven development—writing software that matters” (1/2 day tutorial) and “Adventures of an Agile Architect”

Dan North, author of The RSpec Book

Oct 4-7, Los Angeles, CA

Adobe MAX

Oct 4-7, Orlando, FL

VSLive

Oct 4-9, Denver, CO

Models 09 (ACM/IEEE 12th International Conference on Model Driven Engineering Languages and Systems)

Oct 5, Supreme Court chambers, Washington, DC

Season Opener: South Carolina v. North Carolina

Oct 5-7, Bloomington, IN

SIGDOC 2009

Oct 7-9, Mumbai, India

Interop Mumbai

Oct 10-16, Atlantic City, NJ

2009 IEEE Information Visualization Conference (InfoVis)

Oct 11-14, San Antonio, TX

2009 IEEE International Conference on Systems, Man and Cybernetics (SMC)

Oct 11-14, Big Sky, MT

22nd ACM Symposium on Operating Systems Principles

Oct 11-15, San Francisco, CA

Oracle OpenWorld

Oct 11-16, Grenoble, France

International Conference on Hardware-Software Codesign and System Synthesis

Oct 12, Skills Matter, London

“Pragmatic Thinking and Learning”

Andy Hunt

Oct 13

Arthur Burks’ birthday

Oct 13-14, Rails Summit Latin America, Sao Paulo, Brazil

Topic TBD

David Chelimsky

Oct 15-16, Lake Buena Vista, FL

International Symposium on Empirical Software Engineering and Measurement

Oct 17-18, Voices That Matter, Boston, MA

Topic TBD

Marcus S. Zarra

Oct 19-21, Denver, CO

Advanced Ruby Studio

Oct 19-22, SpringOne and Groovy/Grails Experience, New Orleans LA

“Grails Quick-Start” and “Not Your Father’s Custom Tags”

Dave Klein

Oct 19-22, SpringOne and Groovy/Grails Experience, New Orleans LA

“That old Spring magic has me in its SpEL: DI Wizardy with the Spring Expression Language”

Craig Walls

Oct 19-22, San Jose, CA

ZendCon

Oct 19-23, Cambridge, MA

Software Test & Performance Conference

Oct 19-24, Beijing, China

ACM Multimedia 2009

Oct 20-22, San Francisco, CA

Web 2.0 Summit

Oct 20-22, ZendCon, San Jose CA

“Getting Git”

Travis Swicegood

Oct 20-22, San Francisco

Web Squared

Oct 21, JUG, Richmond VA

“Refactor Your Wetware”

Andy Hunt

Oct 21-23, Santa Clara, CA

ARM techcon3

Oct 22, Compaid webinar

“Pragmatic Lessons Learned in Project Management”

Johanna Rothman

Oct 22-23, New York, New York

HotNets-VIII

Oct 22-24, Fairfaz, VA

SIGITE 2009

Oct 23, University City, MO

Strange Loop Conference

Oct 24, The Twin Cities Code Camp, Minneapolis MN

“Meet Ruby”

Brian Hogan, author of Web Design for Developers

Oct 24, The Twin Cities Code Camp, Minneapolis MN

“No More Excuses: Test Your Javascript!”

Kevin Gisi

Oct 25-28, PNSQC, Portland OR

Topic TBD

Johanna Rothman

Oct 25-29, Orlando, FL

OOPSLA

Oct 26-29, Biloxi, MS

OCEANS 2009

Oct 28

Ted Hoff’s birthday

Nov 2, Skillsmatter, London

“In the Brain of Dan Haywood”

Dan Haywood

Nov 6-8, Las Vegas, NV

7th Annual UFO Crash Retrieval Conference

Nov 7, Bletchley Park, Milton Keynes, UK

ACCU Security Yesterday, Today, and Tomorrow

Nov 9-13, Agile Development Practices, Orlando FL

“Pragmatic Thinking and Learning”

Andy Hunt

Nov 9-13, Agile Development Practices, Orlando FL

Several topics in the testing sub-conference

Jared Richardson

Nov 14-15, Greater Boston Chapter of ACM, in Boston, MA

Craig Walls is sharing the stage for two days with Ken Sipe doing an overview of Spring, covering much of the Spring portfolio.

Craig Walls

Nov 17-19, Los Angeles

Microsoft PDC09

Nov 20, Redmond, WA

Microsoft Windows’ birthday