PragPub 2013-11: Issue #53

Table of Contents

Features

- Namespaces in Clojure

by Michael Bevilacqua-Linn

Michael gives you “just enough about namespaces to get you going.”

- Rediscovering Awk

by Derrick Schneider

A tedious task sends Derrick digging through the musty man pages of an archaic language, where he unearths some gems.

- Good and Bad Technical Debt

by Henrik Kniberg

Sometimes technical debt is good. The trick is knowing how much and when.

- Seven Coaching Patterns

by Portia Tung

Talk less, listen more, and other habits to cultivate.

- Nostalgia for the Future

by Michael Swaine and Paul Freiberger

An excerpt from the upcoming Fire in the Valley 3rd edition.

Departments

- On Tap

by Michael Swaine

Languages old and new, career and productivity wisdom from experts.

- Choice Bits

Flotsam on the Twittertide, tech books of note, and other goodies we snared in our net.

- Rothman and Lester

by Johanna Rothman and Andy Lester

Should I stay or should I go?

- Pragmatic Bookstuff

Want to meet one of the Pragmatic Bookshelf authors face-to-face? Here’s where they’ll be in the coming months. Also, find out what are the top-selling Pragmatic Bookshelf books and what new books are coming out.

- Shady Illuminations

by John Shade

John reflects on the Post-Post-PC Era, Jony Ive’s fingerprints, and just-in-time deconstructionist iconography.

- Rear Window

The first computer mouse, invented by Doug Engelbart at SRI in 1964, was carved from a block of wood.

Except where otherwise indicated, entire contents copyright © 2013 The Pragmatic Programmers.

Feel free to distribute this magazine (in whole, and for free) to anyone you want. However, you may not sell this magazine or its content, nor extract and use more than a paragraph of content in some other publication without our permission.

Published monthly in PDF, mobi, and epub formats by The Pragmatic Programmers, LLC, Dallas, TX, and Raleigh, NC. E-Mail support@pragprog.com, phone +1-800-699-7764. The editor is Michael Swaine (mailto:michael@pragprog.com). Visit us at http://pragprog.com

ISSN: 1948-3562

On Tap

Getting Clojure, but Feeling a Little AWKward

by Michael Swaine

This month we dive into languages old and new, and get some career and productivity wisdom from experts.

The “old” language is AWK, which was created in the 1970s by Alfred Aho, Peter Weinberger, and Brian Kernighan (the name is their initials). So it’s not only old but has a heck of a pedigree. Wikipedia will try to tell you that it has been largely supplanted by Perl, but don’t be fooled. It may be the most widely-available Turing-complete language in existence, and it’s actually fun to use. Derrick Schneider recently immersed himself in a study of AWK and shares some nifty AWK tricks for tasks as diverse as analyzing data from an iPhone app, managing podcasts, and checking for null parameters on public methods.

The “new” language is Clojure. Michael Bevilaqua-Linn continues his series on the Clojure language with an insightful exploration of Clojure namespaces this month.

Henrik Kniberg has some advice on managing technical debt. You start out by realizing that not all technical debt is bad, he says. Portia Tung, author of The Dream Team Nightmare, has something to say to coaches. And Rothman and Lester are back this month with some career advice on when to stay and when to go. Also, your editor shares another excerpt this month from his and Paul Freiberger’s upcoming third edition of their seminal history of the personal computer, Fire in the Valley.

What’s Next?

Next month we’ll have an article on profiling by Dmitri Sotnikov, author of Web Development with Clojure and another excerpt from Fire in the Valley, Michael Bevilaqua-Linn will return with more Clojure explorations, and Rothman and Lester will tackle the question, “How do I get my boss to do X?” Plus John Shade and more.

On the delivery front, you can now subscribe or purchase single issues without going through PayPal, via the Stripe payment infrastructure.

Finally, feedback. We have a magazine forum, but it isn’t seeing much use. We have some ideas about how to change that, one of which is simply to invite you to drop in. Please consider yourself invited.

Cover and p. 17, istockphoto.com/george tsartsianidis; p. 5, istockphoto.com/hrhportia; p. 14, istockphoto.com/greenjellybean; p. 26, istockphoto.com/Feverpitched; p. 36, istockphoto.com/Kali Nine LLC; p. 40, Bob Marsh; p. 49, Doug Engelbart.

Choice Bits

Chosen just for You

Flotsam on the Twittertide, tech books of note, and other goodies we snared in our net.

Blogging isn’t dead. If you try to keep up a blog on a regular basis for a long stretch it might kill you, though. So those of us on the lookout for good blogs have to keep rebuilding our lists, because good bloggers get tired and stop, either for good or for a while. But new ones keep coming on. And some bloggers have staying power. “Uncle Bob” Martin doesn’t post regularly, but when he does he always has something to say—and says it engagingly. And Caterina Fake’s blog, with posts like, “How the World Will End: Possibilities,” is constantly thought-provoking. What are your favorite blogs?

“Most engineers like to proceed from A to B to C in a series of logical steps. I’m the rare engineer who says the answer is obviously Z and we will get on with that while you guys work out how to do all the intermediate steps. It makes me a dangerous person to employ in IT but a useful one.” —Sophie Wilson, designer of the Acorn microcomputer and the instruction set for the ARM processor, among other things.

It’s a Good Life

In some cultures, this is the season of reflecting that you haven’t got it so bad.

It finally happened at Fenway!!! The only boyhood Red Sox fantasy remaining left for me now is to become their starting catcher. — ‪@Ihnatko‬

I liked how the Japanese translator of Swing Hacks picked up on my anime references and fixed them up for the native market. — @invalidname

Fantastic Ruby Conference in Beijing. Good place, good people. — ‪@pragdave‬

It’s such a pleasure to watch a smalltalk program instantly begin breathing and grow gently from there. — ‏‪@KentBeck‬

For those asking, yes, that’s a treadmill desk. I’ll be writing about life with it very soon. Short answer—it’s awesome. — ‪@dannysullivan‬

FAA to allow tablets and e-readers during all phases of flight. — ‪@davewiner‬

Love that feeling when a tool that’s taken awhile to come together starts to come together. Former long, manual process just took minutes! — ‪@tswicegood‬

For my birthday I got an HTTP status 500 from an app I just pushed to production. — ‪@marick‬

My new favorite subreddit: ‪http://www.reddit.com/r/whatisthisthing/ … People post pictures of weird objects for identification.‬ — ‪@petdance‬

My favorite issue of Tiger Beat has Crockford on the cover. — ‪@SaraJChipps‬

“I was a bit of an artist, and somewhere along the way had gotten the idea that computers could be used for animation and artists, because in-betweening was so tedious.... Of course, everyone thought I was nuts.” —Carla Meninsky, video game designer, entrepreneur, and intellectual property lawyer.

Antonio on Books

Hi there, I'm Antonio Cangiano, an author and developer who runs Any New Books, a weekly new book notification service. Here in this space each month, I share a selection of some of the most interesting newly released programming books with PragPub readers specifically in mind.

This month there are quite a few interesting releases, particularly from The Pragmatic Bookshelf and O’Reilly Media. Among them, my pick for the month is the fourth edition of the award-winning Rails book by Sam Ruby et al., which now covers Rails 4. For those who aren’t into Rails web development, you’ll still find plenty of good titles this month. In particular, on the more discursive front, I’d like to point out both Minecraft and Smart Machines. The first features the story of the game and its author, Markus “Notch” Persson, programmer turned multi-millionaire entrepreneur. The second delves into the new cognitive frontier of computing, and how it was popularized by such innovations as IBM’s Watson.

Antonio's Pick: Agile Web Development with Rails 4 By Sam Ruby, Dave Thomas, David Heinemeier Hansson • ISBN: 1937785564 • Publisher: Pragmatic Bookshelf • Publication date: October 8, 2013 • Binding: Paperback • Estimated price: $23.51

Node.js in Action By Mike Cantelon, Marc Harter, TJ Holowaychuk, Nathan Rajlich • ISBN: 1617290572 • Publisher: Manning Publications • Publication date: October 28, 2013 • Binding: Paperback • Estimated price: $29.76

The LEGO MINDSTORMS EV3 Laboratory: Build, Program, and Experiment with Five Wicked Cool Robots! By Daniele Benedettelli • ISBN: 1593275331 • Publisher: No Starch Press • Publication date: October 25, 2013 • Binding: Paperback • Estimated price: $17.75

iOS 7 Programming Cookbook By Vandad Nahavandipoor • ISBN: 1449372422 • Publisher: O'Reilly Media • Publication date: October 30, 2013 • Binding: Paperback • Estimated price: $35.53

Android Application Development in 24 Hours, Sams Teach Yourself (3rd Edition) By Carmen Delessio, Lauren Darcey, Shane Conder • ISBN: 0672334445 • Publisher: Sams Publishing • Publication date: October 26, 2013 • Binding: Paperback • Estimated price: $24.96

Test-Driven Infrastructure with Chef: Bring Behavior-Driven Development to Infrastructure as Code By Stephen Nelson-Smith • ISBN: 1449372201 • Publisher: O'Reilly Media • Publication date: October 24, 2013 • Binding: Paperback • Estimated price: $27.82

Minecraft: The Unlikely Tale of Markus "Notch" Persson and the Game that Changed Everything By Daniel Goldberg, Linus Larsson • ISBN: 1609805372 • Publisher: Seven Stories Press • Publication date: October 17, 2013 • Binding: Hardcover • Estimated price: $11.90

Smart Machines: IBM's Watson and the Era of Cognitive Computing By John E. Kelly III, Steve Hamm • ISBN: 023116856X • Publisher: Columbia University Press • Publication date: October 15, 2013 • Binding: Hardcover • Estimated price: $16.18

Getting Started with BeagleBone: Linux-Powered Electronic Projects With Python and JavaScript By Matt Richardson • ISBN: 1449345379 • Publisher: Maker Media, Inc • Publication date: October 18, 2013 • Binding: Paperback • Estimated price: $8.49

A Tour of C++ By Bjarne Stroustrup • ISBN: 0321958314 • Publisher: Addison-Wesley Professional • Publication date: October 3, 2013 • Binding: Paperback • Estimated price: $19.77

Mining the Social Web: Data Mining Facebook, Twitter, LinkedIn, Google+, GitHub, and More By Matthew A. Russell • ISBN: 1449367615 • Publisher: O'Reilly Media • Publication date: October 17, 2013 • Binding: Paperback • Estimated price: $24.25

Learn Python the Hard Way: A Very Simple Introduction to the Terrifyingly Beautiful World of Computers and Code (3rd Edition) By Zed A. Shaw • ISBN: 0321884914 • Publisher: Addison-Wesley Professional • Publication date: October 11, 2013 • Binding: Paperback • Estimated price: $23.76

Think Bayes By Allen Downey B. • ISBN: 1449370780 • Publisher: O'Reilly Media • Publication date: October 1, 2013 • Binding: Paperback • Estimated price: $22.83

The Lean Mindset: Ask the Right Questions By Mary Poppendieck, Tom Poppendieck • ISBN: 0321896904 • Publisher: Addison-Wesley Professional • Publication date: October 4, 2013 • Binding: Paperback • Estimated price: $23.15

3D Game Programming for Kids: Create Interactive Worlds with JavaScript By Chris Strom • ISBN: 1937785440 • Publisher: Pragmatic Bookshelf • Publication date: October 23, 2013 • Binding: Paperback • Estimated price: $24.39

Pro jQuery 2.0 By Adam Freeman • ISBN: 1430263881 • Publisher: Apress • Publication date: October 16, 2013 • Binding: Paperback • Estimated price: $47.99

Practical Programming: An Introduction to Computer Science Using Python 3 By Paul Gries, Jennifer Campbell, Jason Montojo • ISBN: 1937785459 • Publisher: Pragmatic Bookshelf • Publication date: October 1, 2013 • Binding: Paperback • Estimated price: $27.64

Java Network Programming By Elliotte Rusty Harold • ISBN: 1449357679 • Publisher: O'Reilly Media • Publication date: October 14, 2013 • Binding: Paperback • Estimated price: $23.95

Modern C++ Programming with Test-Driven Development: Code Better, Sleep Better By Jeff Langr • ISBN: 1937785483 • Publisher: Pragmatic Bookshelf • Publication date: October 17, 2013 • Binding: Paperback • Estimated price: $31.30

Play for Scala: Covers Play 2 By Peter Hilton, Erik Bakker, Francisco Canedo • ISBN: 1617290793 • Publisher: Manning Publications • Publication date: October 11, 2013 • Binding: Paperback • Estimated price: $27.92

“Any girl can be glamorous. All she has to do is stand still and look stupid.” —Hedy Lamarr, co-developer of a frequency-hopping idea that underlies current spread-spectrum communication technology, including Bluetooth, COFDM, and CDMA.

On November 4, 2013, “human calculator” Shakuntala Devi would have turned 84. Google honored her birthday with a Google Doodle. I bet she would have liked that.

Living in the Future

Code behind this comp-bio paper‪‪ is on github, discussion on reddit, science‬‬’‪‪s future is now.‬‬ — ‪@judell‬

What if you could swap out a fresh battery for your electric vehicle at 7-11? In Japan, it’s happening for scooters. — ‪@mims‬

If Mad Men were set in current times, they’d just need to have everyone on their phones rather than smoking cigarettes. — ‪@dannysullivan‬

Bicycles are outselling cars in Europe and that might not be just a blip. — ‪@mims‬

Apparently asking someone to marry you at the Apple Store is a common thing. — ‪@migueldeicaza‬

I hear internet-cable-gnawing raccoons are the Oregon version of flying monkeys. — ‪@lxt‬

“I think men will always find an excuse for keeping women in their ‘place.’ So, let’s make that place the executive suite and start more of our own companies.” –Jean Bartik, one of the original programmers for the ENIAC computer.

Who Are Those Guys?

First, they’re not all guys. Second, we have to confess that we cleaned up the punctuation and stuff in the tweets. Here’s the list of tweeters we followed this month: Chris Adamson, Kent Beck, Sara Chipps, Jim Dalrymple, Miguel de Icaza, Andy Ihnatko, Andy Lester, Brian Marick, Christopher Mims, Danny Sullivan,
Travis Swicegood, Dave Thomas,
Laura Thomson, Jon Udell, and Dave Winer.

Antonio Cangiano does our books; he also wrote the excellent Technical Blogging and you can subscribe to his reports on new books in technology and other fields here.

Okay, so I got the new Apple Time Capsule; buried it in the yard; nothing happens. I don't get it. — ‪@jdalrymple‬

Fair’s fair. You can follow us at www.twitter.com/pragpub.

Rothman and Lester

Unhappy at Work?

by Johanna Rothman and Andy Lester

Should I stay or should I go?

Andy: This month’s topic is dealing with unhappiness and frustration on the job, and how to know when it’s time to move on to another job. The issues of leaving a job are bigger than unhappiness and cover things like career planning, but I’ve seen many people, myself included, quit jobs that were worth keeping because they were dealing with some temporary unhappiness on the job.

The one thing I want to shout loudest is: Never quit your job because of another person at the company. So often there will be someone else at the company that makes our day-to-day lives not as good as they could be. Maybe it’s a boss that’s a micromanager, or maybe it’s someone obnoxious on your team, or maybe it’s someone in another department that is making unreasonable requests of you. The thing to remember is that there will be jerks always, and leaving one place to go to another will just take you from one jerk to another.

(Now, of course, I’m not talking about extreme cases where you’re dealing with sexual harassment or you feel unsafe.)

Johanna: I agree. You want to separate all the things that are making you unhappy. That can be tough! Sometimes you have things going on at home, too. Remember, you’re a whole human being. One thing I suggest is to make a list of everything that you enjoy about your job, the pros. And make another list, the cons. Now, I bet there are things you can do about the cons. If you haven’t talked to that obnoxious person on your team and provided feedback, in a peer-to-peer way, it’s time.

Maybe that person doesn’t know what he or she is doing that’s making you berserk. It’s time to tell that person nicely. Maybe it’s even time to get the entire team to tell that person. Not ganging up, but when each person tells a team member, that person realizes it’s serious. You can give your boss feedback, too. If you’ve established a relationship with your boss via one-on-ones, and you keep asking for those one-on-ones, you’ll be in great shape. If you haven’t, it’s time.

Andy: Keep in mind when you’re dealing with the problem people that it’s not the person himself (or herself) that is the problem. It’s that the company/department/group allows that person to be that way.

Johanna: Right, who’s placating this person, who’s allowing this bad behavior to exist for so long? I see “bad” or weak management all the time.

Andy: Say you’ve got some people in your group who are hostile and rude. They may be insulting in meetings or when working on projects. The problem isn’t that Dave is a jerk, but that Dave’s boss allows Dave to be a jerk. The end result remains the same, but it’s important to recognize where the core problem is.

Johanna: I start by providing Dave feedback and see if that works. You want to start with the root of the problem. But if Dave’s boss sees Dave as “a star performer,” that’s a huge problem. I always say, “Fire the indispensable employee!” They always cause more problems than they are worth. Mostly because managers screw things up. The managers mollycoddle those stars and placate them, so their behavior is bad in any number of ways.

Andy: Right, if Dave’s a jerk, but management sees Dave as a star performer, then what do you know? You know that management values individual jerks over the rest of the team. THAT is a problem worth finding another job over.

Johanna:Yes, because software development is a team sport. Period. There are no one-person projects. You can try to have heroes save the day, but it’s not worth it. I don’t care what approach to software you have, agile, waterfall, anything in between—any approach to software is a team sport.

On the other hand, it might be that Dave is a person who is a little clueless about checking his voicemail in cubicle-city, as one of my colleagues was, a number of years ago. He used his speaker phone. Everyone for 3 rows and 3 columns over could hear his voicemails. It was quite irritating. What about that? He’s not a jerk. He’s just clueless.

Andy: Let’s look at another source of frustration: Technical stagnation. “We’re still stuck on PHP 4, and I hate it.” “We’re still using CVS as our version control system,” etc. PHP 4 is not the problem. That the company is willing to live with outdated technology is. Why do they do this? Is the work you’re doing seen as not worth the time to upgrade? Maybe the company sees your work as expendable, or they know that in a year it’s going to be replaced with something else, and so will you. Or maybe they don’t realize the mountain of technical debt they’re under by relying on ancient technology. Either of those reasons might well be indicators it’s time to start actively looking.

Johanna: One question for you, if you have this on your list of Cons: have you tried to change anyone’s mind about this? Or, are you only complaining to your fellow techies? If management doesn’t realize the cost of staying behind on outdated technology, they may not realize the cost of their decisions. But if they DO realize the cost, and they still make the decision, leave. Pronto. Why? Because you are not working for a company that is making decisions about their project portfolio in a way that makes sense for the long term. They are making decisions based on cost, not on value. Bad move.

Andy: Also, consider if the unhappiness is localized to you, or if it’s everyone. If you’re the only person who’s doing scutwork projects and everyone else is getting the fun stuff, find out why that is. Tell your boss that you understand that boring maintenance projects are part of programming life, but that you’ve felt left out for a while. And then listen.

If your entire group is doing maintenance and nothing new is happening, that can point to something much larger and much more deadly. Why is your department not getting assigned new projects? Does the company not value your department any more? Does company vision no longer include expansion and new frontiers?

Johanna: I’ve done my share of maintenance work. Once, I had THE job of maintenance for a small company and a small development group. I’d been stuck on maintenance for six months. Everyone else had moved on to the new cool stuff. I was still stuck on the old stuff.

I told my boss I was going to leave if I didn’t get to work on the new stuff. He said, “But who will work on the maintenance?” I told him that wasn’t my problem. If I left it was going to be his problem. He didn’t believe I was going to leave. I did. They figured it out.

If the entire group is only doing maintenance, look around and see who is doing new work. Where are they located? That tells you more about the strategy for the organization than you might want to know. A number of my colleagues learned that their organizations were moving to off-shoring before the organizations announced it, just by looking at where the new projects were going.

Andy: Well, you DO want to know, you just might not like what it says.

One other common aspect of job dissatisfaction: Boredom. “I’ve been working on the WangoBongo website for three years now, and I feel like I want to do something else.” Boredom’s an entirely legitimate reason to want to quit.

The first step, of course, is to talk to your boss. Maybe you can transfer to another department, or another unit of the company. Although that has risks—because if you do it wrong you can be seen as a complainer. But the key with boredom is that not all of your excitement necessarily needs to come from work. Part of the reason I spend so much time in open source is that it allows me to scratch itches after hours that I can’t get scratched at work.

Johanna: There’s project boredom, product boredom, and company boredom. It depends on how big your organization is. It also depends on how much autonomy you have at your job.

Maybe your project really is boring! In that case, see if you can move around in the organization to another project. But maybe the organization doesn’t do it for you anymore. In that case, you need another product to work on. Does the organization have a different product you can work on?

If your organization has one type of product, and you’re tired of it, it’s time to look around for something else. Or, maybe it’s time to take a class in something else, to see if you like it. You don’t have to leave to see if you like something else. Maybe you need to expand your options first. As you say, you spend time in open source to scratch that creative itch you can’t scratch at work.

But one of the things that’s a big red flag: if you’re bored by the company’s products, maybe the customers are too. That’s a sign. Think about it.

Andy: It’s also entirely possible that the open source work you do can feed back to what you do for your day job.

Johanna: Yup. All your learning can feed into all parts of you.

Andy: I tell the story over and over about how I eagerly took a job doing new (to me) cool tech on systems I would get to learn on, and for more money, too. Problem was, it was in the financial industry and I had absolutely no interest in the work that they were doing. That’s why the entire first chapter of my book is about knowing what is important to you.

Johanna: That’s a problem. I guess you and I are similar in that we need to care something about the products we work on. I’ve learned new tech at all the jobs I’ve had, and I guess I’ve been lucky. I’ve always enjoyed learning about them. Maybe I intuitively understood what I would like and wouldn’t like. Because I didn’t have your book!

Andy: There’s no right or wrong about what matters. It’s not “wrong” or “selfish” or “short-sighted” to value interesting work. I know that for some older generations, the idea of leaving a job because of boredom seems appalling, but it really isn’t.

Johanna: We spend at least one third of our lives at work. We should enjoy it!

Andy: I phrase it that you spend more hours at work than you do awake with your spouse.

The flipside of the boredom issue is that it’s unrealistic to expect your job to always be exciting. We can’t expect that every project will be a green field project with all new requirements and you conjure magic out of the ether. Sometimes you have to do the grunt maintenance work. That’s to be expected.

Johanna: I don’t think I want my job to be exciting every day. I do want to have satisfaction every day. There’s a difference. I own my own business, and I do grunt maintenance work. I expect that!

Andy: It’s all about balance. It’s about how much of the fun stuff is there with the not-so fun stuff. It’s about how many of your coworkers are a joy to get along with and how many aren’t.

Johanna: And, if you have a reasonable relationship with your manager.

Andy: One more story, this one about my being tempted to quit. We were at an off-site IT planning retreat: My four peers, our boss, and my grandboss. We were in a Holiday Inn conference room all day with butcher paper taped to the wall, planning our strategy for the coming year. At one point, while I was discussing some part of the website strategy (this was back in the 90s) my boss laid into me. Told me my ideas were stupid. How could I think that that made sense? It was pointed and hostile, and I was astonished. I’d never seen anything like that before. I was stunned into silence, and I let the meeting go on without me.

As we broke at midday and made our way down to lunch, I saw grandboss and said “Did I do something there I missed?” and he said “You’re fine. Don’t worry about it.” so I muddled through the rest of the day feeling a bit better about what happened.

Boss was fired the following week. I’m certain that he was on his way out anyway, but this illustrates in a micro sense what we’ve been talking about at a macro level. There are things that may be bad, but they in themselves may not be worth leaving over. Certainly after I was degraded in front of my peers I had flashes of wanting to quit. Turns out that it was a temporary hiccup that I just needed to ride out.

Johanna: It’s a good thing you checked with the grandboss. That’s why building your relationship with management, not just your direct manager, is a good thing.

Andy: Exactly. Grandboss could also have said “I don’t know what you were doing in there, Lester. Those ideas are stupid.” And I’d have known it was time to go.

Johanna: So, if you’re feeling dissatisfaction at work, articulate it, by writing it down. Don’t just complain. Make a list of Pros and Cons. Now you have something you can do something about. What can you do something about directly? What do you need management for? Attack the ones you can do something about. Provide feedback to the people you haven’t given feedback to.

Andy: Next month in this space, we’ll continue with dealing with management and answer the age-old question “How do I get my boss to do X?” X can be “use Ruby” or “let us telecommute” or “buy us new chairs” or “send me to a conference,” but the answer is always the same. We’ll see you in 30.

Johanna Rothman helps leaders solve problems and seize opportunities. She consults, speaks, and writes on managing high-technology product development. She enables managers, teams, and organizations to become more effective by applying her pragmatic approaches to the issues of project management, risk management, and people management. She writes the Pragmatic Manager email newsletter and two blogs on www.jrothman.com.

Andy Lester has developed software for more than twenty years in the business world and on the Web in the open source community. Years of sifting through résumés, interviewing unprepared candidates, and even some unwise career choices of his own spurred him to write his nontraditional book on the new guidelines for tech job hunting. Andy is an active member of the open source community, and lives in the Chicago area. He blogs at petdance.com, and can be reached by email at andy@petdance.com.

Send the authors your feedback or discuss the article in the magazine forum.

Namespaces in Clojure

Organizing Code into Simpler Building Blocks

by Michael Bevilacqua-Linn

Michael gives you “just enough about namespaces to get you going.”

Welcome again to the pragmatic series on Clojure!

Up until now, we’ve seen plenty of examples of programming in the small, but what happens when we need to organize the functions, macros, and protocols we’ve seen into a larger unit? This time, we’ll take a brief tour through Clojure’s namespace system, which gives us a way to organize our code at a higher level.

Let’s start by taking a look at a namespace created using the ns macro and some of its most common options. There are several ways to create a new namespace, but ns is the most common and convenient way to do so in a Clojure source file.

(ns example.namespace

"Look at me, I'm a docstring!

In the real world, I'd be useful documentation."

(:require [clojure.string :as string])

(:use [clojure.pprint :only [pprint]])

(:import [java.util BitSet Calendar]))

(def a-string "this is a string")

(defn split-on-commas [str]

(string/split str #","))

Here, we’ve created a namespace called example.namespace. Clojure namespaces’ names usually consist of several parts separated by periods, somewhat like Java classes, but they do not follow the convention of starting off with a reversed domain name.

We’ve included code from other Clojure libraries with the :require and :use directives, and a Java class with the :import directive. Finally, we’ve created a string and given it the name a-string and created a function named split-on-commas.

We’ll dig into the two directives that let us use code from Clojure libraries, :require and :use first. The :require directive loads a Clojure library from another namespace. Here, we’ve loaded Clojure’s built-in string library.

By default, items in the loaded library need to be referenced by their qualified name, which means that they’re prefixed with the full name of the namespace they were loaded from followed by a forward slash. For instance, we’d refer to the split function in the string library as clojure.string/split, as we show below.

=> (clojure.string/split "foo,bar,baz" #",")

["foo" "bar" "baz"]

Since this can be verbose, especially for long namespaces, :require lets us provide an alias using :as. Here, we alias clojure.string to string, so we can refer to split as string/split.

=> (string/split "foo,bar,baz" #",")

["foo" "bar" "baz"]

If we’d like to load multiple libraries with the same prefix, we can do so in one shot by pulling the common prefix out, as we demonstrate below.

(:require [clojure [string :as string] [xml :as xml]])

This loads the clojure.string library and aliases it to string and the clojure.xml library, which it aliases to xml.

It’s often a good idea to load libraries using :require and :as to alias the namespace. Doing so not only ensures that we avoid clashes if we load libraries that have functions with the same name, but can help us write more descriptive code. The function name split is fairly ambiguous; string/split is clearer.

However, sometimes having to preface everything with a namespace, even a short alias, is a pain. Lets take the pprint function, which pretty-prints Clojure’s built-in data structures and lives in the clojure.pprint library as an example.

This function is unlikely to collide with a function in another namespace, and doesn’t gain anything from a descriptive alias. It would just be cumbersome to have to call it namespace qualified all the time, even with an alias this would result in calls that look like (pprint/pprint some-data-structure).

In these cases, Clojure’s :use directive comes to the rescue. It loads the library and refers it into the current namespace so that symbols can be used directly as we show below.

=> (pprint

{:foo "foo"

:bar "bar"})

{:foo "foo", :bar "bar"}

nil

When using the :use directive, it’s a good idea to include :only, which specifies a vector of symbols to refer into the current namespace. Anything not in the vector will still be loaded and available by its qualified name, but not but its unqualified one.

Finally, let’s take a look at :import. This is much like Java’s :import statement, but it allows us to import multiple classes easily using prefixes, much as we saw with Clojure’s libraries and :require.

The (:import [java.util BitSet Calendar]) form that we saw earlier imports java.util.BitSet and java.util.Calendar. We can then use those classes using Clojure’s java interop features. For instance, to create a new BitSet class we append the classname with a period, which calls its constructor.

=> (def a-bitset (BitSet.))

#'foo.bar/a-bitset

We’ve only touched on the most commonly used parts of Clojure’s namespaces here. Both :use and :require have several other useful options. All three options we’ve seen for loading code are also available as the functions, use, require, and import for easy use in the REPL.

While namespaces may seem boring, I think there’s one fairly fascinating thing about them in that they provide a way to organize code into larger units than functions without using objects. Objects, at least as implemented by mainstream object-oriented languages, combine many features.

Package names in Java, for instance, are a form of namespacing. In addition objects in Java are polymorphic, and they’re intended to form composable, reusuable software modules, amongst other things. Clojure teases these features apart. We get polymorphism with protocols and multimethods, which are split out from the namespace system.

By doing so, Clojure gives us simpler building blocks than the traditional object oriented approach.

Thanks for reading, and hope you join us next time when we introduce Clojure’s Java interop features in more detail!

Michael Bevilacqua-Linn has been programming computers ever since he dragged an Apple IIGS that his parents got for opening a bank account into his fifth grade class to explain loops and variables to a bunch of pre-teenagers. He currently works for Comcast, where he builds distributed systems that power infrastructure for their next generation services, and wrote Functional Programming Patterns for the Pragmatic Bookshelf. He tweets occasionally at @NovusTiro.

Send the author your feedback or discuss the article in the magazine forum.

Rediscovering Awk

Sometimes Older Programs are the Right Tool for the Job

by Derrick Schneider

A tedious task sends Derrick digging through the musty man pages of an archaic language, where he unearths some gems.

I had a problem. I needed to analyze some Java thread dumps from production servers. The Ruby script I used was on my work machine, but I wasn’t at work; I was at home. I had to log in via VPN to download the thread dump and run it through my script. It worked, but it was clumsy.

“This would be easier if the script were on the machine,” I groused to a teammate. But Ruby wasn’t installed on the servers.

“You should rewrite it in awk,” he said.

Like most people who have worked with UNIX, I had used awk before, usually in “one-liners” that squirted data through a chain of shell commands. Intrigued by his suggestion, I trawled through man pages and websites and realized I had been missing out on awk’s full power.

Needless to say, awk was already on my server. In fact, awk’s been included with UNIX since the late 1970s. It was on my high-end production server; it was on my MacBook. It’s on my Raspberry Pi, and it’s on the bare-boned Micro Instance in Amazon’s EC2 offerings.

While awk’s original authors—Alfred Aho, Peter Weinberger, and Brian Kernighan, from whom the program gets its name—built everything from simple one-liners to full-fledged relational databases on top of awk, it’s best to use it not as a general-purpose language but as a domain-specific language for processing text. It handles the boilerplate of reading and parsing from a file or standard in; you focus on your problem.

One-liners are a common use for awk, but it can also load scripts saved in files via the command-line option -f. It’s in those scripts where awk shines.

Since I fumbled my way through rewriting my thread-dump scripts, awk has snagged a permanent place in my arsenal. Readers of this magazine know the latest and greatest tech, but sometimes an older program is the right tool for the job. Here are a few ways I’ve used awk since I rediscovered it, which I hope will inspire you to give this ubiquitous language a deeper look.

Sleep, Baby, Sleep

Crying. Feeding. Burping. Bouncing. Patting. Changing. Anyone with a baby will recognize the pattern. My wife and I were in the thick of it. Like every parent everywhere, we wanted our baby to sleep. We wanted to sleep. We used an iPhone app to record her habits, but it was better at capturing metrics than analyzing them. The app would tell you the number of hours the baby had slept, but not when those hours had happened. There’s a big difference between two six-hour stretches and twelve one-hour stretches. I wanted better charts.

Fortunately, the app can export data. Unfortunately, it formats for humans, not machines. Here’s an excerpt:

- Start Time 3/30/13, 5:41 AM

- Duration 1 hr 51 min

- Start Time 3/30/13, 5:25 AM

- Duration 7 min

Try slurping that into your favorite visualization tool. Date and duration are on different lines, a tab separates the columns while spaces separate the pieces in the right column, and the duration format varies depending on the amount. A simple awk script unscrambled the text into comma-separated values.

function quote(string) {

return "\"" string "\""

}

function print_csv_line(key, date, timestamp, minutes) {

print quote(key), quote(date), quote(timestamp), quote(minutes)

}

BEGIN {

FS = "[, \t]"

OFS = ","

print quote("line"), quote("start_date"), quote("start_time"),

quote("duration")

}

/Start Time/ {

date = $3

full_timestamp = (date " " $5 " " $6)

}

/Duration.*hr.*min/ {

print_csv_line(NR, date, full_timestamp, ($2 * 60) + $4)

}

/Duration.*hrs?$/ {

print_csv_line(NR, date, full_timestamp, $2 * 60)

}

$3 ~ /min$/ {

print_csv_line(NR< date, full_timestamp, $2)

}

Don’t worry if this script seems indecipherable. Awk is easy once you know the basics: An awk script contains a list of conditions and actions that act on the data you provide.

For each input record, a line by default, awk checks all the conditions—the pieces outside the curly braces. For every condition that matches the record, it runs the associated actions—the pieces inside the curly braces. If you omit a condition, awk runs those actions for every record. If you omit actions for a condition, awk prints the record. If a record matches multiple conditions, their actions will be run in top-down order. Awk defines two special conditions, BEGIN and END, that match before any input is read and after every record is read, respectively.

When awk reads a record, it creates some variables for you. $0 is the entire record. $1, $2, and so on are the individual fields, split on spaces by default—awk counts from one, not zero, even with arrays. You can modify $0 and see changes reflected in $1 and so forth, and you can change the value of any field and see it reflected in $0. NF contains the total number of fields in that record, NR contains the record number of the current record across all the files awk is processing, and FNR contains the record number within the current file.

Note that, while I’ve removed them for brevity, awk supports comments that start with the hash character.

Now you can dissect my script. Before awk reads any lines, the BEGIN actions are run. In this case, I change the default field separator (FS) from space to any one of comma, space, or tab. I include comma to ensure that it’s removed from the end of strings such as "3/30/13,". I also set the output field separator (OFS) to comma. Then I print out the headers for the file. In awk, strings separated by commas are joined together with the value of OFS (which in this case happens to also be a comma but could be anything).

I’ve defined a quote function to refactor and clarify some code. This function concatenates strings without a separator by writing them in sequence. I’ve also defined a print_csv_line function that quotes and joins all the arguments with OFS.

The next condition finds lines that match the regular expression Start Time. The actions capture the date by itself and the date with timestamp as two separate variables. This made things easier for my visualization script. Variables in awk are typically global, so these variables will be waiting for the next record. You can use variables before defining them: They default to the empty string.

The next three conditions each capture a different form of the Duration line. A mix of hours and minutes will match the regular expression Duration.*hr.*min. Hours without minutes matches the regular expression Duration.*hrs. Less than an hour means that the third field will match the regular expression min followed by the end of the line. (You can use ! to specify records that don’t match.) In each case, I call print_csv_line with the record number (to provide a primary key for the row), the date, the full timestamp, and the total number of minutes. You can see that awk converts back and forth between strings and numbers as needed.

Here’s what the script produces:

"line","start_date","start_time","duration"

"4697","3/30/13","3/30/13 5:41 AM","111"

"4700","3/30/13","3/30/13 5:25 AM","7"

This format made it a snap to craft a lovely chart showing how little sleep we were getting.

KPDC

Even before I had a baby, I couldn’t keep up with radio programs I wanted to hear; they always happened at inconvenient times. I set up a server to snag those programs via the Internet and created a personal podcast of shows I want to hear.

Various scripts capture the streams and upload them to Amazon.com’s S3 service, but I needed a properly formatted podcast XML to subscribe to.

I use s3cmd for talking to S3, and here’s the sample output from its ls command:

2013-09-22 14:20 172800000   s3://mypodcastmusic/aida.mp3

2013-08-05 15:41 229968000   s3://mypodcastmusic/peter_grimes.mp3

2013-09-23 01:00      1521   s3://mypodcastmusic/podcast.xml

2013-09-22 15:31 172560000   s3://mypodcastmusic/turandot.mp3

At least its output is better than the baby-monitoring iPhone apps! But it’s still not what I need. This awk script changes that output into correct XML.

BEGIN {

"date -R" | getline d

print "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"

print "<rss version=\"2.0\">"

print "<channel>"

print "<title>My Radio Podcast</title>"

print "<description>Slurped internet radio</description>"

print "<link>http://programmingobsession.blogspot.com</link>"

print "<language>en-us</language>"

print "<copyright>Copyright 2013</copyright>"

print "<lastBuildDate>" d "</lastBuildDate>"

print "<pubDate>" d "</pubDate>"

print "<webMaster>derrick.schneider@gmail.com</webMaster>"

}

/\.mp3/ {

print "<item>"

line_date = $1" "$2

date_cmd = "date -d \" " $1 " " $2 " -0000\" -R"

date_cmd | getline item_date

close(date_cmd)

size = $3

url = substr($0, index($0,$4)) # get the last n fields

split(url,url_components,"/")

file = url_components[4]

directory = url_components[3]

print "<title>" file "</title>"

print "<link>" url "</link>"

print "<guid>" url "</guid>"

print "<description>" line_date "</description>"

print "<enclosure url=\"http://s3.amazonaws.com/" directory "/"

file "\" length=\"" size "\" type=\"audio/mpeg\"/>"

print "<category>Podcasts</category>"

print "<pubDate>" item_date "</pubDate>"

print "</item>"

}

END {

print "</channel>"

print "</rss>"

}

Obviously, awk is here filling in a template. It begins by printing out the podcast metadata. Then for each mp3 in the listing, it generates the XML for that file. But there are some extra aspects that weren’t in the first script.

The first is using shell commands to do work. In awk, you can write any string to the shell with the | operator. And you can use getline to pull a line out of that command’s output (without the pipe operator, getline snorts up the next input record without processing it normally). getline by itself overwrites $0, but if you pass it a variable name, it will write into that variable. While awk will close those shell processes when it quits, you should use close to manually close them if you expect that action to get run many times: The operating system won’t let you spawn processes indefinitely. In the BEGIN block, I use date -R to get the current date. Within the actions for the .mp3 pattern I again use date to reformat what s3cmd gives me into the appropriate format (RFC-822). I call close with the exact same string I used to open the process, since this action will get run numerous times.

Second, you can see some of awk’s built-in string commands, such as substr, which slices a substring out of a source (remember that everything in awk counts from one).

Finally, this script also uses awk’s arrays. Arrays in awk are associative arrays, meaning that you can use any key you want. The split() command the script uses—which can be read as “split this string into this array using this delimiter and return the number of items”—happens to insert items using numerical indices. With numerical indices, you can use a standard for (i = 1; i <= count; i++) syntax; for non-consecutive indices, awk provides a for (variable in array) syntax.

Once a cron job uses this script to churn out the XML and upload it to S3, my personal podcast is ready to play.

Check Your Parameters

At my current job, checking for null parameters on public methods is a constant refrain during code reviews. Rather than wait for reviewers to spot lapses, I wrote a script that could find them all before a review.

function last_index_of(complete_string, search_string,

_next_found_index, _prev_found_index) {

do {

_next_found_index = index(complete_string, search_string)

if (_next_found_index > 0) {

_prev_found_index += _next_found_index

complete_string = substr(complete_string, _next_found_index + 1)

}

} while (_next_found_index > 0)

return _prev_found_index

}

BEGIN {

check_not_null_string = "checkNotNull("

}

FNR == 1 {

printed_filename = 0

}

/public.*\(/, /\{/ {

sub(/[ \t\r\n]*/, "")

if (aggregate_method_declaration == "") {

aggregate_method_declaration = $0

method_line = FNR

} else {

aggregate_method_declaration = (aggregate_method_declaration " " $0)

}

if (index(aggregate_method_declaration, "{") == 0) {

next

}

$0 = aggregate_method_declaration

gsub(/[{}]/,"")

aggregate_method_declaration = ""

printed_method = 0

method_start = match($0, /^[^ \t\r\n]*/)

method_definition = $0

in_public_method = 1

prefix_spaces = substr($0, 1, method_start - 1)

left_paren = index($0, "(")

right_paren = last_index_of($0, ")")

vars = substr($0, left_paren + 1, right_paren - left_paren - 1)

if (vars == "") {

next

}

split(vars, declarations, ",")

for (var_index = 1; var_index <= length(declarations); var_index++) {

if (match(declarations[var_index],

/(int|long|boolean|short|double|float|byte)/) > 0) {

continue

}

split(declarations[var_index], tokens, " ")

variable_name = tokens[length(tokens)]

has_check[variable_name] = 0

}

}

in_public_method == 1 && /checkNotNull/ {

check_string_index = index($0, check_not_null_string)

variable_name_start =

check_string_index + length(check_not_null_string)

variable_name =

substr($0, variable_name_start, length($0) - variable_name_start)

variable_name_end = index(variable_name, ")") - 1

variable_name = substr(variable_name, 1, variable_name_end)

gsub(/[ \t\r\n]/, "", variable_name)

has_check[variable_name] = 1

}

in_public_method == 1 && $0 ~ (prefix_spaces "}") {

in_public_method = 0

for (variable_name in has_check) {

if (has_check[variable_name] == 0) {

found_unchecked = 1

if (!printed_filename) {

print FILENAME

printed_filename = 1

}

if (!printed_method) {

print "    " method_definition " (" method_line ") "

printed_method = 1

}

print "        " variable_name " has no null check"

}

delete has_check[variable_name]

}

if (found_unchecked) {

print ""

}

found_unchecked = 0

printed_method = 0

}

This big script is easy to figure out when you break it down. The overall flow is as follows: When there’s a public method, figure out all the variables in its signature. Assume each one is not checked. When you find a call to checkForNull (a method in Google’s Guava library), figure out which variable is being checked and record it. When reaching the end of a method, if any variables weren’t checked for null, print them out, along with the filename, the method, and the method’s line number.

The last_index_of function finds the last instance of a given character in a string. The code is straightforward: It tries to accumulate the index of the character in progressively smaller strings. If it finds the character, it chops the string from that character on. If it doesn’t, it returns the last index it did find.

But there are three items to note in it. First, awk has the standard do-while construct which ensures the loop’s code runs at least once. Second, because strings are passed by value to functions—arrays are passed by reference—the loop can change the string without affecting the larger script. Finally, if you want functions to have local variables—as opposed to awk’s norm of global variables—you declare them at the end of the arguments to the function.

Let’s look at the conditions defined by this script.

I use the condition FNR == 1 to record when I enter a new file; this script often gets run against a set of Java files, and FNR is set to 1 at the beginning of each one.

The main condition introduces another useful feature of awk: The ability to define ranges of matches. By separating conditions with a comma, you tell awk to start matching when the first condition is met and stop matching when the second condition is met (which could be in the same record).

Remember that a Java method could start like this:

public int setX(Object x) {

or like this:

public void updateUser(@PathParam("user") long userId,

@QueryParam("email") String email) {

Starting the match when the line matches public.*( and stopping when it finds the left curly brace causes awk to give you every line in the method signature. But what the script wants is the full signature without line breaks. So it accumulates the lines in the signature into the variable aggregate_method_declaration until it finds a curly brace, using awk’s next command to force a new record to be read and processed. At the first line, it captures the line number within that file.

The prefix_spaces and in_public_method variables track state used below. You can also see more built-in string functions to clean up the string and parse out variable names. The sub function replaces the first match of the regex with the provided string. If you pass a third argument, sub changes the value in that variable; otherwise it works against $0. The gsub function does the same except that it replaces every instance of the regex. The match function returns the location of the first match of the given regex within the given string.

To parse the variable names, I first find everything between the first (" and the last ") in the complete method signature. The updateUser method above would yield @PathParam("user") long userId, @QueryParam("email") String email. Split that on commas, and you produce an array where item 1 is @PathParam("user") long userId and item 2 is @QueryParam("email") String email. For each of those items, split on space—producing a new array named tokens where the first item is @PathParam("user") and so on—and grab the last item in that array, which must be the variable name.

For each variable name, set an entry in the has_check array that assumes the variable is not checked.

Because we use Google’s Guava library, the script can just match on the function name checkNotNull. If we find the string �checkNotNull�, and we’re in a public method, we parse the variable name and record the null check.

Our coding standards require that the final right curly brace in a method line up with the first character of the declaration. When parsing the method, the script captured the number of leading spaces in the variable prefix_spaces. The end of the method is thus easy: If we’re in a public method, find lines where $0 matches prefix_spaces.s followed by a right curly brace. Check the has_check array to find unchecked variables, print them (along with the file name and the method if they haven’t already been printed) and use delete to clear the array element. The result is something like this, run against a personal project of mine:

async_requests/src/main/java/com/derrickschneider/async/

AsynchronousController.java

public void handleAsynchronousRequest(HttpServletRequest request,

HttpServletResponse response) throws Exception  (13)

request has no null check

response has no null check

async_requests/src/main/java/com/derrickschneider/async/

AsyncRequestsService.java

public void addAsyncContext(AsyncContext ctxt) throws Exception  (32)

ctxt has no null check

public RequestProcessingRunner(AsyncContext ctxt)  (38)

ctxt has no null check

Good thing those aren’t going into code review!

Awkward and Upward

I hope this glimpse gives you an idea of the power of awk and encourages you to dig in, as I did months ago. I’ve used awk to create simple DSLs, and I even used it to work with this draft, checking word counts of non-source-code lines and the like. Once you start down this path, there’s a lot more to explore. While awk is installed on virtually everything, GNU awk, or “gawk,” is the default awk implementation on Linux and adds a number of useful features to the core language: include files, native date-time functions, networking support, and more. If you’re processing very large data sets, you should check out “mawk,” a high-performance implementation of awk that often trounces any other text processing tool.

Have fun!

Derrick Schneider is a Berkeley-based writer and software engineer. He currently works at Opower, and can be reached at derrick.schneider@gmail.com. He blogs erratically at programmingobsession.blogspot.com. And he’s still not getting a lot of sleep.

The editor feels compelled to add that Derrick has also had An Obsession with Food and Wine since 2002. Hence (and whence) the picture. Send the author your feedback or discuss the article in the magazine forum.

Good and Bad Technical Debt

And How TDD Helps

by Henrik Kniberg

Sometimes technical debt is good. The trick is knowing how much and when.

Technical Debt is usually referred to as something bad. In another article of mine, The Solution to Technical Debt, it certainly implies that, and most other articles and books on the topic are all about how to get rid of technical debt.

But is technical debt always bad? When can technical debt be good? How can we use technical debt as a tool, and distinguish between good and bad debt?

Exercise: Draw Your Technical Debt Curve

Think of technical debt as anything about your code that slows you down over the long term: hard-to-read code, lack of test automation, duplication, tangled dependencies, etc.

Now think of any system you’re working on. Grab a piece of paper and draw a technical debt graph over time. It’s hard to numerically measure technical debt, but you can think of it in relative terms: How is the relative amount of technical debt changing over time? Going up? Down? Stable?

Sadly, in most systems tech debt seems to continuously increase.

How Do You Want Your Debt Curve to look?

Next question: What if you could choose, in a perfect world, what you would like that curve to look like instead? It might sound like an obvious question, and your spontaneous thought may be something like “Zero tech debt! Yay!”

Because, heck, didn’t I just say that technical debt is anything that slows you down? And who wants to be slowed down? If we could have zero technical debt throughout the whole product lifecycle, wouldn’t that be the best?

Actually, no. Having zero technical debt at all times will probably slow you down too. Remember, I said “Think of technical debt as anything about your code that slows you down over the long term.” The short term, however, is a different story.

Let’s talk about good technical debt.

When Is a Mess Good?

Think of your computer and desk when you are in the middle of creating something. You probably have stuff all over the place, old coffee cups, pens, and notes; and your computer has dozens of windows open. It’s a mess isn’t it?

Same thing with any creative situation: While cooking, you have ingredients and spices and utensils lying around. While recording music, you have instruments and cables and notes lying around. While painting, you have pens and paints and tools lying around.

Imagine if you had to keep your workspace clean all the time: every time you slice a vegetable, you have to clean and replace the knife. After each painting stroke, you have to clean and replace the brush. After each music take, you have to unplug the instrument and put it back in its case. This would slow you down and totally kill your creativity, flow, and productivity.

In fact, the mess is what allows you to maintain your state of flow: you have all your work materials right at your fingertips.

When is a Mess Bad?

A fresh mess is not a problem. It’s the old mess that bites you.

If you open your computer to start on something new, and discover that you still have dozens of windows and documents open from the thing you were working on yesterday, that will slow you down. Just like if you go to the kitchen to make dinner and find that the kitchen is clogged up with old dishes and leftovers from yesterday.

Same with technical debt. Generally speaking, old debt is bad and new debt is good.

If you are coding up a new feature, there are lots of different ways to do it. Somewhere out there is probably a very simple elegant solution, but it’s really hard to figure it out upfront. It’s easier to experiment, play around, try some different approaches to the problem, and see how they work. Any technical debt accumulated during that process is “good debt,” since cleaning it up would restrict your creativity. In fact, somewhere inside the messy commented-out code from this morning you may discover the embryo to a really elegant solution to your problem, and if you had cleaned it up you would never have found it.

Another thing. Sometimes early-user feedback is a higher priority than technical quality. We’re worried that people might not want this feature at all, so we want to knock out a quick prototype to see whether people get excited about it. If the feature turns out to be a keeper, we go back and clean up the code before moving on to the next feature.

However, old debt is bad debt. All that old experimental messy code, duplicated code, lack of test coverage—all that stuff will really slow you down later when you build the next feature. So regardless of your reason for accumulating short-term debt, make sure you actually do pay it off quickly.

But wait. How short is “short term?”

When Does Good Debt Turn into Bad Debt?

My experience is that, in software, the “good mess” is only good up to a few days, definitely less than a week. Then it starts going stale, dirty dishes clog up the kitchen, the leftovers start to stink, and both inspiration and productivity go downhill.

So it’s really important to break big features into smaller sub-features that can be completed in a few days. If you need practice doing that I can highly recommend the elephant carpaccio exercise.

The Ideal Technical Debt Curve

If new debt is good, and old debt is bad, then the ideal curve should look something like this, a sawtooth pattern where each cycle is just a day or two.

That is, I allow myself to make a temporary mess while implementing a new feature, but then make sure to clean it up before starting the next feature. Sounds sensible enough, right?

Just like the kitchen; It’s OK to cause a creative mess while cooking, but clean it up right after the meal. That way you make space for the next creative mess.

The More Realistic Ideal Technical Debt Curve

In theory, it would be great to get down to zero technical debt after each feature. In practice, there’s an 80/20 rule involved. It takes a reasonable amount of effort to keep the technical debt at a low level, and an unreasonably high amount of effort to remove every last crumb of technical debt.

So a more realistic ideal curve looks like this, with a baseline somewhere above zero (but not too far!).

That means our code is never perfect, but it’s always in good shape.

The “debt” metaphor works nicely because, just like in real life, most people do have some kind of financial debt (like a house mortgage) on a more or less permanent basis. It’s not all bad. As long as we can afford to pay the interest, and as long as the debt doesn’t grow out of control.

Have a Debt Ceiling, Just in Case

Now, even if we do clean up after every feature, we are likely to forget small pieces of technical debt here and there, and it will gradually accumulate over time. Like this:

So it’s best to introduce a “debt ceiling.” Just like certain governments ...

When debt hits the ceiling, we declare “debt alert!” The doors are closed, all new development stops, and everybody focuses on cleaning up the code until they’re all the way back down to the baseline.

The debt ceiling should be set high enough that we don’t hit it all the time, and low enough that we aren’t irrecoverably screwed by the time we hit it. Maybe something like this over a half-year period:

How to Set the Baseline and Ceiling

All this invites the question: “Yes, but how?” It may seem hard to quantify technical debt. But actually, it’s not. Anything can be measured, as long as it doesn’t have to be exact.

Just ask the people on the team: “How do we feel about the quality of our code?”

Pick any scale. I often use 1 to 5, where 5 is “beautiful, awesome code with zero technical debt,” and 1 is “a debt-riddled pile of crap.” With that scale, I would set the debt baseline to 4, and the debt ceiling to 3 (think of debt as the inverse of quality). So quality will mostly be around 4, but if it hits 3 we will stop and yank it back up to 5.

Of all the possible metrics that can be used, I find that teams often like this subjective quality metric. It’s simple, and it visualizes something that most developers care deeply about—the quality of their code.

Other more objective metrics (such as test coverage, duplication, etc.) can be used as input to the discussion. But at the end of the day, the developer’s subjective opinion is what counts.

Use the Debt Ceiling to Avoid a Vicious Cycle

The debt ceiling is very important! Because once your debt reaches a certain tipping point, the problem tends to spiral out of control, and most teams never manage to get it back down again. That applies to monetary debt as well. And governments....

One reason for this “tipping point” effect is the “broken window syndrome.” Developers tend to unconsciously adapt the quality of new code to the quality of the existing code. If the existing code is bad enough (many, many “broken windows”), new code tends to be just as bad or even worse: “Oh that code is such a mess, I don’t know how to integrate with it so I’ll just add my new stuff here on top.”

Think of a kitchen, at home or at the office. If it’s clean, people are less likely to leave a dirty cup on the counter. If there are dirty cups everywhere, people are very much more likely to just add their own dirty cup on top. We are herd animals after all.

Make Quality a Conscious Decision

My experience is that a quality level of 4 (out of 5) is a good-enough level of quality; clean enough to let the team move fast without stumbling over garbage, but not so overly clean that the team spends most of its time keeping it clean and arguing over code-perfection details.

The key is to take a stand on quality. Regardless of how you measure quality, or where you place your debt baseline and ceiling, it’s very valuable to discuss this on a regular basis and make an explicit decision about where you want to be.

How Definition of Done Helps

Here’s a suggested “Definition of Done” for a feature:

- In production

- User accepted

- Code clean

The sequence will vary from case to case; every sequence has its advantages and disadvantages. Sometimes we’ll want to put something into production, then get user feedback, then clean up. Sometimes we’ll want to clean up first, then get user feedback, then put it into production. But the feature isn’t Done until all three things have been done. Here’s a sample board to visualize features going through this process.

- Feature A: in production.

- Features B & C: Current focus is getting user feedback (the features may or may not have clean code right now).

- Feature D: Current focus is cleaning up the code (it may or may not have been accepted by the user yet).

- Features E & F: Currently being developed, trying to quickly get to a point where we can get user feedback.

- The rest of the features are in the idea pool (most teams call that a “backlog,” but I prefer the term “idea pool”).

How TDD Helps

Acceptance Test-Driven Development is a really effective way to put these ideas into practice. All features are developed in three distinct steps:

The first step is to write a failing (“red”) acceptance test. When doing that, we focus exclusively on the question: “What does this feature aim to achieve, and how will I know when it works?” We’re setting a very clear goal and embedding it in code, so at this point we don’t care about quality, or how the feature will be implemented.

The second step is to implement the actual feature. We know when we’re done because the executable acceptance test will go from red to green. We’re looking for the fastest path to the goal, not necessarily the best path. So it’s perfectly fine to write ugly hacks and make a creative mess at this point; ignore quality and just get to green as quickly as possible.

The third step is to clean up. Now we have a working feature and a green test to prove it. We can now clean up the code, and even drastically redesign it, because the running tests (not just the newest test) act as a safety harness to alert us if we’ve broken or changed anything.

This process ensures that we don’t forget the purpose of the feature (since it forces us to actually write that in code), and that we don’t forget to clean up before moving on to the next feature.

To reflect this, we can update the board with an “acceptance test written” column, just to make sure that we don’t start implementing a feature before receiving a failing acceptance test.

The acceptance test doesn’t necessarily have to be expressed at a feature level (“if we do X, then Y should happen”). Some alternatives:

- Lean Startup style: “We have validated or invalidated that users are willing to pay for premium accounts.” Maybe rename the “User feedback” column to “Validating Assumption.”

- Impact Mapping style: “The feature is done when it increases user activation rate by 10 percent.” Maybe rename the “User feedback” column to “Validating Impact.”

Either way, we just need to make sure cleanup is part of the process somewhere.

TDD can be done at multiple levels—at a feature level (acceptance tests) as well as at a class or module level (unit tests). Think of the unit tests as a bunch of triangles inside the acceptance test triangle. One loop around the big triangle involves a bunch of smaller loops inside.

After each unit test goes green, do minor cleanup around that. When the acceptance test goes green, do a bigger cleanup. Then move on to the next feature.

The key point is that each corner of the TDD triangle comes with a different mindset, and each one is important.

Good quality = Happier People

At the end of the day, technical debt is not about technology. It’s about people.

A clean code base is not only faster to work with, it is more fun (or less annoying, if you prefer seeing things that way). And motivated developers tend to create better products faster, which in turn makes both customers and developers happier. A nice positive cycle :o)

Henrik Kniberg is an Agile/Lean coach at Crisp in Stockholm, working primarily with Spotify. He enjoys helping companies succeed with both the technical and human sides of software development, as described in his popular books Scrum and XP from the Trenches, Kanban and Scrum, making the most of both, and Lean from the Trenches.

Send the author your feedback or discuss the article in the magazine forum.

Seven Coaching Patterns

Tapping Human Potential

by Portia Tung

Talk less, listen more, and other habits to cultivate.

Well-known for bringing fun and fairytales to Agile conferences around the world, Portia Tung is a consultant-coach, storyteller, and game maker who combines business strategies with play to bring about positive organizational change.

One: Lead by Example

A coach doesn’t just talk the talk, they walk the walk. This means living by the values and principles they espouse as well as applying the tools and techniques they know to themselves and to their work. They know when they’ve failed to lead by example and always strive to get themselves back on track. A coach knows their limits and asks others for help. A coach appreciates when others stretch their knowledge, skills, and experience beyond their comfort zone because they know that’s where the real learning happens.

Exercise: The Flaw Workout

No one’s perfect. Everyone has at least one flaw. Share one of your personal flaws with someone you’re coaching. Ask them for ideas on how you can address that flaw in order to improve yourself. Commit to actively improve regarding that particular flaw over a certain time period of your choice. After that time has passed, ask that other person to rate your behavior before you tried to improve. Then ask them to rate your behavior after a period of attempting to improve. Inquire about the person’s thinking behind the ratings to come up with more ideas for improving yourself.

When rating, I suggest using a range of 0-10 where “0” means “There’s tons of room for improvement” and “10” means “What flaw? You’re already behaving perfectly!”

For more information, see: What Got You Here Won’t Get You There by Marshall Goldsmith where he provides similar exercises for personal and professional development for highly successful individuals.

Two: Begin with the End in Mind

A coach knows they have to keep their eye on the goal to score. Question them and they will always know why they’re doing what they’re doing. They keep the end goal at the forefront of their mind so they don’t get lost in the minutiae. A coach works backwards from the goal to figure out the most effective and efficient way of getting from A to B.

Exercise: The 5 Whys for Goal Discovery

The 5 Whys is a popular Lean tool for discovering the root cause of problems. Start with a problem and ask why it exists. Repeat asking “Why?” to each subsequent answer. Ask “Why?” for a maximum of 5 times to discover the root cause of a problem.

In the context of goal discovery, ask “Why?” up to five times in the same way you do for root cause analysis to find out what the root or real goal is to understand why you’re doing what you’re doing.

Three: Set a Sustainable Pace

A coach works at a sustainable pace so they can think and act clearly at all times. A coach stays calm when others around them lose their heads. They know when to say “Yes” and when to say “No.” They know their physical, cognitive, social, and emotional limits and when those limits can be stretched without compromising their personal effectiveness and efficiency.

Exercise: Coach’s Log

Write a daily log to keep track of what you do as well as make you mindful of how the day has affected you. Consider recording the day’s activities, things that went well that day, things that went wrong and personal improvement actions. You can even rate the day out of 10, where 10 is “It’s been a fantastic day!” and 0 is “I wish I’d never got out of bed.” Begin the next day by reviewing the improvement actions and choose at least one to work on to keep you improving.

For more information, see: The Dream Team Nightmare by Portia Tung.

Four: Think with Your Head and Feel with Your Heart

In solving problems, people typically favor one or the other, thinking or feeling. A coach balances thinking and feeling. They apply logical thinking as well as empathy when solving problems. A coach is both the Scarecrow and the Tin Man. A coach understands that change begins with the individual and that only you can change yourself for the better. They speak and act with understanding, compassion, and determination.

Exercise: Facts and Feelings

Think of a problem you’re trying to solve. On the left side of a sheet of paper, write down up to seven facts about the problem. On the right side of the same sheet of paper, write down your feelings about the problem.

If you find it easier to come up with facts, it’s likely your preference is thinking over feeling. If you find it easier to describe how you feel about the problem, it’s likely you prefer to feel than think. Neither of these is good nor bad. The exercise simply increases your self-awareness when it comes to achieving a thinking-feeling equilibrium.

For more information, see: What Type Am I? Discover Who You Really Are by Renee Baron, a book that examines the Myers-Briggs Type Indicator, including the two preferences of thinking versus feeling.

Five: Pull, Don’t Push

As the saying goes, “the teacher appears when the student is ready.” A coach waits and is always ready when someone asks for help. A coach creates and offers learning opportunities instead of thrusting their ideas, advice, and views onto others.

Exercise: Coaching Offer

Make yourself available by offering 20-30 minute coaching sessions over coffee for free. Advertise by word of mouth and poster or wiki page so people can contact you to arrange a session. Explain up front how the session serves two goals. The first is for you to assist the person asking for help. The second is for that person to give you feedback on your coaching so that you can improve. If no one takes you up on your offer, reflect on why that is, as this may provide a useful lesson in itself.

Six: Talk Less, Listen More

A coach listens more than they say. A coach postpones judgment on what they hear and lets others talk while they listen with care. To get to the heart of a problem or gain a deep understanding of a situation, a coach asks open questions, questions that cannot by answered by a mere “Yes” or “No.” Open questions begin with words like “What happened?”, “How would you do things differently?”, “Why does the same problem keep recurring?”

Exercise: All Ears

Imagine all you can do is ask questions and listen. And, when asking questions is inappropriate, the only option is to listen.

What do you notice about the quality of conversations you have in comparison with your usual way of communicating? And what about the differences in the level of engagement you get from those you usually interact with?

My grandmother used to say, “There’s a reason why we have two ears and one mouth.” Help people answer their own questions and maybe they’ll help answer yours.

For more information, see: The Yellow Brick Road—Fresh Insights Through Peer Coaching by Portia Tung and Pascal Van Caunwenberghe.

Seven: Flow Like a Stream

A coach works with ease like running water. They are patient, pragmatic, and present. The effectiveness and efficiency with which they work inspires others to strive to achieve the same. To coach with ease is the result of years of conscious practice, perseverance, and a promise to be better.

Exercise: Pick a Tool

Pick a tool you’d like to improve in your use of. Set aside seven days. Identify improvement ideas, then implement them in those seven days. Measure and reflect, then repeat as desired.

I suggest short improvement time boxes such as seven days because the act of measuring progress in small increments and often not only keeps it in the forefront of your mind but it also encourages you to keep moving forward.

For more information, see: Outliers by Malcolm Gladwell in which he describes the “10,000-Hour Rule,” which states that the key to success in any field is through practice for approximately 10,000 hours.

About the Author

Portia is an international speaker and the creator of concepts such as Agile Fairytales and Enterprise Gardening. She shares her insights and experiences as a coach and parent on her blog Selfish Programming. She is the author of The Dream Team Nightmare, the first ever business novel about Agile where the reader’s decisions determine the outcome of the story. The Dream Team Nightmare is the first in the Agile Adventure gamebook series, where the journey matters as much as the destination.

Send the author your feedback or discuss the article in the magazine forum.

Nostalgia for the Future

A Story from Personal Computing’s Early Days

by Michael Swaine and Paul Freiberger

There’s a lot more to the story than Apple and Microsoft. In the 1970s, a motley group of counterculture engineers were starting computer companies in their garages. This is a snapshot of one of those companies and the entrepreneurial environment in which it grew. In the photo: Bob Marsh is in the center with the cool mustache; Gary Ingram is on the right.

In June 1975, entrepreneur Bob Marsh and Popular Electronics technical editor Les Solomon were contemplating an “intelligent terminal” kit. It would consist of a terminal with semiconductor circuitry that would perform certain display and keyboard decoding functions that another computer attached to it would have otherwise handled. Marsh had some ideas from his own experience and from discussions with his friend Lee Felsenstein about a device Lee called the Tom Swift Terminal. “If you can get me a working model in 30 days, I'll give you a cover story,” Solomon said.

A Terminal with a Brain

Marsh put the proposition to Felsenstein this way: “Do you think it’s impossible?” Felsenstein appreciated Marsh’s careful phrasing of the question. To dodge the job, he would have to pronounce it impossible, a distasteful act to any self-respecting engineer.

Marsh said he would pay Felsenstein to design the video portion of his dream machine, the convivial terminal that Felsenstein saw as essential to releasing the power of computers to everyone. Felsenstein liked the idea, and agreed to do it. It soon became apparent that Marsh had a different project in mind. What he wanted was a terminal with a brain—the same Intel 8080 chip that was the brain of the Altair. They argued over the details of the design, with Marsh usually getting the better of the arguments. Neither Felsenstein, Marsh, nor Les Solomon realized it then, but the product they were designing would become something more than just a terminal.

Felsenstein had to withdraw from another project when he agreed to design the intelligent terminal. “The roof is falling in again,” he told his ex-customer. Up until then, he had paid his share of the rent by consulting for various people. But Processor Technology was expanding to take up the whole garage—all 1100 square feet of it. Felsenstein was gradually being absorbed into Marsh’s enterprise.

Marsh had already developed the terminal’s architecture and continued to change the design requirements as Felsenstein worked. Felsenstein had enjoyed consulting, in part because he could get some distance between him and the person he was working for and concentrate without interruption on a problem. This advantage evaporated when he began to devote most of his time to the Proc Tech terminal. Marsh insisted on design changes on a daily basis and repeatedly forced Felsenstein to junk much of his careful work and start over. “The situation,” Felsenstein later said, “did call heavily on my sense of futility, absurdity, and ultimate irrelevance.”

Despite his complaints, Felsenstein was enjoying himself. His grumbling about being manipulated was more of a jab at himself than it was at Marsh, who, for all his entrepreneurial energy, was at least partly involved for the fun of it. At one point in the project, Felsenstein said, “Let’s advertise it as having ‘the wisdom of Solomon.’” He meant it as a sly reference to Les Solomon, and this whimsical slogan soon inspired them to name the machine “the Sol.”

Marsh and Felsenstein argued ceaselessly over the design at Felsenstein’s workbench at one end of their garage and at the makeshift Proc Tech offices housed at the other end. They argued about it over meals and while driving across the San Francisco Bay to Homebrew meetings. Despite their constant wrangling over design, they got the goods out. On one drive to a Homebrew meeting, they redesigned an entire internal bus.

A Real Computer

It eventually dawned on Marsh and Felsenstein that they were designing a real computer. After all, it had an 8080 in it. But clearly it was also a terminal. Until then, computers typically consisted of rectangular boxes with accessory connections to terminals of some kind— Teletypes, cathode ray tubes, typewriters, or printers. But theirs was a screen, a keyboard, and a computer all in one. Could they really pull this thing off?

The question had both technical and political implications. At this point, the Altair dominated the tiny microcomputer industry, and IMSAI had not yet made its entry. And here they were, developing this terminal under the auspices of the Altair’s biggest booster outside of Albuquerque—Leslie “Uncle Sol” Solomon. Would he rescind the cover story agreement if they told him they were concocting a computer instead of a terminal?

They decided not to tell him.

And they continued to work. Despite all the arguments, Marsh, his partner Gary Ingram, and Felsenstein were enjoying themselves. “This is a company that’s going to have fun,” Felsenstein said, “no matter how miserable I have to be.” He described his partners as “nostalgic for the future,” like many computer hobbyists of the day, and their discussions were frequently that of visionaries. But the mundane, day-to-day decisions also had to be made. Marsh’s friend still had all that cheap walnut originally slated for the digital clock business, and it seemed a shame to let it go to waste, so Marsh incorporated walnut side panels into the Sol’s design, giving it the appearance of a 1950s station wagon.

Felsenstein had originally expected to hand the finished schematic to a layout artist. As it turned out, he was the chief layout artist. Because they had long since filled all the available floor space in the garage, a light table for the layout work was installed in a loft above the Proc Tech offices. Felsenstein padded the forehead-level conduit, but couldn’t keep from bumping his head on the rafters as he worked with the other layout artist 14 to 17 hours a day, seven days a week. The other artist, pumping himself up with cola, dropped out before the end of the project, and Felsenstein had to finish the job alone, on orange juice.

Marsh kept the pressure on, and within 45 days of his initial discussion with Les Solomon, he had a circuit board. But Solomon had given the team a 30-day deadline, so as they neared completion, Marsh booked a flight to New York and informed a bleary-eyed Lee Felsenstein that he was going, too. They stuffed the Sol into two brown paper bags and carried it with them on the plane.

The demonstration for Solomon at (New York-based) Popular Electronics was an utter disaster. The thing just didn’t work. They made what excuses they could and, feeling hopeless at this point, flew to an appointment at (Framingham, Massachusetts-based) Byte, where the presentation was even more disastrous. Felsenstein, dead on his feet from the grueling work schedule, fell asleep during the Byte demonstration.

Well-rested and back in California at his workbench, he quickly located the problem, a short circuit. Marsh promptly put Felsenstein back on a plane to New York to demonstrate a working Sol with strict instructions to not reveal that it was actually a computer.

Felsenstein kept his mouth shut, but Solomon was no dummy. When Felsenstein showed him the Sol terminal, he watched it work for a while, and then asked Felsenstein what was to stop him from plugging in a memory board with BASIC on it and running the Sol as a bona fide computer.

“Beats me,” Felsenstein deadpanned.

Who Owns the Software?

Of course the Sol was a computer. And that meant, Marsh and Ingram realized, that it needed software, particularly BASIC. The two contracted with Chuck Grant and Mark Greenberg to write it. Everybody at Homebrew had a computer or software company, it seemed, and Grant and Greenberg were no exception. They had started a company with George Morrow, but had a falling out because Morrow didn’t think they were taking their oral agreement with distributor Bill Godbout seriously enough. Morrow decided to deal with Godbout alone, while Grant and Greenberg began doing business as G & G Systems.

As they worked on the BASIC, Grant and Greenberg found they were having the most trouble with the floating-point routines: arithmetic on real numbers, not integers. They simply couldn’t process the operations as quickly as they wanted to. They finally decided to build the floating-point math into the hardware, and hired George Millard to help design a floating-point board.

Around this time, the issue of proprietary software arose. Conflict arose over ownership of an implementation of the BASIC computer language. Marsh asserted that the software was being developed for Proc Tech, whereas Grant and Greenberg, with growing ambition, insisted that it was theirs, and began soliciting other customers for their BASIC. Proc Tech took Grant and Greenberg to court, and the case lumbered through discovery and delay, doing neither company any good.

The Problem of Storage

Grant and Greenberg had other hot projects going. They developed a cassette-tape interface that would allow microcomputers to save data to tape by using cheap audio tape recorders. But then Shugart, a Silicon Valley minicomputer disk drive manufacturer, announced the introduction of a drive that used 5-1/4-inch disks—smaller than the 8-inch disks commonly used on big computers—that cost less than any other disk drive. Disk drives were the obvious answer for data storage, if they could be made affordable. So Grant and Greenberg dropped their interest in cassette storage and started designing a controller board to make the Shugart disk drive work with microcomputers.

When they had their disk system together, they gave themselves a new name, North Star, perhaps echoing the name Altair, another bright star in the sky. Simultaneously, as Applied Computer Technology, they contracted to sell IMSAIs bundled with their own BASIC and cassette interface to universities. But the market, they discovered, did not want configured systems, but raw computers, so they began selling IMSAIs out of Mark Greenberg's garage. This operation, at Grant's suggestion, was called Kentucky Fried Computers.

Meanwhile, their ex-partner, George Morrow, had bought an Altair, studied it, and decided not to imitate it. He shared Godbout’s estimation of the Altair. The computer that he and Godbout planned to build, and that he began to design, would definitely be better. He would base the computer on National Semiconductor’s PACE, a microprocessor they hoped to get for $50 from National.

Godbout, however, had reservations about the project. He meditated over Altair’s sales figures and decided that memory boards for the Altair might do well after all. Morrow, with some reluctance, put aside the PACE machine and commenced designing a 4K memory board with his own name on it, joining Proc Tech and Bruce Seals in the memory market. Godbout sold the board for $189, well under Proc Tech’s price, and Morrow suddenly found himself making $1,800 a month in royalties.

Godbout now became intensely interested in selling microcomputer boards. But when he vetoed one of Morrow’s ideas, Morrow reevaluated their relationship. Couldn’t he sell his boards just as well as Godbout, he asked himself. The only difference, he decided, lay in who placed the magazine ads. Thus was born Morrow’s Microstuf.

There was no telling who would be successful among these entrepreneurs. Godbout already had a successful electronics business. Morrow was a bit older than some of these players, and came across as more of a serious businessman. Marsh, on the other hand, looked like a kid, and Felsenstein was staunchly anti-corporate. But Proc Tech was starting to look really promising. The market was crazy, according to Morrow. “You could start a company, announce a product, and people would throw money at you.”

Bob Marsh had already learned this lesson with Proc Tech’s memory boards, but was more than willing to take a refresher course. Marsh and Felsenstein took the Sol to the PC Computer show in Atlantic City, New Jersey, in June 1976 to unveil it to the world. It went over big.

When they returned to California, they continued to enhance and modify the Sol. While writing tutorial articles on computer design for PCC, Felsenstein added what they called, to use writer Don Lancaster’s term, a “personality module.” This tiny circuit board had a ROM (read-only memory) chip and could be plugged into the back of the machine, enabling its “personality” to be changed in a second. Felsenstein wryly imagined employees popping in game modules for the business modules while the boss was out of the office.

Competition

By late 1976, DEC was selling its LSI-11 bottom-of-the-line minicomputer for slightly over $1,000. In Southern California, Dick Wilcox gave hard thought to a suggestion in Dr. Dobb’s Journal about interfacing the LSI-11 with an Altair or IMSAI. What he came up with was the Alpha Micro, an LSI-like multiuser CPU board, which he demonstrated to Homebrew in December 1976.

New microprocessors continued to arrive. Toshiba released the first Japanese chip, the T3444. National Semiconductor issued a new microprocessor and also supplied the development tools hobbyists needed to start building real computers and writing software.

Scores of new microcomputer companies began to appear, like Vector Graphic in Thousand Oaks, California. Vector’s Lore Harp quickly showed she was more than just a business manager as she guided the company with a shrewd sense of the market’s needs and the possibilities for potential growth.

However, Vector was not doing any better than Proc Tech. During the winter of 1976–1977, Proc Tech moved to a much larger facility, 14,000 square feet next to a beef-rendering plant in nearby Emeryville. The atmosphere was uninviting, but the new location was far roomier than their former digs.

A month after Proc Tech moved out of the (Berkeley-based) Fourth Street garage, Grant and Greenberg took over two-thirds of the space of the garage, Felsenstein reclaimed the other third, and their three company names were placed on the door: North Star, Applied Computer Technology, and Kentucky Fried Computers. As the last company, they were now marketing IMSAIs, Polymorphic and Vector Graphic boards, and an Apple I kit that they were persuaded to take on consignment by a scraggly-bearded young man named Steve Jobs. But soon, sales of their North Star disk system soared, and they closed Kentucky Fried Computers to concentrate fully on North Star. A letter from a certain fast-food chain that demanded they cease and desist from using the name Kentucky Fried Computers made the decision easier.

By the end of 1976, Processor Technology, Cromemco, North Star, Vector Graphic, and Godbout Electronics were prominent among the Silicon Valley enterprises, building an entire industry where none had existed two years before. And that industry was growing with amazing speed.

Paul Freiberger and Michael Swaine wrote Fire in the Valley, the seminal history of the personal computer and the source for the movie Pirates of Silicon Valley. This article is adapted from the upcoming Third Edition from the Pragmatic Bookshelf. Send the authors your feedback or discuss the article in the magazine forum.

Pragmatic Bookstuff

Here’s what’s up with the Pragmatic Bookshelf and its authors.

On this day in 1938, Orson Welles broadcast the radio play “War of the Worlds,” leading many listeners to believe that our planet had actually been invaded by aliens. We’re required to remind you that was a complete fiction. No aliens here. Not at all. Go back to your business....

Which brings us to our latest title now in print and shipping, Modern C++ Programming with Test-Driven Development: Code Better, Sleep Better. Also...

Coming Soon: Crafting Rails 4 Applications: Expert Practices for Everyday Rails Development, Developing Android on Android: Automate Your Device with Scripts and Tasks, Create Mobile Games with Corona: Build with Lua on iOS and Android, Seven Concurrency Models in Seven Weeks: When Threads Unravel, Node.js the Right Way: Practical, Server-Side JavaScript That Scales.

Recently Released: 3D Game Programming for Kids, iPad and iPhone Kung Fu, The Coding Dojo Handbook, The Dream Team Nightmare.

What’s Hot

1^

3

Agile Web Development with Rails 4

2^

4

Programming Ruby 1.9 & 2.0

3v

2

Seven Concurrency Models in Seven Weeks

4^

11

Programming Erlang

5^

NEW

3D Game Programming for Kids

6

6

Programming Elixir

7v

5

Web Development with Clojure

8v

1

Practical Programming

9^

NEW

The Coding Dojo Handbook

10

10

The Definitive ANTLR 4 Reference

11^

NEW

The Dream Team Nightmare

Who’s Where When

- 2013-11-01, London, UK

Agile Tour London - The Emperor's New Clothes: Meaningful Interactions in Stressful Situations

Portia Tung (author of The Dream Team Nightmare )

- 2013-11-08, Software Craftsmanship North America, Chicago

Talk, to be determined

Dave Thomas (author of Programming Ruby (2nd edition) , Agile Web Development with Rails (3rd edition) , The Ruby Object Model and Metaprogramming , Agile Web Development with Rails 4 , Programming Ruby 1.9 & 2.0 (4th edition) , Agile Web Development with Rails 3.2 , and Programming Elixir )

- 2013-11-11, Better Software/Agile Development East, Boston MA

Tutorial: Congruent Coaching: An Exploration

Johanna Rothman (author of Behind Closed Doors , Manage It! , Manage Your Project Portfolio , and Hiring Geeks That Fit )

- 2013-11-11, Better Software/Agile Development East, Boston MA

Tutorial: Agile Program Management: Networks, Not Hierarchies

Johanna Rothman

- 2013-11-13, Better Software/Agile Development East, Boston MA

Software Manager’s Lab

Johanna Rothman

- 2013-11-14, CocoaConf Atlanta

iOS 2D Game Development Workshop

Jonathan Penn (author of Test iOS Apps with UI Automation )

- 2013-11-14, AnDevCon

Dynamic Audio for Apps and Games - In this talk you'll get a brief introduction to the field of sound design and different techniques that can be used to create sound. You'll then be introduced to PureData, an application and platform for working with

Tony Hillerson (author of Programming Sound with Pure Data )

- 2013-11-14, Better Software/Agile Development East, Boston MA

Talk: Program Management: Collaborating across the Organization

Johanna Rothman

- 2013-11-14, CocoaConf Atlanta

Core Audio All-Day Tutorial

Chris Adamson (author of iOS SDK Development )

- 2013-11-15, CocoaConf Atlanta

Testing in Xcode 5

Jonathan Penn

- 2013-11-15, CocoaConf Altanta

Get on the Audiobus

Chris Adamson

- 2013-11-15, CocoaConf Atlanta

Glitch-Free A/V Encoding

Chris Adamson

- 2013-11-16, CocoaConf Atlanta

NSIncrementalStore - Bet you didn't think Core Data could do *that*!

Jonathan Penn

- 2013-11-17, Problem Solving Leadership Workshop, Albuquerque, NM

5.5 days of leadership training for people who want to learn to be problem solving leaders.

Johanna Rothman

- 2013-11-18, BITS - Software Developer Conference, Denmark

I'll be speaking on TDD, and have a hands-on workshop. We'll also offer a 2-day public training class for TDD and embedded C/C++

James Grenning (author of Test Driven Development for Embedded C )

- 2013-11-20, Agile Iceland

Coaching Teams Through Change talk. Come to get some tips for getting to the heart of resistance and how to dissolve barriers to adopting new working practices.

Rachel Davies (author of Agile Coaching )

- 2013-12-05, Core Vim Class Online

Core Vim Class Online

Drew Neil (author of Practical Vim )

- 2014-01-08, CodeMash 2014

Introduction to the Roku SDK (Half-Day Class)

Chris Adamson

What’s Happening

But to really be in the know, you need to subscribe to the weekly newsletter. It’ll keep you in the loop, it’s a fun read, and it’s free. All you need to do is create an account on pragprog.com (email address and password is all it takes) and select the checkbox to receive newsletters.

Shady Illuminations

The Post-Post-PC Era

by John Shade

John reflects on the Post-Post-PC Era, Jony Ive’s fingerprints, and just-in-time deconstructionist iconography.

So is this the end of line for the Apple iPod? Some pundit, I forget which one, recently argued that if Apple was ever going to introduce a new iPod model, it surely would have done so by now. Because I forget why.

Although I seem to have failed to follow the pundit’s logic, I’m perfectly willing to follow his conclusion and see where it leads.

I think it leads to the end of the Post-PC Era. Which, surely, means the beginning of the Post-Post-PC Era. After all, we have left the Postmodernist Era and entered the Post-Postmodernist Era.

Post-Postmodernism apparently means just-in-time capitalism. Whatever that means. And Postmodernism was all about deconstructionist art criticism. So I guess there doesn’t have to be any comprehensible connection between Post-Post-something and Post-something. But I’ll press gamely on and try to glean the outlines of the Post-Post-PC Era from the Post-PC Era because I really have nothing better to do with my time.

Jony Ive’s Fingerprints Are All Over This

So what will the Post-Post-PC Era look like? I think we can assume that Apple is going to design it, so let’s take a sighting through what Apple was doing to what Apple’s been doing lately and see what we can see.

The Post-PC Era, of course, was all about fear. Fear and fuzzies.

People fear the unfamiliar, and the Post-PC Era had all these devices that were not computers. How scary. So Apple did what you do to ease people’s fears, they gave them fuzzy phoney familiarity. Also known as skeumorphism. To comfort users in this scary Post-PC Era, Apple created icons that looked like buttons and engaged in other crimes of skeumorphism, to allay users’ fears in this unfamiliar Post-PC world. Naturally users would shy away from phones that surfed the Web, and MP3 players that talked to their feet. They would need for these strange new devices to look comfortingly familiar.

They had to be told, for example, that it was OK to touch the screen. But won’t it get all smeared with fingerprints, they fretted. Don’t worry about it, Apple explained.

So in the Post-Post-PC Era? In the Post-Post-PC Era, icons don’t look like buttons any more. They look like roadkill buttons. Because users don’t need to be reassured any more. Skeumorphism is becoming a thing of the past in the Post-Post, where new pastel icons follow some just-in-time deconstructionist esthetic that only a curmudgeon would claim goes beyond the pale.

And those messy fingerprints? Apple will collect those for you in the bright new world of Post-Post-PC. In the Post-PC Era fingerprints were a bug, but in the Post-Post-PC Era, they are a feature.

There’s much more to Post-Post-PC, of course. Is wearable technology Post-Post-PC? Sure it is. Is implanted technology Post-Post-PC? Of course. Post-Post-PC technology is personal, it’s ready to wear (or as the French say it, porta-potty), and it will get under your skin. Doubtless your local tattoo parlor will soon be where you go to get your software upgraded.

OK, that’s all I’ve got. Also, Swaine’s bugging me for the column, so I’m going to wrap this up and post it. Or post-post it.

John Shade was born under a cloud in Montreux, Switzerland, in 1962. Subsequent internment in a series of obscure institutions of ostensibly higher learning did nothing to brighten his outlook. He firmly maintains that stickball is crack croquet. Follow John on Twitter, send him your feedback, or discuss the article in the magazine forum.

Rear Window

A Parting Shot

The first computer mouse, invented by Doug Engelbart at SRI in 1964, was carved from a block of wood. Photo courtesy of Douglas Engelbart.

The computer mouse will turn 50 next year. You don’t see as many of the little critters around any more; there are new ways of controlling computers and computerish devices. But while the mouse may be on the way out, computing hasn’t even begun to realize some of the innovations of its inventor, the legendary Douglas Engelbart.

In a legendary demo at the Fall Joint Computer Conference in San Francisco in 1968, he demonstrated what his team at the Augmentation Research Center Lab at SRI had been working on: the mouse, a workgroup-focused network of computers, bitmapped screens displaying a precursor to GUIs, and a form of hypertext. He blew the room away. Many of his ideas were later worked out at Xerox PARC.

But all of the technology was in service of Engelbart’s bigger idea: the augmentation of human intellect. He believed that technology could dramatically improve our ability to solve big problems collaboratively.

We lost Doug Engelbart this year, but his genius lives on.

Table of Contents

PragPub 2013-11: Issue #53

On Tap

Choice Bits

Rothman and Lester

Namespaces in Clojure

Rediscovering Awk

Good and Bad Technical Debt

Seven Coaching Patterns

Nostalgia for the Future

Pragmatic Bookstuff

Shady Illuminations

Rear Window