PragPub 2011-02: Issue #20

Table of Contents

Features

Agile @ 10

by Andy Hunt, Kent Beck, Ron Jeffries, Jon Kern, Ken Schwaber, James Grenning, Arie van Bennekum, Stephen J. Mellor, Ward Cunningham, and Dave Thomas

Ten authors of the Agile Manifesto celebrate its tenth anniversary.

Abstraction

by Tim Ottinger, Jeff Langr

The third in this series of Big Ideas in software development.

Refactoring Your Job

by Craig Riecke

Forget the self-help books. Here’s practical advice on making the best of bad economic times.

De Morgan to the Rescue

by Staffan Nöteberg

Sometimes a little math can substitute for having the right tool.

Agile in PragPub

Looking back on two years of Agile articles in PragPub

Departments

Up Front

by Michael Swaine

Ten Agile Voices Celebrate Ten Agile Years

Choice Bits

What’s hot—and what can be said in 140 characters.

Way of the Agile Warrior

by Jonathan Rasmusson

Words have power, and the wrong word can steer your whole project wrong.

Calendar

After a slow-ish winter, things are really heating up this spring.

Shady Illuminations

by John Shade

John explains the values and principle on which his writing is based.

Except where otherwise indicated, entire contents copyright © 2011 The Pragmatic Programmers.

Feel free to distribute this magazine (in whole, and for free) to anyone you want. However, you may not sell this magazine or its content, nor extract and use more than a paragraph of content in some other publication without our permission.

Published monthly in PDF, mobi, and epub formats by The Pragmatic Programmers, LLC, Dallas, TX, and Raleigh, NC. E-Mail support@pragprog.com, phone +1-800-699-7764. The editor is Michael Swaine (mailto:michael@pragprog.com). Visit us at http://pragprog.com

ISSN: 1948-3562

Up Front

The Agile Issue

by Michael Swaine

We’re pleased to welcome you to this, our Agile issue, celebrating the tenth birthday of the Agile Manifesto, a document that changed the world of software development. To mark the occasion, we invited the authors of the Manifesto to share their thoughts. Ten of them agreed to do so. We think you’ll find their reflections thought-provoking.

In this spirit of Agile reflection, we’ve also included a list of every Agile-themed article ever published in PragPub. They’re all available and clickable, thus (you might say) adding 48 more Agile articles to this already-packed Agile issue.

In addition to these Agile reflections, this issue contains another article in the series on big ideas in software from Jeff Langr and Tim Ottinger, an article on “Refactoring Your Job” by Craig Riecke and a cool math article by Staffan Nöteberg. Then there’s Jonathan Rasmusson’s latest “Way of the Agile Warrior” column, our regular Choice Bits and Calendar departments, and the latest of John Shade’s “Shady Illuminations,” in which he writes a letter to the editor. The editor of PragPub, that is.

For the record, the 17 authors of the Agile Manifesto are: Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, and Dave Thomas.

Choice Bits

We thought you might like to know...

A few selected sips from the Twitter stream.

What’s Hot

Top-Ten lists are passé—ours goes to 11. These are the top titles that folks are interested in currently, along with their rank from last month. This is based solely on direct sales from our online store.

1^ | 2 | Agile Web Development with Rails
2v | 1 | Crafting Rails Applications
3^ | 7 | HTML5 and CSS3
4v | 3 | The RSpec Book
5v | 4 | Programming Ruby 1.9
6^ | NEW | Agile in a Flash
7v | 5 | Seven Languages in Seven Weeks
8v | 6 | Hello, Android
9^ | NEW | Arduino
10^ | NEW | Using JRuby
11v | 8 | The Agile Samurai

Outage Outrage

Internet outage at home. Oh noes. — @shelly

Sorry to hear that, Shelly. Here’s something to read while you wait for your connection to come back.

Things to Do in Denver When You’re Offline

RT @JackieBaisa: Drinking a 1795 Terrantez Madiera at the Herbfarm. Oldest wine in the world. Sweet, full-bodied, fig, caramel. #herbclub — @Herbguy

Was not aware that kielbasa is a breakfast food. Or at least it is in Ohio. — @invalidname

Clear night in Shanghai so we went up into the bottle opener building. — @jwgrenning

A squirrel just dropped am acorn on me at the Capitol grounds. — @tswicegood

I am in hell. It’s full of Swedish furniture. — @alancfrancis

Yes, travel is broadening—especially breakfasting in the Midwest.

Google This when Your Connection Comes Back

Eric Schmidt as Google’s ambassador? Here are his five greatest public bloopers. http://bit.ly/hMAaCc — @pgralla

A “Baby bump” is something one “shows off.” And sports fans like to set fire to things. #thingsilearnedingooglereader — @shelly

[I’m] running MacPorts. When it’s done, I’ll have a local cache of the Internet if anyone needs it. — @neal4d

Ummm, why would Goog PR put out a photo of the management team in a robot car, suggesting that nobody is at the wheel? — @markoff

Offline Observations

I think that ADD is eating my TDD today. — @tottinge

I am Jack’s waning concentration. — @benrady

Other than this couch, I may well be the oldest thing in this frat house. — @gvwilson

Extrapolating from today, I will have a cold all of 2011. — @marick

It’s Girl Scout swarming season! #fb — @shelly

Pearls of Wisdom Washed Down the Twitterstream

“Discipline is the bridge between goals and accomplishments.”—Jim Rohn — @venkat_s

Like the silent fog-bird, tweets appear like magic, then fade away. Unexpected, seen for a moment, instantly released. — @mindheart

Some jobs are thankless for good reason. — @chadfowler

Lil Wayne looks and sounds too much like Lisa Simpson for me to take him seriously. — @therealadam

We need a Zen of Scala. — @rit

Texting while driving should carry a heavier punishment than DUI. You’re sober when you choose to text and drive. — @benrady

So, the light on top of a taxi is on when can *not* hail it? Seems like it should be the opposite. — @jeffcohen

The next great productivity innovation will be to eliminate Outlook calendars. — @mtnygard

Welcome to My Paradigm

Amazing the amount of bad PHP string handling code created when the original programmer doesn’t know about regular expressions. — @petdance

Every time you save a selenium script as html, a baby panda cries. — @adamgoucher

I’d love to see someone submit a book proposal on XSLT for #pragpub cos that’s a book I could use right about now. — @bphogan

@bphogan But... “for the love of God, please don’t use XSLT” wouldn’t make much of a book. — @noelrap

I suspect that many people who “like the notion of being a blacksmith” have never seen a blacksmith’s day. — @adamgoucher

Thought gleaned from @chadfowler’s keynote: the code is always cleaner on the other side of the ocean. #codemash — @daveklein

The Coding Life

@jaredrichardson When you’re stuck on a hard problem, the worst place to be is in front of your keyboard. — @PragmaticAndy

@PragmaticAndy No, no. That’s for other people. I need to sit here and glare at the screen while furiously typing!!! ;) — @jaredrichardson

After a few months of on-and-off viewing, @saranutter and I have now re-watched all of ST:TNG in order. Next: DS9. — @headius

http://locksoflove.org Gains 10 inches of hair, and my neck sees daylight for the first time in years. http://yfrog.com/h5cw3mergdj — @scottdavis99

...and now my fans are crying sellout, they say I’ve lost my touch... — @bdudney

I am personally insulted that they’re removing Narcissistic Personality Disorder from the DSM IV. — @marick

This month we were following Chris Adamson, Jeff Cohen, Scott Davis, Bill Dudney, Neal Ford, Chad Fowler, Alan C. Francis, Adam Goucher, Preston Gralla, James Grenning, Brian P. Hogan, Andy Hunt, Adam Keys, Dave Klein, Andy Lester, Brendan W. McAdams, Brian Marick, John Markoff, Charles Nutter, Michael Nygard, Tim Ottinger, Ben Rady, Noel Rappin, Jared Richardson, Venkat Subramaniam, Travis Swicegood, Nick Turner, Greg Wilson, and Herb Farm restaurant owner Ron Zimmerman, not to mention the editor’s old friend Shelly Brisbin. You can follow us on twitter.

My Internets are back. Please feel free to proceed with your day. — @shelly

Way of the Agile Warrior

They’re Not Requirements

by Jonathan Rasmusson

Words have power, and the wrong word can steer your whole project wrong.

Requirement: something essential to the existence or occurrence of something else.

Few words have done more harm to our industry than this single word “requirement.” The very word invokes an aura of non-negotiation and permanence.

We all know it’s impossible to gather all the requirements before the start of a project.

We all know they are going to change.

And we all know there are always going to be more things to do than time and money allow.

So why do we insist on calling them requirements? And if they are not “requirements,” what are they?

Let’s take a closer look at what a software requirement really is, and how it maps into our world of delivery.

Throwing in the Kitchen Sink

At XP2002, Jim Johnson, the chairman of the Standish group, shared a study that showed what percentage of features companies typically used on software they had built.

While surprising to some, the study confirmed what many had long suspected: companies regularly build features that their customers seldom or never use, and most of the value of the software comes from a really small subset of features.

So why would companies do this? Nobody sets out build a product with dozens of features their customers will seldom or never use. So why do intelligent people do this?

Mostly it has to do with how our industry has historically trained customers to communicate what they need to get out of their software.

“Hi, customer. Can you write down everything you could possibly need in your software? We ask you to do this because once we start delivering, we are going to be on a tight budget and schedule. Oh, and if changes do come up, please forward your requests to the change control board. They would be happy to assist you in handling any changes you’d like to see made to your software.”

This is what we used to tell our customers. If you were a customer, and you knew that a gang of thugs was going to charge you an arm and a leg every time you wanted to make a change to the requirements, what would you do?

You’d throw in the kitchen sink! Because you’d know that this is your one chance to get everything in there.

With so much focus on schedule and money, it’s ironic that most companies don’t realize they could save 50% of their budgets simply by dropping the seldom- or never-used features of their systems.

Kent Beck (the founder of Extreme Programming) says it best:

“Software development has been steered wrong by the term ‘requirement,’ defined in the dictionary as something that is mandatory or obligatory. The word carries a connotation of absolutism and permanence, inhibitors for embracing change. And the word ‘requirement’ is just plain wrong.

“Out of the thousands of pages used to describe requirements, if you deliver the right 5, 10, or 20 percent, you will likely realize all of the business benefit envisioned for the whole system. So what were the other 80 percent? Not requirements—they weren’t mandatory or obligatory.”

OK. So the word “requirement” is plain wrong. But if they’re not requirements, what are they?

Let’s Call Them Features

I like the word feature:

It doesn’t have any of the baggage carried by “requirement.”

Everyone knows what a feature is.

It is something you can have a conversation about with your customer.

And it just avoids all the drama that comes with “requirement.”

Now as much as I like the word features I don’t necessarily use it all the time (at least not initially). There’s simply too much history wrapped up in the word, so it’s sometimes handy just to go ahead and use it for the sake of communication or to make a point.

But the sooner you can get your customer thinking in terms of features they would like to see in their software (and less in terms of absolute requirements) the better. It will give you and your team the wiggle room you’ll need to balance all the conflicting trade-offs and forces that come with building interesting products, while enabling you to focus on the really important stuff first.

So Let’s Drop the Word “Requirement”

It’s plain wrong. It runs counter to the spirit of discovery that comes in building great software and it sends the wrong message (that everything has to be in there).

If you can get your customer talking about the “features” they would like to see in their software and then focus on how to best deliver the most important 20%, you’ll go a long way to ensuring that your customer gets the biggest bang for their buck—and you’ll never have to worry about not delivering something of value.

Jonathan Rasmusson is the author of The Agile Samurai. As an experienced entrepreneur and former agile coach for ThoughtWorks, Jonathan has consulted internationally, helping others find better ways to work and play together. When not coaching his sons’ hockey teams or cycling to work in the throes of a Canadian winter, Jonathan can be found sharing his experiences with agile delivery methods at his blog, The Agile Warrior. For more information on the Agile Inception Deck check out The Agile Samurai.

Send the author your feedback or discuss the article in the magazine forum.

Agile @ 10

Ten Authors of The Agile Manifesto Celebrate its Tenth Anniversary

by Andy Hunt, Kent Beck, Ron Jeffries, Jon Kern, Ken Schwaber, James Grenning, Arie van Bennekum, Stephen J. Mellor, Ward Cunningham, and Dave Thomas

One thing, at least, is clear from these reflections: What was defined at the Snowbird meeting remains a work in progress. It wouldn’t be agile otherwise.

When we realized that February 2011 was the 10th anniversary of the Agile Manifesto, we suggested to all of the 17 people who were present at that historic meeting that it would be appropriate for them to put down their thoughts, and send them to us for inclusion in this Agile Issue of PragPub.

Pretty much all of them responded, mostly with some variation of, “Yeah, you and every other technical magazine, blog, podcast, or conference organizer.” So we nagged them some more. In the end, ten of the authors of the Agile Manifesto sent us something. Some wrote full essays, others wrote a few sentences. Some reflected on progress made, others on challenges remaining. Here are their thoughts on the Agile Manifesto’s tenth birthday.

Ten Down, A Thousand to Go

Andy Hunt

“We shall do a much better programming job, provided that we approach the task with a full appreciation of its tremendous difficulty, provided that we stick to modest and elegant programming languages, provided that we respect the intrinsic limitations of the human mind and approach the task as Very Humble Programmers.” Edsger W. Dijkstra offered that sage advice nearly forty years ago, at the conclusion of his Turing Award lecture in 1972. Apparently, as an industry, we’re slow learners.

The ensuing programming languages over the years, from Modula 2 and Ada to Java and C#, were neither modest nor elegant. Instead of respecting and working with our innate limitations, methodologies seemed to assume that people could make great decisions despite a complete absence of fact; that we could could plan months and years in advance with prescient clarity; that we could create working code that would, as if by magic, answer every need of an unseen and unknown customer. We thought we could even write code that did what exactly what we intended, more or less the first time. Well, at least as soon as it compiled successfully. Maybe with a few print statements for good measure.

Time passed, and some thirty years after Dijkstra’s gentle polemic, the Agile Software Development movement reached critical mass in Snowbird, Utah. The seventeen of us interested parties who gathered on that date had different views on how software development should best be conducted, but tacitly agreed on one very important point: the need, as Dijkstra put it, to respect the intrinsic limitations of the human mind and approach the task as Very Humble Programmers.

A bunch of us (for some large value of bunch) recognized what it meant to be “humble,” in the sense that Dijkstra meant it. To take small bites, and not try and do too much at once, or to try to prognosticate too far out. To verify and validate what we’d written, not trusting our own overrated coding prowess. To openly and honestly discuss what we’d do next, not assuming that we knew better than anyone else. We recognized that software development is like trying to run through a very large dark cave with a very small weak flashlight. Raw speed will just kill you faster if you’re headed over a steep cliff.

Now, exactly ten years to the month after the catalytic coining of the word “agile,” where do we stand? Did the world at large catch on that programming was a might trickier than it seemed, and that you needed the support of modest and elegant tools, flexible and adaptable methods, and experienced, enthusiastic team members and customers in order to succeed?

Not exactly.

Meet the New Boss, Same as the Old Boss

Let’s start with education. Or rather, the lack of it. The better software engineering techniques promoted by agile methods still aren’t widely taught in college and university. There are some bright spots, and some excellent programs that actually “get it,” but we’re far from universal acceptance. There are plenty of degree programs where graduates have never worked in teams, never used version control, and never written a unit test. So academia isn’t quite on board yet. What about the corporate world?

Last summer, I had the good fortune to visit a “very advanced” agile shop. These folks really did embrace agile methods with a discipline, completeness, and a zealous fervor that would be hard to match. In many ways, they could have been a poster child for agile methods. For such a magnificent, large-scale operation to succeed, perhaps we had won the war after all. Perhaps the agile ideals really had taken hold, and manifested themselves here in this largish company, freed from the shackles of mindless dogma that had held previous generations of programmers in thrall.

But these weren’t productive developers freed from mindless process dogma. They were agile slaves. The dogma they followed was ours, and they followed it well. And as with many organizations in a similar position, they saw some promising results. Continuous integration, refactoring, unit tests, pair programming—all these techniques yielded some benefit. But they weren’t thinking, they weren’t reacting, they weren’t being agile. When problems came up, they addressed them with all the grace and elegance of a deer caught in the terrifying blaze of alien headlights. They knew how to do agile; they didn’t know how to be agile.

And that’s part of the continuing challenge. We’ve made great headway in getting better basic techniques adopted. Years ago you still found shops that didn’t use any version control system at all—just source code files on a shared disk, and “last one in wins.” That’s much less common now. Unit testing and its descendants are increasingly accepted and practiced, and even pair programming, often viewed with malignant suspicion by facilities managers and stakeholders alike, isn’t quite as foreign and scary as it once was.

But folks need to understand what it means to be agile; how to successfully solve the new problems that arise constantly. There’s no formula for that, and never will be. As Dr. Patricia Benner puts it so eloquently, “Practices can never be completely objectified or formalized because they must ever be worked out anew in particular relationships and in real time.” That’s a big part of the picture we haven’t got quite right yet: helping people work out new practices in particular contexts that work well for them. That’s the secret to real agility; it’s not about doing pair programming, or having stand-up Scrum meetings. Anyone can dogmatically follow the practices prescribed by others.

But true agility goes beyond the dogma, beyond the practices. Agility is about adapting; adapting your process, your language, your tools, your team, and yourself to respond to the situation at hand. We need to be positioned to be able to do that in order to fully realize the promise of agility, and maybe even move beyond it.

Beyond Agile

So what’s next? I humbly suggest we take the direction for the future from Dijkstra: stick to modest and elegant programming languages, and acknowledge and respect the intrinsic limitations of the human mind.

Java is neither modest nor elegant. But what about Ruby, Clojure, or Erlang? Are these languages modest or elegant enough? Can we come up with better programming languages, languages that make it easy to do the right thing? How many programming languages do you know? If it’s only two or three, grab another dozen before you answer.

As far as our limitations go, I’ve hit on a few of those in the PragPub “Guru Meditation” column (named for the old Amiga OS error messages.) Our brains aren’t wired optimally for this sort of work in general, and not well for agile methods in particular. But that’s just a start. Here are a few other interesting topic areas that we could pursue to go beyond agile:

The Dreyfus Model. Experts are not merely smarter novices. There are many changes on the road from novice to expert, from how you approach problem solving to how and what you communicate. Yet on most teams, we tend to treat all team members equally. This is unfair to both novice and expert, in different ways. We can be more effective by using the lessons of the Dreyfus Model of skill acquisition to guide teaching, learning, and teamwork.

Communications. In the 2011 State of the Union address, President Obama spoke of education, competitiveness, and inspiration. But the number one word that listeners heard? “Salmon.” The president made a joke about overlapping government responsibilities (which involved salmon), and that’s what stuck with the audience. People don’t always hear what you want them to hear, and we don’t communicate requirements, technical needs, or opportunities all that well. We can do better. From interviewing and working with users to help them determine their needs, to teaching the dangers of technical debt and the need to refactor, we can use what we know of cognitive limitations to communicate the important stuff more effectively.

The Nature of Thinking. But improving communications is just the tip of the iceberg. We can improve our creativity and inventiveness—two critical components for successful software development. I touch on these topics in Pragmatic Thinking & Learning, but that’s only a start. A lot of the research in cognitive- and neuroscience can be applied to improve software development and other “thinking” professions. So far, we ignore most of it, because some of the lessons seem counter-intuitive to what we’ve been taught. But our brains are plastic and pliable, so we can change how our brains work, for better or worse. Let’s deliberately try for better.

Something to think about.

To Agility... and Beyond!

Kent Beck

I took the Manifesto in one direction with my To Agility... and Beyond! talk. The point there was that the Manifesto assumes that there is someone “out there” who knows (or at least will tell you) what to do next. What do you do if that isn’t the case? Increasingly it seems to me that this will be the case for interesting development. How do you develop when it’s everybody’s job to figure out what to do next?

This is not the only direction the Manifesto could go. In general, I’d like to see it talk about increasing responsibility and accountability, productive conflict, and personal growth, but that’s probably because these are the issues I struggle with. :)

We Really Changed the World—and Yet…

Ron Jeffries

I was excited to be at Snowbird for at least two reasons. First of all, it was a meeting of important and interesting people, some of whom I knew, some of whom I did not, most of whom I at least knew of and had reason to admire. I had been noisy about Agile but did not and do not consider myself to be an inventor of it, so I felt honored to be there.

I was also excited because I’ve always been interested in doing good software for people who need it, and Agile—especially Extreme Programming—holds out the prospect of doing good work in a context of delivering great results to someone who actually wants them. I had found the ideas and practices to be incredibly energizing, even at my then-advanced age.

It seemed to me that we had a handle on something great, and that we were going to change the world. And, in fact, I guess we did.

Today, there are a lot of people doing Agile, and a lot more saying they are doing Agile while really doing something else. What we see, however, is that if an organization makes a good-faith effort to do what we have suggested, they do get benefit. There are over 100,000 “Certified ScrumMasters,” so surely there must be a million people trying to do what we offered, and I believe that most of them are probably getting some benefit.

Frankly, though, I had imagined more. I had hoped that many people would adopt these ideas, and I had imagined a significant step upward in project success among those who did adopt them. I had imagined the industry really moving up a notch. What happened is that many have adopted the ideas, at least in name, but that few of them have attained anything like the benefit that is possible.

That leaves me pleased in my mind, and less than pleased in my heart.

So what to do? I try to review and adjust course every year and maybe as this is a decade marker for the Manifesto I should take a harder look.

If there’s something new and amazing on the horizon, I do not yet see it. Lean and Kanban aren’t it. Fuzz your eyes right, and they are the same as Agile, or Agile is the same as them if you prefer. Mostly cut from the same cloth, that’s what I’m saying. Still, if something newly great comes along, surely I’ll have a hard look and dig in if I can.

Meanwhile, my first mission will remain the same: eradicate all errors on the Internet, at least regarding Agile. I’ll surely continue my appearances and rants on as many mailing lists as will have me. I do observe that I’m laying back a bit, letting other younger, less bruised individuals take more of the lead. But I don’t expect that I’ll stop.

In the past 18 months or so, I’ve primarily been working with relative newcomers to Agile, teaching “Certified ScrumMaster” courses. That has been fun, most of the time, and fairly lucrative, but it is not fulfilling. Many of the students are in organizations where they cannot possibly succeed, and many more are just in the class because they were sent to it. It furrows into my heart.

In the time remaining before me, I want to work more with people who truly want to excel. I don’t imagine for a moment that I can teach them all they need to do to excel, but I do believe they can profit from working with me and observing how I do this stuff. And we’ll all have a good time.

Finally, I’ve been “working” on a book for quite a while. It is time to bear down and finish it. I am confident that it’ll be interesting to people, and more important, it will say something that I want to say.

Change Begins with Me

Jon Kern

I’m all over the “agile sucks” thing. In general, the collective “we” suck at software development, agile or otherwise. Most of the folks paying for software have not a clue whether they are getting their money’s worth or not. There’s no Car & Driver or Consumer Reports showing the stark contrasts that exist between software teams/products. Or John Stossel showing the incredible waste in one software dev organization versus another.

I’ve pretty much given up on preaching to companies how to change their ways. Often, the software bit of what they do is so far down the chain of bureaucratic flotsam as to matter very little. I’ll happily work with small teams to help them solve real problems the best we can, and do it as “agile” as we can—in context, of course. As long as we have the chance to have an impact and change what needs changing in order to succeed, that’s where you will find me toiling away.

I’ve pretty much resolved to change... me. Yup, just me. I want to be the best that I can be at software development and producing deployed code. To that end, I am thankful for Ruby and all the good published works on pragprog—I confess, I am an addict, with a stack “hits” I am still waiting to ingest.

Maybe through changing my ways to be as good as they can be, I’ll change those around me. Together, other teams might take notice of what we’re doing and think about their own journey, starting at themselves. Just maybe.

Part of me knows that this is a cop-out. Part of me says that I should strive to help shine light on the dark recesses of the truths behind software development in the wild. The other part of me says, let the market do its thing. For now, the libertarian in me is winning. To each their own. You can believe that scrum in a vacuum will save the day, you can believe that agile sucks, you can believe that waterfall is for features and agile is for bugs (yeah, that’s a real quote). Whatever. All I ask is that you question with boldness everything that you and your software organization do. Since you all might be zombies, you should even allow outsiders to question with boldness what you do. Following that, you’re free to continue or change as you see fit.

After all, agile is a state of mind, an attitude—nothing more, nothing less.

It’s About People

Ken Schwaber

Agile is a neat term. Someone once said that all this would have never happened if we called it the “Fragile Manifesto.” It certainly achieved one of its goals, to drive a stake into the heart of Waterfall. Using the wrong process was nothing but frustrating and demeaning.

However, agility required people to retool their practices, tooling, and most significantly—their skills, beliefs, and habits. The desire is strong and the vision is clear, but the path is not straight, and the effort and courage to achieve agility is phenomenal. I think we have started a quiet revolution toward people no longer being thought of as resources. Instead, paraphrasing an old IBM slogan, “People are our most valuable asset,” I would say, “we are our people.”

I Went for the Skiing

James Grenning

When I first learned about extreme programming back in 1999 I was very excited about it. It provided logical solutions to many of the problems I had experienced over the prior two decades of software development. I was not quite sure about pair programming and test-driven development; that all seemed promising but a bit… extreme. But after a little experience I found both practices productive, and fun.

In 2000, while I was with Object Mentor, we were early adopters and advocates of extreme programming. We knew that we had discovered something that could help many development teams get out of their rut. Organizations hired us to help them change. We had visions of walking in the door, explaining XP and having everyone embrace it like we did. Wow, what a shock: they were not ready to change.

Current practice was not delivering quality software on time, but XP was too extreme. Just look at the name. “Will we need elbow pads and helmets?” our clients would joke. Developers and product managers of the day thought that the solution to their current woes was to do more upfront work, not less. The idea of fast feedback along with evolving design and requirements was too foreign. Well, we persisted and helped many organizations get started. XP was new then. Agile did not even exist by that name.

In the beginning of 2001 Bob Martin invited me to join him at Snowbird for a meeting with the guys that started XP and some of the other “Lightweight” development processes. My answer was, “Yes, I want to go (skiing).”

The snow was fantastic, with two feet of fresh powder and plenty of avalanche danger. Hanging out with the software development personalities was affirming, inspiring, and thought-provoking. There were many competing and complementary perspectives discussed: code-centric, model-centric, and people-centric. Rather than trying to define the one true software development methodology and practice, someone had the bright idea that we should make a statement of the things we have in common. With some good discussion, careful word tweaking, and the snow falling, the four values were distilled.

Before we could hit the slopes, we needed one more thing, a name. Lightweight was the banner we flew under on our way to Snowbird, but who wants to be known as a lightweight? Good point; we could not have that. After a while we settled on Agile Software Development. Of course organizations would want to be agile. You can explain that to your CEO or customer far more easily than proclaiming you are extreme.

We had a name, we made a statement, but we were not sure if anyone would know or care. Over the following years we found that people did care. Many creative and passionate people were attracted to Agile, and Agile continues to grow in popularity and depth. After ten years, you can see that much has changed, but much remains the same.

Many developers are still unaware of Agile, or only know the misconceptions. The idea that more upfront work is needed is deeply ingrained in the software development mindset. Code bases are a mess, dragging teams and products to a standstill.

Happily, people are discovering agile everyday. Embedded development, where I spend much of my time, is awakening to Agile with interest—and skepticism. The questions and concerns are essentially the same that non-embedded application developers cite, though on the surface they sound very different. Dependence on hardware is a big conceptual hurdle for embedded developers, but non-embedded developers have dependencies on UIs and databases. Those sound like very different problems, but they have similar solutions in the technical practices of XP and Object Oriented Design.

Incremental delivery is another conceptual hurdle. How can you deliver a washing machine incrementally? Embedded development is not the only place where product completeness requires that the top ten (priority 1) features must be in the product for it to be useful or marketable. But even when value cannot be delivered incrementally, we can still benefit from measurable, visible, and incremental progress.

There is at least one new problem ten years after. Now we have organizations that have a few Scrum Masters and proclaim themselves Agile, but that continue to spend months in analysis and design, and similar amounts of effort in test and fix. They have stories and iterations, but ignore relative effort estimates and velocity. Code is deteriorating. Tests are not written. And they wonder why Agile is not working for them.

Change is difficult, but the only way to improve is to change. I show teams the careful way of working, with clean code, tight feedback, and plenty of tests. They say “That takes so long. We want to know the fast way, show us the fast way,” as if I am holding back some secret. The careful way is the fast way. Get good at careful; that’s how you can go fast. My hope for the next ten years is that more people accept this tough lesson. Now it’s time to get out the ski gear for a tenth anniversary trip to Snowbird. I’m going for the skiing and looking forward to the next ten years. Will there be another revolution? Maybe, but I know for sure there will be more evolution.

Hard Times Call for Agility

Arie van Bennekum

Ten years ago I was part of the initiative that led to the Agile Alliance. At that time I was already totally into DSDM (since 1994) and what we now call “agile” had become part of my way of thinking. Not only in software projects, but also in other kind of projects and even outside work —things like prioritizing, prototyping, and iterating to converge to a well-fitting solution.

At the customer site the business case way of thinking, including prioritization techniques like MoSCoW, were understood, but the urgency was not felt. I remember for me it felt like being on trial again, explaining to the customer how he could and should save lots of money and time in using agile methods…. And the answer was always “Well, this is not really for us,” or “Well, it works well but it is to difficult for us.”

Now we have a credit crunch torturing the world. Business and government feel they really have to execute projects in a slim way. Nowadays where I live the need for doing the right things instead of doing things right surfaces a lot. Nevertheless, old ways of thinking in project management are still in the lead. “Just squeeze your project members and you will get there” seems to be the line.

Agile has evolved. DSDM has grown into Atern. It covers business-case thinking in a fast-changing and agile environment. Agile principles in projects need to be applied more than ever. We have many miles ahead of us on this road....

I Hoped to Derail Their Evil Plans

Stephen J. Mellor

I was astonished to be invited to what became the meeting that originated the Agile Manifesto because my work had always been based around building models.  The Structured Development for Real-Time Systems trilogy with Paul Ward and the pair of Object-Oriented Analysis books with Sally Shlaer all focused on analysis and design, placing much less emphasis on the coding and testing favored by agile practitioners.

Moreover, I had recently read Kent Beck’s Extreme Programming and I was horrified by the lack of emphasis on up-front thinking, abhorrence of models and the deprecation of documentation. However, it was clear that lightweight, extreme, or agile approaches were gaining traction, so I resolved to attend the meeting. The fact that it was in the Rockies in winter had nothing to do with it.

I introduced myself as a spy who hoped to derail their evil plans. While I found many of the positions taken regarding process to be sound (talking to the customer, for example, or timeboxing), the rejection of models was surely unjustified. Yes, there had been an overemphasis on “big design up front,” but surely models had some value. Yes, writing documents (and models) had come to be seen as an end in itself, but surely the correct response was not to throw the productivity gains from modeling out with the dirty documentation.

So just why were models so bad? Because they don’t execute, I was told. Yet my work over the preceding decade had been all about building executable models. Over the few days of the meeting I had a similar conversation with almost all of the signatories, sometimes more than once.

This intellectual passing-in-the-night came about because we had differing ideas about what the word “model” means. Some signatories viewed models (if they thought about them at all) as sketches, to be drawn on napkins for communication purposes and then thrown away. What drew the most ire was the view of a model as a blueprint to be thrown over the wall to developers who did as they were told. I held neither of those views. I thought models ran.

Though we were executing models in 2001, it was with our own action language. I heard repeatedly at the meeting that you could not write a program to say “Hello World!” in the Unified Modeling Language (UML). And although you could, in fact, say “Hello World!” in UML, it certainly wasn’t easy. In fact, it was an advertisement for writing code. This had to be addressed if modeling was to become widely viewed as executable.

In 1997, when the UML was accepted as a standard, there were but seven actions you could carry out in a model, and one of those was “uninterpreted string.” (You can certainly forgive the signatories for thinking that models couldn’t execute!) I proposed a standard Action Language for UML. Some folk immediately suggested standardizing on Java. Or Smalltalk. Or....

But an existing programming language is insufficient because it works at too low a level of abstraction. It therefore mixes (to some degree) application concepts with their implementation. For example, when we build a particular data structure (a list of Customers, say, with pointers to lists of Accounts), we have made a decision about implementation. In the UML, we would instead express this is as an association, but we would not specify its implementation.

Moreover, when we come to operate on this data in code, we have to take into consideration the implementation we have selected. For example, to sum up all the account balances for a specific customer, we are required to traverse the linked list. It would be simpler to say (somehow) “sum all of the account balances for this customer.”

It’s not so much that we want to be able to execute models in UML—we could just add code to achieve that. It’s that we want the higher productivity that comes from operating at a higher level of abstraction away from the implementation. We also want to be able to change our minds about the implementation without changing the description of the problem. The UML model would have the same application meaning if we put Customers and Accounts into a database implementation.

Over the following twelve years, we (the members of the Object Management Group, the organization that standardized UML, and especially Ed Seidewitz from Model Driven Solutions) have produced a series of standards that define the kinds of actions permissible in UML; defined a subset of UML for execution; defined the semantics of that subset precisely; and (finally!) defined a standard Action Language for UML.

In that language, the example above would be written:

myCustomer.account.balance -> reduce ’+’

where “reduce” is an operation that “reduces” the collection of balances produced by traversing from myCustomer to all their accounts by applying a binary operation, in this case ‘+’ repeatedly until only a single value remains. Of course, that can then be translated into an implementation using linked lists. Or a database. Or anything else. There are no lists in the specification, nor any traversal of them, just a very compact statement of what needs to be done. (By the way, one design requirement for the language was that it be easily accessible to Java programmers, so much of the syntax looks like Java. The example here was chosen not to obfuscate but to illustrate the power of moving away from the implementation.)

The other signatories were kind enough, back in 2001, to write the Manifesto using the word “software” (which can include executable models), not “code” (which is more specific.) So I felt able, in good conscience, to become a signatory to the Manifesto while continuing to promote executable modeling. Ten years on we have a standard action language for agile modeling.

We rarely see the words “agile” and “model” in the same sentence, but they are not at all in conflict. Rather, modelers can learn a lot from the agile folk (building tests for models early, for example) and those following an agile process can benefit from the increased productivity and eased customer communication that come from sharing executable diagrams with their pair-modeling customers. Surely that’s a win for everyone.

For more detailed information, including critical references, look here.

How Will It Survive?

Ward Cunningham

We wanted people to read code and learn from it. We wanted a canon for our industry. We asked that people program together and reflect on the experience, to learn a new vocabulary from the experience, a vocabulary of patterns, a vocabulary of collaboration.

Now everyone wants to collaborate with programmers. Programmers have a new-found reputation for getting things done. It’s refreshing. But it’s diluting the canon, our young canon. How will it survive?

You Can’t Do Agile

Dave Thomas

Mike Swaine contacted me for my thoughts on “Agile at 10.” My initial thoughts were slightly churlish—there’s no such thing as Agile, and whatever there is, it isn’t 10 years old.

First, there’s this growing tendency to treat the word “Agile” as a noun. People say “we’re doing agile.” But friends, agile is not a noun. It’s an adjective, meaning to be able to move quickly and easily. When we first coined the term back in our Snowbird meeting as a way of describing what we did, we felt that the word “agile” also carried overtones of grace.

So, for me at least, you really can’t use “agile” without having it qualify some kind of noun—it’s the process, or the thinking, or the interactions that are agile. And that’s something I’d like us all to refocus on as we reflect back and plan forward. Being an agile software developer means doing things in an agile manner, not just naming things that you do “agile.” And that’s what makes this all so tough, but also so effective. You can’t just buy an agile toolkit, or stick up agile posters, or pay for some agile consultant. Ultimately what makes the things you do agile is the way you do them. And, in my experience, if you continue to work to make the doing of those things quicker and easier (over the long term), you’ll be making yourself more agile.

Think about that for just a second. You can’t do “agile” and make yourself better. You can only do things better and in the process become more agile.

The other problem with “Agile at 10” is that agile processes are not really ten years old. When we seventeen middle-aged white guys met in Snowbird in February, 2001, we didn’t invent agility. All we did was give a name to a set of values that encompassed the way a whole bunch of people (including ourselves) were already producing software. I dare say much of the work at Xerox PARC 25 years before our meeting would be considered agile. I know that much of the software development that was being done in the 90s was. So, in a way, all we did was to provide the seed crystal of a name—the subsequent growth came from all those thousands of teams that had already been doing things in an agile way. Our seed gave them a name and a set of four values on which to hang their existing practices.

I’m proud that much of our industry has used the Agile Manifesto as the starting point from which to explore better ways of creating software. I’m impressed by people who have used the term to help convince their companies to try new techniques. But mostly, I’m just blown away by the enthusiasm with which the people in our industry constantly strive to do things better. These are the people we should be celebrating on the 10th anniversary of the Snowbird meeting. Thank you all for continuing to make it interesting and fun.

Abstraction

How to Tell a Cat from a Dog

by Tim Ottinger, Jeff Langr

There is a very deep antipathy between duplication and abstraction.

This continues our series on the four Big Ideas in software development. Be sure to check earlier issues for articles on Cohesion and Coupling. This month, our goal is to cast new light on abstraction.

A naive approach to object-oriented design is to create a system using classes that model real-world things. If you learned about object-oriented programming in the 1990s, chances are someone had you model Dog, Cat, Mammal, and Animal classes as an exercise. Abstraction meant implementing the parts that related to your software needs. You might have designed a bark() behavior, but not a tailWag() behavior, along with a few supporting attributes such as size and speed. Your classes were a straightforward abstraction of the real world, each with as many attributes and behaviors as made sense for that real-world element.

From this introduction to abstraction comes quite naturally a mindset that the best way to create an an object-oriented design is to model the real world, leaving a few bits out. This is not necessarily wrong, but it is misleading. Abstraction is deeper and more profound than this mindset makes it sound.

We base our primary focus on abstraction on a definition by Uncle Bob Martin: Abstraction is the elimination of the irrelevant and the amplification of the essential. See how we just emphasized both essential phrases in that definition and eliminated the BS about “the real world?”

Abstract and Concrete

We start our discussion of abstraction with the concept of abstract types vs. concrete types. Abstract types do not completely specify behavior, whereas concrete types contain specific code details for all behaviors. Purely abstract types in C#, Java, and the like (where absolutely no behavior is defined) are known as interfaces.

From concept to code, then, abstraction is directly implemented in the form of interfaces. The set of behaviors supported by a class appears as a standalone declaration, a contract of sorts:

| public interface FineCalculator {
| BigDecimal charge(int daysLate);
| }

The FineCalculator interface captures the concept of determining how much to charge library patrons for borrowed materials that they return late. The interface captures only this singular concept and no implementation details (other than the argument and return types). A fine calculator implementation will complete the interface by implementing charge(). You might imagine BookFineCalculator, MovieFineCalculator, and NewReleaseFineCalculator implementations.

Though implementations may vary, the abstract concept of determining an appropriate fine charge for a given number of late days is likely to remain unchanged from the point of view of a FineCalculator user. Among the benefits of having this abstraction are:

The specific portion of the client code that must obtain fines can be written once, regardless of the material types involved. “If” statement logic isn’t littered throughout the client: “if the material is a book, calculate the fine using this algorithm, otherwise if it’s a movie, calculate it that way, otherwise….”

New FineCalculators can be introduced, and existing algorithms changed, without touching virtually any other code elsewhere in the system (a great example of Bertrand Meyer’s open-closed principle).

The interface isolates the client software from any changes to the implementation details of each FineCalculator algorithm, as long as it continues to meet its contract (which is assured by unit tests).

The client software can be unit-tested in isolation and thus not have to depend on interacting with any one specific material type. Tests for the client can substitute a test-double that implements the same abstraction solely for purposes of testing. This ability to test against a simple, in-memory construct isolates the client code being tested from dependency on a collaborating class that might be volatile, slow, or even non-existent.

Without the interface, the client is dependent on concrete details of the algorithms, which are likely to change over time. Introducing an abstraction layer, in the form of an interface, basically nets you all the positive benefits of reduced coupling.

Generalization is also Abstraction

It would be possible to name the charge() method something like CalculateChargeForBooksOverTenDaysLate(), but that has a problem of over-specification and implementation exposure. It is not an essential feature of FineCalculator that the charge is for a book (modern libraries lend a variety of materials), nor that the charge is only calculated for lateness over 10 days. A name that reveals only relevant and correct information is an abstraction. One may back into abstraction by stripping irrelevant details from names in the system.

With a name like FineCalculator a developer can know in an instant if this is a class he wishes to subclass or not. Generalization has limits, though. The simpler name “Calculator” lacks evocative value. Uncertain whether to implement its interface, a developer may create a new interface (duplication!), hack new behavior into existing code (complication!), or directly modify the caller of the existing FineCalculators with code for calculating his specific fine (duplication and coupling!)

The TDD community has been recently buzzing with the realization that code becomes more general as tests become more specific, revealing that test-driving code alone will push it to a more appropriate level of abstraction. It is still up to the human(s) at the keyboard to change the class and method names to match.

Data Duplication vs. Abstraction

There is a very deep antipathy between duplication and abstraction.

One frequent example we’ve encountered is the pervasive use of a parameterized collection object. For example, the library system works with lists of holdings:

List<Holding> holdings = new ArrayList<Holding>();

Throughout the code, you’ll find dozens of references to the List<Holding> type, often in signatures or method calls:

| List<Holding> holdings = findHoldings(patron);
| public List<Holding> findHoldings(Patron patron);
| // ...
| }

This is a subtle form of duplication: We have to specify two pieces of information—the collection type and the type to which the collection is bound—in every appropriate code place. Suppose we must now associate additional characteristics with the collection of holdings as a whole, such as a date stamp to indicate when the collection was created. We can pass this date stamp around as an additional argument here and there where appropriate:

public void archiveHoldings(List<Holding> holdings, Date created)

This opens up the door to increasingly long method signatures over time, instead of helping the system to evolve gracefully. The date is really an attribute of the list of collections as a whole—yet we have no abstraction in which we could capture that information.

Prefer instead to create an abstraction that simply encapsulates the two:

| public class HoldingSet {
| private List<Holding> holdings = new ArrayList<Holding>();
| private Date created;
| // …
| }

This amplifies what’s important—the collection of holdings—and buries the irrelevant fact that holdings are stored as a sequential list.

As you need, you can easily incorporate new behaviors into HoldingSet without having to revisit numerous method signatures throughout the application. The abstractions become richer over time instead of the parameter lists becoming more cumbersome. Abstraction drives out duplication.

The same principle applies to a loose collection of primitive parameters. Perhaps a repeating set of (latitudeHours, latitudeMinutes, latitudeSeconds, longitudeHours, longitudeMinutes, longitudeSeconds) might indicate a missing map coordinate abstraction? Do the coordinates have related methods scattered about the code?

Code Duplication vs. Abstraction

You may frequently find two-line or even single-line duplications. In the Risk game implementation we’ve looked at, there is a large class named Risk which looks to control everything about the game. Within this multi-thousand-line class are numerous methods and lines of code involving both an offensive player (attacker) and a defensive player:

| public boolean isValidAttack(Player attacker, Player defender);
| if (display(attackingPlayer) || display(defendingPlayer)) {
| // ….
| }

The class that controls the game includes additional information related to making attacks:

| int[] attackerResults = game.rollDice(game.getAttackerDice());
| int[] defenderResults = game.rollDice(game.getDefenderDice());

Similar code is sprinkled through the Risk class. Virtually every place there is code relating to an attacker, there is also code relating to a defender.

The related code can be rolled into a single abstraction, an Attack:

| public class Attack {
| // ... fields here ...
| public Attack(Game game, Player attacker, Player defender) {
| // ...
| }
| public boolean isValid() {
| // ...
| }
| public void rollDice() {
| attackerResults = game.rollDice(game.getAttackerDice());
| defenderResults = game.rollDice(game.getDefenderDice());
| }
| public int[] getAttackerResults() {
| return attackerResults;
| }
| // ...
| }

With this design change, you see very subtle bits of unnecessary (duplicate) code disappear. For example, we were able to change the method name isValidAttack to isValid, once we moved it into the Attack class.

The client code becomes simpler overall. We’ve moved two lines of complexity involving interaction with a game object into a single method in the Attack class, rollDice. That change didn’t eliminate any duplication yet, but it did simplify the client and achieved command-query separation (i.e. we can ask for attacker and defender results multiple times without having to re-roll the dice):

| Attack attack = new Attack(game, attacker, defender);
| attack.rollDice();
| int[] attackerResults = attack.getAttackerResults();
| int[] defenderResults = attack.getDefenderResults();

Further, we made it possible to change the implementation of how dice are rolled without having to open and touch the client class. The game object is now referenced in the client only when constructing the Attack object. The design isn’t yet “perfect”—perhaps we should move the rollDice, getAttackerDice, and getDefenderDice methods into the Attack class itself—but we now have a new home into which we can relocate attack-related code.

With the introduction of this previously missing abstraction, our many-thousand-line blob class shrinks by perhaps a few dozen lines of code. As the Risk class shrinks over time, additional opportunities for abstraction become more obvious. Abstraction begets abstraction.

Spotting “missing” abstractions takes a bit of practice. Here are a few smells that might point to the need for additional abstractions:

Code chunks that seem to repeat (perhaps not exactly) throughout the code.

ctrl-c / ctrl-v

“I know I saw something similar somewhere else in the code.”

Extensive detailed test setup

Tiny Abstractions

Sometimes, you’ll spot two lines, or even a single line, that redundantly specifies code. Here’s a bit of ugliness used to add two new menu items, and corresponding actions, to an existing menu:

| // clean
| this.clean = new MenuItem(this.menu, SWT.PUSH);
| this.clean.addSelectionListener(
| App.instance().getAction(CleanAction.NAME));
| // remove
| this.remove = new MenuItem(this.menu, SWT.PUSH);
| this.remove.addSelectionListener(
| App.instance().getAction(RemoveAction.NAME));

Don’t hesitate to factor these couplets into a single method! While they may not represent a top-level abstraction like a class, helper methods in the same class are still abstractions—you’re replacing a complex implementation detail with a simple declaration:

| this.clean = createMenuItem(menu, CleanAction.NAME);
| this.remove = createMenuItem(menu, RemoveAction.NAME);

And once you’ve created such methods, you may start to notice that they too may be better suited in another class, whether existing or new. Further, you might recognize that things are a bit disjoint and implicit—it seems as if there’s an action object somewhere that is associated to the key identified by CleanAction. A good goal for this code might be to shape it into something like:

| this.clean = menu.addItem(new CleanAction());
| this.remove = menu.addItem(new RemoveAction());

Of course, the library type for menu may not support this—perhaps it’s time to create your own abstractions that wrap the third-party types.

We hear the same resistance to these ideas all the time: “But all these new method calls and object instantiations are going to degrade performance.” When we hear this, we recommend that the programmers try and measure. You will be surprised to find what is fast, what is slow, and why. The world changes too fast to blindly follow rules of thumb about performance.

Test Abstraction

Unit tests, particularly those created as a virtue of doing test-driven development (TDD), must document the essence of what’s going on:

What data is being created for purposes of the test?

What behavior is being executed?

How do we know that the expected behavior happened?

It’s far too easy to drown these three key test elements in a sea of difficult-to-understand test code.

Tests must amplify what’s essential and bury what’s not relevant to understanding the requirement. Tests that are not sufficiently abstract will be difficult to understand and will break for all the wrong reasons. Test abstraction is such a significant element of doing TDD well that we’ve chosen to discuss it in an upcoming article.

Conclusion

Abstraction is where object-oriented software design starts. We strive to build a system that presents straightforward concepts to the reader, not overwhelming masses of detail.

The process of abstracting drives out duplication and reveals more natural abstractions over time, making the code easier to read and easier to test.

A well-abstracted design imparts meaning and provides easy navigation. We can deftly navigate the system through its simple abstractions, and push them aside when we need to get to the nitty-gritty implementation details.

Jeff Langr has been happily building software for three decades. In addition to co-authoring Agile in a Flash with Tim, he’s written another couple books, Agile Java and Essential Java Style, contributed to Uncle Bob’s Clean Code, and written over 90 articles on software development. Jeff runs the consulting and training company Langr Software Solutions from Colorado Springs.

Tim Ottinger is the other author of Agile in a Flash, another contributor to Clean Code, a 30-year (plus) software developer, agile coach, trainer, consultant, incessant blogger, and incorrigible punster. He writes code. He likes it.

Send the authors your feedback or discuss the article in the magazine forum.

Refactoring Your Job

Finding the Upside of the Downturn

by Craig Riecke

You say your department was downsized from 100 programmers to 60? That doesn’t have to be a disaster; it can be an opportunity, if you make it one.

Is it me, or has this economic downturn been... uhhhh, somewhat less than fun? I mean really. The chances are very good you’re a programmer working in a smaller department than you were 5 years ago. You’re in maintenance mode. The tasks you perform are more break-fix, less cutting-edge, less intellectual challenging. Your budget is smaller, so training and conferences are less accessible.

“At least you have a job,” someone says. That might make you feel good for a while, but you’re not the kind of person who thrives on standing still. In the 2010 TechRepublic IT Skills and Salary Report, 80% of participants cited “Opportunity to Increase Skills” as a major job satisfaction contributor. And you feel that might be slipping away.

It feels like someone is pushing the brakes on your car. You feel you’re no longer in control. Your morale plunges, and it gets harder to go to work every day.

So how do you get through this?

Turn Things Around

I hate self-help books. I especially hate ones that minimize one’s pain and suffering—the ones that offer pithy sayings like “Look at this as an opportunity. Be positive. Let go of your anger and frustration.” Blah blah blah. It’s as if now, on top of feeling crummy, you’re now a failure for not making yourself feel un-crummy. Gee thanks.

Here’s the thing: you’re an IT person, and you like to be somewhat in your control of your destiny. When something is boring, you write a batch job to do it for you. When you encounter complexity, you draw diagrams and write comments to help you understand it. In short, you act, you don’t just close your eyes and try to picture things differently. That’s why vague self-help books don’t work on you.

I look at it this way—I love to refactor code. Refactoring’s a necessity when the surrounding code changes, you have new programmers working on it with different points of view, you change the environment underneath it, etc. Good code is in a constant state of flux, but you always get the sense that your code progresses, because you’re a better programmer today than you were yesterday.

So it is with your work life. When the environment around you changes, you can refactor it to make it more harmonious. Programming is not a cookie-cutter job. It allows you a great deal of flexibility because... well, if you could write down the exact steps needed to do it, you would’ve automated it by now! You have some power to shape your career, make it fit the environment, yet reflect the intellectual athlete you are.

Slash and Burn

First let’s be realistic. If you have 40% fewer programmers around you, you should have 40% less code to maintain. And the chances are very good that 40% of your code can be cut. Much of it is dead, unreachable code. Some of it lies in redundant or seldom-used features.

You might counter, “Well, if I don’t touch that 40% extra code, what does it matter?” It matters.

That 40% extra creates noise, masks true problems, and invites you down dead-end paths.

When you Global Search, you’re searching 40% more code.

When you run your test suite, you’re testing 40% more code, which takes 40% longer.

When a company downsizes, it often stops doing whole classes of activities. Perhaps it stops selling a particular product line, it simplifies a 5-tier pricing structure down to 1, and so on. That’s the perfect opportunity for slashing out this code. If you ever take these duties back in, you can always fall back on the code in your Source Control system.

OK, but here’s the problem. Slashing code takes time, which you have less of. I use these ideas to help:

Since no one gives you explicit time to cut code, factor it into the velocity of a project. If you spend 10% of your day cutting code that’s tangentially connected to your project, the project goes a little slower, but not prohibitively so.

Take the opportunity to heavily instrument code. In my case, I created a special logging level called DEPRECATE, then threw logging in suspect procedures.

Cutting dead code is not only practically good, it gives you a psychological boost. It’s like the TV show Hoarders, where at the end, everyone’s overjoyed in their new, cleaned up house. You can be like that!

Get Really, Really Good at Prioritizing

Oiling the Squeaky Wheels in your company is easy when you have enough resources. So one luxury of having too few resources is that you now have a fail-safe excuse not to do useless projects. When you’ve stopped doing the obviously bad projects, and you start having to cut worthwhile endeavors, prioritizing becomes a deeper, more interesting task. All of those things you should have been doing all along—watching people work, understanding the business, and talking to people who are not always the most vocal, but maybe the most perceptive—are the things that will save your bacon now.

Not to get all Utilitarian on you, but there’s a moral calculus at work here. (John Stuart Mill is a good read while your test suite is running!) Seek out pain points—the things in your company that are the biggest hindrance to the greatest number of people. Then formulate projects that’ll solve them.

Why do this? First, it keeps IT visible, making it a problem solver rather than a cost center. People who don’t normally think of IT will now become your champions. Second, it’ll give you a boost of confidence. Every IT person remembers the first few programs they wrote, and how elated people were to use them. Getting back that feeling will remind you why you got into IT in the first place.

If you’re not solely in charge of prioritization, you must get better at convincing others to adopt yours. Terrence Ryan’s Driving Technical Change (Pragmatic Bookshelf, 2010) has some really good advice on this.

Misery Loves Company

There’s another reason to choose pain-point alleviators. Psychologically it may help to hang around people who are as miserable, or more so, than you. Suffering is bad but suffering alone is even worse, and by talking to others, you relieve a lot of the pressure and isolation.

However, and this is very important, once you find suffering you must find glimmers of hope. Seek out people who believe things can get better, and are willing to work a little harder and think a little deeper to get it. These are people who you can form a symbiotic relationship with, where each of you feeds off the positivity of the other.

Don’t let negative people drag you down further. You are not their psychologist or their spiritual advisor. Talking helps, but talking is only the first step. At your core you are an IT person, and your toolbox is IT. If they believe only a bigger paycheck or more hired help will solve their problems, then it’s best to minimize contact with them. (OK, maybe you can give them a bigger paycheck with a little well-placed COBOL, but let’s stick to legal tactics).

The Next Big Thing

In any other business, you might think, “Now I can slow down and concentrate on doing my job instead of finding the next big technological change.” But that’s not the IT Way. People get into IT because they love doing the next big thing. And let’s face it. Compare 2008 to 2010 and you see just as many technological advances—from consumer electronics to architecture to programming languages. It isn’t slowing down.

When you had more programmers, you could factor learning the next programming environment into your project plans. Now you have no time, and projects tend to be more maintenance-oriented than new code. So how do you keep up?

First, you can take advantage of what I call Cooperative Computing. Modern systems are less monolithic, more best-of-breed. The idea is to build projects in the new style, then connect them with common, well-debugged mechanisms like Service Oriented Architecture. You can also look for non-traditional ways to glue new architecture to old, like JRuby melds Ruby to Java. Instead of building a new building, think of buildings that learn and gain additions while keeping their character. Stewart Brand’s How Buildings Learn (Penguin, 1995) is a great inspirational book on this subject.

Second, if you have to build in the old paradigm, always keep the new paradigm in mind. If you must write in JSP knowing you will move to Rails later, use JSP conventions that will make the shift easier. Use more data-driven code generation. Then you can simply write new generators when you have time to move to Rails. Or you can even use a product like ANTLR to write transformation grammars. But the more regular you build your code now, the easier the shift will become.

Third, keep reading and researching! It’ll help you even if you can’t put it into practice right away. One of the best recent books I’ve read is Seven Languages in Seven Weeks (Pragmatic Bookshelf, 2010) by Bruce Tate. I may not have time to adopt any of the languages there, but it got my mind to thinking in new ways, which eventually changed the way I write Java. The Sapir-Whorf hypotheses says that different languages inspire different ways of perceiving and thinking about the world. That’s certainly true for computer languages.

Get Plucky

Let’s say your department was downsized from 100 programmers to 60. It looks like, by all accounts, you are at a relative disadvantage now. But here’s the thing. At one point (unless your company formed from a miraculous, virgin birth) your company had less than 60 programmers. These programmers were doing amazing work.

It helps to adopt that mindset. Forget about what life was like with 100 programmers. Instead, remember (or imagine) what it was like to work with those 60 programmers, working your way up, dreaming big. You can’t always act accordingly, but it helps to break your thinking out of depressing boxes.

Let’s say, for example, you have a web application and the navigation is clunky and inelegant. This is the way it’s been from Way Before The Jurassic Period, and there’s no way you’re ever going to change. Because you don’t have 100 programmers anymore, right?

But let’s say you had nothing. A blank HTML page. Tabula Rasa. How would you design navigation? Would you use tabs, or a search box, or drop-down menus? And how would you design this navigation so you could easily change it later? Would you use centralized templates? So now you have a better navigation system, and a way to refactor your current navigation system to something easily changeable. Two short hops. Hmm. That sounds almost doable.

We like to think we’re objective, logical, and focused. But IT is as much a craft as a science, and requires a bit of passion. Do whatever you can to rekindle it.

Hang in There

Sure, it’s hard to stay mentally afloat during hard times. But if you do, the rewards are significant. In a rebounding economy such as ours, employment is always the last to rebound fully, and when it does happen, you have two options:

You can leave for greener pastures, having kept your previous business going. That always looks good on a résumé.

-OR- You can reap the rewards of having stayed. More new hires will look to you for guidance, and you’ll have a cleaner, easier environment to train in. And management will hopefully remember your commitment, and reward it.

It’s remarkable that people adapt to adverse conditions. They rethink their values, and re-prioritize their work. This is the natural order of things. So using bleeding edge technology may be cool. But re-connecting with people, alleviating their suffering, and making your surroundings cleaner, more elegant, and streamlined is just as cool. It’s all IT, it’s all good!

Craig Riecke is a Dojo committer and a writer and editor for the Book of Dojo, Dojo’s online documentation. Holding a BA in English and an MS in Computer Science from the University of Nebraska, he is currently Chief Software Architect for CXtec in Syracuse, NY. While programming he listens to old, scratchy blues music on his iPod. His motto is “I’d rather drink muddy water and sleep in a hollow log than write a redundant line of code.”

Send the authors your feedback or discuss the article in the magazine forum.

De Morgan to the Rescue

Sometimes the Solution Is Just a Little Math

by Staffan Nöteberg

Sometimes you think you don’t have the right tool for the job—but it’s really right there in the theory you learned in that math class.

An old Chinese proverb states that learning is a treasure that will follow its owner everywhere. You may need some of that treasure following you when you’re dealing with regexes (Regular Expressions). The outstanding trait of regex standards is that there are plenty of them. Old school regex tools like grep and sed, lack many operators present in regex of Java, Ruby, Perl, C#, Python, etc. But we’re about to see that modern operators are sometimes no more than syntactic sugar.

You’ve got loads of old C files. Unfortunately they are in Kernighan-Ritchie style and your C compiler only accepts ANSI C. You’re concerned about declarations of parameterless functions: in Kernighan-Ritchie C you write foo(), but in ANSI C you write foo(void). You want to find all lines ending in empty parentheses ()—ignoring all other lines—and insert “void” between the parentheses.

However, the problem is harder than that. You also want to ignore lines where the ending pair of parentheses are preceded by the C inline comment token: //. It’s easy to write a regex that solves this problem using negative lookahead. But you’ve got the tool sed built into your operating system (GNU/Linux as well as BSD/OS X) and sed doesn’t support such fancy things as negative lookahead. Is there a sed solution?

There were a lot of buzz words in the last paragraph. Let’s start from the beginning: What is negative lookahead and why is it useful in solving this problem?

You want to find lines ending in empty parentheses, but not if they’re preceded by slash-slash. We can match all lines ending in empty parentheses with the regex \(\)$—i.e. a left parenthesis pre-escaped by backslash, a right parenthesis pre-escaped by backslash and finally a dollar to assert it’s at the end of the line. Parentheses must be escaped, otherwise they are interpreted as group boundaries. Regex assert operators, like the dollar symbol, don’t consume anything from the source string (the string that we’re scanning); they just tell us if a certain condition holds.

Negative lookahead is another assert operator. It checks in advance that a pattern is not present, without consuming anything. It’s written as question mark followed by exclamation mark. The regex (?!.*//) is true if it’s impossible to find a bunch of characters followed by slash-slash. When we prefix with the caret symbol, to assert it’s at the start of the line, then we get the regex ^(?!.*//).*\(\)$—which is correct in, e.g., Ruby:

| irb(main):001:0> "xxx foo()"[%r{^(?!.*//).*\(\)$}]
| => "xxx foo()"
| irb(main):002:0> "//xxx foo()"[%r{^(?!.*//).*\(\)$}]
| => nil
| irb(main):003:0> "xxx // foo()"[%r{^(?!.*//).*\(\)$}]
| => nil

Ruby promises that the first attempt above is a perfect match—a line that must be redesigned in ANSI style. In the other two cases, Ruby says nil, which in Rubyish means that these lines have slash-slash and should not be touched.

Nice, but not usable since sed—and not Ruby—is our target tool right now. Negative lookahead is not the answer, when ERE is a part of the pre-conditions.

To see what a solution to our problem would be like, look at the first figure. This clock-shaped machine solves the slash-slash problem. The square window at the top of the machine reads a tape—from left to right—consisting of the input string, i.e., the C code. We have a state machine of type Nondeterministic Finite Automaton (NFA). To visualize and represent the automaton, we use a transition graph, in which the vertices represents states and the edges represent transitions. The initial state is identified by an incoming unlabeled arrow not originating at any vertex. The acceptance state is surrounded by a circle. This is the graph that you see printed on the clock dial. Now, whenever a new symbol is read from the tape, the clock dial rotates so that the drooping peak, just below the reading window, points at the current state. Ah, and this particular machine also has a special lookahead feature: it’s a long arm with an eye in the end and a light bulb. This eye can look ahead and tell if there’s any double slash. If there is, the bulb will glow and the machine will understand that it doesn’t matter if the input ends in a pair of parentheses—it’s in a comment anyway.

Regex syntax differs from tool to tool. The Basic Regular Expressions (BRE) standard and its more clever brother the Extended Regular Expressions (ERE) standard make regexes more compatible. The tool sed (as well as grep) implements both. By default, sed uses the simple BRE, but if we add the –r flag in GNU or –E flag in BSD, we get all the extra goodies in ERE. That’s exactly what we’ll do. Do we get an operator that says “don’t touch lines with slash-slash” in that case? I’m sorry to say we don’t. There are these things called character classes. If we write [/], we tell the engine to match a slash. And if we write [^/], we tell the engine to refuse lines with slash. But character classes only focus on one single position in the string, and your problem is to only ignore strings when they have a particular two-character-long substring: slash-slash—the evil twins. We’re just about to give up when we remember that set theory class we went to while studying mathematics at University.

Take a look at the Venn diagram. Two sets A and B have some area in common and each of them also has its unique area. Then there’s area outside both of them. Let’s use the following notation:

A: the area inside A

A': the area outside A—in other words, the complement of A

A u B: the area inside A, inside B, or inside both of them—this is the union of A and B

A n B: the area inside both A and B—we call this the intersection of A and B

(A n B)': the complement of the A and B intersection—everything except the area shared by A and B

Now, let’s say that A represents that there is a slash in position k in the source string (a line in the C code), for a given k. Let’s also say that B represents that there’s a slash in position k+1—and we’re talking about the very same k and the very same source string. We’re interested in (A n B)', which is everything except the intersection of A and B. Does this sound like gibberish? Then hear the translation into English: “The area where the following is not the case: there’s a slash in position k, as well as in position k+1.” I.e., assert it’s not two consecutive slashes.

You may not realize it, but we’re close to the solution now. Please meet Augustus De Morgan, British mathematician and logician during the 19th century. De Morgan renovated algebra and logic. He gave an inventory of the fundamental algebraic symbols (0, 1, +, - , ×, ÷, ()⁽⁾), and letters—these only, all others are derived) and he gave an inventory of algebraic laws. Wait, that sounds familiar. Isn’t there a De Morgan’s Law? Yes, indeed. As a matter of fact, there are two of them. The second one goes like this:

(A n B)' = A' u B'

Yes! The complement to the intersection in the Venn above can be called A' u B', or in plain old English: “there’s no slash in position k or there’s no slash in position k+1.” This seems like a reasonable statement when we want to assert that we don’t have slash-slash. Either there’s no slash here or else there’s no slash in the next position. If our sed regex can repeat that statement from the start of the line, up to the ending empty parenthesis, then we’ve got a candidate for replacement surgery.

Take a look at the third figure. Here’s another clock-shaped abstract machine. It solves the slash-slash problem without the lookahead-eyebulb-arm the other machine required. The square window at the top of the machine reads a tape—from left to right—consisting of the input string, i.e., the C code. As in the previous machine, there’s an NFA graph on the clock dial. And each time a new symbol is read from the tape, the clock dial rotates so that the drooping peak just below the reading window points at the current state. If the machine ends up in the state that is surrounded by a circle, then this is a perfect match. Right now the abstract machine is in the state where one slash has just been read.

In order to close this case, we need to combine the information above in a sed regex, do some testing and then add some final food for thought. First, the regex ([^/]|.[^/])* repeatedly asserts that we don’t have slash-slash. Second, we prefix it with caret, to assert it starts at the beginning of the line, then end it with backslash-escaped parentheses and finally get ^([^/]|.[^/])*\(\)$, when we also suffix it with dollar to assert that the parentheses are at the end of the line. That last regex finds all lines we want to touch—with no negative lookahead used. To touch the line is to keep what we got up to and including the left parenthesis, then add the ANSI C keyword void and finally append a right parenthesis. The back-reference symbol \1 refers to everything inside the first parentheses group in the find string:

Replace ^(([^/]|.[^/])*\()\)$ with \1void)

Let’s do some back-of-the-envelope calculation with the unix tool sed:

| ~>echo "foo()" | sed -re "s/^(([^/]|.[^/])*\()\)$/\1void)/g"
| foo(void)
| ~>echo "foo() bar" | sed -re "s/^(([^/]|.[^/])*\()\)$/\1void)/g"
| foo() bar
| ~>echo "foo(bar)" | sed -re "s/^(([^/]|.[^/])*\()\)$/\1void)/g"
| foo(bar)
| ~>echo "// foo()" | sed -re "s/^(([^/]|.[^/])*\()\)$/\1void)/g"
| // foo()
| ~>echo "foo // bar()" | sed -re "s/^(([^/]|.[^/])*\()\)$/\1void)/g"
| foo // bar()
| ~>echo "foo bar()" | sed -re "s/^(([^/]|.[^/])*\()\)$/\1void)/g"
| foo bar(void)

It looks great! Notice that the first and the sixth C lines were changed, while the second, third, fourth, and fifth were left untouched. Mission completed—and we learned some valuable lessons. The rules of Regular Expressions are based on set theory, predicate logic, automata theory, and other standard mathematics. If we have the base theory in our repertoire, we can manage even without fancy operators like negative lookahead.

See, investment in knowledge always pays the best interest. And I also want to add four final words.

Four final words: Thank you De Morgan!

Staffan Nöteberg is the author of the focus booster Pomodoro Technique Illustrated. He loves programming, baklava and Turkish coffee—in that order. And he perseveres in the idea that finite automata are the coolest gadgets ever invented.

Agile in PragPub

An Agile Retrospective

As we look back on the first ten years of Agile, these four dozen articles reflect PragPub’s Agile coverage.

In the two years of PragPub’s existence, we’ve touched on a wide variety of topics—mostly related to software development, but occasionally veering off into beer-making, dieting, and bad movies.

A lot of it, including the essay on dieting, has been Agile-themed. To be precise, we found forty-eight articles on Agile topics as we looked back over our two years. We thought you might like to have the opportunity to take a look at some of them, so we’ve put together this list, along with the links so that you can jump right into one if it sounds interesting.

Or to put it another way, here are forty-eight more articles on Agile for this, our Agile issue.

Pragmatic Life: Andy Hunt Talks about Life and Everything Else

by Andy Hunt in PragPub August 2009

A pragmatic chat with Andy Hunt about life in general.

All Your Context: You Are a Product of Your Times

by Andy Hunt in PragPub August 2010

Context is king, but context extends further than you think. There are some subtle biases that might affect your judgement.

Guru Meditation: The Right Thing at the Wrong Time and Place is Still Wrong

by Andy Hunt in PragPub September 2010

Context is everything, especially with agile methods.

Guru Meditation: The Programmer’s Rememberall

by Andy Hunt in PragPub October 2010

“Oh I’ll Remember That.” No, actually, you won’t.

Guru Meditation: Avoiding the Infinite Abyss

by Andy Hunt in PragPub November 2010

Avoiding the Infinite Abyss: constraints foster ingenuity.

Guru Meditation: The Perfect Deadline

by Andy Hunt in PragPub December 2010

The Perfect Deadline: how to avoid both cognitive shutdown and endless procrastination.

Guru Meditation: Why Johnny Can’t Be Agile

by Andy Hunt in PragPub January 2011

Why adopting agile methods might be harder than you think.

In Defense of Mediocrity: Is Your Personal Best the Enemy of Your Personal Good?

by Ben Scofield in PragPub August 2010

There are different levels of expertise, and it’s smart to know what level to aspire to.

Interrupt Mood: Simple Strategies for Avoiding Dumping Your Mental Stack

by Brian Tarbox in PragPub November 2009

Are people always interrupting you with dumb questions?

Testing as Contract: A Modest Proposal

by Brian Tarbox in PragPub March 2010

Why not use testing as a guard against changes to the design?

The Floating Finish Line: Programmer Education Should Be Less Fair and More True to Life

by Brian Tarbox in PragPub May 2010

Life isn’t fair, information is never complete, and goals change in mid-project. So why does programmer education pretend otherwise?

Better Tools and Training: A Different Spin on Static Analysis and Some Thoughts on Developer Internships

by Brian Tarbox in PragPub June 2010

Our current set of static analysis tools is working on the wrong level.

The Key to Better Software: Training, not Enforcement, Is the Key to Quality in Software

by Brian Tarbox in PragPub September 2010

The approach to quality taken by most companies is misguided and based on a flawed model. This is the reason why so many companies are having such trouble increasing the quality of the software they deliver.

Clone Yourself: Destroy Your Job Through Automation and Outsourcing

by Chad Fowler in PragPub September 2009

Thinking of your career as a system leads to some surprising conclusions.

Writing and Performing: Thoughts on Software Development as Performance

by Chris McMahon in PragPub November 2010

Agile software development has all the hallmarks of an artistic performance like those of music, theater, or dance. It requires practice; it requires rehearsal; it requires a team to be constantly negotiating their own personal and professional relationships among themselves.

Get a Life: Holiday Eating

by Daniel Steinberg in PragPub December 2009

Daniel gets a jump on his January diet with an agile approach.

Pragmatic Publishing: Dave Thomas Talks about Ebooks, Agility, and the Future of Books

by Dave Thomas in PragPub July 2009

Dave Thomas talks about the decisions behind Pragmatic Bookshelf’s comprehensive ebook program, and how book publishing is undergoing a transformation, from electronic versus paper to direct sales versus the channel.

TDD on iPhone: DIY: You Really Can Use Test-Driven Development in Creating iPhone Apps

by Eric Smith in PragPub July 2010

If you’re trying to do quality iPhone development, TDD is not optional. Fortunately, it is also not impossible.

The Red Dirt Interview: Chatting with James and Dana Gray about the Red Dirt Ruby Conference

by James Edward Gray II & Dana Gray in PragPub April 2010

Far out in the remote red dirt country of Oklahoma—well, actually in Oklahoma City—the Red Dirt Ruby Conference, a unique experience in professional education and networking.

Agile Flash Cards: Quite Possibly the Most Versatile Tool on the Planet

by Jeff Langr & Tim Ottinger in PragPub September 2010

Jeff and Tim have been capturing the wisdom of the Agile community and their own combined 20+ years of Agile and XP experience on index cards. Here they share a few Agile in a Flash cards with you.

What Agile Is Not: Dealing with Cowboys, Dogmatists, and Authoritarians

by Jeff Langr & Tim Ottinger in PragPub October 2010

Jeff and Tim shuffle the Agile deck and come up with some jokers.

Shu Ha Ri: Learn, Detach, Transcend: Steps to Agile Mastery

by Jeff Langr & Tim Ottinger in PragPub November 2010

Our well-meaning agile coach’s maturity model was based, either deliberately or subconsciously, on Shuhari, a Japanese martial arts concept involving three stages of mastery: “first learn, then detach, and finally transcend.”

Cohesive Software Design: Cohesion Makes Code Easier to Understand, Debug, and Test

by Jeff Langr & Tim Ottinger in PragPub December 2010

Those big software design concepts like coupling, cohesion, abstraction, and volatility have real practical value. In this article, Jeff and Tim talk about how cohesion can make your life as a programmer easier.

Code Coupling: Reducing Dependency in Your Code

by Jeff Langr & Tim Ottinger in PragPub January 2011

Following up on their article on cohesion, Tim and Jeff talk about what coupling is, why it’s necessary, and how you can reduce it to just that necessary amount.

Managing Your Life Projects: Surprise! You’re Making Dinner Tonight

by Johanna Rothman in PragPub August 2009

How to be more effective in your daily life by applying the principles of managing your project portfolio.

Barriers to Agility: How to Thread the Barriers to Agility when Failure Is Not an Option

by Johanna Rothman in PragPub February 2010

Johanna shows how to thread the barriers to Agility when failure is not an option for your project.

Shady Illuminations: Toward a Better Class of Manifestos

by John Shade in PragPub August 2010

John surveys the leading examples of software development manifestos and makes some suggestions.

Agile Microsoft: An Introduction to ASP.NET MVC

by Jonathan McCracken in PragPub June 2010

Microsoft’s Model-View-Controller web presentation framework is being called “Rails for .NET.”

The Indispensable Developer: How to Keep Yourself Indispensable

by Jonathan Rasmusson in PragPub June 2010

Jonathan takes a break from writing The Agile Samurai to share the four things that will make you indispensable.

Way of the Agile Warrior: Ten Questions You’d Be Crazy not to Ask at the Start of Your Project

by Jonathan Rasmusson in PragPub October 2010

It starts out so hopefully. As you begin the project, you and your team are all on the same page. Or so it seems. Then you start actually building, and you realize that you all had something different in mind. Has it happened to you?

Way of the Agile Warrior: Production Readiness

by Jonathan Rasmusson in PragPub November 2010

Once regularly producing shippable code becomes a habit to you and your team, it’ll just feel good. You are delivering something of value every week and your customer is seeing steady progress in a live system.

Way of the Agile Warrior: The Way of the Spartan Warrior

by Jonathan Rasmusson in PragPub December 2010

Jonathan explains the benefits of startin’ Spartan.

Against SEMAT: Thinking Differently about a Proposed Software Engineering Revolution

by Jorge Aranda in PragPub January 2010

You say you want a revolution? We’d all love to see the plan.

Responsive Design: Coming to Grips with the Chaos that Is Software Design

by Kent Beck in PragPub September 2009

Kent reflects on his years designing software and concludes that we need a new approach, one that embraces ambiguity and tension.

Swaine’s World: A Team of Two

by Michael Swaine in PragPub February 2010

Working with another person can change your perception of time.

Swaine’s World: Multitasking Considered Harmful

by Michael Swaine in PragPub June 2010

Does multitasking battle with our ability to turn repetitive tasks into habits?

The Quiz: Playing Cards in the Bizarro World

by Michael Swaine in PragPub September 2010

A card-based quiz inspired by Jeff Langr and Tim Ottinger’s article on Agile in a Flash.

A Troubleshooter’s Tale: Three Principles and a Big Ball of Mud

by Mike Hostetler in PragPub April 2010

Sometimes fixing the problem is the last thing you should do.

Tangled Up in Tools: What’s Wrong with Libraries, and What to Do about It

by Mike Taylor in PragPub April 2010

Surely “Just learn these 57 classes” is not the re-use we were promised?

The Virtues of Mockery: A Mock Is Like a Stub with Attitude

by Noel Rappin in PragPub May 2010

Mock objects can be very useful in test-driven design, but they need to be used wisely.

The Mikado Method: Small- and Large-Scale Refactorings

by Ola Ellnestam & Daniel Brolund in PragPub June 2010

The Mikado Method can help you find the troubling dependencies that affect any large refactoring.

Forgive and Remember: Learning from Your Mistakes without Blame

by Paul Butcher in PragPub December 2009

Paul offers some advice to make debugging a learning experience.

Great Expectations: Better Testing Through DSLs with Ruby and Scala

by Paul Butcher in PragPub August 2010

Improve your tests with custom expectations.

Why ASP.NET MVC?: An Interview with Phil Haack of Microsoft

by Phil Haack in PragPub June 2010

A chat with the Project Manager for the ASP.NET MVC project at Microsoft.

The Agile Coach: An interview with Rachel Davies and Liz Sedley

by Rachel Davies & Liz Sedley in PragPub November 2009

Agile coaches Rachel Davies and Liz Sedley share advice on coaching your Agile team.

Building Trust: The Equation for Successful Agile Teams

by Rachel Davies in PragPub September 2010

When you’re coaching Agile teams, it’s essential to work on building trust from the get-go. Without trust, Agile simply doesn’t work. So how can we build trust on our teams so that Agile practices can take root?

BDD on iPhone: iCuke: iCuke Gives iPhone the Cucumber Treatment

by Rob Holland in PragPub July 2010

In an earlier article in PragPub, Ian Dees showed how to use Cucumber to test your iPhone apps. Now iCuke makes it even easier.

The Pompatus of Pomodoro: Can a Cute Kitchen Timer Change the Way You Work?

by Staffan Nöteberg in PragPub November 2009

Make time an enabler rather than the enemy who always wins.

Calendar

Author sightings, partner events, and other notable happenings.

Author Appearances

Feb 2–4, Magic Ruby, Orlando, FL

Keynote

Chad Fowler, author of The Passionate Programmer

Feb 4–5, Magic Ruby, Orlando, FL

Speaking

Dave Thomas, Pragmatic Programmer

Feb 4–5, Agile Coaches Gathering, Madrid, Spain

Sessions

Rachel Davies, co-author of Agile Coaching

Feb 8, Cologne.js, Cologne, Germany

A talk about accessing the Arduino using a web browser.

Maik Schmidt, author of Arduino: A Quick-Start Guide

Feb 14–16, Belgium Testing Days, Brussels, Belgium

“Tutorial: Becoming a Great Test Manager” and “Keynote: Lessons Learned from 20 Years of Managing Testing”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Feb 16, Jfokus 2011, Stockholm, Sweden

“Regex—the Future Programmer’s Best Friend” (this event was moved from Feb 15)

Staffan Nöteberg, author of Pomodoro Technique Illustrated

Feb 19, MobiDevDay Detroit 2011, Detroit, MI

“iOS Multimedia APIs”

Chris Adamson, author of iPhone SDK Development

Feb 22–25, CodeRanch, online

Jeff Langr and Tim Ottinger answer your questions about their new PragProg book Agile in a Flash, at CodeRanch (aka JavaRanch) in the “Agile and Other Processes’ forum. Ask questions to be qualified to win one of four copies being given away by the publisher!

Jeff Langr and Tim Ottinger, authors of Agile in a Flash: Speed-Learning Agile Software Development

Feb 25–26, Greater Wisconsin Software Symposium, Madison, WI

“Grails Deep Dive,” “Hello! Groovy,” and “Rediscovering Apprenticeship in the 21st Century”

Dave Klein, author of Grails: A Quick-Start Guide

Feb 28, Agical Geek Night, Stockholm, Sweden

“Regex—the Future Programmer’s Best Friend”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

Mar 4–5, Codepalousa, Louisville, KY

Speaking

Jeff Cohen, co-author of Rails for .NET Developers

Mar 4–5, Twin Cities Software Symposium, Minneapolis, MN

Talks on JVM languages, concurrency, and testing

Venkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

Mar 4–6, Twin Cities Software Symposium, Minneapolis, MN

“Rediscovering Apprenticeship in the 21st Century” and “Grails: Bringing Radical Productivity to the JVM”

Dave Klein, author of Grails: A Quick-Start Guide

Mar 7–10, Pragmatic Studios, Santa Clara, CA

Pragmatic Rails II Studio with Dave Thomas

Chad Fowler, author of The Passionate Programmer

Mar 7–10, Pragmatic Studios, Santa Clara, CA

“iPhone/iPad Programming” with Matt Drance

Daniel Steinberg, co-author of iPad Programming: A Quick-Start Guide for iPhone Developers and author of Cocoa Programming: A Quick-Start Guide for Developers

Mar 7–11, QCon London, London, England

Running my Secrets of Agile Architecture tutorial on March 7 and speaking on the Enterprise Agile Transformation track (Topic TBD).

Dan North, co-author of The RSpec Book

Mar 10, Crisp Rocket Day, Stockholm, Sweden

“Regex—the Future Programmer’s Best Friend”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

Mar 10, DevLin 2011, Linköping, Sweden

“Pomodoro Technique”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

Mar 11–12, No Fluff Just Stuff, Boston, MA

Keynote plus talks on JVM languages, concurrency, and testing

Venkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

Mar 16, MountainWest RubyConf 2011, Salt Lake City, UT

“Improve Your Ruby Skills” and “Get Started With TDD in a Legacy Environment”

Noel Rappin, author of Rails Test Prescriptions

Mar 16, Turku Agile Day 011, Turku, Finland

“Regex—the Future Programmer’s Best Friend”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

Mar 17, Turku Agile Day 011, Turku, Finland

“Pomodoro Technique”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

Mar 21–22, DevNexus, Atlanta, GA

Keynote and talk on Programming Scala

Venkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

Mar 21–22, SDC, Wellington, NZ

“Keynote: Creating an Adaptable Life” and “Keynote: Who’s On Your Team?”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Mar 24–25, SDC, Sydney, Australia

“Keynote: Creating and Adaptable Life” and “Keynote: Who’s On Your Team?”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Mar 28–29, SDC, Sydney, Australia

“Working Effectively With Distributed Agile Teams”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Mar 29–31, Pragmatic Studios, Reston, VA

Teaching Rails Studio with Dave Thomas

Chad Fowler, author of The Passionate Programmer

Apr 1–2, No Fluff Just Stuff, Orlando, FL

Talks on JVM languages, concurrency, and testing

Venkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

Apr 1–3, Greater Florida Software Symposium, Orlando, FL

“Hello! Groovy,” “Rediscovering Apprenticeship in the 21st Century,” and “Grails: Bringing Radical Productivity to the JVM”

Dave Klein, author of Grails: A Quick-Start Guide

Apr 4, SDC 2011, Gothenburg, Sweden

“Regex—the Future Programmer’s Best Friend”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

Apr 4–6, iPhone/iPad DevCon East, Boston, MA

“Using Core Animation to Build Complex and Attractive Interfaces,” “iOS 4 Event Handling Using UIGestureRecognizer,” and “Practical UIKit: Creating Custom Views and Components”

Nathan Eror, co-author of Core Animation for Mac OS X and the iPhone

Apr 5–7, QAI, QUEST 2011, Boston, MA

Tutorial: “Coaching as a Two-Way Relationship,” and talk: “It’s the People; It’s Always the People”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Apr 6–8, 33degree , Krakow, Poland

Keynote plus talks on JVM languages

Venkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

Apr 7, Scottish Ruby Conference, Edinburgh, Scotland

Charity tutorial

Chad Fowler, author of The Passionate Programmer

Apr 9–10, Voices That Matter: iPhone Developers Conference, Seattle, WA

“Advanced Media Manipulation with AV Foundation”

Chris Adamson, author of iPhone SDK Development

Apr 11–13, JAX London, London, England

Keynote (Tuesday 12) on “Deliberate Discovery and Programming Patterns”

Dan North, co-author of The RSpec Book

Apr 15, GLSEC Conference, Grand Rapids, MI

Keynote: “Thriving in a Competitive Marketplace”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Apr 15–16, Southern California Software Symposium, Los Angeles, CA

“Rediscovering Apprenticeship in the 21st Century” and “Grails: Bringing Radical Productivity to the JVM”

Dave Klein, author of Grails: A Quick-Start Guide

Apr 17, Chennai Java User Group, Chennai, India

Talk on JVM Languages

Venkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

Apr 19–22, Great Indian Developer Summit, Bangalore, India

Keynote plus talks on Java, RichWeb and .NET related topics

Venkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

Apr 22–23, RedDot RubyConf, Singapore

Speaking

Chad Fowler, author of The Passionate Programmer

Apr 22–23, RedDot Ruby Conference, Singapore

Speaking

Dave Thomas, Pragmatic Programmer

Apr 26, Mongo Philly, Philadelphia, PA

Speaking on MongoDB Administration and possibly one or two other talks.

Brendan McAdams, author of the forthcoming MongoDB: A Quick Start Guide

Apr 27–28, Emerging Tech Conference, Philadelphia, PA

“Say Yes—or Say No? What to Do When Faced with the Impossible”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Apr 30, No Fluff Just Stuff, Reston, VA

Talks on JVM languages, concurrency, and testing

Venkat Subramaniam, the author of Practices of an Agile Developer, Programming Groovy, and Programming Scala

O’Reilly Events

As publishers who think of ourselves as being on the cutting edge, we’re always interested in O’Reilly’s Tools of Change conference.

Feb 1–3, Strata, Santa Clara, CA

O’Reilly Strata Conference: “Get control of the new data opportunity at Strata—immerse yourself in three full days of hands-on training, information-rich sessions, and a sponsor pavilion filled with the key players and products. This new O’Reilly conference brings together the people, tools, and technologies you need to make data work.”

Feb 14–16, TOC, New York, NY

O’Reilly Tools of Change Conference: “Join us as we explore this new world of ‘Publishing Without Boundaries’ at the fifth annual O’Reilly Tools of Change for Publishing Conference.... This premiere event provides an unparalleled opportunity for stakeholders from across the book publishing and tech industries to network, share ideas, engage in lively debate, and consider the many issues we face individually and collectively.”

Mar 28–31, TOC, San Francisco, CA

O’Reilly Web 2.0 Expo SF: “The program will spotlight experts, leaders, and under-the-radar innovations, and in the spirit of Web 2.0, there will be ample opportunity for attendees to connect, contribute, and collaborate. Web 2.0 Expo will be a place for creativity, engineering, and innovation.”

Apr 11–14, MySQL Conference & Expo, Santa Clara, CA

MySQL Conference & Expo: “The O’Reilly MySQL Conference & Expo is committed to spreading the latest and best knowledge on MySQL and related technologies to the global community of DBAs, developers, and managers. You’ll find answers to your questions about MySQL in an independent and inclusive environment. You’ll also find practical guidance to help you add value and manage costs to your project or organization.”

Apr 19–21, Where 2.0, Santa Clara, CA

O’Reilly Where 2.0 Conference: “Where 2.0 brings together the people, projects, and issues building the new technological foundations and creating value in the location ecosystem. Join with developers, technologists, CTOs, researchers, geographers, academics, business developers, and entrepreneurs to debate what’s critical now, and what’s pushing the boundaries of the location frontier.”

USENIX Events

What’s coming from our USENIX friends.

Feb 15–18, FAST ’11, San Jose, CA

9th USENIX Conference on File and Storage Technologies: “FAST ’11 brings together storage system researchers and practitioners to explore new directions in the design, implementation, evaluation, and deployment of storage systems.”

Mar 29, Hot-ICE ’11, Boston, MA

Workshop on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services: “The first Hot-ICE workshop seeks to bring together researchers and practitioners working on network and service management in the Internet, cloud, and enterprise domains. The scope of Hot-ICE includes all aspects of network and service management.”

Mar 29, LEET ’11, Boston, MA

4th USENIX Workshop on Large-Scale Exploits and Emergent Threats: “LEET aims to be a true workshop, with the twin goals of fostering the development of preliminary work and helping to unify the broad community of researchers and practitioners who focus on worms, bots, spam, spyware, phishing, DDoS, and the ever-increasing palette of large-scale Internet-based threats.”

Mar 30—Apr 1, NSDI ’11, Boston, MA

8th USENIX Symposium on Networked Systems Design and Implementation: “NSDI ’11 will focus on the design principles, implementation, and practical evaluation of large-scale networked and distributed systems.”

Apr 10–13, EuroSys11, Salzburg, Austria

European Conference on Computer Systems: “The EuroSys conference series brings together professionals from academia and industry. It has a strong focus on systems research and development: operating systems, data base systems, real-time systems and middleware for networked, distributed, parallel, or embedded computing systems. EuroSys has become a premier forum for discussing various issues of systems software research and development, including implications related to hardware and applications.”

Other Happenings

February

In February, 1986, Richard Stallman published the first official definition of Free Software. In February, 1992, Linus Torvalds first placed Linux under the Gnu Public License. In February, 1998, the Open Source Initiative was founded. And in February, 2001, at The Lodge at Snowbird ski resort in the Wasatch mountains of Utah, seventeen people wrote the Agile Software Development Manifesto.

Feb 2

On this date in 2002, the award-winning anime film “Voices of a Distant Star” is released in Japan. The 25-minute film was created entirely by one person, graphic designer Makoto Shinkai, on a Power Mac G4. [Thanks and a tip of the Pragmo Hat to Chris Adamson.]

Feb 4

Ken Thompson is 68.

Feb 5

Brad Fitzpatrick, author of memcached, is 31.

Feb 7

Leslie Lamport, creator of LaTeX, is 70.

Feb 8

MIT professor Gerald Jay Sussman is 64. Among his other accomplishments, he is a bonded locksmith, so he can stay out till quarter to three and not worry about anyone locking the door.

Feb 12

Phil Zimmerman, author of PGP, is 57.

Feb 15

Niklaus Wirth is 77.

Feb 19

CGDA Hall of Fame game developer Danielle Bunten would have been 62 today.

Feb 22

Bill Jolitz is 54.

Feb 24

The Ruby programming language was conceived on this date in 1993.

Feb 27

Grady Booch is 56.

Feb 29

Seymour Papert would be 83 this year if it were a leap year.

March

Maybe you knew that Edsger Dijkstra’s famous letter “Go To Statement Considered Harmful,” appeared in Communications of the ACM in March of 1968. But maybe you didn’t know that the phrase was not in fact Dijkstra’s, but was attached to the letter by the publication’s editor, Niklaus Wirth.

Mar 1

Yahoo (as a corporation) is 16.

Mar 3

GNOME first came out of its burrow on this date in 1999.

Mar 9

Jef Raskin would have turned 68 today.

Mar 11

J.C.R. Licklider would have turned 96 today.

Mar 13

On this date in 1986, Microsoft went public, creating four billionaires.

Mar 15

A-life pioneer Craig Reynolds, who worked on the first Tron, is 58.

Mar 15

It’s the Ides of March. Beware.

Mar 16

Richard Stallman is 58 and MINIX author Andy Tanenbaum is 67.

Mar 24

It’s Ada Lovelace Day, honoring women in technology and science.

Mar 24

Mac OS X is X today.

Mar 25

On this day in 1995, Wiki went worldwide as Ward Cunningham installed WikiWikiWeb on the Web.

Mar 26

Larry Page is 38.

Mar 31

Evan Williams, co-founder of Twitter, is 39.

Mar 31

It’s the second anniversary of Steve Wozniak’s last appearance on “Dancing with the Stars.”

Apr 4

Larry Page takes over as Google CEO.

Apr 5

SATAN’s father, Dan Farmer, is 49.

Apr 6

MacOS co-creator Andy Hertzfeld is 58.

Apr 5

Inventor Dean Kamen is 60.

Apr 9

ENIAC co-inventor J. Presper Eckert was born on this date in 1919.

Apr 9

SQLite architect D. Richard Hipp is 50.

Apr 14

Ruby designer Yukihiro Matsumoto (Matz) is 46.

Apr 18

PUGS project initiator Audrey Tang is 30.

Apr 20

Chief Yahoo David Filo is 45.

Apr 22

Psychedlic ruminant-fancying game designer Jeff Minter (Yak) is 49.

Apr 24

Champion of modelessness Larry Tesler is 66.

Apr 28

Mathematician Kurt Gödel was born on this date in 1906.

Apr 30

Information theory pioneer Claude Shannon was born on this date in 1916.

Apr 30

Colossal Cave Adventure creator Don Woods is 57.

Apr 30

The World Wide Web is 18.

Shady Illuminations

My Letter to the Editor

by John Shade

John explains the values and principle on which his writing is based.

Mike,

I like to think that by writing this column every month, I am uncovering better ways of writing satirical back-page columns for software development magazines. I hope you agree.

I also hope, as a result of our working together these past two years, that we have come to share some core values. I hope that you have learned to value me and our interaction over my actually delivering the column and putting it in the right format. I hope that you agree with me that delivering a working column, by which I mean enough text to fill at least one page in the pdf version of the magazine, is more important than comprehensively documenting any meaningful topic. I hope that we have come to an understanding that your adopting a collaborative attitude toward my compensation is preferable to the usual boring ritual of contract negotiation. And I hope that I have convinced you that I am always going to react to ever-changing circumstances rather than following any plan you may have for me, and there’s not a darned thing you can do about it.

Good so far? Great, then moving on, I’m sure we can agree on a few basic principles:

Our highest priority for this column, surely, is to satisfy the reader with regular, even if last-minute, delivery of my valuable thoughts. At least one pdf page of them.

We should welcome change, by which I mean that you should be happy when I throw out the column I told you I was going to write and substitute something else at the last minute.

Thirty days is a perfectly reasonable delivery cycle. You really need to get over this idea that you need to see a draft any time before the last possible moment, because, seriously, it’s not going to happen.

It is desirable that you work with me daily throughout the writing of the column, specifically by not bugging me for the column.

Motivated individuals do great work. So send me goodies. I need an iPad. But wait until the new version is out. What I’m saying is, if you give me the environment and support I need, including that iPad, you can trust me to get the job done. So if you want to see next month’s column, you know what to do.

Face-to-face conversation is a highly efficient way of sharing information. If I need any information from you, I’ll drop by your office for a face-to-face. Also if I need an iPad, which I do. If you need something from me, email me. Or just trust that you’ll get it; that would be better.

Clear, unambiguous measures of progress are critical to projects. Filling up that pdf page should be the primary measure of progress on my column.

I guess what I’m doing here is describing a process, although let’s not call it a process, because that’s a bad word. But it’s a process that I believe promotes sustainable column writing. By which I mean, if you let me do it my way, I can keep up the pace indefinitely.

Trust me, Mike, I’m continuously thinking about excellence in my work, even when you think I’m procrastinating. And actually you’re right, I am procrastinating. I’m procrastinating so that I don’t do something non-excellent.

Somewhere I read that simplicity, defined as the art of maximizing the amount of work not done, is essential. I’m so there.

Also, it’s well established that the best work emerges from self-organizing teams, so let’s move beyond this idea you have that you should set my deadlines for me, OK?

Finally, I think you’ll agree that it’s good to reflect periodically on how to be effective at one’s job. And if I can do that while at the same time delivering this month’s column, that’s a win-win, right? It’s—what would you call it? Nimble? Spry? Something like that.

-John

John Shade was born under a cloud in Montreux, Switzerland, in 1962. Subsequent internment in a series of obscure institutions of ostensibly higher learning did nothing to brighten his outlook. When informed that this was our Agile Issue, he said that everything he knows about agility he learned from watching expert practitioners of three-card monte.

Send the author your feedback or discuss the article in the magazine forum.