PragPub 2011-10: Issue #28

Table of Contents

Features

Lightsabers, Time Machines, & Other Automation Heuristics

by Adam Goucher

Automation, like all of testing, is an inherently heuristic activity. Adam reveals some of the most powerful heuristics of automated testing.

Open Source Community Values

by Jeff Cohen

When you get a new job, you often receive a handbook. Don’t open source communities also need a handbook of community values?

Scala for the Intrigued

by Venkat Subramaniam

In this second installment of this series on the Scala programming language, Venkat shows how Scala’s static typing leads to low ceremony programming.

But We Have These Distributed Folks

by Tim Ottinger, Jeff Langr

Is “distributed agile” an oxymoron? Tim and Jeff explain how to deal with the costs of going distributed.

When Did That Happen?

by Dan Wohlbruck

On another journey into the history of technology, Dan shows that a lot can be done with string, rubber bands, and a macaroni box.

Departments

Up Front

by Michael Swaine

What’s PragProWriMo? Do open source communities need a handbook? And other burning questions.

Choice Bits

For every question in twitterspace, somebody has tweeted an answer. Just not necessarily to that question.

Meet the Team

Meet Susannah Pfalzer, Pragmatic Bookshelf’s Managing Editor.

Shady Illuminations

by John Shade

John shares his values, and they turn out to include pessimism, procrastination, and paranoia.

Calendar

Author sightings, upcoming conferences, and other events of note.

But Wait, There’s More...

Coming attractions and where to go from here.

Except where otherwise indicated, entire contents copyright © 2011 The Pragmatic Programmers.

Feel free to distribute this magazine (in whole, and for free) to anyone you want. However, you may not sell this magazine or its content, nor extract and use more than a paragraph of content in some other publication without our permission.

Published monthly in PDF, mobi, and epub formats by The Pragmatic Programmers, LLC, Dallas, TX, and Raleigh, NC. E-Mail support@pragprog.com, phone +1-800-699-7764. The editor is Michael Swaine (mailto:michael@pragprog.com). Visit us at http://pragprog.com

ISSN: 1948-3562

Up Front

It’s PragProWriMo Time!

by Michael Swaine

We’re counting down to our annual orgy of authorship, when we invite you to write that book.

Lots of good stuff this month. Adam Goucher on automation, Jeff Cohen on developer community values, Tim Ottinger and Jeff Langr on distributed computing, and Dan Wohlbruck on tech history. Venkat Subramaniam continues his series on the Scala language. John Shade has a little list. And in the latest installment of our series of staff profiles, you get to meet another member of our team.

You’ll find something of interest in this issue, I’m sure. But now let me tell you about a little program we’ve got going on next month.

We call it PragProWriMo, short for Pragmatic Programmers Writing Month, and this November will be our third PragProWriMo. We were inspired by (we ripped off) NaNoWriMo, National Novel Writing Month.

PragProWriMo is all about helping you write that technical book you’d really like to write but for some reason haven’t been able to get started.

We’ll provide supporting materials and encouragement and all you have to do is to write 60 pages toward that book during the month of November.

To help you along, we’re setting up a forum and a Twitter account. Follow us on Twitter at @pragprowrimo to stay up to date. Join the forum at forums.pragprog.com/forums/190 for more detailed writing advice, answers to your writing questions, and progress reports from participants. And when you finish your 60 pages, you might even get some special recognition from us.

And of course we’d love for you to submit a proposal for that book to us. But it’s your work. You can publish it for free, you can do print on demand, you can hide it from the world and keep it to yourself, or you can take it to another publisher. What we’re really trying to do is to help you write the book you’ve always wanted to write.

Choice Bits

What’s hot on the bookshelf and what’s new in twitterspace.

For every question in twitterspace, somebody has tweeted an answer. Just not necessarily to that question.

What’s Hot

Top-Ten lists are passé—ours goes to 11. These are the top titles that folks are interested in currently, along with their rank from last month. This is based solely on direct sales from our online store.

1^ | 6 | Agile Web Development with Rails
2^ | NEW | Technical Blogging
3^ | NEW | Programming Clojure
4v | 1 | CoffeeScript
5v | 4 | Web Development Recipes
6v | 2 | The Cucumber Book
7^ | 8 | Programming Ruby 1.9
8^ | NEW | Programming Concurrency on the JVM
9^ | 10 | Pragmatic Guide to Sass
10v | 9 | The RSpec Book
11^ | NEW | Cutting an Agile Groove

You Learn Something New Every Day

Today I learned that there are people whose job it is to play with baby otters all day. — @lxt

Apparently people are moving to West Virginia to flee wi-fi and cell phones: gaw.kr/p6H7aB — @kitalooclef

100% test coverage does not help you find a failure to increment a loop variable. — @headius

Magical: mysterious paper sculptures appearing in Edinburgh: bit.ly/n9zaV9 /via @harryharrold @finiteattention /cc @timoreilly — @SarahM

Paris apartment unlocked after 70 years, a time capsule from 1940. http://ow.ly/6upVG /cc: @studioloraine @voocreative — @irishgirl

Sometimes it’s good to see all 24 hours of a day. Gives you a chance to thoroughly evaluate it. — @chadfowler

You know what’s impossible? Finding a non-skanky Halloween costume. #firstworldproblem — @smoozle

If This Then That

IFTTT may be the hot new cloud app, but If This Then That is nothing new in Twitterspace.

If You Already Hate Windows 8 Then You Hate Technology. — @Gizmodo

If iOS is Post-PC OS then Windows 8 is Post-Post-PC OS. — @TaimurAsad

If I didn’t work in an English Dept., I wouldn’t have to worry about the office coffee lounge randomly closing for a 2-hour poetry reading. — @rjreporter

Yes, IFTTT is a long-established meme:

“If particular care & attention is not paid to the ladies, we are determined to foment a rebellion."- Abigail Adams, 1776 http://ow.ly/6vvgl — @msmagazine

If If-Then has come to the cloud, can Case statements be far behind?

If plan A fails, just remember you still have 25 letters left. — @TEFBOYZ

Cue an Eh?

For every question in twitterspace, somebody has tweeted an answer. Not necessarily to that question, but we can fix that.

How many “best practices” are really coping mechanisms for poor practices? — @tottinge

In startup we practice all of Agile best practice: standoff meeting, backburner chart, and automated trolling. — @DEVOPS_BORAT

After accepting the first few people, I’ve never gone back to Google+. Am I missing out on something? — @marick

I forgot about Google Plus quicker than I forgot about Dre. — @inthefade

Even elephants have forgotten about Google Plus. — @ladymisskate

Always wonder why the grubhub cartoon guy is naked in this flow chart. img.skitch.com/20110914-e7cu1… — @amyhoy

I’ve been neglecting grubhub and netflix lately. Today seems to be a perfect day to make up for lost times. — @JenneNelson

Hi! I can help you with that, can you DM me your order number? — @GrubHub

@GrubHub Putting the “buzz” in busy since 1994! — @coreyhaines

It’s just coincidence that the page on Amazon showing a new kids’ book is also displaying a hip flask, right? — @KentBeck

I don’t like only ordering one thing because it’s not worth the postage so I’m going to buy a hip flask. — @LibbyJenkinson

Umm, so why does the city of Toronto need remote control lawnmowers? — @adamgoucher

My backyard here in Toronto looks like a jungle someone needs to cut the grass man! — @antony_671

Aw c’mon, you’re embarrassing us.

“@jlangr: new PragPub article, “The Only Agile Tools You’ll Ever Need” lnkd.in/YK_Jrn” [Right on!] — @RonJeffries

Developer and don’t read PragPub Magazine? If that’s the case then fix that now. bit.ly/oJpDbE #pragprog — @jockeholm

“You Don’t Pick Scala, Scala Has to Pick You.” Added PragPub#27 to Kindle, excited about @venkat_s’s new series about Scala. — @longyiqi

Wow, the @pragpub was pretty fantastic this month. I enjoyed the Scala articles, without being a Scala guy, and the agile tools article. — @JEG2

Did you know that Dr Dobbs hero Michael Swaine is chief editor of PragPub? A wonderful free mag by the PragProg guys! pragprog.com/magazines — @mtevruchte

When I see a book published by The Pragmatic Programmers, it’s very likely it’s a must read book. I’m very happy there is such a series. — @screencustoms

Post Partum

When I’ve gotten something out the door, and experiencing mixed feelings of euphoria and nothingness—that’s my cue to start reflecting. — @ellnestam

I love coming back to code I wrote a few months ago and having to rewrite big parts of it. It means I learned something since them. :) — @jaredrichardson

Yup. Legacy code is anything you wrote more than a few weeks ago... — @PeterBell

OH: “The guy whose code I’m cleaning is me 45 seconds ago... and that guy’s an idiot.” — @tottinge

Who Are Those Guys?

We told you last month, they’re not all guys. This month we followed Taimur Asad, Kent Beck, Peter Bell, DevOps Borat, TEF BOYZ, Ola Ellnestam, Chad Fowler, Gizmodo, Adam Goucher, James Edward Gray II, GrubHub, Corey Haines, Joakim Holm, Kate Horton, Amy Hoy, Ron Jeffries, Elizabeth Jenkinson, Kristi McKinney, Ms. Magazine, Brian Marick, Sarah Milstein, Jenne Nelson, Aleksej Nesterow, Charles Nutter, Tim Ottinger, Longyi Qi, Jared Richardson, Rebecca J. Ritzel, Antony Rodriguez, susan, Laura Thomson, MC Thumbtack, Marc te Vruchte, and Meghan Wilker. All tweets are the responsibility of their respective tweeters. Some capitalization and punctuation regularized to protect the negligent. Contents sold by weight, not volume. You can follow us at www.twitter.com/pragpub.

Meet the Team

Susannah Pfalzer

Each month we profile a member of the Pragmatic Bookshelf team. This month it’s Susannah Pfalzer, our Managing Editor. With November coming up, Susannah will be facilitating our annual PragProWriMo, which is a great way for our readers to try their hands at writing a tech book or a book proposal. Just one more hat that our busy Managing Editor wears.

Susannah Pfalzer, Managing Editor

I’ve been working with the Prags since 2006. I was among the first development editors that Andy and Dave brought on. Development editors work with our authors as they write their books, guide them on the book content, style, and organization—and manage the book process from start to finish, too. We get to be very closely involved with a book as it evolves, which is really rewarding.

Now, as managing editor, I wear a few different hats. I work with the editors and authors to keep things running smoothly, train new editors, and get our new authors up to speed on our tools and processes. I also manage the new book proposals that come in to us. Plus, I keep my editing hand in by continuing to do development editing—most recently, I’ve edited New Programmer’s Survival Manual and Web Development Recipes. My roles here are so varied, I always have something going on!

As for my favorite way to spend my downtime, people tend to be surprised when they hear how much I read—after reading and editing for work, I still love to immerse myself in a good book at the end of the day. Right now I’m working my way—verrrry slowly, because they’re long books—through George R. R. Martin’s Game of Thrones series.

Email: susannah@pragprog.com

Twitter: @spfalzer

G+: Susannah Pfalzer

Lightsabers, Time Machines, & Other Automation Heuristics

Plus a Few Anti-heuristics

by Adam Goucher

Automation, like all of testing, is an inherently heuristic activity.

Heuristics are used in testing as rules of thumb or prompts for solving a particular problem or class of problems. An important characteristic of heuristics is that they are fallible in certain situations, though correct in most situations. And then there are anti-heuristics. They are fallible in most situations.

Automation, like all of testing, is an inherently heuristic activity. Some of the risks associated with it can be managed by making conscious decisions about which heuristics to employ and which anti-heuristics to avoid.

This article identifies a handful of heuristics that apply to automation. While not an exhaustive set, it is a useful one, and it will put you on the path to identifying and collecting your own set of automation heuristics.

Lightsabers

In the Star Wars stories, the lightsaber is the physical tool the Jedi turn to when their primary tool, the Force, proves insufficient for the task at hand. In testing, we use automation when our primary tool, our brain, is insufficient for the task.

An important step in the apprenticeship of a young Jedi is the creation of their first lightsaber. This happens towards the end of their apprenticeship. Prior to that they use either a blunted lightsaber or one that is not attuned to their own energies.

This lack of attenuation is exactly what happens when using “generic” automation tools. It is not the vendor’s intent to be malicious or to do harm, but by their very nature these tools need to support a lot of situations to be viable in the market. That support leads to generic approaches, and that sometimes means you have to adapt the way your application is built to accomodate your automation tool.

When using a closed-source tool the challenges are even greater, as you are dependent on the vendor for any fixes to problems you might discover. And what is a problem in your context might not be one to them, so they might be unwilling to fix what they don’t see as a problem.

Just as a Jedi solves the problem of attenuation and bug fixing by building their own lightsaber, building your own automation tool or framework using Open Source components addresses these problems for your automation. No longer shackled to generic solutions to specific problems, you can use one that solves your specific problem precisely. And if some difficulty occurs, you have access to the source code and can produce a fix to it immediately.

The Lightsaber heuristic is of course fallible. A Padawan (apprentice Jedi) does not just wander out on their own and build a Lightsaber; they have a Master who guides them through the process the first time—in part to protect them from themselves. So too with building automation. Someone who does not have experience with a number of different automation tools or techniques could stumble upon something that works, but is more likely to head down false paths, run into dead ends, and end up causing more harm than good. My own current automation framework, Saunter is the 5th or 6th incarnation of these ideas and is only now at the point where it is really useful.

With Open Source, as with commercial automation, you still need to be aware of the “Know Where The Sun Is” heuristic when fixing problems you encounter. For example, the Selenium WebDriver API is Open Source and anyone can build custom copies, but if you have a question on the Java, Ruby, or Python implementations, the best way to get information is during European business hours. For C#, though, it is business hours on the east coast of the US. During WinRunner’s heyday if you tried to call in the (really) early morning North American time you hit the support center in Israel.

There will be times when technology choices dictate going with a commercial, closed source tool or framework. But teams that really succeed with automation tend to be ones that build their own tools using Open Source components.

One final thing to remember with lightsabers is that they are wielded for both good and evil purposes.

Time Machines

It is impossible to determine the Return on Investment (ROI) for automation without access to a time traveling device. This is a fairly outrageous claim to make in a world where budget justifications need to be made quarterly, but it really strikes at the core of why we use automation in the first place.

Automation is not a technique to be used to increase some metric in a spreadsheet tallying the number of scripts executed or coverage percentage. It’s not really even a technique for doing things faster than humans or that humans are not good at (like boring data input). Automation is just a technique for gathering information. And that information is consumed by humans who make decisions with it.

It is this “usage in decisions” that is the tricky bit. Consider the following situations:

Based on a series of automation results, a tester decides to investigate a particular part of the product and finds a bug.

Based on a series of automation results, a tester decides to investigate a particular part of the product and does not find a bug.

Based on a series of automation results, a tester decides not to investigate a particular part of the product and misses a bug.

Based on a series of automation results, a tester decides not to investigate a particular part of the product and does not find a bug.

Anyone who has worked in/with automation has found themselves in each of these situations. And what was the outcome? And over what time horizon is that outcome being measured? The only way to do this measurement is to look into the future at every single decision either directly or indirectly made with input from automation and see how it affected the company.

And of course, if you have a time machine at your disposal, your competitors do as well. Which means you have to check the impact of decisions against the information received from not only your automation but also from your competitor’s reactions to those decisions, and your responses to them, and....

Automation ROI Calculators, such as the one from ATI should of course themselves be considered heuristic (at best, and an example of poor metric collection and management at most cynical). But the problem with them is that they do not have an input field for “future money earned by using automation” or “future money lost by using automation.” Because both are impossible to measure—without a time machine.

The whole notion of ROI often comes up as part of the justification process for using expensive, commercial automation platforms. Building your own lightsaber can offset part of the need for a time machine, but a better solution is to approach automation with the realistic understanding that its purpose is to provide information. Some of that information will of course be useful, but some will not, just as is the case with manual testing. And exactly as you do with manual testing, when the cost of acquiring that information outweighs its value to the organization, you stop. No time machine required.

The One Ring

Tolkien’s books revolved around a magical artifact, the One Ring, which was intended for use by Sauron to control all the peoples of Middle Earth. Inscribed on the ring are two of the more famous lines from the books:

One Ring to rule them all, One Ring to find them,

One Ring to bring them all and in the darkness bind them.

With automation, this focus on One is a symptom of homogenous thinking and is an anti-heuristic. That is, it is a bad idea most of the time.

I primarily see two variations of The One Ring when talking with companies:

Automation language choice

Automation tooling choice

The automation language choice version is perhaps the most insidious and seems to happen more in companies whose primary development language is Java (which usually also means that they are large companies organizationally). But it can happen in a one-person company as well.

The question to ask in order to determine whether The One Ring has started to spread its corrupting influence on the organization is “why are you using language x for your automation?” The answer all too frequently is “because the application is written in language x.” And it may be exactly the correct choice, but too often it was made without any consideration for:

who will be creating the automation,

what their skill sets are, and

what the automation (really) needs to do.

It is not uncommon to see excellent non-programming manual testers “converted” to being writers of automation only to flail and ultimately fail. This is because automation is programming and it is a skill that needs to be learned.

It is also easy to choose the same language for both the application and automation because this should offer ease of integration and the ability to share code. But if the integration is through a REST interface then you might have fallen under the sway of The One Ring since that interface is accessible from any language with an HTTP library.

The second variation of this anti-heuristic occurs more in places that have implemented some sort of commercial Application Lifecycle Management System (ALMS) such as Microsoft’s or HP’s, but sometimes it is an in-house tool (Lightsaber) that was built by a team member. These often cost a non-small amount of money to purchase and involve significant workflow disruption to implement. Having incurred both produces a mandate to use the tool. In this situation it is important to remember that these costs, while non-trivial, are sunk costs. No matter how much you use the tool, they cannot be recovered.

An argument against calling these sunk costs is that by using the tool you are amortizing the cost of it over a larger period of time. Which is correct from a financial reporting perspective, but if you are not getting the information the business requires in a timely fashion, you are actually adding more cost to the tool by using it.

The solution is of course to throw The One Ring into the fires of Mount Doom where it was forged. But if that isn’t convenient, then honestly assessing tooling and language decisions both at the beginning of a project and periodically during it to check that they are still correct will go a long way.

The First Hit Is Free

One of the more trendy “movements” in software these days is the Lean Startup, which has a number of important and very cool ideas that any team should consider. But one idea it includes is Continuous Deployment: the process wherein code travels through a series of automated gates and makes its way to production.

On.

Every.

Commit.

The purpose and net effect of this is that new features and bug fixes reach the audience in the shortest possible time. But Continuous Deployment also removes the human testing process from the equation—a step I consider unethical. Instead, one should look at Continuous Delivery, which uses automation where it is appropriate, and manual steps when they are appropriate. And installing new builds for a functional automation run is absolutely one of those manual step places.

Consider Continuous Delivery as Continuous Deployment with ethics.

Implementing either is a disruptive event, both from a process perspective and a political one, and often meets resistance. Automation, especially of the Functional variety (Selenium, Watir, etc.) is a fantastic way to introduce Continuous Delivery into an organization through a back door.

What Continuous Delivery practices are required by Functional Automation?

Automatic Resource Provisioning: Both physical and cloud machines need to have all their packages and configurations controlled by some central means. Once this is in place, there should never be a need to log into a machine (to do maintenance).

Application Deployment: The application needs to be able to be installed and configured on the target machine without human interference. This could mean enabling “silent mode” for installers or scripts for moving files from a build server to the one being configured.

Database Management: All interactions with the database from a management perspective including schema and data changes need to be handled as migrations. This allows for testing of changes before production and removes the risk of type-o’s happening on the management console during deployment. As with provisioning, once this has been implemented there should be no need to ever issue any SQL commands into the management console.

Continuous Integration: The CI server becomes the heart and mind of the entire process. Not only does it control the entire build pipeline but it informs the user about past trips along the pipeline.

Depending on the organization, each of these will have a different effect on the efficiency of its operations, but I have found that the Database Management practice actually has a greater effect on perceived application quality than does the execution of the functional automation it is being implemented for.

Once all these pieces are in place and debugged for the automated functional tests, it is not much of a leap to implement them for the manual testing environment(s). And once any new problems are discovered in the process and resolved, then you roll it out for pre-production. And then production.

Functional automation that is executed without any intervention is fantastic in reducing the number of “broken builds” that end up in front of humans. Not having to rely on the Operations team to constantly prepare and maintain the testing machines also means more efficiency. And it just so happens that the techniques to do this are the same ones that are needed to do automated pushes to production, but by doing it in the early environment the functional automation acts as a gateway drug towards Continuous Delivery.

Shanhaiguan

Shanhaiguan is the city in China where the Great Wall meets the Pacific Ocean and is an anti-heuristic that deals with the most challenging part of automation—the people writing it.

Walls serve two primary purposes; to keep people out and to keep people in. In too many organization charts they are also used to consolidate control. In every such situation, it comes at the expense of communication. And communication between parts of the organization attempting automation is vitally important to the success of the effort.

Automation requires the entire team to be successful.

Product Owners: They need to communicate their desires for what the product should do. Without their initial expression of capability there is nothing to compare against to ensure that the scripts do what they are supposed to do.

Application Developers: Today’s applications are too complex not to have automation hooks provided by the development team. They may not know which hooks are needed, though. Whoever is automating needs to be able to talk to the developer to request these—and have those requests be honored.

Operations: Even though the configuration and deployment of hardware and software resources throughout the build pipeline is automated, someone needs to maintain that automation.

Script Developers: The hub of the automation wheel, they need to be able to talk to anyone in the organization when it relates to getting the automation as effective as possible.

Being able to succinctly place people into these buckets could itself a symptom of other organizational disfunction. In the Agile community there is the Whole Team concept where everyone involved in a product is a member of the product team first and their “other” role second. The difference between identifying as “a scripter for project x” and “a member of project x who happens to script” is subtle, but important. Everything done on a project should be done for the project’s wellbeing. Including automation.

When everyone is working for the betterment of the project as a single team, there tend to be fewer walls in place between people of various specialties. But even when there are fiefdoms in which communication in discouraged by formal processes, informal communication will eventually start to sneak into the system either at the proverbial water cooler or via instant messaging or email. One just needs to be aware that those imposing the restrictions will sometimes not look fondly upon them being avoided. So weigh the risks—and rewards.

If walls, either physical or imaginary, are placed between the team members, their ability to communicate is impaired. But like the Great Wall, these walls can be worked around—it just might be extremely unproductive. And you might get your feet wet.

Switches

Adding the ability to switch something on or off depending on a configuration setting is not a new technique, but it is experiencing a revival from the DevOps movement, as it can decouple production releases from feature availability. But when you start to bring automation into the mix, it is useful to put switches on everything.

Especially third-party content.

The primary goal of (functional) automation is to gather information about the current behavior of the application. Too often, though, that information gathering is held hostage by a slow or unresponsive third-party widget or advertisement. Rather than being a victim it makes sense to flip the switch that disables that widget in the automation environment.

Simply disabling the loading of something might throw off the overall look and feel of the application, though, if the design relies on a certain size of item in a certain location. For situations like that we need a smarter switch. In the case of an ad or widget the server could present to the user a static, local image in place of the expected content. Now the speed penalty of the third-party content is not paid but the application looks the way the designer intended.

Another type of smart switch is one that replaces the behavior that will end up in production with one that is useful for automation perspectives. Consider the automation required to verify the error messages for a typical payment flow. There are a lot of scenarios that not only require the third-party credit card processor to be available but also presumes that you can trigger all the possible responses from within the scripts. Both are pretty big assumptions to be hanging an automation attempt on. A better option would be have a switch that can be thrown that will respond with a predetermined behavior given a specific input (such as a CVV code in the credit card example). With that in place you can easily automate checking of all the error conditions even when the payment provider’s system is unavailable.

When doing this sort of switch, however, it is especially important that your automation do a full verification of the event. If it does not, you could find yourself pushing to production with the switch thrown (and that’s not good; trust me) and no obvious indicator that key parts of the flow are being short circuited.

Switches are often also a leading indicator of whether or not an application has been built using TDD techniques. These switches and their ability to isolate and control functionality constitute one of the tricks used to get a high degree of coverage by the unit tests. As a result, applications built via TDD tend to be easier to automate than those that are not.

The Second Noble Truth

One of the core principles of Buddhist philosophy are the Four Noble Truths. The second of these is roughly translated as

The origin of suffering is attachment.

Yet when it comes to automation, attachment is common. And then we wonder why we suffer.

The first type of attachment involves the actual hardware that the automation runs on/against. Configuring machines is a time-consuming activity, and once it works you don’t want to sacrifice the time it takes to re-do it. But this leads to compromises in machine freshness, patch level compliance, and quick hacks directly made to the machine. Thankfully, automated configuration management (see “The First Hit Is Free” above) provides relief from that pain, and so from the attachment.

The second revolves around functional automation scripts and is far more engrained in teams that do automation.

If the point of testing is to gather information and thus the point of automation is to facilitate the gathering of information, then the value of the automation drops significantly when there is no more information to be gathered. You should release yourself from the attachment to the script and delete it from the repository.

With functional automation, the part of the process that generates the most value for the organization is the creation of the script itself—not the however-many-thousand times it subsequently gets executed. It is during the creation of the script that the author is actively engaged with exploring the system, following data, asking questions of the product and development teams, etc. It is not surprising then that this is when the most bugs are found. And when bug fixes have unit tests created for the fixes, then the creation step of a script will likely be the only time a bug is found (by the script).

If a script is unlikely to provide new information once it has been created, why keep running it on every commit? Or at all? The answer is attachment.

Baby Steps

“Baby steps get on the bus, baby steps down the aisle, baby steps....” - Bob Wiley

In the movie What About Bob, Bill Murray’s character comes to grips with his OCD by latching onto the mantra of “Baby Steps”—taking life in small, manageable slices only as they occur. It is important to approach automation projects with this mantra as well.

Consider the activity of creating Page Objects for an application. It is tempting to go off and create POs for every element and every action on every page and then just glue them together in interesting ways through scripts. This is the very opposite of the Baby Steps approach and pushes out the time when the scripts can produce information (much in the same way that excessive test documentation prevents actual testing).

Rather than building everything out at once, create the script and the PO(s) in tandem as you need them. By taking this approach you are getting information from the automation immediately. Which is, again, the purpose of this kind of automation. It also means that isolating problems both discovered by and created by the automation is much easier as the changes are smaller.

The Baby Steps approach also minimizes the amount of re-work necessary when something changes, as the PO and the application are in sync from the start.

To some degree, this heuristic brings to automation the YAGNI and iterative development that have been popularized by Agile.

Don’t Kill Unicorns

There are certain things—like killing unicorns or making baby pandas sad—that you really should not do. When creating automation there are lots of unicorns around, but one that I keep seeing is the automation of sites that are not under your (company’s) control.

The scenario this most often comes up in is a registration flow for applications in which an email is sent containing a verification link the user is supposed to click. Automating the first part is simple, as is the last part, but the retrieval of the verification link is where unicorns get involved. And often takes the form of automating GMail or similar sites.

The only people who should automate GMail are those employed by Google and working on the Gmail team. Anyone else is killing unicorns.

There are two ways to avoid killing a unicorn in this particular situation.

Access the site’s POP3 or IMAP interface directly via your language’s libraries. Both are standardized protocols and this removes any interaction with the interface from your scripts. Instead, it is just another service the script is using.

Reach into the application’s database to retrieve the code and build the link in the script. Your application’s server has no way of knowing if the link originated from a physical click inside an email or a select statement in the code. Nor does it care.

Both these solutions will work, though the second one is preferred as it removes the requirement of the third-party system being available and behaving in the expected way.

Aside from automating third-party services, another unicorn comes in the form of automating things that are more efficient to do manually such as page layout/content testing.

There are services that can do this for you, but it is somewhat akin to hiring a Big Game Unicorn Hunter: you get the trophy but don’t get your hands dirty.

Using a Switch to disable third-party items in the application is a good first step to avoid the needless slaughter of magical creatures. Constantly reflecting on whether you should be automating something, however, remains the most effective method of preventing wholesale slaughter of unicorns.

If you meet the Buddha

“If you meet the Buddha, kill him.” - Zen Master Linji

This Buddhist koan does not of course mean that you should literally commit murder. What it is saying is that if you believe you have a correct image of the Buddha or the nature of Enlightenment then you should kill that idea, as it is false.

Killing false ideas is important in all testing, including automation. Recall that both activities are heuristic based. Which means that any “best practices” or “expert advice” is fallible. It is therefore important to constantly look at the path you are on and determine whether the practice is one that you should continue, or should kill. And that includes every single heuristic mentioned in this paper.

But of course, even that is a heuristic.

Adam Goucher has been testing professionally for over 12 years at a range of organizations from national banks to startups. A large part of that time has been spent augmenting his exploratory testing with automation. He is the maintainer of Selenium IDE and consults on automation through his company, Element 34.

Send the author your feedback or discuss the article in the magazine forum.

Open Source Community Values

Welcome to Our Community. Here Are the Ground Rules.

by Jeff Cohen

When you get a new job, you often receive a handbook. Don’t open source communities also need a handbook of community values?

When you get a new job, you often receive a company handbook. The purpose of the handbook is to articulate commonly-held expectations and set forth various guidelines. These are (supposedly) intended for the common good of everyone in the company. The very existence of the handbook says to everyone, “We have some ground rules around here.” In many companies, your agreement to comply with the handbook is so important that you have to physically sign a piece of paper promising that you will join the company's culture and act accordingly.

In the land of open source, we don't all belong to a single organization. Instead, multitudes of smaller “communities” are formed. Like organizations, they have a set of cultural norms. However, they don't usually come with their own written handbook. The unique expectations and guidelines often go unspoken. And this is where the trouble can start.

In 2005, I made a rather brisk career change and left an established career centered around Microsoft technologies. Intrigued by the Rails framework, I began learning about open source languages and found myself knocking on the door of the Ruby community. I soon recognized that my behavior and approach inside an open source community was very different than the place I had come from. It was not better or worse, just different.

Fortunately, once I figured out how to adjust, I found myself in the middle of a rather exhilarating experience. At its best, open developer communities are a joy to be a part of. Unfortunately, I've also seen many folks have a hard time integrating themselves, and it can sometimes cause havoc on both the newcomer and the established veterans.

More Than Code

One of the first lessons I had to learn is that any developer who wants to be part of a community is judged on more than their code. You are not your code. You can be an awesome coder and solve difficult technical challenges, and still fall short of the unspoken expectations and unwritten guidelines held by the community at large.

Most online communities are characterized by their use of a mailing list, or Google Groups, or IRC channel. The Ruby community has had a long-held reputation of being a “nice” community. There's a cultural value that says that a developer's standing in the community is based on more than just their raw coding skills. You're expected to be nice to others.

If you choose to answer a question on the mailing list, it's to be done with professional courtesy regardless of how many times that particular question has been asked before.

The same goes if you’re the one seeking help: be considerate. By carefully watching other conversations on the Ruby mailing list, I learned that before I ask any question, I should:

First look for the answer myself.

If I can't find the answer, then I should try some experiments on my own and see what I can figure out.

If I'm totally stuck, then I can submit a question to the mailing list, enumerating everything I have tried so far.

If I need to show some of my code, I should boil it down to the smallest code sample possible that reproduces my problem.

Remember, a community's online group is not your lazyweb. Respect everyone else's time as much as you want others to respect yours.

Don't Be a Consumer

There's one aspect of my involvement with open source communities that took me a while to figure out: don't act like a consumer.

Consumers pay for products or services, and expect a certain level of support in return. They expect software releases on a predictable schedule. They expect those who are writing the software to do what their users want. None of these expectations are appropriate in open source. If you want a certain feature, don't wait for someone else to build it, complaining all the while until it's finished. Instead, write it yourself and contribute it.

Contributions go beyond just code, too. You can contribute by writing documentation. You can spend time answering questions on the mailing list, Google Group, or IRC channel. You can help that next release by going through a list of open issues, finding reproducible steps, and fixing bugs.

Open source works when those who benefit from it “pay” for it with their time and energy. When that next version of your favorite framework didn't ship when you expected it to, ask yourself: did I help get it ready for release, or was I acting like a spoiled consumer, expecting other people to work for free on my behalf?

Repository Responsibility

Recently, social networking and source code control seem to have met in a particle collider, giving birth to sites like GitHub and BitBucket. These sites make it possible for individual developers to publish their code as first-class, open source projects, with the same online presence, functionality, and potential as any other project. I can upload my HockeyStats framework to GitHub this afternoon, and folks around the planet can discover it, access it, contribute to it, manage it, and fork it. My tiny project would have seemingly equal footing with the largest and most popular projects on the web. This is one of the things that makes open source so powerful, and why sites like GitHub have been a tremendous boon for the open source movement at large.

There's a growing trend nowadays for employers to judge applicants based on their quantity (and hopefully, quality) of GitHub projects. The notion that working code trumps résumé acronyms is something I think we can all get behind.

Often repositories are pushed to GitHub, but activity ceases soon after. Perhaps the project wasn't as worthwhile or as interesting as was first thought. Without active maintenance, all projects fall out of date and decay. They become incompatible with newer versions of some dependent components, or simply become obsolete in the presence of similar efforts that are more active and gain a larger following.

These two phenomena can sometimes combine. Developers want to post as many projects as they can to demonstrate their coding skills to potential employers, but they don't want to assume the role of a dedicated maintainer. Rather than enjoy the benefits of more active destinations, such projects become desolate islands, rarely visited by anyone except those perusing the developer's new “résumé.”

I think some responsibility rests on the shoulders of each developer who publishes a public repository:

When you open your code and post it publicly on a site like GitHub, be ready for contributions. If you don't want pull requests, say so in your README file.

If you will accept requests, be sure to actively maintain your project and acknowledge pull requests. You don't need to accept every request, but you do need to respond in a timely and professional manner one way or the other.

Provide documentation, at least a starter set, to help others understand your code. In turn, this will enable others to contribute improvements and new features. Remember, you'll be receiving these benefits for free.

Don't trash-talk other people's code. If you think your project is better, that's great. You can still tout your project without mentioning or criticizing someone else's hard work.

I also believe that contributing to active, maintained projects should be valued above forking and creating a near-duplicate project. If the author of the project isn't responsive to pull requests or patches, then by all means fork the project and start your own. But this should be a last resort. Everyone benefits from actively maintained and open projects; few benefit from a graveyard of similar-yet-different projects that violate the spirit of community collaboration.

Reach Out, and Reach Back

Finally, I've found that the community members I look up to follow a pattern I call “Reach Out, and Reach Back.”

Beyond coding contributions, they find ways to reach out to those who might otherwise feel excluded from the community. Minorities and women have historically had a near-impossible time finding their rightful place in open source. What can you do to bridge the gap?

They are also willing to cheerfully help newcomers. It's easy for those of us who are entrenched to forget what it was like to be new. But we were all new once, and we were helped by those who came before us.

Programming communities are made of real people, and you're invited to become a vital part. This can lead to a sense of fulfillment that goes way beyond what any company handbook could ever hope to offer.

Jeff has worked with object-oriented programming languages and application frameworks for more than 15 years. In 2008 he started Purple Workshops, conducting beginner-level workshops in Ruby, Rails, and agile software techniques. He co-wrote the book Rails for .NET Developers; speaks at technical conferences throughout the year; and is a Ruby on Rails contributor, teacher, and writer. This fall he begins a new adventure as the head instructor for Code Academy in Chicago.

Send the author your feedback or discuss the article in the magazine forum.

Scala for the Intrigued

Sensible Typing and Optional Items

by Venkat Subramaniam

In this second installment of this series on the Scala programming language, Venkat shows how Scala’s static typing leads to low ceremony programming.

In this second part of the series we will look at the statically typed nature of Scala and its low ceremony.

Java programmers are used to static typing. Scala is also statically typed. However, it employs type inference to figure out types at compile time. So for the most part, you don’t have to specify type information. While Scala provides type inference, it does not go to extremes to determine the type (like F#, for example, which will analyze the method body to figure out types of parameters). Scala type inference reduces the number of keystrokes you have to type and reduces noise in code without compromising readability or understandability.

Let’s first look at how we’d define a variable, set its type, and assign it an initial value. Then we can look at what type inference buys us.

var greet : String = "hello"

You use var to define a mutable variable, a variable whose value can change (true to its varying nature). When defining a variable, use a colon to separate the name of the variable from its type. Finally you can assign a value to the variable using the all-too-familiar equals operator.

Let’s extend that example to change the variable’s value and print the new value.

| var greet : String = "hello"
| println(greet)
| greet = "howdy"
| println(greet)

We initialized the variable greet to “hello” and then changed it to “howdy.” The output from the code is shown below:

| hello
| howdy

At this point if you listen keenly you may hear Scala laughing at the redundancy in typing. Scala already knows that greet is of type String from the fact that its value was initialized to “hello.” So, the : String part is redundant, and you can omit it:

| var greet = "hello"
| println(greet)
| greet = "howdy"
| println(greet)

Scala infers the type of the variables at compile time. If Scala notices any ambiguity or errors, it will report right away and will not run any part of the code.

Let’s see what happens if we try to set a value of the greet variable to something not compatible with its type:

| var greet = "hello"
| println(greet)
| greet = 1
| println(greet)

When you try to run this script, you will get an error:

| error: type mismatch;
| found : Int(1)
| required: java.lang.String
| greet = 1

Scala is quick to tell us that the value 1 we’re assigning to the variable does not match with the type String, the type which it inferred at the point of declaration.

Scala’s type inference plays well far beyond simple variable declarations, too. For instance, in the declaration

val numbers : List[Int] = List(1, 2, 3)

the type declaration is redundant and you can write it as:

val numbers = List(1, 2, 3)

I must acknowledge that this can be a blessing or a curse, depending on how you look at it. numbers is inferred as a List[Int] based on the values you present to the list. If all the values are of the same type, then this is not a concern. However, if you mix the values of different type, you will not get any error. For example:

val list1 : List[Int] = List(1, "2", 3)

will result in an error:

| error: type mismatch;
| found : java.lang.String("2")
| required: Int
| val list1 : List[Int] = List(1, "2", 3)

However, if you leave out the type in the declaration, as in:

val list1 = List(1, "2", 3)

you will not get any errors and the type of the list1 reference will be inferred as List[Any]. If you did not expect that and you mixed the type for the value in error, the type inference will not have alerted you and will not have acted in your favor. So use caution when relying on type inference.

Type inference is also convenient when defining function values. We will discuss function values later in this series, but for now think of them as anonymous code blocks. To iterate over a list of numbers you can pass a function value to the internal iterator:

numbers.foreach { e : Int => print(e) }

With numbers defined as List(1, 2, 3), this code will print

123

as output. Scala is performing type checking to ensure that the type of the parameter e in the anonymous code block (function value) matches the type of the elements in the list numbers. If you declare a different type, you will get a clear compilation error, as in:

numbers.foreach { e : Double => print(e) }

and the error reported by the compiler is:

| error: type mismatch;
| found : (Double) => Unit
| required: (Int) => ?
| numbers.foreach { e : Double => print(e) }
| ^

Scala reports that it found a parameter of type Double when Int was required.

You can avoid this journey entirely by letting Scala infer the type for the parameter of the function value:

numbers.foreach { e => print(e) }

This code will also produce the output

123

however it is less noisy, since we let Scala infer the type. At this point Scala checks the signature of the function that the foreach function expects as parameter and infers the type of the parameter e to be the same. You will appreciate the benefit this conciseness offers even more when the anonymous code block has more parameters.

If you make a mistake and treat the parameter as some other type, Scala will give you an error at the point of usage. For example, suppose you mistook the parameter to be a String and decided to call the length method on it:

numbers.foreach { e => println(e.length) }

Scala will give you an error that clearly explains the problem as it sees it:

error: value length is not a member of Int

When defining anonymous code blocks or function values, you can safely leave out the type declaration. When you are defining functions or methods, however, you’ll find that Scala requires you to specify the type. Where possible, I prefer to provide meaningful variable names and let the context tell me what the type is, allowing Scala to infer the type. Coming from Java you may find that this takes a bit of getting used to, but once you get a handle on it, you’ll find the reduced noise quite refreshing.

We saw how specifying the type is optional as Scala infers the type for us wherever possible. Quite a few more things are optional in Scala, depending on the context:

class

dot and parenthesis

parameter names

return

semicolon

try-catch

Let’s quickly look at each of these.

optional class: In the first article of this series we saw how writing a class is optional. Scala allows you to write simple scripts and does not force you to create a class.

optional dot and parameter: We’re used to following a target object with a dot to indicate method invocation, as in lower.to(upper);. If we follow that convention, a simple loop will look like this:

for(index <- lower.to(upper)) { print(index) }

However, that’s quite noisy and not as readable as the following:

for(index <- lower to upper) { print(index) }

When invoking methods on instances, you may drop the dot and the parentheses and enjoy the added fluency that Scala provides.

optional parameter names: When defining simple function values or code blocks, if you are merely passing the parameters to another function, you can let Scala do that for you. For example, instead of writing

numbers.foreach { e => println(e) }

you can write

numbers.foreach { println }

Scala analyzes the parameter your code block should receive and analyzes the function you call in it and chains the parameter of your code block as the arguments for the function you call within the code block.

You can go a step further and make use of the optional dot to write

numbers foreach println

optional return: The last expression executed within a function is returned and you don’t have to place an explicit return. So the following

def doubleValue(number : Int) : Int = return number * 2

is equivalent to

def doubleValue(number : Int) : Int = number * 2

Scala punishes us a little for placing the return by requiring us to specify the return type of the function. However, if you drop the return keyword, you can also drop the type information:

def doubleValue(number : Int) = number * 2

optional semicolon: You’re not required to end each statement or expression with a semicolon, Scala inserts semicolons automatically. Unlike Java code, Scala code usually has few semicolons.

optional try-catch: The Scala compiler does not distinguish checked vs. unchecked exceptions. If you’d like to handle an exception, place a try-catch block around the piece of code. If you’d rather let the caller of your code deal with it, simply do nothing and the exception will propagate to the caller. This eliminates clutter in the code and also avoids the temptation to suppress exceptions to silence the compiler.

When programming in Scala you will see these optional items interplay nicely to reduce code and clutter and make the code more expressive.

Until next time, I hope you play with Scala to enjoy its expressive and concise style.

Dr. Venkat Subramaniam is an award-winning author, founder of Agile Developer, Inc., and an adjunct faculty at the University of Houston.

He has trained and mentored thousands of software developers in the US, Canada, Europe, and Asia, and is a regularly-invited speaker at several international conferences. Venkat helps his clients effectively apply and succeed with agile practices on their software projects.

Venkat is the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy: Dynamic Productivity for the Java Developer and Programming Scala: Tackle Multi-Core Complexity on the Java Virtual Machine. His latest book is Programming Concurrency on the JVM: Mastering synchronization, STM, and Actors.

Send the author your feedback or discuss the article in the magazine forum.

But We Have These Distributed Folks

Can Distributed Teams Be Effective

by Tim Ottinger, Jeff Langr

Have you worked on a distributed team where management apparently thought it should hobble local members to make everybody equally frustrated and ineffective?

Last month, we discussed appropriate reasons for acquiring an agile project management tool. One common reason is to help consolidate project information for environments with distributed team members. Our controversial take is that you’d have one less reason to invest in an agile PM tool if you had no distributed team members.

Quoting the Agile Manifesto again this month, we reiterate that its very first value is of individuals and interactions over tools and processes, with supporting principles that tell us what the agile signatories really meant:

Business people and developers must work together daily throughout the project.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

There have been attempts, possibly misguided, to create a “distributed agile” manifesto. The problem is, the proposed new principles or values contradict this value and the corresponding principles. A revision that violates the intent of the original is a poor extension, indeed.

Developing in distributed teams is an issue close to our hearts. Both of us have worked as remote developers, including being full-time remote pair-programmers. We have worked with scores of teams that included distributed members. We have seen distributed development work, and have seen how it can fail, and we are devoting this month’s article to an eyes-open look at agile tooling and distributed teams.

Distributed Teams Are a Choice

The choice to create a team that is not co-located will challenge the team’s ability to follow the above values and principles. This departure from agile in turn detracts from the team’s ability to be successful—unless they can find a way to return the emphasis to individuals communicating face-to-face on a daily basis.

Distributed teams are a choice, not a circumstance forced upon a company by chance. A company may find that they can attract professionals of a higher calibre if they don’t insist that those professionals pull up roots and move. Likewise, a company may have chosen an implementation technology that does not have a rich user base in their local area. These are decisions that value people and interactions.

In other cases, executives buy into the promise of reducing costs by employing lower-priced and perhaps lower-capability programmers through off-shoring. CIO’s article “The Hidden Costs of Offshoring,” by Stephanie Overby makes it clear that off-shoring must be viewed as “a long-term investment with long-term payback.” It’s not as simple a matter as finding talent elsewhere—it takes a long time and continual effort to succeed with distributed teams. Executives must be willing to invest in additional travel and technology. They must also demonstrate patience in the early stages, and create a culture that accommodates many hiccups.

Whether the draw is greater talent or lower cost, distributed development incurs some intangible costs. One significant difference with distributed development is that remote folks are usually considered peripheral to the team. Since they’re not physically present, they miss out on important ad-hoc conversations. Teams may view them as an annoyance:

Jeff: “Nuts! We forgot to start the phone bridge to include Curt.”

Tim: “Oh! ... Oh well.”

Jeff: “Oh well.”

There is no question that the potential impact of Curt is diminished because he’s remote. Curt knows this too, and often feels distanced from the team as much emotionally and mentally as physically.

We worked with a team that had a business analyst in Sydney, a project manager in Dallas, two developers and a tester in Dallas, two more developers and two testers in Krakow, and another developer in Bangalore. The time zone issues alone meant that it was impossible to hold a meeting with all team members without encroaching on the sleep habits of at least one participant. When there was a question, it could take 12 to 24 hours to get an answer.

If you are in a sizable organization with distributed teams, the question to ask is, “What can we do to minimize the isolation of team members and maximize face-to-face conversation?”

The best answer? Don’t do that. In larger organizations, it’s usually possible to create entire project teams in many of your offshore locations, particularly if you’re willing to send out your expertise to help remotely grow the capabilities and culture you need to succeed. We hear “we can’t do that” far more often than we actually believe it.

Employing remote developers in order to save on costs is not a necessary condition of development, but instead a choice. This choice not only diminishes from an optimal communication environment, but also emphasizes negotiation and contracts instead of incremental and iterative exploration.

Employing remote developers in order to retain a stellar performer is also a choice. Our take: We’d rather have an enthusiastic, co-located team with an average performer in place of the headaches associated with dealing with a remote “rock star.”

We produced a fairly popular card to help organizations cope with having distributed teams, presented here for your consideration.

The discussion that accompanies the original card stands alone, and is worth a quick read, but since we’ve gotten to this discussion through a discussion of tool purchases, let’s see how these principles guide us in the selection and deployment of agile project management tools.

Don’t

Try pushing distributed management tool purchases off to the last responsible moment. First consider all other tools: development tools, distributed version control, continuous integration tools, remote desktop sharing, voice-and-video tools, shared whiteboards, telepresence devices, conference systems, testing hardware, and so on. If the need for an agile management tool outweighs all of these, then buying an agile management tool is reasonable. Be sure that the need is immediate and significant, the use of the tool is a good long-term decision, and that it does not further burden the remotes.

As the zen of Python says, “never is often better than right now.”

Don’t Treat Remotes As If They Were Local

Beware how you firewall in your teams. If the remotes can’t access email, intranet resources, file shares, version control, messaging, and the like then you won’t have much fun trying to collaborate with them. Locking out tools like Skype may be a bad idea.

If you choose any collaboration or documenation tools that present themselves as file systems or shares, will the remotes have ready access to them?

Remember that VPN is “the next best thing to being there” but also that it isn’t the most efficient way to communicate—it’s a distant second place. Often even simple screen sharing apps will have a half-second or more of lag, so that typing a word or editing a line of code can be tedious.

Don’t Treat Locals As If They Were Remote

Well-meaning companies use various communication and coordination tools to help the remotes, and then make the locals use those tools as well. Local developers don’t need such tools because they can more easily and fluidly communicate face-to-face. The goal of tools (management, bug tracking, or whatever) is not to give all members of the team common hardships, but to empower them to do work well.

Remote developers know that their team experience is degraded, and expect some sacrifice to be a part of the bargain. They learn to communicate with the local developers better, to keep back channels of communication open, and to deal with lag and network droppage. Constraining locals to the hindrances of being remote only serves to diminish productivity.

Latitude Hurts, But Longitude Kills

Most agile management tools are not affected by time zones, but a communication among widely-distributed team members may routinely be an overnight thing. Expect that your emails will be answered tomorrow, and that any status updates to the management system will not be immediate like it will with local people. Expect this delay to be a part of the daily routine.

For God’s sake, don’t check in code that breaks the build right before going home. Knowing the time zone difference, be sure that you leave your work day with a clean build that is ready for the remote team to use. Failure to observe this warning has caused teams to develop disdainful views of each other. Be wise, be warned.

Don’t Always Be Remote

A team whose members understand each other and can work together well is a successful team.

If you are just starting an agile effort, start with everyone in one place. If you have remote team members, fly them in, and insist everyone situate in a single room to feel each other out. Not literally of course—that’s the stuff of lawsuits—but ensure that they come to understand each others’ motivations, cultures, and quirks. Only then should you fling them back to remote corners of the earth.

To kick the team off initially, a week is not enough; you’ll want them in close quarters for at least a few weeks—ideally a month to six weeks. If you have the budget, then invite spouses as well, since locals don’t have to leave their families to go to work and it may be helpful for families to experience the local culture.

Sound costly? It is, but it’s less costly than a non-team unable to deliver quality software because they can’t communicate and don’t understand one another.

You’ll want your distributed team to occasionally reacquaint themselves with each other, perhaps for a week or two at a time. Twice a year is a reasonable goal.

Tools That Work

Our tools must lead us toward agile values and principles, not away from them. If a tool detracts from our ability to communicate face-to-face with our teammates on a daily basis, it is not a good agile tool. Conversely, we seek tools that help us make up for any loss in daily face-to-face communication that we might suffer due to being distributed.

“Face-to-face” means just that: When we can converse with a visible individual, observe their facial expressions, hear the tone in their voice, and read their body language, we have the best opportunity for understanding them. Email strips all of these important facets of communication. Text messaging returns only the ability to converse. Voice chat allows us to hear tone of voice, but hides body language and facial expressions that inform our reactions and interpretation. Was the remote person being sarcastic or enthusiastic? Was it a joke? Was the last remark condescending or just ill-phrased?

Audio/video chat begins to return us to the realm of face-to-face. The camera lens is limited in scope, however, and can still prevent us from seeing body language. It’s also usually fixed in focus, preventing us from following a teammate into the hallway, or from seeing that someone else just entered the room.

Here are some specific guidelines for communicating within a distributed team:

Everyone has a pretty good camera and microphone or headset available on their computer.

Everyone is available on a single, standardized team chat channel. At GeoLearning, some developers used Jabber, some used Skype, many didn’t log into their chat clients until reminded, and some weren’t on chat at all. It was very frustrating when you needed a question answered but couldn’t find anyone to ask.

Everyone is automatically logged into chat.

The team area hosts at least two always-on cameras, positioned to cover as much of the workspace as possible.

The team area hosts a high-quality conferencing microphone that is also always on.

The team either focuses another always-on camera at the card wall, or uses online distributed card wall software.

The team uses a scribe to capture and record important conversations when not everyone can be present (perhaps due to time zone issues). Conversation records must be broadcast to all team members.

The team creates and follows a process that makes it clear when and how each distributed conversation gets closed out. Otherwise, distributed conversations can easily drag on without resolution.

Conclusion

There is no reason that a company with distributed membership cannot be agile (Industrial Logic is largely a distributed organization) but it is considerably more work. Distance makes communication both less natural and more important. Misunderstanding is more likely. It is harder for local teams and local team members to appreciate the contributions of remote teams.

You can of course succeed with distributed team members (and we have!), but we ask that you recognize that the loss of free-form conversation compromises the agile ideal. We don’t care whether this means you can no longer call yourselves “agile”—that’s not what’s important. What’s important is that you understand the values you are controverting, so that you can seek remedies and help improve your chances of success.

Tim Ottinger is the originator and co-author of Agile in a Flash, a contributor to Clean Code, and a 30-year (plus) software developer. Tim is a senior consultant with Industrial Logic where he helps transform teams and organizations through education, process consulting, and technical practices coaching. He is an incessant blogger and incorrigible punster. He still writes code, and he likes it.

Jeff Langr has been happily building software for three decades. In addition to co-authoring Agile in a Flash with Tim, he’s written over 100 articles on software development and a couple books, Agile Java and Essential Java Style, and contributed to Uncle Bob’s Clean Code. Jeff runs the consulting and training company Langr Software Solutions from Colorado Springs.

Send the authors your feedback or discuss the article in the magazine forum.

When Did That Happen?

The Adding Machine

by Dan Wohlbruck

The road to entrepreneurial success has many potholes.

On October 11, 1887 US Patent 371,496 was awarded for an invention whose prototype was built from a macaroni box, string, and some rubber bands. Dorr Eugene Felt, the inventor, called it a Comptometer but we would know it better as an adding machine.

Dorr Felt grew up in Wisconsin where at the age of 14 he went to work in a machine shop in Beloit. He was a mechanical whiz and by the time he was 20 he had moved on to Chicago. After trying his hand at sales, he found a job working with machines for Ostrander and Huke. In 1884, using a wooden macaroni box, meat skewers, rubber bands, and string, he built a machine that he called a Comptometer. Felt interested a local machine shop owner, Robert Tarrant, who financed his endeavors with an initial investment of $5,000. In November 1887, a month after Felt received his patent, the two entered into a partnership which matured a year later into the Felt and Tarrant Manufacturing Company. It was Felt and Tarrant who built and sold the world’s first mechanical adding machine.

Success—and Competition

When the U.S. Treasury bought four Comptometers, others took notice. Among them was one William S. Burroughs who was inspired to design a machine that not only added but also recorded or, as he put it, “listed,” the result of the calculation. Where the Comptometer’s result had to be read from a set of dials and then written on paper, Burrough’s machine had a printout.

The difference in the two machines mirrored the difference in the two men. Burroughs was a bank clerk in New York who became a machine designer only after seeing the need for an adding machine. Once his machine was designed, patented, and successful, he sold out to the American Arithmometer Company. It was only after Burrough’s death in 1898 that the company was renamed Burroughs.

The two machines battled it out during the 1890’s but by the turn of the century it was clear that the Burroughs adding machine was the winner. In 1902, Felt gave up trying to design a “lister” for his Comptometer.

The Burroughs Company sold the one millionth adding machine in 1926 and continued their manufacture until the age of transistors.

But, of course, that’s not the end of the story. In 1951, Burroughs began the development of its own electronic accounting machines and started to move toward computers. In 1953, seeing the handwriting on the wall, the company changed its name from the Burroughs Adding Machine Company to the Burroughs Corporation. Burroughs did build and market mainframe computers during the 1950s and 1960s but neither they nor anyone else could compete with IBM. In fact, in 1982 Burroughs built the B20 personal computer with similar lack of success. Four years later the end came when Burroughs and Sperry merged to form Unisys, a company that continues to this day.

Winners and Losers

The story of the two adding machines is one that we’ve seen repeated as we’ve toured the history of technology. Namely, the best machine doesn’t always win but the best design often does. Users will not long support something that doesn’t mesh with the way that they perform their tasks.

Similarly, the story of the Burroughs Corporation has a familiar ring. The company struggles, succeeds, prospers, falls behind, and, ultimately, succumbs to the next wave of technology.

But, then, that’s why we study history.

It began over 120 years ago with macaroni and inspiration—and that’s when it happened.

Dan Wohlbruck has over 30 years of experience with computers, with over 25 years of business and project management experience in the life and health insurance industry. He has written articles for a variety of trade magazines and websites. He is currently hard at work on a book on the history of data processing.

Like this article? Hate it? Send the author your feedback or discuss the article in the magazine forum.

Shady Illuminations

Subvirtues and Pseudovirtues

by John Shade

If your world seems out of kilter, your values may need to be adjusted—so that they are out of kilter, too.

This is a list of what I call subvirtues and pseudovirtues. Subvirtues are subversive: they subvert conventional wisdom. Pseudovirtues are conventional wisdom, and I don’t like them. In my world, they are antipatterns. In your world, some of my pseudovirtues may be virtues and many of my subvirtues may be antipatterns. Suit yourself. This is my list.

Paranoia

If you write security software, you have to think like the bad guys. You have to adopt the mindset that they’re out to get you. You have to be paranoid. But is any other area of software development all that different? Instead of bad guys trying to break your security, you have stupid users seemingly trying to break your software. Your best strategy is to assume they are doing it deliberately: be paranoid. In software development, paranoia is a subvirtue.

Procrastination

This was not my first topic for this month’s column. I was going to devote the column to ridiculing NetFlix for their really abysmal handling of the infamous price increase. That it was seen primarily as a price increase is itself the best evidence of how badly they messed up. I was going to remind NetFlix customers that many of them signed up for a DVD-by-mail program and then started getting streaming videos as a freebie, and now that the streaming library was starting to be marginally adequate, they were being asked to pay for it, but were being given a discount on the original DVD-by-mail deal—and permitted to choose either delivery mode or both. I was going to ask how many of them really needed both the DVD and the streaming services, and how many were leaning toward streaming-only anyway, and were now being nudged to do what they would have done anyway, and being rewarded for it by having their monthly bill decreased. I was going to offer the opinion that NetFlix management almost looked like they wanted to drive the stock price down, the way they so messed up the framing of the policy change.

And then as I was starting to write the column, which, as usual, I had put off until the last possible moment, NetFlix management dropped another shoe in its corporate mouth, explaining how they were separating the two delivery modes and giving the DVD business its own name and website, in apparent preparation to sell it off. More classic public relations hara-kiri. Good thing I didn’t write that column early, or I’d have been stuck with an out-of-date diatribe. So I’m now holding off until the self-inflicted wounding abates. Procrastination is a subvirtue.

Perseverance

Perseverance is only a virtue for people who are not borderline OCD. Ask the people you live with if they think you have a problem sticking to the task when they want you to break for dinner, shopping, conversation, or whatever stupid thing they keep yammering on about. Oh, you live alone? I rest my case. Anyway, popularity is a pseudovirtue. Like perseverance.

Power

Power is overrated. With great power comes great responsibility, the noted scholar Uncle Ben Parker tells us, and who needs more responsibility? Also, with great power comes great complexity, and you know that’s trouble. Somebody, probably Uncle Ben, said that the devil is in the details, but that’s wrong. Details are nodes. The devil is in the connections.

Ah, connections. Wouldn’t it be wonderful if we could connect all our social networking activity with if-then statements? That would be so powerful. Yeah. And how long will it be before somebody creates the first IFTTT infinite loop? Oops: it’s already happened, according to one tweeter. And that’s authoritative enough for me. Power is a pseudovirtue.

Pessimism

A dedicated pessimist will tell you that Murphy was an optimist. I’ll tell you that. And Murphy is the name of the devil who lives in the connections. This discipline, if you can call it a discipline, of software development is afflicted with more such deviltry than any other. Hence more opportunities for things to go wrong. Exponentially more opportunities. We live in Mr. Murphy’s neighborhood. Pessimism is a survival skill. Pessimism means never having to be disappointed. Pessimism is a subvirtue.

Porn

As Brian Hogan points out in HTML5 and CSS3: Develop with Tomorrow's Standards Today, porn is the leading testing ground for cutting-edge technologies. Some academic from Kinsey’s alma mater says, “Pornography... is a positive good that encourages experimentation with new media.” Whenever a new technology comes along, people use it to make porn. It drives innovation. Porn is a subvirtue.

I hope my little list has been helpful to you. But I doubt it.

John Shade was born under a cloud in Montreux, Switzerland, in 1962. Subsequent internment in a series of obscure institutions of ostensibly higher learning did nothing to brighten his outlook. The thought that his pessimism, procrastination, and paranoia may be virtues depresses him. Send the author your feedback or discuss the article in the magazine forum.

Calendar

Author sightings, partner events, and other notable happenings.

We often want to call your attention to a particular event, and this month it’s this Clojure Conference in Raleigh. Lots of good people there and all the Clojure expertise in the world, apparently.

Nov 10–12, clojure-conj, Raleigh, NC

clojure-conj

Stuart Halloway and Aaron Bedra, co-authors of Programming Clojure, 2nd Edition (among others)

Author Appearances

Who’s where, and what for.

Oct 4–6, Agile Business Conference, London, UK

“Agile Retrospectives”

Rachel Davies, author of Agile Coaching

Oct 5, Java One, San Francisco, CA

Script Bowl

Aaron Bedra, co-author of Programming Clojure, 2nd Edition

Oct 10, Web 2.0 Expo, New York, New York

“The New Startup Stack”

Trevor Burnham, author of CoffeeScript: Accelerated JavaScript Development

Oct 10–12, Pacific Northwest Software Quality Conference, Portland, OR

“Dirty Tricks in the Name of Quality”

Ian Dees, author of Scripted GUI Testing With Ruby and co-author of Using JRuby

Oct 11–13, Scrum Gathering, London, UK

“Open Space Facilitation”

Rachel Davies, author of Agile Coaching

Oct 14, MagRails, Manchester, UK

“The Quest for Agility”

Rachel Davies, author of Agile Coaching

Oct 17–20, Zend PHP Conference, Santa Clara, CA

“SQL Injection Myths and Fallacies” and “MySQL 5.5 InnoDB Tuning”

Bill Karwin, author of SQL Antipatterns: Avoiding the Pitfalls of Database Programming

Oct 24–27, Pragmatic Studios, Reston, VA

iOS Studio

Daniel Steinberg, co-author of iPad Programming: A Quick-Start Guide for iPhone Developers and author of Cocoa Programming: A Quick-Start Guide for Developers

Oct 24–28, Much Ado ABout Agile 2011, Vancouver, BC

Keynote: “Where is Agile Going,” Tutorial: “Influence and Authority: Using Your Personal Power to Get Things Done”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Oct 25, SenchaCon, Austin, Texas

“Sencha on Rails”

Drew Neil, author of Practical Vim

Oct 26–28, QA&Test, Bilbao, Spain

“Taming the Android Unit Testing Tiger”

Paul Butcher, author of Debug It!: Find, Repair, and Prevent Bugs in Your Code

Oct 30–Nov 4, AYE Conference, Cary, NC

“Project Bloat Doesn't Float,” “Replacing Management Myths with Effective Practices,” “Influence and Authority: Using Your Personal Power to Get Things Done,” “Adapting to Change in Your Life,” and a one-day workshop: “Making Geographically Distributed Projects Work”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Nov 3, RedDirt.JS, Oklahoma City, OK

Intro to backbone.js

Derick Bailey, who’s working on a screencast on backbone for us

Nov 4, Midwest Ruby Conference, Kansas City, MO

Midwest Ruby Conference Keynote

Andy Hunt

Nov 6–11, Better Software Conference East, Orlando, FL

Three tutorials: “Tuning and Improving Your Agility,” “Agile Project Design: Building Strong Backlogs,” “Agile Estimation and Planning: Scrum, Kanban, and Beyond”

David Hussman, author of Cutting an Agile Groove

Nov 7–9, Clojure/conj, Raleigh, NC

“(with-training Clojure/conj)”

Aaron Bedra, co-author of Programming Clojure, 2nd Edition

Nov 7–11, Oredev, Malmo, Sweden BC

Tutorial: “Influence and Authority: Using Your Personal Power to Get Things Done,” “Managing for Collaboration,” “Agile Portfolio Planning: Managing Your Project Portfolio”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Nov 12–13, Voices That Matter: iOS Developers Conference, Boston, MA

“Core Audio Cranks It Up” and “Capturing Stills, Sounds, and Scenes with AV Foundation”

Chris Adamson, co-author of iPhone SDK Development

Nov 14–17, Agile Testing Days, Potsdam, Germany

Keynote: “Agile Testing and Test Management,” Tutorial: “Making Geographically Distributed Projects Work”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

Nov 16, QConSF, San Francisco, CA

“One () to rule them all”

Aaron Bedra, co-author of Programming Clojure, 2nd Edition

Nov 16, Agilis, Reykjavik, Iceland

Workshop

Rachel Davies, author of Agile Coaching

Nov 18, SCNA, Chicago, IL

“Simplicity”

Aaron Bedra, co-author of Programming Clojure, 2nd Edition

Nov 22, Best of DevSum, Stockholm, Sweden

“Regex in .NET”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

Nov 29-Dec 2, Reston, VA

Pragmatic Studios: “iPhone/iPad Programming”

Daniel H Steinberg, co-author of iPad Programming: A Quick-Start Guide for iPhone Developers and author of Cocoa Programming: A Quick-Start Guide for Developers

Dec 2–3, Cocoaconf, Raleigh, NC

“Core Audio Cranks It Up” and “Capturing Stills, Sounds, and Scenes with AV Foundation”

Chris Adamson, co-author of iPhone SDK Development

Dec 2, Cocoaconf, Raleigh, NC

Cocoaheads Keynote

Andy Hunt

Dec 3-4, North Carolina

CocoaConf:Keynote—“Objective-C: the Director’s Cut”

Daniel H Steinberg, co-author of iPad Programming: A Quick-Start Guide for iPhone Developers and author of Cocoa Programming: A Quick-Start Guide for Developers

O’Reilly Events

Upcoming events from our friends at O’Reilly.

Oct 9–11, Android Open, San Francisco, CA

Android Open Conference: “Android Open is the first conference to cover the entire Android ecosystem. Whether you’re a developer, IT pro, business decision-maker, or marketer, you’ll find the latest and best information for maximizing the power of the Android platform.”

Oct 11, TOC Frankfurt, Frankfurt, Germany

Tools of Change Frankfort: “Tools of Change Frankfurt returns for a third year on... the eve of the Frankfurt Book Fair (12-16 October)—connecting the people, content, and conversations along the cutting edge of global publishing and technology.”

Oct 10–13, Web 2.0, New York, New York

Web 2.0 Expo NY: “At the next Web 2.0 Expo, we’ll reveal the key pieces of the digital economy and the ways you can leverage important ideas for your own success. We’ll look specifically at web success stories in the New York startup ecosystem.”

Oct 17–19, Web 2.0 Summit, San Francisco, CA

Web 2.0 Summit: “Once each year, the Web 2.0 Summit brings together 1,000 senior executives from the worlds of technology, media, finance, telecommunications, entertainment, and the Internet. For 2011, our theme is ‘The Data Frame’—focusing on the impact of data in today’s networked economy.”

Nov 8–9, Velocity Europe, Berlin, Germany

Velocity Europe: “Velocity Europe is the convergence of performance and site reliability experts and rock stars who share the unique experience that can only be gained by operating at scale. At Velocity, leading experts share their knowledge about how you can make your site scalable, reliable, and fast.”

USENIX Events

What’s coming from our USENIX friends.

Oct 23–26, SOSP 2011, Cascais, Portugal

23rd ACM Symposium on Operating Systems Principles: “The biennial ACM Symposium on Operating Systems Principles is the world’s premier forum for researchers, developers, programmers, and teachers of computer systems technology. Academic and industrial participants present research and experience papers that cover the full range of theory and practice of computer systems software.”

Dec 4–5, ACM Symposium on Computer Human Interaction for Management of Information Technology, Boston, MA

CHIMIT 2011: “At CHIMIT, researchers and practitioners share issues, solutions, and research drawing upon fields such as human-computer interaction, human factors, computer systems, and management and service sciences to address cognitive, social, and technical issues in managing the increasing complexity of modern IT systems.”

Dec 4–9, 25th Large Installation System Administration Conference, Boston, MA

LISA ’11: “LISA ’11 will break out the time machine and take you on a journey of system administration through the years via top-notch training, innovative research, Practice and Experience Reports, and that all-important face time with others confronting the same challenges you battle everyday.”

Dec 12–16, ACM/IFIP/USENIX 12th International Middleware Conference, Lisbon, Portugal

Middleware 2011: “The Middleware conference is a forum for the discussion of important innovations and recent advances in the design, construction, and uses of middleware.”

Other Happenings

Oct 4

The Free Software Foundation is 36.

Oct 4

Birthday of John Vincent Atanasoff, one of the inventors of the computer.

Oct 5

Linux is 20!

Oct 6

Adobe co-founder John Warnock is 71.

Oct 12

Birthday of Ole-Johan Dahl, co-creator of the Simula language and thus of object-oriented programming.

Oct 16

Mike Muuss, who invented Ping, was born on this date in 1958.

Oct 25

Peter Naur, the N in BNF, is 83.

Oct 28

Former software executive Bill Gates is 56.

Oct 29

On this date in 1969, ARPAnet first linked Doug Engelbart’s lab at SRI with Leonard Kleinrock’s at UCLA.

Oct 31

The traditional date for the leaking of confidential internal Microsoft documents outlining strategies for undermining open source software.

Nov 1

This month the OLPC is 6 years old.

Nov 1

Mitch Kapor is 61.

Nov 8

Bill Joy is 57.

Nov 6

Jerry Yang is 43.

Nov 10

Bert Bos, co-creator of Cascading Style Sheets, is 48.

Nov 10

Richard Brodie, creator of Microsoft Word, is 52.

Nov 18

Pamela Kyle Crossley, historian and creator of SIMPLE, is 56.

Nov 19

Dries Buytaert, Drupal founder and lead developer, is 33.

Nov 22

Randall Schwartz is 50.

Nov 22

PHP creator Rasmus Lerdorf is 43.

Nov 22

And it’s Charles Simonyi’s wedding anniversary.

Dec 3

John Backus would have been 87.

Dec 4

Eric Raymond, aka ESR, is 54.

Dec 9

Grace Hopper’s 105th birthday.

Dec 10

Ada Lovelace’s 196th birthday.

Dec 10

Ada, the programming language, is 31 today.

Dec 16

John Bardeen and Walter Brattain created the first transistor on this day in 1947.

Dec 17

Ken Iverson, creator of APL, would have been 91 today.

Dec 17

Les Earnest, inventor of first spell checker and of the finger protocol, is 81.

Dec 23

Birthday of Bob Miner, who wrote Oracle’s original database software.

Dec 25

Richard Greenblat, famed MIT hacker, is 67.

Dec 26

Charles Babbage’s 220th birthday.

Dec 28

Linus Torvalds is 42.

Dec 30

Bjarne Stroustrup is 61.

Dec 31

Leonard Adleman, the A in RSA encryption, is 66.

But Wait, There’s More...

Coming Attractions

This may be the last page, but that doesn’t mean it’s the end of the conversation. Keep up with what’s going on in the Pragmatic universe by subscribing to our newsletter and drop by the Pub again next month for more Pragmatism. We’ll keep a seat warm and a mug chilled and a candle burning in the window.

Coming Soon in PragPub

Bill Dudney will tell us what the news about iOS5 will mean to developers. Jonathan Rasmusson will warn us about the perils of self-inflicted scope creep. David Hussman’s video series is now out, and I’m going to pin him down for that interview on going beyond Scrum, Kanban and other brand warfare. And Venkat will continue his popular series on Scala.

Coming Soon on the Bookshelf

Watch for The Developer’s Code, Programming Node.js, and Programming Your Home.

Right. But to really be in the know, you need to subscribe to our weekly newsletter. It’ll keep you in the loop, it’s a fun read, and it’s free. All you need to do is create an account on pragprog.com (email address and password is all it takes) and select the checkbox to receive newsletters.

While you’re waiting for the next issue of the newsletter or of PragPub to come out, you can follow us on twitter at @pragpub, @PragmaticAndy, or @pragdave. Or on Facebook at facebook.com/PragmaticBookshelf. Or email us at michael@pragprog.com.