PragPub 2012-05: Issue #35

Table of Contents

Features

Scala for the Intrigued

by Venkat Subramaniam

Venkat shows how, in Scala, tail recursion is like a metabolic miracle.

Beyond the Bit Bucket

by Jim R. Wilson

The decision about where your data’s structure resides has become a lot more complicated.

Effective Emails

by Josh Carter

That email you just sent says a lot about your professionalism. Here’s how to be sure your emails say what you want.

What Makes an Awesome Command-line Application?

by David Copeland

Command-line applications are applications, and should be written with the care you bring to all your app development work.

Departments

Up Front

by Michael Swaine

Cutting through the complexity of command-line apps, data storage structure, tail-call recursion optimization, email, and privacy.

Choice Bits

Drinking from the Twitter firehose.

Meet the Team

Meet Michael Swaine, PragPub’s editor.

Calendar

Author sightings, upcoming conferences, and other events of note.

Shady Illuminations

by John Shade

Anthony Bourdain’s habits, the Junior Ganymede Club, and the Lidless Eye all find their way into John’s exploration of privacy.

But Wait, There’s More...

Coming attractions and where to go from here.

Except where otherwise indicated, entire contents copyright © 2012 The Pragmatic Programmers.

Feel free to distribute this magazine (in whole, and for free) to anyone you want. However, you may not sell this magazine or its content, nor extract and use more than a paragraph of content in some other publication without our permission.

Published monthly in PDF, mobi, and epub formats by The Pragmatic Programmers, LLC, Dallas, TX, and Raleigh, NC. E-Mail support@pragprog.com, phone +1-800-699-7764. The editor is Michael Swaine (mailto:michael@pragprog.com). Visit us at http://pragprog.com

ISSN: 1948-3562

Up Front

It’s Complicated

by Michael Swaine

Developing software is complicated. This month, as every month, PragPub brings together some bright author/programmers to cut through some of the complexity.

David Copeland is on a campaign to raise the quality of command-line apps. He wants everyone to stop writing scripts and start writing applications. Calling something a script, he says, can be an excuse not to care about writing good code. In “What Makes an Awesome Command-line Application?” he shows how to approach command-line programming with respect.

Jim R. Wilson points out that things have gotten more complicated when it comes to the structure of your application’s data. Do you put all the intelligence about that structure in the application, or in the data store? Today, more often than not, the answer is some combination of the two. And, as Jim details in “Beyond the Bit Bucket,” you have a lot of options.

Josh Carter’s contribution to this issue is called “Effective Emails.” A lightweight article, you figure? Guess again. The various things you need to consider in writing and sending emails in a professional setting are almost as tricky as anything you’ll encounter in writing command-line apps or designing data structures.

Thinking recursively can be tricky until you get the concept. Then it’s simple—until you run into the limitations of pure recursive algorithms and start trying to optimize the code. But if you’re working in Scala, “Recursions and Tail Call Optimization” will ease your mind, because both the language and author Venkat Subramaniam have got your back.

On a lighter note, John Shade shows that privacy may be more complicated than you think.

Choice Bits

Life Lessons and Tech Rants

The second rule of Cite Club is ibid.

What’s Hot

Top-Ten lists are passé—ours goes to 11. These are the top titles that folks are interested in currently, along with their rank from last month. This is based solely on direct sales from our online store.

1^ | NEW | Practical Vim
2v | 1 | The Rails View
3^ | NEW | Rails Recipes
4v | 3 | Agile Web Development with Rails
5^ | NEW | Programming Clojure
6v | 2 | tmux
7^ | 9 | Seven Databases in Seven Weeks
8 | 8 | Programming Ruby 1.9
9v | 5 | The Cucumber Book
10 | 10 | Deploying Rails
11^ | NEW | iOS SDK Development

In April we watched the season change, we learned that a sunglasses-wearing, bored-looking, texting Secretary of State is the current definition of cool, and we continued to discover what Twitter is really for.

What is Twitter for, you ask?

It’s for Deep Thoughts

Reading physical book. Haven’t done that in ages. It’s so heavy! And where is the built in light?! — @catehstn

How come you never find someone showing “mitigated gall?” It’s always the full-on variety. — @PragmaticAndy

@PragmaticAndy Likewise, I've never heard of gruntled employees... — @bdstevens

It’s for Revealing Your Hidden Talent

Have you seen my latest party trick? Don’t try this at home, folks! http://virg.co/sbr #talentschool — @richardbranson

I’m racing the #37 @ConquestRacing LMP2 in the Long Beach Grand Prix tomorrow at 4:30pm. Watch it live on ESPN2 or http://alms.com. — @dhh

First day of rowing lessons. Ended up in the water and surprised my instructor by being able to get back in the boat. — @tswicegood

It’s for Your Tech Rants

Batteries are my Curse and Damnation. — @cloudforest

*shakes fist* Nested HAML partials *shakes fist* </rant> — @johnsonch

I was a really nice guy until I used Rails. Now I kick puppies on my way to set fire to orphanages as I rant on Hacker News while driving. — @jm

1. I have to *reboot* my mac before it recognizes the TV on the end of the HDMI cable? 2. No sound? Isn’t that point of HDMI? — @marick

When the revolution comes, those who provide XML configuration and build files will be first against the wall. — @rit

Hey @nabshow tried to update my profile only to find Ms. is not an option. Only Mr., Miss, Mrs., Dr. What year is this is? 1955? — @jenred

Grrr. Technology is awesome except when it isn’t. — @bphogan

Great...have started getting recaptcha’s with non-Latin scripts. Today it’s Arabic. — @headius

It’s for Status Reports

It just took me 15 minutes to disentangle my bra from a USB cable. There’s something oddly appropriate about that. — @snipeyhead

Our washing machine just sent a status report to my cell phone. — @pragdave

My friend Scott just used big-O notation to describe the length of a relationship, and I didn’t even blink about it until 5 min later. — @demew

1. My chess rating has dropped 1500 points since I started traveling for work. 2. Politicians spend a lot of time on the road. #hmm — @gvwilson

I wasn’t wearing the bra at the time, although the mental picture is much funnier that way. — @snipeyhead

It’s for Sharing Lessons Learned

Observation: being left handed works well when doing book signings. Shake right, sign left, don’t put down pen. — @adamgoucher

How you get your geek to do anything for you: “@chrisjpowers: Just got this text message from @sarahapowers: ‘Sudo! Bring me some pie!’” — @redsquirrel

When you come to a fork in the road, pick it up and keep it in inventory. Eventually you’ll need it to prop open a door or stab a dragon. — @pragpub

Don't mistake your email inbox for a to-do list. — @staffannoteberg

Here’s an idea: Comment forms have a captcha, not to prove the commenter is a human, but that the commenter has read the damn article. — @petdance

Remember kids, there are no problems that can’t be solved by creatively piping Unix commands together. — @bketelsen

Miles to Kilometers conversion is a Fibonacci sequence... #cool http://www.mnn.com/earth-matters/wilderness-resources/blogs/cool-math-trick-converting-between-miles-and-kilometers — @scottdavis99

The game where you try to chase your cats off of the carpet as they throw up is a game everyone loses. — @tekniklr

The second rule of Cite Club is ibid. — @dwineman

Who Are Those Guys?

They’re not all guys. Also, we have to confess that we cleaned up their punctuation and stuff a little. OK, who they are: Addie Beseda, Richard Branson, Scott Davis, Adam Goucher, David Heinemeyer Hansson, Brian P. Hogan, Dave Hoover, Andy Hunt, Cate Huston, jenred, Chris Johnson, Brian Ketelsen, Andy Lester, Brendan W. McAdams, Jeremy McAnally, Brian Marick, Staffan Nöteberg, Charles Nutter, Nadine Schaeffer, snipe, Teri Solow, Brian Stevens, Michael Swaine, Travis Swicegood, Dave Thomas, Greg Wilson, and Dan Wineman. You can follow us at www.twitter.com/pragpub.

Meet the Team

Michael Swaine

Each month we profile a member of the Pragmatic Bookshelf team. This month it’s Michael Swaine, the editor of PragPub.

Michael Swaine, Editor

Every month Michael pulls together an issue of PragPub, the magazine of the Pragmatic Bookshelf. He’s been doing this sort of thing for three decades, first as part of the launch team for InfoWorld, then as editor-in-chief of Dr. Dobb’s Journal, and now with the Prags. He thinks he’s starting to get a handle on it.

Michael also manages to fit in a little bookwork, both editing and writing books. He and Paul Freiberger are writing the third edition of Fire in the Valley, their seminal history of the personal computer, which was adapted for the screen as the movie Pirates of Silicon Valley. They’re publishing with the Prags, naturally. Michael also claims to be working on a detective novel, but nobody’s seen that yet.

“I love the concept of PragPub,” Michael says, “ad-free, DRM-free, as-in-beer-free, and written by programmers for programmers. And I love working with smart people, both the editorial team at the Pragmatic Bookshelf and the writers. I have a dream job.”

Michael may also have a dream 411. He lives on an organic farm in southern Oregon with his partner, Nancy Groth. The farm also has a gourmet restaurant on site.

“When Nancy told me I needed to get out and enjoy the beauties of this special corner of the world,” Michael says, “I responded in the only way I knew: I became the editor of a regional magazine focused on southern Oregon. If all you have is a hammer—”

Email: michael@pragprog.com

Twitter: @pragpub and @mswaine

Blog: Swaine’s World

Scala for the Intrigued

Recursions and Tail Call Optimization

by Venkat Subramaniam

To tame recursion, Scala catches it by its tail.

Recursion, a powerful technique in programming, allows one to build up the solution to a problem using the solutions to sub-problems. Recursive functions call themselves, an apparent paradox. Recursion can be a bit bewildering if you’ve never encountered it, but once we get a handle on it, its expressive power can serve us well. In this article we’ll look at an example of writing recursive methods in Scala, discuss issues with simply using this powerful feature, and then look at the solutions Scala offers to the problems.

Computing the factorial, which is a product of all integers from 1 to a given number, has a familiar recursive solution. We can easily write a recursive function in Scala to compute the factorial of a number n.

| def factorial(n : BigInt) : BigInt =
| if(n == 1) 1 else n * factorial(n - 1)

Since the factorial of a number can be pretty large, we use a BigInt rather than an Int. Given a number, we compute the factorial for it using a recursive call to the factorial function. The factorial for 1 is the number itself. For all numbers greater than 1 the value is computed as the number times the factorial of the number minus one. In this example we assume the given number is greater than or equal to 1.

Let’s use this function to compute the factorial of a small number, like 5.

println(factorial(5)) //120

That was easy. And the code is concise and expressive. On first sight, the recursive solution looks quite desirable, so why would we not want to use solutions like this wherever possible? But appearance can be deceiving. While this code worked well for this small argument, it will unfortunately not yield the desired result for a large argument. For example, try invoking the function with 10,000 as a parameter.

| try {
| factorial(10000)
| } catch {
| case ex => println(ex) //java.lang.StackOverflowError
| }

As the code runs, each call to the function will push the parameter onto the call stack. The stack will rapidly build up as each call continues to hold its space on the stack while it waits for the subsequent call to complete. We will soon run into a StackOverflowError exception.

Recursive solutions are fine for small parameters but quickly turn unusable as the parameter size grows. This can force us to fall back on iterative solutions, like:

| def factorial(n : BigInt) : BigInt = {
| var result = BigInt(1)
| for(i <- BigInt(1) to n) result *= i
| result
| }

The iterative solution will yield the right results for both small and large parameters:

| println(factorial(5))
| println(factorial(10000))

The iterative version, however, does not have the charm of recursion. It’s relatively verbose and makes use of a mutable local variable that the recursive solution nicely avoided. You can avoid the mutable variable in the iterative solution by using other functions like foldLeft. In any case, now that we have a taste of recursion, it’d be nice to use it without the negative consequences.

Scala makes that possible, as we will now see.

The solution is highlighted in a fantastic book called Structure and Interpretation of Computer Programs (SICP) by Abelson and Sussman (The MIT Press, 1996, Cambridge, Massachusetts). The key is to think differently about the code that we write and the code that we execute.

The first solution we saw is a recursive procedure (structured as a recursion in code) and a recursive process (exercised as a recursion at runtime). The second solution is an iterative procedure and an iterative process, that is written as iteration and runs the same way.

It would be great if we could have the best of both worlds. That is, if we could write a solution as a recursive procedure, and then, using compilation techniques, transform the code so it runs as an iterative process. This is the computing equivalent of having a metabolism we can only dream of, letting us eat all we want without adding an extra pound around the midsection. The midsection here is the call stack created during the execution of code.

Let’s trim the fat from the recursive procedure’s midsection.

Let’s examine that original recursive call more closely.

if(n == 1) 1 else n * factorial(n - 1)

The obvious detail here is that the function calls itself, but the thing to notice is that the last operation performed in the call is not the call to the function itself. The multiply operation has to be performed once the recursive call returns. As a result, the stack that is built has to grow while we wait for the recursive call to complete and for the multiply operation to exercise. That’s where the fat comes from.

If we can structure the recursive call so that the final operation we perform is not the multiply but the call to the function itself, then a smart compiler (like Scala’s of course) can make use of that to transform the recursion into a simple iteration under the covers—and eliminate the fat. Let’s examine this with another example before we return to the factorial problem.

| object Sample {
| def madMethod1(n : Int) : Int =
| if(n == 1) throw new RuntimeException("mad") else 1 * madMethod1(n - 1)
| //
| def madMethod2(n : Int) : Int =
| if(n == 1) throw new RuntimeException("mad") else madMethod2(n - 1)
| //
| def main(args : Array[String]) = {
| try {
| madMethod1(5)
| } catch {
| case ex => ex.printStackTrace
| }
| try {
| madMethod2(5)
| } catch {
| case ex => ex.printStackTrace
| }
| }
| }

The Sample singleton has two methods, madMethod1 and madMethod2 both of which throw an exception. The last operation in the madmethod1 is the multiply operation, but the last operation in the madMethod2 is a call to itself, making it tail recursive. Tail recursive methods get a special treatment by the Scala compiler. They’re transformed into simple iteration under the covers—just what we were looking for. If we run the sample, we can see the difference in execution.

The call to madMethod1 results in a stack trace that is five levels deep, as we’d expect.

| java.lang.RuntimeException: mad
| at Main$.madMethod1(Sample.scala:3)
| at Main$.madMethod1(Sample.scala:3)
| at Main$.madMethod1(Sample.scala:3)
| at Main$.madMethod1(Sample.scala:3)
| at Main$.madMethod1(Sample.scala:3)
| at Main$.main(Sample.scala:10)
| at Main.main(Sample.scala)

The recursive calls to madMethod2, on the other hand, stays at one level deep, as its stack trace reveals.

| java.lang.RuntimeException: mad
| at Main$.madMethod2(Sample.scala:6)
| at Main$.main(Sample.scala:16)
| at Main.main(Sample.scala)

This is due to the special treatment I mentioned: the Scala compiler transforms madMethod2 into a simple iteration while it leaves madMethod1 as a recursion. You can take a peek at this by compiling the code using scalac Sample.scala and running a javap -c Sample$ command on the generated bytecode.

| public int madMethod1(int);
| Code:
| 0: iload_1
| 1: iconst_1
| 2: if_icmpne 15
| 5: new #17; //class java/lang/RuntimeException
| 8: dup
| 9: ldc #19; //String mad
| 11: invokespecial #22;
| //Method java/lang/RuntimeException."<init>":(Ljava/lang/String;)V
| 14: athrow
| 15: iconst_1
| 16: aload_0
| 17: iload_1
| 18: iconst_1
| 19: isub
| 20: invokevirtual #24; //Method madMethod1:(I)I
| 23: imul
| 24: ireturn

| public int madMethod2(int);
| Code:
| 0: iload_1
| 1: iconst_1
| 2: if_icmpne 15
| 5: new #17; //class java/lang/RuntimeException
| 8: dup
| 9: ldc #19; //String mad
| 11: invokespecial #22;
| //Method java/lang/RuntimeException."<init>":(Ljava/lang/String;)V
| 14: athrow
| 15: iload_1
| 16: iconst_1
| 17: isub
| 18:istore_1
| 19: goto 0

The invokeVirtual in the madMethod1 shows the recursive call, and that’s been replaced with an iterative call at the end of the madMethod2.

Writing a recursion as tail recursion gives us the maximum benefit: expressive, concise code that does not tax the stack and so won’t result in a StackOverflowError.

Let’s shift our focus to the factorial example. We have the given number on hand and we need to multiply that value by the result of the recursive call to the method with a parameter of the given number minus one. The challenge now is to write it as a tail recursion instead of a regular recursion.

A straightforward way to do this is to add a parameter. Rather than holding onto the partial result in the current method invocation, we can pass on the partial result as a parameter to the method so it can perform the computation, like so:

| def factorial(fact : BigInt, n : BigInt) : BigInt =
| if(n == 1) fact else factorial(fact * n, n - 1)

This version of the factorial method takes two parameters instead of one. The first parameter is the partial computation of the factorial, and the second parameter is the number for which we want to compute the factorial. If the given number is 1, we simply return the value of the partial result fact. Otherwise, we multiply the value of fact by the given number, and then send off to the recursive function. Since the last expression we perform is the call to the method itself, this implementation is tail recursive and will enjoy the tail call optimization.

Let’s invoke this method for the two parameter values we tried with the earlier implementations.

| println(factorial(1, 5)) //120
| //
| factorial(1, 10000)
| println("no exception") //no exception

The tail recursive version produces the desired result and is able to handle parameters of large size as well.

By running this example we can see that the optimization took effect. For a compile-time confirmation of such optimization, Scala provides a special annotation that you can place on the recursive method.

| @scala.annotation.tailrec
| def factorial(fact : BigInt, n : BigInt) : BigInt =
| if(n == 1) fact else factorial(fact * n, n - 1)

If the method is not really tail recursive, then this annotation directs the Scala compiler to give us a compilation error.

We benefited from the tail call optimization but we lost a bit of clarity in code. Now, to make use of this method, the caller has to pass two parameters instead of one. In addition to this burden, this opens the door for errors. Someone could send an incorrect value, like 0 as the first parameter, and jeopardize the correct execution of the method.

The need to send two parameters instead of one is an implementation detail to facilitate tail recursion. In programming we’ve learned to encapsulate or hide implementation details. We can achieve a lightweight encapsulation by defining this function within another function that takes the necessary single parameter.

| def factorial(n : BigInt) = {
| @scala.annotation.tailrec
| def factorial(fact : BigInt, n : BigInt) : BigInt =
| if(n == 1) fact else factorial(fact * n, n - 1)
| factorial(1, n)
| }

The two-parameter method is hidden from the view of callers. The users of this version can invoke the single-parameter method, which in turn invokes the two-parameter tail recursive implementation of the factorial method.

Recursion is a pretty cool programming technique. Scala not only supports recursion but optimizes the resource usage. This allows us to make good use of this powerful technique without the worries of stack overflows.

Dr. Venkat Subramaniam is an award-winning author, founder of Agile Developer, Inc., and an adjunct faculty at the University of Houston.

He has trained and mentored thousands of software developers in the US, Canada, Europe, and Asia, and is a regularly invited speaker at several international conferences. Venkat helps his clients effectively apply and succeed with agile practices on their software projects.

Venkat is the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy: Dynamic Productivity for the Java Developer and Programming Scala: Tackle Multi-Core Complexity on the Java Virtual Machine. His latest book is Programming Concurrency on the JVM: Mastering Synchronization, STM, and Actors.

This series started in the September 2011 issue and has been running continuously since then. If you’d like to read the whole series, here are the links to the articles published so far:

9/11: The Elegance of Scala

10/11: Sensible Typing and Optional Items

11/11: Cute Classes and Pure OO

12/11: Functional Style of Programming

1/12: Working with Collections

2/12: Creating Higher Order Functions

3/12: Pattern Matching

4/12: XML as First Class Citizen

5/12: Recursions and Tail Call Optimization

Send the author your feedback or discuss the article in the magazine forum.

Beyond the Bit Bucket

3 and 1/2 Ways to Let the Database Handle It

by Jim R. Wilson

In designing applications, there are now more choices in where the structure of the data should reside.

No matter what kind of application you write, at some level you must deal with data. It’s what programs do. But when it comes to organizing and persisting your data, you have some new decisions to make.

For decades, relational databases and filesystems were the go-to options for storing pretty much everything. What we’re seeing now with the rise of NoSQL is the filling-in of the space between. What was once a binary choice is now a continuum.

This article talks about these new options and what they mean for your projects.

Defining the Terms

We can group data storage systems by how much structure we expect from them. We’ll explore these data-organization options by grouping them in this way and looking at them from least structured to most structured. Armed with this knowledge, you should be better able to make informed database genre and schema design decisions.

Before we dive into the first structure grouping, let’s define some terms. For the sake of this discussion, we’ll assume that the whole system can be practically divided into two major components: the application and the datastore. In reality these roles may be serviced by the same software, but we’ll find it useful to make the functional distinction. On the far other end of the spectrum, your application may live on a smartphone with data stored in the Cloud. What matters is this functional distinction between what the final system does (the application) and what it stores (the datastore).

Ready? Let’s start with the zero case.

Option 0: Completely Unstructured

The first and most basic data storage option is to put all the logic about your data’s structure into your application. All of it. Under this model, the only thing your application needs for storage is a place to shove a big blob of bytes and get them back out again later.

This is what a filesystem does; the filesystem application presents folders, files, and permissions through its API. But underneath the filesystem is the datastore, the physical disc.

Practically speaking it’d be overkill most of the time to implement your own filesystem or the comparable application functionality, but it bears mentioning as one extreme option along the continuum of choices.

Option 1: Minimally Structured

The first baby step up from one big bit bucket is a set of labeled buckets. In other words, a key/value (KV) store.

Some key/value stores offer additional options, but at heart KV stores are basically just persistent maps. You provide a key and a value (both blobs) and the KV store puts them on the shelf for you to use later.

Option 2: Advanced Datastructure Support

Your applications will usually benefit from offloading some or all of its data organization responsibility to the datastore. This means you’ll write, test, and maintain less code, giving you more time to focus on the core value your application delivers. Just as you wouldn’t write your own sorting algorithm when a perfectly good library was at hand, you don’t need to resign yourself to working at low levels with your data.

Stepping up the structure ladder from KV stores we find many design archetypes you might choose. These include document datastores, column-oriented databases, and graph databases. Each of these genres offers different viewpoints and tradeoffs with respect to the amount and kind of structure enforced by the system, as well as additional tooling for finding and operating on stored data.

We won’t delve into the details of these options in this brief article, but you should know that this rung of the structure ladder is a broad one, with many choices.

Option 3: Rigorous Schema Enforcement

There is one more option still, and that is to put a considerable amount of structure into the database. We’re now basically at the top of the ladder. Under this model, you expect the database to enforce specific rules that you establish in advance.

By forcing this up-front structure on your data, you free up the database to reason fairly intelligently about your data at the system level. This reduces the amount of intelligence you have to implement in client applications.

Relational Database Management Systems (RDBMSs) are the canonical example of this kind of structure responsibility. Also here you’d find certain inverted index implementations that require documents to be typed, have certain required fields, etc.

Implications for Your Project

So what does all this mean for your application? Unfortunately, it means more work. You have more decisions to make. In a nutshell, it’s your duty as a creator of software to understand and decide which of these tiers makes sense for your use case and then find something that fits.

The once simple line between structure imposed by the database and structure maintained by the application is shifting and expanding. That means new decisions for you to make, but it also means more power in your hands. On the one hand, it’s now more complex to decide what your stack is going to look like. But this plethora of options means there are probably better choices for your applications than you might have considered before.

And that, ultimately, means better apps.

Jim R. Wilson started hacking at the age of 13 and never looked back. He began tinkering with NoSQL databases in 2007 and has contributed code to large-scale open source projects such as MediaWiki and HBase. You can learn much more about the various genres of databases and what they offer and how to use them from his book Seven Databases in Seven Weeks.

Send the author your feedback or discuss the article in the magazine forum.

Effective Emails

Think Before You Send

by Josh Carter

That email you just sent says a lot about your professionalism. Here’s how to be sure your emails what you want.

We’re called upon to communicate many things to many people via email. Doing this clearly and efficiently is an essential skill to master.

As you pop open a new mail window, the first questions to ask yourself before typing anything should be:

Who am I writing this to?

What do they need to know?

Answer these up front and you’ll immediately save everyone, including yourself, a lot of time. Your audience determines the length of your message, the topics that are relevant, the amount of detail that’s appropriate, and the humor you can get away with.

Audience

Let’s consider a few audiences and their implications for your email:

Upper Management: These people are busy, they get hundreds of emails a day, and they often read email on a cell phone while traveling. More than anything else, you must be concise to reach this crowd. If an upper-level manager can’t understand your point within the first couple of lines of text, they’ll simply delete your email and move on.

Your Manager: While she might also be busy and reading email on her phone, you need to give your manager enough details to make informed decisions. The appropriate level of technical detail depends on your manager’s background; many are ex-programmers and still want to hear it.

Other programmers: You’re on your home turf, so go technical! If your message could be broadly useful, for example explaining gnarly details of some software component, consider documenting that in an appropriate place—docs in source control, intranet wiki—and email a link instead.

People outside the company: Use your judgement, but with a couple of obvious limitations: don’t mention or imply anything that could be considered a trade secret, and keep a positive, professional demeanor. You may not work in Marketing, but that doesn’t matter: you are a representative of your company, and you need to act like it.

Of course, some email isn’t so simple: managers, programmers, janitors, etc. may all be on the distribution list. What then? My rule of thumb is, write for the most important audience. That’s not always the person with the highest title. Instead, it’s the audience that stands to benefit most from the email. For example, if the topic is gnarly software details, but the company CEO is on cc:, still write for the programmers.

One caveat: if upper managers are on your email’s distribution, do them a favor and include a brief summary at the top. This could be called out as an “executive overview,” or you can be more subtle. Open with, for example, “The component will work once Bug 2112 is fixed. Let me add some background information on the design of that component...” That gives the managers their key information up front.

Addressing

You can address an email using to:, cc:, and bcc:. Your choice sends its own implicit message to the readers.

People on the to: line are the most important audience; you’re saying “this message is to you.” Your message should be directly relevant to everyone on this line.

When you put someone on cc:, you’re stating “this message isn’t to you, but you should be aware of it.” When replying to a message, it’s common practice to reply to all, but bump some people to the cc: line. Those are the people that should still be in the loop, but only as a secondary concern.

One warning: sometimes a person will get huffy when replying to an email, and they’ll add a bunch of managers on the cc: line. Or they’ll email you and cc: your manager. This is a jerk move. Dragging managers into a email thread that’s going downhill doesn’t fix problems—it makes them much, much worse.

Finally, when you bcc: someone, you’re saying, “this message isn’t to you, but I’m slipping you a copy and nobody knows it.” Quite simply, don’t ever use bcc:. If the person on bcc: hits “reply all,” you’re hosed. Just send your message with no bcc:, then forward a copy to that recipient. That’s much safer.

For an already-running email thread, be careful when adding or removing people from the distribution. When adding, include a line near the top such as:

[adding Bob to cc:, this thread is relevant to his team.]

Removing is tricky, as some people may take offense. The most tactful approach is to send a brief reply-all indicating that you’ll start a new thread, and your reason for doing so. Then send a second email to the trimmed-down list.

Content and Structure

An email’s content contains between one and three major chunks: the subject line, the first couple of lines of the message body, and then the rest of the body. Technically, of course, the message body is just one thing. In practice, however, the first couple of lines need special attention.

Subject Line

You must get your subject line right if you want people to read your message. Vague subject lines get an instant delete from busy readers. A concise subject line, on the other hand, is a blessing.

Let’s look at a few examples:

Subject: Replicator software v2.34 rolling out Friday

My take: good. Selects its readers (people who care about the software release) clearly, and communicates the most pressing detail (rollout date) before the reader even opens the message.

Subject: Project delayed!

My take: bad. Fails to indicate who should read it, the nature of the delay, and whether or not the situation is under control. Combined with exclamatory tone, this subject line will wind up a bunch of people—maybe not the right people—and your manager will spend the next week doing damage control.

Subject: Need your approval: new switch for test lab

My take: good. Use of “your” implies one recipient, presumably the person that approves lab equipment. Clearly implies that the recipient needs to take action, the nature of the action, and what it relates to.

Subject: Josh, you have notifications pending

My take: bad. If a notification is important, it should be summarized in the subject line. Instead, this email just sounds like a social networking website desperately wanting to show me ads.

First Couple of Lines

This is “the lead,” your opportunity to capture the audience’s interest. Like the subject line, it needs to capture the essence of your message. Any essential or time-sensitive content needs to be here, or at least mentioned so that readers know to look for it. Any summary information for busy managers needs to be here. For an email with many recipients, your lead should narrow the audience.

Have you ever sent a long email that included a question for the recipient, then not received a response? You can guess what happened: your reader saw the length of the email, scanned the first few lines, then moved on. Maybe he had the intention of coming back to it later... just like the other 2,000 old emails in his inbox.

Many times, when you cut the fluff and commentary from your email, you’re left with just a subject line and one paragraph. That’s great—most corporate emails don’t need to be any longer!

Everything Else

At this point in your message, you’re free to go into details. If you’ve done your job well, most readers will already have the detail they needed, and they’ve moved on. The readers wanting the whole story are the only ones still reading. You don’t have license to ramble, of course, but you don’t need to edit with the same scrutiny as in the beginning.

Damage Control

Earlier we discussed choosing the audience for your email. Let’s add a caveat: you only choose the initial audience for your email. Once you hit “send,” your message can go anywhere from there.

The havoc wrecked by forwarded emails can know no end. Before sending anything, ask yourself:

Who could this email be forwarded to?

What damage would it do if it fell into the wrong hands?

If your email criticizes a person, assume it will be forwarded to them. If you insult a piece of code, assume it will be forwarded to the programmer. If you say the company’s products suck, assume it will be forwarded to the CEO.

Therefore, treat all email as etched in stone and notarized. There’s no delete button, and measures like Outlook’s “recall email” only bring more attention to your mistake. (Note, however, that IT administrators can delete messages from folders on the Exchange server—I’ve seen it happen.)

No-No Email Topics

Given the nature of email, stick to in-person conversations for these topics:

Criticisms of fellow employees. Exception: your manager may ask for a written criticism for that employee’s performance review. In that case, keep it as fact-oriented as possible, and review it in-person with your manager before sending.

Negative personal opinions of company projects and products. Bad-mouthing is generally in bad taste anyway.

Discussions related to patents, especially possible violations of patents. These emails may be subpoenaed by a court.

Any matters of questionable legality. (And in this situation, maybe your conversation should be with your lawyer.)

When in doubt, save your email as a draft. The next morning, read it again, and then decide if it’s appropriate to send.

These rules apply to other electronic formats as well. Your company may be logging instant messages, Twitter, and other communication as it passes through their firewall. Don’t count on privacy while using any corporate network.

Style and Cultural Matters

Many times you need to send email to people you don’t personally know, and they could be anywhere in the world. If you work for a large, multi-national company, even your peers could be scattered across the globe.

In these situations, take a few steps to ensure your message hits its mark:

Beware of American and sports idioms like, “ensure [x] hits its mark.”

Humor is easy to misread, and don’t rely on emoticons (smileys) to fix it. As a rule, don’t use emoticons in professional email.

Some languages have both formal and informal means of addressing a person. Respect their standards, and whenever in doubt, go with formal.

Don’t use ALL CAPS, ever. (But you knew that one already.)

Sure, you might come across sounding stiff. That’s better than sounding disrespectful.

Case Study: Weekly Status Report

Let’s consider a few examples, starting with the humdrum weekly status report.

| To: (Your manager)
| Cc: (Other managers, as needed)
| Subject line: 2012-04-27 Status report, Josh Carter, Project Foo
| [possible delay]

So far nothing special. The subject line includes the date and name, which seems redundant, but I include it because status reports are commonly forwarded. I also added “[possible delay]” as a heads-up. Now for the body:

| Foo project delayed because prototype board won’t boot. I
| suspected solar radiation, so I started designing a test to
| determine the correlation between sun spots and the
| non-workingness of our prototype board. First, I built a
| Faraday cage out of scrap metal from [...]

Whoa, now we’re off track! (In more ways than one.) This level of detail isn’t useful to managers. Let’s try that again:

| Prototype board did not boot, and standard debugging practices
| didn’t resolve the problem. Ordered replacement from vendor;
| arrives Tuesday.

That’s all that needs to be said. If someone wants the full story, let them ask for it. Often, we really want to tell the story so that managers appreciate the amount of work we put in, but your status report isn’t story time. A competent manager will already know the story from day-to-day interactions with the team.

One more item for the status report:

| Bugs 2112, 2113, 2114 resolved. All related to common problem, see
| documentation in wiki [1] and resolution of bug 2112.

[1] http://wiki/engineering/projects/foo/hardware_bugs

This one’s good. Feel free to document useful things you discovered—but do it in the right place. Engineering wikis are good for this, and a link in your status report lets curious readers get the full story.

Case Study: Feasibility Report

Many times we’re asked to investigate some technology and see if it would work with the company’s product. Someone says, “Hey, should we use Ruby on Rails for our web app instead of J2EE?” And then it lands on you to see if that’s feasible and send a summary report to management.

This is a tough one, because while the work itself might be fun, your report needs to distill inherently technical topics into something managers can use to make a decision. Let’s start with the first draft, sent to your manager for initial review:

| To: (Your manager)
| Cc: (Other team members who helped)
| Subject: Draft 1 of Rails feasibility study

Easy enough. Now for the body:

| This document summarizes our study of using Ruby on Rails for
| Project Foo’s web front end. Initial study looks promising, with
| a possible downside of higher latency on certain pages, but still
| within acceptable limits. We recommend moving forward with Rails.

Strong lead! Non-technical readers can stop right there, they’ve got the bit they needed to know. The engineering managers, however, will keep reading.

| Routing is configured using the file config/routes.rb, and you
| create routes by matching lines like match foo/:id => foo#view,
| or you can create REST-style resources with lines like [...]

Stop, stop! Too much detail, this isn’t a book on Rails. Let’s try again:

| Routing is configurable, and while there is a bias toward
| REST-style paths, custom paths are also supported.

That’s all that needs to be said. If someone wants to know how to create a route, they can buy a book on Rails.

Conclusion

While this may seem like a lot of rules about writing email, it all boils down to this: be clear about the audience you’re writing to, and give them the level of information they need. You’ll find your emails getting to the point much faster, and generally getting shorter overall.

You’ll also find, unfortunately, that writing short, concise emails is not faster than writing long ones. Quoting Pascal, “I would not have made this so long except that I do not have the leisure to make it shorter.” It takes effort to put yourself into the mindset of your audience and cut out all the cruft that’s not relevant to them.

But we’re not ones to take the easy way out, are we? Put in the work to write effective email, and your career will benefit accordingly.

Josh Carter is the author of the New Programmer’s Survival Manual. He’s a professional programmer with more than 15 years of experience, both in programming and engineering management. Josh has a passion for programming and keeping on the leading edge of technology, but it’s balanced by the Steve Jobs mantra, “Real artists ship.” These days he’s equally passionate about helping the next generation of programmers make their own mark in industry.

For more pragmatic career advice, be sure to check out the New Programmer’s Survival Manual, now in print. The Survival Manual includes much more discussion of corporate organization, understanding personalities, and working with a team. Acknowledgements: Thanks to Jim Reekes and David Trachy for their contributions to this article.

Josh is on the web at joshcarter.com and Twitter as @joshcarter. Send him your feedback or discuss the article in the magazine forum.

What Makes an Awesome Command-line Application?

Command-line Apps Can Be Machinery of Beauty

by David Copeland

Command-line applications the most utilitarian of programs, but the most utilitarian machinery can be beautiful.

During the first week of a new job several years ago, I was given a spreadsheet that contained SQL queries used in a manual testing process that I was to carry out. Being a good developer, I converted the spreadsheet into a command-line application that ran the queries automatically. Being new, I did it as quickly as I possibly could, to show my new employers how valuable I was.

The script was horrible. It didn’t take normal options, had almost no error handling, output a hodge-podge of debug messages, and broke in mysterious ways if the environment wasn’t set up just right. But I could run it, and it worked. I moved on to other projects, and six months later, another developer came over to ask me to fix a bug in this script.

It was still being used, and it was this poor sap’s job to run it. But my name was on it, so it was my job to fix it. And add features to it. I decided that, instead of hacking out a fix, I would make this application the best automation script I possibly could. It would be easy to use, helpful, well-documented, and easy to enhance. It would be awesome—and if I did things right, I’d never have to touch it again.

Two days later, I no longer had a script, but an application, just as I’d planned. It was in regular use for over a year after that, during which time I never had to help anyone debug it, fix it, or enhance it.

My book Build Awesome Command-Line Applications with Ruby is all about how to do this, and how to do it in Ruby, which is a great language to use for writing command-line apps. While the book goes into great detail, we’ll hit the high points here, focusing on three steps you can take on the path to making awesome command-line apps.

Step 1—Decide to Care

Calling something a script can be an excuse not to care about writing good code. Stop writing scripts and start writing applications. Everything you know about writing good software should be applied to your command-line apps. Why is it OK to skip error handling or duplicate code in a “quick script,” but it’s not OK in the “real” application you work on?

Before you write your next command-line app, take a minute to adjust your mindset. Decide that you’re not going to hack something together, you’re writing a new application, the same as any other application. You’ll find that with this simple change in attitude, you’ll naturally create a more polished app without spending significantly more time than you might have otherwise.

Step 2—Be Helpful

You’ve decided to craft an application instead of hacking together a script. The most important thing to do from this point on is to make your app helpful. This means making it easy for the user to understand how to use your app, and it means not letting them do something wrong and punishing them for it.

To see what I mean, type this into your terminal:

rm

Did it delete your entire hard drive? No. It showed you a very brief usage statement. Your app should behave similarly. It should not do anything destructive or annoying when invoked without any arguments.

It should also accept both -h and --help as options to show more detailed help. There are tools that automate this almost entirely in most languages. In Ruby, for example, OptionParser makes this dead simple.

Also, give the user actionable error messages. You wouldn’t show a user of your web app an error message from MySQL when they try to choose a user name for themselves that already exists, would you? Treat command-line users the same way. Most languages have plenty of if statements to help you get things just right.

If you want to be really helpful, make a man page. A man page (so-named for being used by the UNIX man command, short for manual) is a place to put all the gory details about how your app works. If you are writing a Ruby app and are distributing your app via RubyGems (both excellent ideas), the gem-man gem will install a man page with your gem. Even if you aren’t using Ruby, the Ruby app ronn allows you to write a man page in Markdown, which is a lot easier to use than nroff.

Step 3—Play Well with Others

Being helpful is about interfacing with humans, but it’s also important to interface well with other programs. The tools available on a typical UNIX system are all small and single-purpose, but they are all designed to interface with each other in a way that allows complex operations to be performed. For example, suppose I’m using Subversion and I’ve just done an update that generated conflicts. I need to resolve them, and I’d like to write one command to edit all the files in conflict.

Here is that command:

vi `svn stat | grep ^C | awk '{print $2}'`

I can then mark them as resolved similarly:

svn stat | grep ^C | awk '{print $2}' | xargs svn resolved

If you aren’t familiar with awk or xargs, don’t worry. My point is that the authors of Subversion never sat down and designed the edit all files in conflict command. In fact, if you were working alongside them and asked them to add a feature to edit the files in conflict, they would rightfully refuse, because they’ve designed their app to play well with others.

This playing-well-with-others thing works because of a few conventions adopted by good command-line apps. The first of these is the proper use of streams. Every command-line app has access to three streams: the standard input stream, the standard output stream, and the standard error stream. When you use a pipe in a shell command, the shell pipes the standard output stream of one app to the standard input stream of the other app.

The standard error stream is printed to the terminal. This means you can provide error messages to the user without interrupting the output of your app. You should respect this convention and only message the user on the standard error stream. Your program’s output should go to the standard output stream.

Further, you should accept input on the standard input stream, at least as an option. Most command-line apps operate on files. If your app does this, it should look for its data on the standard input stream if no files were provided. This allows it to operate in a pipeline just like all other apps. In Ruby, the awesome ARGF library makes this completely transparent.

The second way in which apps can communicate is via exit codes. The idea is this: If an app does what it was asked, it exits with zero. If some error occurs, it exits with nonzero. The use of exit codes allows you to do things conditionally on the success of actions. For example, when I’m ready to deploy something, I want to push my changes to the remote git repository, and then use Capistrano to deploy. If the push fails for some reason, I obviously would not want to run Capistrano. Because git exits nonzero when a push fails, I can do this:

git push dev master && cap deploy

The shell treats && as a “short-circuit” operator, like most programming languages.

Your app can examine these exit codes for itself. The UNIX standard function system returns the exit code, and most programming languages have a similar construct. In Ruby, you could do it like this:

| system("some complicated command")
| if $?.exitstatus == 0
| # everything worked
| else
| # something went wrong!
| end

The final way apps can communicate is via signals. A signal is a value that is given to a process that will interrupt whatever the process is doing. The process can then examine the value of the signal and respond to it. Have you ever used the kill command? All kill does is to signal a process. By default, kill sends the signal TERM, and by default, applications, when receiving TERM, exit. Thus, kill tends to kill processes.

You don’t need to exit when you receive this signal, though; you can do anything you want, such as clean up open resources. You can also respond to other signals. Some applications, for example, will reload their configuration when they receive the HUP signal. In Ruby, you can catch signals with Signal.trap:

| Signal.trap("TERM") do
| # run some cleanup code
| exit 1 # notice how we exit nonzero here
| end

Hopefully you’ve seen the value in taking these three tiny steps toward a better command-line app. By having a helpful and easy-to-use command-line syntax, and by making proper use of I/O streams, signals, and exit codes, your apps will be more flexible, easy to use, and easy to maintain. And that will make your life easier, not to mention help you “level up” as a developer.

David Bryant Copeland, author of Build Awesome Command-Line Applications in Ruby, is a veteran professional software developer who lives on the command line. He speaks frequently at national and regional conferences, and spends his working hours as a developer for LivingSocial in Washington, DC. He’s written applications in C, C++, Perl, Java, Ruby, and Scala, believes in clean code, doing it right, providing a great user experience, and using the right tool for the job.

Send the author your feedback or discuss the article in the magazine forum.

Calendar

Author sightings, partner events, and other notable happenings.

Here’s what’s coming up in the next three months:

Author Appearances

Who’s where, and what for.

May 1–3, MarkLogic WorldWashington, DC

“Stupid XQuery Tricks and XQuery Plugin for IntelliJ”

Ron Hitchens, author of Getting Started With XQuery

May 3–4, SourceDevCon, London, UK

“Vim–precision editing at the speed of thought”

Drew Neil, author of Practical Vim

May 4–5, New York Software Symposium, New York, NY

Multiple talks on JavaScript, HTML 5, Scala, Concurrency, and automated testing

Venkat Subramaniam, the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy, Programming Scala and Programming Concurrency on the JVM

May 8 , Uniforum Chicago, Wheaton, IL

“29 Ways to Get Started in Open Source Today”

Andy Lester, author of Land The Tech Job You Love

May 8–12, Agile Africa 2012, Johannesburg, South Africa

“Pomodoro–can you focus for 25 minutes?”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

May 11, PyGrunn 2012, Groningen, the Netherlands

“Optimize and automate your dev environment”

Reinout van Rees, author of the upcoming book about Django

May 16, Turku Agile Day, Turku, Finland

“BDD as if you meant it” (workshop)

Matt Wynne, author of The Cucumber Book

May 18–20, Lone Star Software Symposium, Dallas, TX

Multiple talks on JavaScript, HTML 5, Scala, Concurrency, and automated testing

Venkat Subramaniam, the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy, Programming Scala and Programming Concurrency on the JVM

May 21–22, JRubConf , Minneapolis, MN

Keynote

Venkat Subramaniam, the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy, Programming Scala and Programming Concurrency on the JVM

May 19–24, Problem Solving Leadership, Albuquerque, NM

“Problem Solving Leadership”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

May 22, JRubyConf, Minneapolis, MN

“Deploying with JRuby”

Joe Kutner, Author of Deploying with JRuby

May 22, MarkLogic User Group London, London, UK

TBA

Ron Hitchens, MUGL Founder and author of Getting Started With XQuery

May 23–24, DevSum 2012, Stockholm, Sweden

“Advanced Regex”

Staffan Nöteberg, author of Pomodoro Technique Illustrated

May 29–31, FluentConf, San Francisco, CA

“The Promised Land: Untangling Async Spaghetti Code”

Trevor Burnham, author of CoffeeScript: Accelerated JavaScript Development

May 30, Project Portfolio Management, Stockholm, Sweden

“Project Portfolio Management”

Johanna Rothman, author of Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects, Manage It!: Your Guide to Modern, Pragmatic Project Management, and Behind Closed Doors: Secrets of Great Management

June 6–8, Norwegian Developer Conference, Oslo, Norway

Multiple talks related to Functional programming, F#

Venkat Subramaniam, the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy, Programming Scala and Programming Concurrency on the JVM

June 10–15, Better Software/Agile Development Practices West, Las Vegas, NV

keynote: “Influence and Authority: Using Your Personal Power to Get Things Done”

Johanna Rothman, author of Behind Closed Doors: Secrets of Great Management Manage It! Your Guide to Modern, Pragmatic Project Management Manage Your Project Portfolio: Increase Your Capacity and Finish More Projects

June 12, Connected World Conference, St. Charles, IL

Home Automation Panel

Mike Riley, author of Programming Your Home

June 13, YAPC::NA (Yet Another Perl Conference), Madison, WI

“29 ways to get started in open source today”

Andy Lester, author of Land The Tech Job You Love

June 19–22, UberConf , Denver, CO

Multiple talks related to Scala, HTML5, Concurrency, JavaScript, Test automation

Venkat Subramaniam, the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy, Programming Scala and Programming Concurrency on the JVM

June 26–29, Open Source Bridge, Portland, OR

“Logic Lessons That Last Generations”

Ian Dees, co-author of Using JRuby and author of Scripted GUI Testing With Ruby

June 27, Bootstrapd—a community gathering for software craftspeople with an independent spirit, Edinburgh, UK

Bootstrapd—conference organizer

Matt Wynne, author of The Cucumber Book

June 28–30, CocoaConf, Herndon, VA

“Mobile Movies with HTTP Live Streaming” and “Core Audio Cranks It Up”

Chris Adamson, co-author of iOS SDK Development

June 29, Scottish Ruby Conference, Edinburgh, UK

“TDD your command line apps for fun and profit”

David Copeland, Author of Build Awesome Command-Line Applications in Ruby

June 29–30, Scottish Ruby Conference, Edinburgh, UK

“Hexagonal Rails”

Matt Wynne, author of The Cucumber Book

June 29–30, Utah Software Symposium, Salt Lake City, UT

Multiple talks on JavaScript, HTML 5, Scala, Concurrency, and automated testing

Venkat Subramaniam, the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy, Programming Scala and Programming Concurrency on the JVM

July 19, OSCON 2012, Portland, OR

“Threads and JRuby, the Simple Alternative to Evented”

David Copeland, Author of Build Awesome Command-Line Applications in Ruby

July 20, OSCON, Portland, OR

“Your Own Metric System”

Ian Dees, co-author of Using JRuby and author of Scripted GUI Testing With Ruby

O’Reilly Events

Upcoming events from our friends at O’Reilly.

May 29–31, Fluent Conference, San Francisco, CA

Fluent Conference: “The O’Reilly Fluent Conference is about everything JavaScript. If you’re developing for the Web, desktop, or mobile, knowing the ins and outs of JavaScript and related technologies is critical. Come to Fluent to learn from expert developers who are using JavaScript in all kinds of contexts, to do things that no one ever expected JavaScript could do, and do so well.”

June 25–27, Velocity Conference Santa Clara, CA

Velocity Conference: “Velocity is the best place to learn from peers, exchange ideas with experts, and share what has worked (and equally importantly, what has not worked) in real world applications. The focus is on practical, large-scale web development, presented by experts.”

July 16–20, O’Reilly Open Source Convention Portland, OR

OSCON: “If you work with open source, you’ll want to be here. With a dozen plus tracks and hundreds of sessions, you’ll find practical tutorials, inspirational keynotes, and a wealth of information on open source languages, platforms, and development.”

USENIX Events

What’s coming from our USENIX friends.

May 11–12, PICC ’12, New Brunswick, NJ

Professional IT Community Conference: “The third annual Professional IT Community Conference (PICC) is requesting individuals who are interested in presenting papers, talks, or participating in panels on any aspect of IT and System Administration.”

June 4–6, 5th Annual International Systems and Storage Conference Haifa, Israel

SYSTOR 2012: “SYSTOR 2012, the 5th annual international systems and storage conference, promotes experimental and practical computer systems research.”

June 7–8, 4th USENIX Workshop on Hot Topics in Parallelism Berkeley, CA

HotPar ’12: “HotPar will bring together researchers and practitioners doing innovative work in the area of parallel computing. HotPar recognizes the broad impact of multicore computing and seeks relevant contributions in all fields, including application design, languages and compilers, systems, and architecture.”

June 12–13, 4th USENIX Workshop on Hot Topics in Cloud Computing Boston, MA

HotCloud ’12: “HotCloud brings together researchers and practitioners from academia and industry working on cloud computing technologies.”

June 12–15, 2012 USENIX Federated Conferences Week Boston, MA

FCW ’12: “Back for 2012, USENIX is combining established conferences and workshops into a week of research, trends, and community interaction.”

June 12, 2012 USENIX Women in Advanced Computing Summit Boston, MA

WiAC ’12: “This is an all day event for those interested in advancing women in computing representing both academia and industry.”

June 13–15, 2012 USENIX Annual Technical Conference Boston, MA

USENIX ATC: “USENIX ATC has always been the place to present groundbreaking research and cutting-edge practices in a wide variety of technologies and environments. USENIX ATC ’12 will be no exception.”

June 13, 4th USENIX Workshop on Hot Topics in Storage and File Systems Boston, MA

HotStorage ’12: “Over the past few years, storage has been changing like never before. Architectures are evolving rapidly, with many fundamental shifts. The HotStorage workshop aims to provide a forum for the cutting edge in storage research where researchers can exchange ideas and engage in discussions with their colleagues.”

June 13, 3rd USENIX Conference on Web Application Development Boston, MA

WebApps ’12: “WebApps ’12 is designed to bring together experts in all aspects of developing and deploying Web applications. The program includes refereed papers on topics such as distributed systems and browser extensions, Java and client extensions, and more, in addition to keynotes by Douglas Crockford (Yahoo!), and Yehuda Katz (Tilde, Inc.)”

June 14–15, 4th USENIX Workshop on the Theory and Practice of Provenance Boston, MA

TaPP ’12: “Provenance provides important documentation that is an essential part of the quality of data, and it is essential to the trust we put in, for example, the data we find on the Web and the data that is derived from scientific experiments. TaPP ’12 covers topics related to theoretical or practical aspects of provenance, including but not limited to: provenance in databases, workflows, programming languages, security, software engineering, or systems; provenance on the Web; real-world applications of or requirements for provenance.”

June 15, 6th USENIX/ACM Workshop on Networked Systems for Developing Regions Boston, MA

NSDR ’12: “NSDR ’12 will provide a venue for researchers to propose and discuss ideas concerning the design, implementation, and evaluation of new computing and communications technologies to support the sustainable development of developing regions. NSDR ’12 specifically focuses on the technical networking and systems research challenges that arise in the design, implementation, and deployment of new computing solutions appropriate for developing regions.”

July 23–24, Asia-pacific Systems Workshop Seoul, South Korea

APSYS 2012: “Building on the success of its 2010 and 2011 predecessors in New Delhi, India and Shanghai, China, APSys 2012 will be a lively forum for systems researchers and practitioners across the world to meet, interact, and collaborate with their peers from the Asia/Pacific region.”

Other Happenings

May

In May, 1980, Erno Rubik’s puzzle started selling internationally under the name “Rubik’s Cube.”

May 1

2006 MacArthur Fellow Jim Fruchterman is 53.

May 2

Protoblogger Dave Winer is 57.

May 8

jQuery creator John Resig got to age 28 one day ahead of Mark Zuckerberg.

May 9

Red Hat founder Marc Ewing is 44.

May 11

Edsger Dijkstra’s 82nd birthday.

May 14

Zillionaire entrepreneur Mark Zuckerberg is 28.

May 17

Youthful Alan Kay is 72.

May 19

Today is the eleventh anniversary of the opening of the first Apple retail stores. Also, Java daddy James Gosling is 57 today; Theo de Raadt, founding member of NetBSD and founder of OpenBSD and OpenSSH, is 44; SNOBOL creator Ralph Griswold would have turned 78 today; and CP/M inventor Gary Kildall would have turned 70.

May 26

Ward Cunningham is 63.

May 27

PEZ was trademarked on this date in 1952.

May 29

63rd birthday of shareware co-creator Bob Wallace.

May 31

John Kemeny was born on this day in 1926.

June

In this month in 1957, computer science legend Donald Knuth published his first article, “Potrzebie System of Weights and Measures,” in Mad Magazine.

June 1

Tim Paterson, whose operating system made Bill Gates a jillionaire, is 55.

June 8

Tim Berners-Lee is 57.

June 23

Vint Cerf is 69.

June 23

Mathematica is 24.

June 28

Firefox co-creator Dave Hyatt is 30.

July 2

Gordon Letwin, one of first dozen Microsoft employees and lead Microsoft architect on OS/2, is 60.

July 5

Linguist Victor Yngve, creator of COMIT, the first string processing language, is 92.

July 6

Adi Shamir, the S of the RSA encryption algorithm, is 60.

July 7

Smalltalk co-creator Adele Goldberg is 67.

July 9

Marc Andreessen is 41.

July 10

Legendary game designer Scott Adams is 60.

July 16

Dan Bricklin is 61.

July 22

Alan Cox, early Linux kernel contributor, is 44.

July 24

Al Lowe, creator of Leisure Suit Larry series of computer games, is 66.

July 26

Hakon Wium Lie, creator of Cascading Style Sheets, is 47.

July 29

James Duncan Davidson is 42.

Shady Illuminations

Broken by Design

by John Shade

Anthony Bourdain’s habits, the Junior Ganymede Club, and the Lidless Eye all find their way into John’s exploration of privacy.

Lately I’ve been analyzing the entrails of the blogosphere for intel in the War on Privacy. In this war, acronyms rumble over the horizon one after another like invading tanks, first SOPA, then PIPA, then CISPA, all threatening to “rip apart the fabric of the internet, compromise the planet’s digital security[,] and open the doors for China-class censorship.” And, of course, roll over our privacy like Arthur Dent’s house.

Lately the fighting in this war has taken some interesting turns. Advance troops on a couple of fronts have taken on friendly fire. The CIA is worried that in the pursuit of the War on Terror the governments of the world are getting too sophisticated in their invasion of citizens’ privacy. Yes, the CIA. And in a setback to police privacy, a judge ruled that it actually is legal to shoot the cops, so to speak. This ruling was somewhat balanced by a ruling that police dashcam videos can be kept secret, a decision that, had it come sooner, might have spared us a drunken man singing all of Bohemian Rhapsody on a police dashcam.

Clearly things are getting more complicated for the spies and police, so maybe what we need is an app to help them out. What if we just spy on ourselves and collect the data for them? That would be thoughtful.

Dead Men Are Blabbermouths

And what about when you die? How will you ensure that your heirs or the government or your former employers or, gosh I don’t know, any number of interested parties, can get access to all your emails? Fortunately, there’s an app for that, too. In fact, more than one.

And you want all this. Because what you really, really need is a Jeeves to pick out your socks and handle all the minutia of your daily life so you can focus on the really important things.

“Jeeves lugged my purple socks out of the drawer as if he were a vegetarian fishing a caterpillar out of his salad.” -P. G. Woodhouse

But there’s a cost: personal assistants are personal. So if you want to be Bertie Wooster, you have to put up with Jeeves raining on your inimitable sense of style and walking in on you while you’re playing with the rubber duckie in the bath. And if you steal policemen’s hats or eat seal eyeballs, he’ll put it in that book at the Junior Ganymedes Club.

Thanks for Sharing That

Disgusting stuff Anthony Bordain has eaten: seal eyeballs, sheep testicles, ant eggs, duck embryo, goat head, warthog rectum, a still-beating cobra heart.

I didn’t learn about Anthony Bourdain’s disgusting eating habits from peeking in his kitchen window. He willingly shares this information about himself. Apparently it doesn’t embarrass him. He has a different sense of privacy than I have, because if I ate this stuff I would pay good money to keep it hushed up.

Then again, I read blogs for a living. Much like Anthony Bordain, I also travel the world consuming entrails. And what I read in the blog offal tells me that a lot of people have a real need to violate their own privacy.

But privacy isn't just about being seen, it's about your attention. Today your mental privacy is taking heavy shelling from a constant bombardment of what I suppose we could technically call information. The invasion is dumping on you and taking from you at the same time, and apparently you want it all.

You Are the Product

Publishing is not dying, it's the new software distribution model. Subscriptions, conversions, opt-out, direct withdrawal, the first hit is free: the whole vocabulary of publishing is now the jargon of software sales. In this model, the product is attention. The companies adopting it are selling you. To whom, I’m not sure. Possibly to each other. But their assault on your privacy is driven by a fierce need to know all about you, the better to package you for sale. It’s driven by a terrible hunger for eyeballs.

You know those jokes about the Google Glass project? “The glass is half full because the other half is reserved for ads?” They aren't jokes.

You will give up your privacy because in exchange for your privacy you’ll be offered something that you consider valuable—by companies that deliver services built on data they collect about you.

Broken Windows

I blame windows. Not Windows specifically, but windows generally. Since the early 1980s, user interaction with technology and with the data and connectivity technology provides has been framed in the metaphor of the window. As soon as Tim Berners-Lee gave is the Web, Marc Andreessen and Eric Bina hung a window on it. We’re like Frodo, perpetually exposed to the Lidless Eye. (You’re right; that sounds like something Anthony Bourdain would eat.)

I don’t remember being asked if I wanted windows surrounding me everywhere. And there is now some evidence that these windows may be broken. May in fact, have been designed that way. Broken windows, Lawrence Lessig said, are a form of architecture.

But we may as well accept it. We’ve been naked in front of the window for decades. Google Glass just another pane.

John Shade want you to know that he is a front for a reclusive author of suspicious background named Keyser Soze, which you might keep in mind if you are tempted to send him your feedback or discuss the article in the magazine forum.

But Wait, There’s More...

Coming Attractions

This may be the last page, but that doesn’t mean it’s the end of the conversation. Keep up with what’s going on in the Pragmatic universe by subscribing to our newsletter and drop by the Pub again next month for more Pragmatism. We’ll keep a seat warm and a mug chilled and a candle burning in the window.

Coming Soon in PragPub

On March 28, Google officially announced version 1 of the Go language. Binary distributions are available for Linux, FreeBSD, Mac OS X and Windows. In the next issue of PragPub, we’ll take a look at this interesting new language that, as Rob Pike says, is being used “for real stuff” at Google.

Coming Soon on the Bookshelf

These books recently went to print: Seven Databases in Seven Weeks (Eric Redmond and Jim R. Wilson) and iOS SDK Development (Chris Adamson and Bill Dudney).  

Right. But to really be in the know, you need to subscribe to our weekly newsletter. It’ll keep you in the loop, it’s a fun read, and it’s free. All you need to do is create an account on pragprog.com (email address and password is all it takes) and select the checkbox to receive newsletters.

While you’re waiting for the next issue of the newsletter or of PragPub to come out, you can follow us on twitter at @pragpub, @PragmaticAndy, or @pragdave. Or on Facebook at facebook.com/PragmaticBookshelf. Or email us at michael@pragprog.com.