PragPub 2013-01: Issue #43

Table of Contents

Features

Functional Programming Basics

by Robert C. Martin (Uncle Bob)

“Uncle Bob” Martin strips the paradigm down to its essentials, and explains why you can and must understand functional programming now.

Web Programming in Haskell, Part II

by Paul Callaghan

Paul continues his deep dive into the Haskell language and functional programming with a look at Web frameworks for Haskell.

Using Memoization in Groovy

by Venkat Subramaniam

Venkat explores the immense time-space tradeoffs of memoization, and explains how Groovy makes memoization easy.

Mac Kung Fu: Breaking the Magician’s Code

by Keir Thomas

Keir breaks the magician’s code and reveals useful secrets that Apple has hidden in its code.

Launching a Gonzo Engineering Project

by Steven K. Roberts

The legendary gonzo engineer shares his secrets for pursuing crazy dreams and succeeding in a new series starting in this issue.

Departments

Up Front

by Michael Swaine

If you’re the kind of person who think refactoring code to make it conceptually cleaner is exciting, then consider this issue your guidebook to adventure.

Choice Bits

We follow Twitter so you don’t have to.

The Quiz

by Michael Swaine

A monthly diversion at least peripherally related to programming.

Calendar

Want to meet one of our authors face-to-face? Here’s where they’ll be in the coming months.

Shady Illuminations

by John Shade

By reading this column you agree to abide by incomprehensible terms of use that John reserves the right to change at any time.

But Wait, There’s More...

Coming attractions and where to go from here.

Except where otherwise indicated, entire contents copyright © 2013 The Pragmatic Programmers.

Feel free to distribute this magazine (in whole, and for free) to anyone you want. However, you may not sell this magazine or its content, nor extract and use more than a paragraph of content in some other publication without our permission.

Published monthly in PDF, mobi, and epub formats by The Pragmatic Programmers, LLC, Dallas, TX, and Raleigh, NC. E-Mail support@pragprog.com, phone +1-800-699-7764. The editor is Michael Swaine (mailto:michael@pragprog.com). Visit us at http://pragprog.com

ISSN: 1948-3562

Up Front

Functional Programming and Gonzo Engineering

by Michael Swaine

You can and must understand functional programming NOW.

I’m riffing off Ted Nelson’s challenge from Computer Lib / Dream Machines. A premise of this issue of PragPub is that functional programming’s time has come. And we want to help you get up to speed with this powerful paradigm.

If you already know a fair amount about functional programming but don’t think it has practical value for you, you need to read Paul Calleghan’s article on Web frameworks for Haskell programming. Paul’s been doing a series for us on Haskell, and this month he makes a case for rethinking how you go about Web development.

If you aren’t that familiar with functional programming, our lead feature by “Uncle Bob” Martin will show you why this is a paradigm you can’t afford not to master.

And if Paul and Uncle Bob are sharing knowledge that could be crucial to your career, Steven K. Roberts has some ideas to share that could dynamite your career.

Steven Roberts was the original “technomad,” covering 17,000 miles around the US on a computerized recumbent bicycle from 1983-1991 while publishing tales via CompuServe and GEnie. And he’s kept right on pursuing gonzo engineering dreams. For 30 years Steve has occupied a world where fun is the bottom line and a living can be made on the opportunistic spinoffs of creativity, not by selling one’s life for a salary. In a series starting in this issue, he’ll share everything he’s learned in a lifetime of gonzo engineering, from scrounging parts to finding sponsors. If it’s possible to teach how to pull off the impossible with an insanely audacious project, Steven’s the guy to do it.

As you may have figured out already, this issue of PragPub is packed with good stuff. Hang on, there’s more.

Venkat Subramaniam shares some Groovy code that also serves as an exceptionally clear exposition of dynamic programming, closures, and memoization. Keir Thomas takes you deep inside the Mac OS to reveal some hidden secrets and tricks for discovering your own. Our resident curmudgeon, John Shade, tackles the Instagram snafu. Plus we’ve got a Quiz, our Events Calendar, and some odds and ends in a collection we call Choice Bits.

Choice Bits

A Smattering of Nattering

We follow Twitter so you don’t have to.

What’s Hot

Top-Ten lists are passé—ours goes to 11. These are the top titles that folks are interested in currently, along with their rank from last month. This is based solely on direct sales from our online store.

1^ | NEW | RubyMotion
2^ | NEW | Async JavaScript
3 | 3 | Agile Web Development with Rails
4^ | NEW | Sublime Text 2
5v | 2 | Practical Vim
6 | 6 | Programming Ruby 1.9
7^ | 11 | The Pragmatic Programmer
8^ | NEW | Programming Groovy
9v | 7 | Seven Databases in Seven Weeks
10v | 1 | iOS SDK Development
11^ | NEW | The Definitive ANTLR 4 Reference

End of Year Nostalgia, Wishes, and Prognostications

Childhood toy flashback: Capsela! I just loved Capsela, as much as Lego and Lite Brite. Haven’t seen it in years. Not a thing anymore? — @ginatrapani

Even more nostalgic magic. You can download the ORIGINAL VINTAGE release of Rails 0.5.0 from Rubyforge So slim. — @dhh

I know, first world problem, but... I wish Xbox had an “update all” mechanism. — @marczak

Watching people talk about how validating HTML isn’t important makes me wish I was their boss so I could fire them. — @bphogan

I’m waiting for the Cars prequel which shows the cars gain sentience, rise up, and destroy humanity. — @Its_Death

I guess Ada 2012 will be the new standard that all US Govt contractors get a waiver from using. — @mtnygard

Apple working on 50” wrist-worn HDTV. — @BodyofBreen

Apple working on its Bogart impression. — @BodyofBreen

My grandkids will have jobs whose titles didn’t even exist for my generation. Like drill press operator, farmer, and assembly line worker. — @mtnygard

Can it please to stop snowing now. — @jeff_lamarche

The Wisdom of the Web

@pragdave I’ve tried using logic, with mixed results. — @tottinge

More often than not the creative process is more interesting than the creation. — @adamgoucher

Rails is omakase. You’re free to say no unagi, but don’t expect the chef to stop serving it at his restaurant because you don’t like it. — @dhh

In software, only new stuff is old hat. Anything written over ten years ago is new material. — @tottinge

“Feedback” makes a fabulous transitive verb. — @chadfowler

We really need anthropologist-types observing great designers, rather than relying on designers’ self-reports. cf: http://bit.ly/aKjCli — @marick

Programmers often say “inefficient” when they mean “slow,” sort of like “utilize” instead of “use.” It sounds smarter, but isn’t. — @petdance

Tip: to trick your colleagues into doing something with you, just “-athon”ify it. Testathon, bootstrapv2athon, everything can be -athon’ed. — @pamelafox

Sometimes, waking from a dream is like rolling back a regrettable transaction. — @mtnygard

I’m not going to make my daughter choose a religion, I’ll explain the differences & when the time comes she can choose either Marvel or DC. — @TashyP_

Annual reminder to Hollywood. The people reading your anti-piracy screens paid for the movie. The people torrenting it didn’t. — @raganwald

The English language really needs a good translation of “svårmod.” — @olabini

Who Are Those Guys?

First, they’re not all guys. Second, we have to confess that we cleaned up their punctuation and stuff a little. OK, who they are: Ola Bini, Reg Braithwaite, Chris Breen, Steve Death, Chad Fowler, Pamela Fox, Adam Goucher, DHH, Brian P. Hogan, Jeff LaMarche, Andy Lester, Edward R. Marczak, Brian Marick, Michael Nygard, Tim Ottinger, Tashy, and Gina Trapani.

You can follow us at www.twitter.com/pragpub.

Functional Programming Basics

What’s It All About?

by Robert C. Martin (Uncle Bob)

Confused about functional programming? “Uncle Bob” Martin strips the paradigm down to its essentials, and explains why you can and must understand functional programming now.

By now you’ve almost certainly heard of functional programming. I mean, how could you miss it? Everybody’s talking about it. There are all these new functional languages coming out like Scala, F#, and Clojure. People are talking about older languages too, like Erlang, Haskell, ML, and others.

So what’s this all about? Why is functional programming The Next Big Thing (tm)? And what in blazes is it?

First, it’s almost certainly true that functional programming is the next big thing. There are good solid reasons for this that we’ll explore later in this article. But in order to understand those reasons, we need to know what functional programming is.

I’m going to upset a lot of people with this next statement because I’m going to resort to extreme minimalism. I’m going to reduce functional programming down to its simplest core; and this isn’t really fair because the topic is rich and expressive and full of wonderful concepts. You’ll see hints of some of them here. But for now, I’ll simply define functional programming this way:

Functional programming is programming without assignment statements.

Oh no! Now I’ve gone and done it. The functional programmers out there are gathering their pitchforks and torches. They want my head for uttering such minimalist blasphemy. Meanwhile all the folks who hoped to learn what functional programming really is are about to stop reading because the above statement is so blatantly absurd. I mean: how in the world can you program without assignment?

The best way to explain that is to show an example. Let’s look at a very simple program in Java: the squares of integers.

| public class Squint {
| public static void main(String args[]) {
| for (int i=1; i<=25; i++)
| System.out.println(i*i);
| }
| }

Who hasn’t written that program, or some simple variant of it? I must have written it many hundreds of times. It’s often the second program I write in a new language, and the second or third program I teach new programmers to write. Everybody knows the good old squares of integers!

But let’s look at it closely. It’s just a simple loop with variable named i that counts up from 1 to 25. Each loop through the program causes the variable i to take on a new value. This is assignment. A new value is being assigned to the variable i every pass through the loop. If you could somehow peer into the memory of the computer and stare at the memory location that held the value of i, you would see the value held by that memory change from one iteration of the loop to the next.

If the previous paragraph seemed to have belabored an obvious point, let me point out that whole papers have been written on this topic. The concepts of identity, value, and state may seem intuitive to us, but they are actually a very rich topic in and of themselves. But I digress.

Let’s look at a functional program for the squares of integers. We’ll use the language Clojure for this, though the concepts we’re going to explore work the same in any functional language.

(take 25 (squares-of (integers)))

Yes, you are reading that correctly; and yes, that is an actual program that produces actual results. If you must see the results they are:

(1 4 9 16 25 36 49 64 ... 576 625)

There are three words in that program: take, squares-of, and integers. Each of those words refers to a function. The left parens in front of those words simply mean: call the following function and treat everything up to the closing right paren as its arguments.

The take function takes two arguments, an integer n, and a list l. It returns the first n items of l. The squares-of function takes a list of integers as its argument and returns a list of the squares of those integers. The integers function returns a list of integers in sequential order, starting at 1.

That’s it. The program simply takes the first 25 elements of a list of the squares of sequential integers starting at 1.

Read that sentence again; because I did something important there. I took the three separate definitions of the functions that I gave you in the preceding paragraph and combined them into a single sentence. That’s called: (are you ready for the buzzword?) Referential Transparency. [cue: Fanfare, glitter, ticker tape, cheering crowds]

Referential Transparency simply means that, in any given sentence, you can replace the words in that sentence with their definitions, and not change the meaning of the sentence. Or, more importantly for our purposes, it means that you can replace any function call with the value it returns. Let’s see this in action.

The function call (integers) returns (1 2 3 4 5 6 ...). OK, I know you’ve got questions about this, right? I mean, how big is this list? The real answer is that the list is only as big as I need it to be; but let’s not think about that just now. For the moment just accept that (integers) returns (1 2 3 4 5 6 ...); because it does!

Now, in our program, we can replace the function call (integers) with its value. So the program simply becomes:

(take 25 (squares-of (1 2 3 4 5 6 ...)))

Yes, I did that with copy and paste; and that’s also an important point. Referential Transparency is the same as copying the value of a function call and pasting it right on top of that function call.

Now, let’s do the next step. The function call (squares-of (1 2 3 4 5 6 ...)) simply returns a list of the squares of the numbers in its argument list. So it returns (1 4 9 16 25 36 49 64 ...). If we replace this function call with its value, our program simply becomes:

(take 25 (1 4 9 16 25 36 49 64 ...))

And, of course, the value of that function call is simply:

(1 4 9 16 25 36 49 64 ... 576 625)

Now let’s look at our program again:

(take 25 (squares-of (integers)))

Notice that it has no variables. Indeed, it has nothing more than three functions and one constant. Try writing the squares of integers in Java without using a variable. Oh, there’s probably a way to do it, but it certainly isn’t natural, and it wouldn’t read as nicely as my program above.

More importantly, if you could peer into the computer’s memory and look at the memory locations used by my program, you’d find that those locations would be initialized as the program first used them, but then they would retain their values, unchanged, throughout the rest of the execution of the program. In other words, no new values would be assigned to those locations.

Indeed, this is a necessary condition for Referential Transparency, which depends on the fact that every time you invoke a particular function call, you will get the same result. The fact that computer memory for my program does not change while my program is executing means that the call (f 1) will always return the same value no matter how many times it is called. And that means I can replace (f 1), wherever it appears, with its value.

Or to say this another way: Referential Transparency means that no function can have a side effect. And, of course, that means that no variable, once initialized, can ever change its value, since assignment is the quintessential side effect.

So why is this important? What’s so great about Referential Transparency? Given that it is possible to write programs without assignment, why is it important?

You are almost certainly reading this on the screen of your computer. Or if not, you have a computer nearby. How many cores does it have?

I’m typing this article on a MacBook Pro with 4 real cores. (They say it has 8, but I don’t count all that “hyperthreading” nonsense. It has four.) My previous laptop had two cores. And the one before that had just one. The only conclusion I can draw is that my next laptop will truly have eight cores; and the one after that will likely have 16.

The poor hardware engineers, who have carried us on their backs for the last four decades, have finally hit the speed of light limit. Computer clocks simply aren’t going to get much faster. After doubling every 18 months for longer than most programmers (except me) have lived, the runaway growth in computer speed has slammed to a halt, never to rise again.

So those hardware engineers, in an effort to give us more and more cycles per second, have resorted to adding more processors into our chips, and there seems no end to how many processors that will lead to as the years march onwards.

So let me ask you this, O skilled and competent programmer: How are you going to take advantage of every computer cycle available to you when your computer has 4096 cores in it? How will you marshal your function executions when they must run within 16384 processors all contending for the same memory buss? How will you build responsive and flexible websites when your models, views, and controllers must share 65536 processors?

Honestly, we programmers can barely get two Java threads to cooperate. And threads are baby-food compared to the meat and potatoes of real processors fighting over the buss. For over half a century programmers have made the observation that the processes running in a computer are concurrent, not simultaneous.

Well, boys and girls, welcome to the wonderful world of simultaneity! Now, how are you going to deal with it?

And the answer to that is, simply, “Abandon all assignment, ye who enter here.”

Clearly, if the value of a memory location, once initialized, does not change during the course of a program execution, then there’s nothing for the 131072 processors to compete over. You don’t need semaphores if you don’t have side effects! You can’t have concurrent update (pardon me: Simultaneous Update) problems if you don’t update!

So that’s the big deal about functional languages; and it is one big fricking deal. There is a freight train barreling down the tracks towards us, with multi-core emblazoned on it; and you’d better be ready by the time it gets here.

Robert Martin (Uncle Bob) (@unclebobmartin) has been a programmer since 1970. He is the Master Craftsman at 8th Light inc, an acclaimed speaker at conferences worldwide, and the author of many books, including: The Clean Coder, Clean Code, Agile Software Development: Principles, Patterns, and Practices, and UML for Java Programmers. He is a prolific writer and has published hundreds of articles, papers, and blogs. He served as the Editor-in-chief of the C++ Report, and as the first chairman of the Agile Alliance.

A somewhat different version of this article, as well as much more about functional programming, can be found at the author’s blog. You can send him your feedback on this article or discuss the article in the magazine forum.

Web Programming in Haskell, Part II

Rethinking the Structure of Web Applications

by Paul Callaghan

Paul continues his deep dive into the Haskell language and functional programming with a look at Web frameworks for Haskell.

Last issue, Paul began an exploration of Web programming in the Haskell language with Fay, a CoffeeScript analog for Haskell. This time out, he begins the journey into Haskell frameworks.

Rails Is Not Your Application

Before we jump into concrete details of existing Haskell web frameworks, I want to pause to consider the wider picture of how web apps can be structured and developed.

Many frameworks, like Rails, are some variation on MVC and share certain standard components. Are you happy using these? or do you have some sneaking suspicion that it can still be easier or simpler? I’m sure we can do better.

There are several motivations for wanting to explore such issues. Firstly, I think it’s fair to say there is some dissatisfaction in various communities about the maintainability of code, as reflected by the number of talks in recent conferences about different architectural approaches like hexagons and DCI. Secondly, a slight annoyance at needing to do things like check that my routes file matches my controllers etc., and having too many framework details mixed up with business logic. Thirdly, I want to see where taking a top-down functional approach gets me. It may deliver me exactly where newer frameworks are heading, but it should still be an interesting exercise and promote deeper understanding of the issues.

An important slogan for me is “Rails is not your application.” This phrase comes from Nick Henry in the middle of 2011. It began as a tweet, as many great pieces of wisdom do, and subsequently has been expanded into a blog post—and taken up by other people.

There’s no strong consensus on answers yet—we’re still exploring—though it seems clear that more separation is needed between the facilities provided by Rails and the real business logic of the app. For example, domain models shouldn’t always be subclasses of Active Record and could instead be users of services provided by Rails components. Some interesting further discussion can be found on Nick Henry’s blog page.

From a strategic aspect, I’ll be asking whether we can streamline the building of web apps, leveraging the language facilities to reduce or remove certain of the weak points in current practices.

For example, I’ve never been keen on the separation of routes from controllers: it feels like having related information distributed over different files, with the ever-present risk of getting the two out of sync, or else having to ensure that your tests are strong enough to catch any mismatches—and I don’t accept that it has to be this way. Phenomena like this represent possible failure points, and it’s a good technique in programming to try to eliminate or control such potential failures. If you want a simpler example, consider looping through a list to compute some result—do you write an explicit for-loop or do you instead use a higher order function? The latter has fewer moving parts....

In general, we’ll often ask the question: what do we actually need, and can we express it in a straightforward and safe way? There are also some elements of BDD-ish thinking here too: for example, we’ll be asking “What’s the simplest thing that would work,” without committing ourselves to an existing framework.

Let’s consider the questions in the context of a concrete application as a guard against getting too abstract, and as a source of examples. How about this: a distributed to-do list tool, which allows users to maintain a set of lists, add and remove items from a list, and mark items as completed, maybe with some advanced features like sharing a list between several users. This is something of a test application for several Javascript front-end libraries, too.

Finally, don’t expect too many answers in this article. My main goal is to ask the questions and see what your response is!

Towards a Functional Approach

We’re going to put data first, over process or tradition, and see where it leads us.

Firstly, the bits we can’t easily change—the Http protocol and how it is used by browsers. Browsers make requests to a host with a certain path string and maybe some parameters. Typically, they will also submit session information too. The expectation is that the server will return some text or Html for rendering, or possibly some javascript code to execute. (Let’s ignore static things like images for now.)

Somehow, the app on server will decode the request into something more structured, run some code, and then return a piece of text. Typewise, we can start thinking of String -> String for the type of the transaction, or being a bit more precise or informative with Request -> Response, where Request represents a path and parameters with the relevant Http verb (eg GET, DELETE etc.), and Response can contain various bits of meta-information like mime type in addition to the raw response.

Html is (or is generally intended to be) stateless, and this is reflected well in the simple type above.

However, session information can sometimes muddy the water and make the requests context dependent. For example, it could depend on whether someone is logged in or not. There will also be real-world side effects: doing a DELETE of some resource will affect the outcome of any GET of the same later. A simple way to encode this context dependency is to refine our type to (Sessionable server, Monad server) => Request -> server Response, that is, explicitly requiring the transaction (well, the generation of some response given a request) to be a monadic action and following the usual monad rules and conventions, plus supporting various session operations.

(Note that I’m not committing to a particular monad instance yet—just in vague terms signalling that I intend to use the wider general monad api or framework.)

We should also consider the component that sits between the screen and the chair, the not always perfect user. What’s their type? Ideally they would be Response -> Request, turning server responses into new requests, by processing the information returned and then deciding what to do about it. We should allow some more leeway, though, and let them be monadic too, since side-effects may influence how they act. So let’s give them a potential type of Monad b => Response -> b Request. There may be some unpredictable elements of user behavior too, but we can hide those away in a monad as well(!).

The First Big Question

From the outside, we can conceive of the complete web app as having a type of (Sessionable a, Monad a) => Request -> a Response. What’s inside this monadic function? That is, how does the web-oriented machinery link up with the underlying business app?

Following the “Rails is not your application” idea, we can ask a very important question: can the business application be entirely separated from the web details? I’m going to assume for now that the answer is yes. Does anyone think this is controversial?

It may be that we find aspects of the business app that are hard to divorce from its web presentation, but this for me is a prompt to look for new ways to structure those aspects to reduce the coupling.

Anyway, I’m going to assume that the two are separable, and furthermore that we can structure the web side of the overall application as a wrapper around the core business app.

This begs the question, what does the underlying non-web app look like? I would hope to see data types corresponding to domain concepts, and code that allows business-oriented processing of elements of those types. We could use the code in a REPL to perform certain tasks, as for the running example, to create a list for a user and manipulate the entries in the list.

What’s missing? Clearly it’s not convenient to use, so we would like to add some structure to the interaction side, and eventually to make it look nice and be pleasant to use. The first aspect is fairly independent of whether we’re aiming eventually at a web app, in the sense of it being useful for a stand-alone desktop app as well. The second part is clearly more in the web domain.

Database Aspects

Let’s get this out of the way early. It is a massive topic, where a lot of work has already been done, though one which we should try to disentangle from the business logic. Web apps typically need some kind of persistent storage, firstly to store data between processing of discrete requests, and secondly because the app will typically run as a distributed system on multiple physical servers. But what does the business app actually need? What’s the simplest thing that would work?

Mainly, what we want is a collection of ways to create new data, retrieve it, and modify or delete it. Most of the time, it doesn’t matter what is handling these operations, so that’s a dependency we should try to abstract out. We’ll also need a way to ensure that destructive updates are done in the right order, so monads might be a useful thing to use here too.

We might try something like this: a type class Storage that provides basic storage facilities for data that is serializable (or mappable to some database representation), using globally unique identifiers (GUIDs) to reference particular data. The “store” operation might also want to check for validity of the data before saving it, though an alternative way is to design the business types so that invalid data is excluded from the types (simple example: that 123 would be rejected as a String). A sub-class could provide more powerful search facilities, taking some collection of predicates on some type and returning a list of GUIDs of the objects found. (Note that GUID type has one parameter: this will help to distinguish GUIDs for one type from those of another type, and so help to prevent a GUID for one record type being used to retrieve another. In contrast, most frameworks just use unprotected integers for their GUIDs.)

| class Monad s => Storage s where
| store :: (Serializable a, Validateable a) => a -> s (GUID a)
| fetch :: Serializable a => GUID a -> s a
| delete :: GUID a -> s ()
| class Monad s => Searchable s where
| search :: [PredsOn a] -> s [GUID a]

I envisage these storage facilities being used outside of the core business app, thus being used to store data that is computed by core functions, or to retrieve data that is used in core functions. Until proved otherwise, I’d like to keep this separation strict, and hence keep the business logic as simple as possible.

Identifying the Interactions

Let’s return to the question of how to convert the types and functions on the domain objects into a recognizable application, by considering what extra we have to add to make an application. One approach is to recognize that we’re wanting to construct various user stories and common tasks out of the basic domain operations. For example, allowing a user to create a new list with a name, and to have it saved if valid.

Consider though, that there are certain stories we want to support, and some combination of actions that we do not. You might be thinking that we can use tests as a way to start distinguishing what is allowed and what isn’t, but we can be more functional here. What about representing this in types?

One possibility is to encode allowed actions as a set of values in some type, for example the simple set of actions:

| data UserAction
| = CreateList { list_name :: String }
| | TickList (GUID List)
| | DeleteList (GUID List)
| | ...

Remember a key point of (modern) FP: that types are cheap—easy to create and use—so if it helps our programming to introduce a type, it’s a technique well worth considering.

Another thought is that the application interface, or the interaction that the user has with the application, is not just a random collection of stuff: there is some kind of structure there, for example indicating what is allowed and what isn’t, and we should look to exploit this structure rather than let it be implicit in the documentation or in the tests. (Notice: it’s another example of trying to use the language to lift the level of the programming, turning the implicit into explicit and so to reduce potential failure points.)

You might also consider the interaction as being carried out in some kind of “formal language” or structured game, where some sentences or moves are ok and others are invalid or highly inappropriate(!).

Once we’ve managed to encode the interaction in some concrete way, the next step is to write an “interpreter” for it that will turn the user’s requests into operations on the business concepts, that is, it will “run” the requests.

You can probably have a similar setup on the other side, namely a set of interactions that the system will need to have with the user. For example, reporting successful creation of a named list, or showing a list of results after some search query. Note also that several types will be needed to represent the interactions, such as representing certain modalities or contexts, like certain operations only being valid in certain states. You might consider it as a hierarchical structure of interactions, like the top level of the tree allowing the “big” operations like creating or selecting a list, with a structure under the latter indicating what can be done once the list has been selected.

To recap, since this might be a strange idea for some and it’s a key part in what I’m suggesting: we should attempt to capture what kind of interactions or operations the user can have with the application by representing them via various mechanisms in the language, such as new datatypes, and to link these representations to the underlying business logic via techniques like interpretation. The point is to encode structure where we can, with a view to exploiting it for clear code and to avoid failures.

Navigation and Routing

By navigation, I mean meaningful moves around the application’s interface, typically from a certain context to relevant or useful functionality. In our running example, like from the display of a given list to either back to viewing all lists or to being able to edit the list, or even to deleting it. Effectively, this means identifying which interface actions are relevant or useful to the current action, and this is easily represented in code.

This brings us nicely to the issue of routing. Traditional frameworks identify various formats of Http path and map them to calls of certain code, usually methods inside certain controllers. There are various potential points of failure here, such as the routes spec not matching up with the controllers, or mismatches between parameters extracted from the route versus those expected in the controller. It’s a bit boring having to test this if we can do better.

What do we actually need, though? Basically, we want to map a string from an Http request into an invocation of certain code. But where do these strings come from? Typically, they come from the routes spec being used in reverse, where some view or controller code references a high-level name for a certain piece of functionality. This does the job, mostly, of ensuring that the relevant functionality is called.

How about a different idea: it doesn’t really matter what string is generated, as long as it gets resolved to the right call in the end! That is, the path string just needs to contain enough information to ensure the relevant interface action is executed, and nothing else. (We could also encrypt whatever representation we use, if we want extra confidence.) What information do we need to include? One possibility is just some encoding of the actual interface operation the link represents, like some element of the UserAction type above.

We probably also need some context information too; for example, if working on a selected list then we need to know which list. Here’s a very useful FP concept that works well here: the Zipper. It’s an idea due to Gerard Huet that has been around for most of my lifetime. It helps with traversal and editing of functional data structures by representing a kind of cursor for where one is in a data structure, together with information for reconstructing the value after traversal. As a simple example, if one has traversed a binary tree from the root node by following left, right, right, then the zipper structure for the tree will contain the current subtree plus information for reconstructing the tree already traversed. The zipper concept can be applied to many types, demonstrated by my great friend Dr Conor McBride as the operation of taking the derivative of a datatype (yes: the concept from calculus). See this early draft for a good introduction.

So, the suggestion is for each “path” in the system to be some encoding of the corresponding “state” of the interface. No separate routes file required!

Controllers and Views

If we can (it seems) dispense with routes, then what happens with controllers? Mostly, they will be subsumed into the interpreters for the interface actions, and so play a lesser role than in current frameworks. I would hope that much of the code seen in controllers—particularly the boilerplate code for decoding parameters, updating models and checking the results of saves—will mostly be abstracted away, possibly replaced by calls to suitably overloaded library code.

We still need views, though. These will be the main way of rendering domain concepts into Html or related formats, though their use might change. I would like to see most of the rendering work done with visibly pure functions (obviously no side effects, so 100% cosmetic), preferably called through a single overloaded function. That is, all viewable things should implement the member functions of an interface like the following for standard rendering of their data, analogously to the Show class for rendering to strings.

(The Html type will be provided by one of Haskell’s several libraries for Html, such as Blaze. See last month’s article for some examples of the cool features of Blaze.)

| class Htmlable a where
| to_html :: a -> Html

Sometimes more control is needed, for example rendering of some thing might be needed in two different contexts, with different rules. There are several options, including a subclass that takes additional context information, like this:

| class Htmlable a => HtmlableWithContext c a where
| to_html_c :: c -> a -> Html

or we could wrap up our basic values into more complex types and get finer control via the more complex type’s instance definition, for (a contrived) example, to prefix the rendering of a value wrapped with strong tags with a number of stars, where we would call

to_html (Stars "foo" 3)

to get 3 stars in front of

<strong>foo</strong>

| data Stars a = Stars { value :: a, num_stars :: Int }
| instance Htmlable a => Htmlable (Stars a) where
| to_html (Stars v n)
| = text (take n $ repeat '*') <+> strong (to_html a)

Similarly, we can control rendering of collections by wrapping them in a different type, where the type’s instance applies the relevant rendering of the container.

| data Unordered a = Unordered { collection :: [a] }
| instance Htmlable a => Htmlable (Unordered a) where
| to_html (Unordered xs) = ul << map (li . to_html) xs

Notice: we’re using types again (as wrappers) to represent the concepts we want, rather than passing around some ad hoc flags, and this can help to ensure consistency and avoid silly mistakes. As a final example, we can use a similar mechanism to define table formats, perhaps storing a list of values together with a list of (column title, extraction function) pairs that control how the columns are rendered. The extraction functions could themselves return Htmlable values so that relevant rendering of links or plain data can be handled.

Forms and Updates

This is one of the parts of Rails that make me nervous, particularly when a form should only update part of an object. All that hassle with protected fields, nested attributes, etc.

In light of the previous discussion, my suggestion is to capture the relevant user story as part of the app’s interface description (that is, as a type), and generate the form and the handling code from this. So in the context of viewing a certain list, if (say) we wanted to edit its title and date, then the interface description would contain this as a possible step for a given list:

... | ChangeTitleAndDate String Date

And the interpreter for this would (depending on context) either yield a form for changing those fields, or perform the relevant update. As with routes, it may be possible to avoid the usual string encoding of fields of nested attributes and use some representation that only the form generator and results receiver need know about. (To recap: the key requirement is that the coding used by the form achieves the desired effect when the form submission is processed. The rest, like human readability, is optional, so we could use quite interesting coding schemes.)

Forms for nested data could be handled by including structured data in the interface representation and adjusting the interpreter, etc.; say the following, which could allow the collection of items attached to the current to-do list to be edited, for example only keeping ones that are ticked.

... | ChangeTitleAndDate String Date [GUID ToDoItem]

In summary, the interface description should explain precisely what should be changing for a given step, from which certain code and html can be generated exactly. Hopefully, this removes a few more potential failure points and reduces reliance on tests.

Final Thoughts

The above may have been a bit of a ramble, but hopefully you saw one or two interesting things along the way. To be honest, I wasn’t quite sure when I started where I would end up, but am pretty pleased with the result and it seems to be worth further thought and experimentation.

The key idea here is to capture the possible user actions more clearly as code, in particular here as (hierarchical) types whose elements indicate what users are allowed to do and what kind of things the system will return. In Rails apps, this information is almost always buried under other details, and has to be teased out with a range of tests (with varying degrees of success). I expect other mainstream frameworks have the same issues. So my suggestion is to make these key details more visible and to get the details to work for the programmer, for example helping them to construct the code that provides the functionality. A side-benefit is to identify Huet’s zipper concept as a powerful way to represent a user’s journey through the structure of an application’s interface.

There’s also a corollary to the story, with several hints as to where and how further use of appropriate data types can help to clean up and simplify an app. A key point of these articles is about using data types more fully, and about how modern functional languages make it easier to create and use new datatypes in the service of simpler programs. So the final word is, instead of going for more complex processes, try to tease out the structure of what you are doing and see if any parts of it can be represented in a simpler form as data. It might just work.

Dr Paul Callaghan finds map reading and motorbike riding a little difficult to combine. Instead of frequent stops, he likes to carry on riding, randomly taking interesting-looking roads just to see where he ends up. It’s a good thing to try intellectually as well, and warmly recommended: you never know what you might find. Bits of his bio can be seen on earlier articles. Paul also flies big traction kites and can often be seen being dragged around inelegantly on the beaches of North-east England, much to the amusement of his kids. He blogs at free-variable.org and tweets as @paulcc_two.

Send the author your feedback or discuss the article in the magazine forum.

Using Memoization in Groovy

Dynamic Programming in a Dynamically Typed Language

by Venkat Subramaniam

Venkat explores the immense time-space tradeoffs of memoization, and explains how Groovy makes memoization easy.

“Dynamic programming” is intriguing for a few reasons. The combined words do not directly refer to anything dynamic or programming, at least not in the sense attached to those words individually today. It is an algorithmic technique that takes a counterintuitive approach to realizing blazingly fast computation. We will explore this technique here and quickly implement an example using the facility available in Groovy.

In a recursive solution we solve a problem using solutions to subproblems. Dynamic programming takes this to the extreme, by using overlapping recursive solutions. In other words, the solution to a sub-problem is used multiple times in the solution to the problem. Take the Fibonacci series, 1, 1, 2, 3, 5, 8, 13, 21, ..., for example. The number at any position in this series can be expressed as a recursion:

| Fib(position) is
| 1 if position < 2
| Fib(position - 1) + Fib(position - 2) if position >= 2
| for position >= 0

This recursion is simple to express, but terrible in performance if naively implemented, because it uses overlapping solutions. For example, to find the value at position 5, we’d perform Fib(4) + Fib(3). But Fib(3) would also be computed in the solution to Fib(4). Likewise, Fib(2) would appear in the solution to Fib(4) and also in solution to Fib(3). These overlapping reevaluations make the computational complexity of this recursion exponential, that is, O(2^position).

That’s bad, but only if we actually perform the reevaluations. If instead we store a computation the first time and reuse it each time it’s needed, we can reduce this to linear complexity, that is, to a time complexity of O(position). It’s a space vs. time tradeoff, in favor of gaining speed and implementation simplicity. And this is memoization, a way to store away the solutions and reuse them to speed up the dynamic programming algorithmic technique.

Though we look at a simple problem here, the Fibonacci series, dynamic programming and memoization are used in a number of day-to-day problems, such as finding the shortest driving distance between cities, maximizing profit in sales, minimizing cost and reducing waste, etc. It’s used in a broad range of optimization problems where the goal is to choose a best solution among different possible solutions.

Let’s implement the algorithm for the Fibonacci series as a simple recursion first, and measure its time. Then let’s use the memoization technique to speed it up.

Here’s the function to compute the Fibonacci value at a given position.

| def fib(position) {
| if (position < 2)
| BigInteger.ONE
| else
| fib(position - 1) + fib(position - 2)
| }

It’s a direct implementation of the recursive algorithm in Groovy. Since the values in the series would grow bigger than an integer or long can hold, we need to use BigInteger for the series values. Due to the dynamically typed nature of Groovy, the only impact of that decision is in the return where BigInteger is used in code: pretty low ceremony.

Let’s create a function to help measure the execution time for various position values.

| def timeit(position, function) {
| def start = System.nanoTime()
| def result = function(position)
| def end = System.nanoTime()
| def time = String.format("%.2f sec", (end - start)/1.0e9)
| println "Fib($position): $result, time: $time"
| }

The timeit function receives a position and a reference to a function—a closure—as a parameter. It forwards the position to the closure and reports the time it took to execute.

We have everything in place to try out the simple recursion. We’ll start with a small position value first.

timeit(5) { position -> fib(position) }

We called the timeit function and passed it two arguments, a position value of 5, and a closure that receives the position and invokes the fib function.

The value at the given position in the Fibonacci series and the time taken is reported by this call.

Fib(5): 8, time: 0.02 sec

The result is correct and the code took about two hundredths of a second to run. Let’s increase the input to a position of 40 and see how this code performs.

timeit(40) { position -> fib(position) }

The value and the time reported are:

Fib(40): 165580141, time: 20.01 sec

That took well over twenty seconds: that’s long. But this algorithm will get worse rapidly due to the exponential time complexity. For example, let’s increase the position just slightly by 2.

timeit(42) { position -> fib(position) }

Here are the value and time reported:

Fib(42): 433494437, time: 50.42 sec

A small variation in the input size but a large degradation in performance.

Imagine this was a problem like the rod cutting problem (the Groovy solution to which is presented in Programming Groovy 2nd Edition) where we want to measure the maximum profit from the retail sale of different sizes of rods. For a large size of rod, say 500 inches, it would take hours or even days to compute, during which time the market may well fluctuate before we find out the best ways to sell the inventory on hand, rendering the solution totally useless.

The degradation in performance came from calling the functions over and over. For example, just for the small position of 5, the fib function was invoked 15 times, and for the position of 40, it was called a whopping 331,160,281 times. Let’s turn to memoization for help.

When the fib function is invoked, we could look up the value in a hashmap and evaluate the logic only if the value is not present. First we need a hashmap to store the values; let’s create it and seed it with the two initial values.

fibSeries = [0: BigInteger.ONE, 1: BigInteger.ONE]

Now, we can modify the fib function to use this hashmap.

| def fib(position) {
| if(!fibSeries.containsKey(position))
| fibSeries[position] = fib(position - 1) + fib(position - 2)
| fibSeries[position]
| }

If the value for the given position is not in the hashmap, we compute it and store it in the collection. Then we simply return the value stored. If the value is already present, then we skip the computation. Let’s invoke the modified version of the function and measure the time to execute.

| timeit(5) { position -> fib(position) }
| timeit(5) { position -> fib(position) }
| timeit(40) { position -> fib(position) }
| timeit(42) { position -> fib(position) }

The code to call the functions are the same as before, but the time it takes is quite different.

| Fib(5): 8, time: 0.02 sec
| Fib(5): 8, time: 0.00 sec
| Fib(40): 165580141, time: 0.00 sec
| Fib(42): 433494437, time: 0.00 sec

The first call for position 5 took about the same time as the previous version. The subsequent calls for positions of 5, 40 and 42 took really no noticeable time. They appear faster than the first invocation, but that’s because the JVM has warmed up and taken care of JIT optimizations in the first call.

If we compare the results of the two versions side-by-side, the values at respective positions are the same, but the computation time is a world apart.

Compared to the 15 times the fib function was called for the position of 5, in this version it is called only 9 times. For the position of 40, on the other hand, the number of calls went down from 331,160,281 times to a mere 79 calls, thanks to memoization.

We can use the above technique in any language, so what’s special about Groovy? The answer is that in Groovy, memoization is already baked into the library as a function on closures. We don’t have to create the hashmap and look it up, that’s taken care under the covers.

To make use of this built-in facility in Groovy, we have to transform the function fib into a closure. Let’s transform the original non-memoized version.

| def fib
| fib = { position ->
| if (position < 2)
| BigInteger.ONE
| else
| fib(position - 1) + fib(position - 2)
| }.memoize()

We first defined a variable named fib. This initial definition is needed to bring the variable into the lexical scope of the closure to enable the recursive call. Then we created a closure, the body of which is pretty much the body of the original function. Finally, on the closure we call the memoize function, which is a Groovy added function on closures.

The memoize function under the covers creates a wrapper function that looks up a hashmap for the value and does the memoization for us.

Let’s invoke this modified version with some position values, and let’s be bold and increase the position size much further this time.

| timeit(5) { position -> fib(position) }
| timeit(5) { position -> fib(position) }
| timeit(40) { position -> fib(position) }
| timeit(42) { position -> fib(position) }
| timeit(500) { position -> fib(position) }

The larger position value is something we can’t even imagine with the poor performance of the overlapping recursion, but we’re confident the memoization has turned this into linear time complexity and so will handle this with reasonable speed. Let’s run the code and look at the speed and the results.

| Fib(5): 8, time: 0.02 sec
| Fib(5): 8, time: 0.00 sec
| Fib(40): 165580141, time: 0.00 sec
| Fib(42): 433494437, time: 0.00 sec
| Fib(500): 22559151616193633087251269503607207204601132491375
| 8190588638866418474627738686883405015987052796968498626,
| time: 0.01 sec

The output confirms that the results of the implementation are consistent with the slow version, but the speed is simply remarkable. We hardly made a dent in time even for a large position.

We made use of the memoize function in Groovy. That means less effort on our part, as we did not have to create and maintain the hashmap. But that’s just the beginning of the benefits. The underlying implementation of memoize is quite efficient. Recall that memoization is a time-space tradeoff—we use more memory—to store the values—in exchange for faster computation. If the input size is very large, we have the risk of running up the memory usage beyond reasonable levels. Groovy provides variations of the memoize method to better handle situations like this. To this method, we can recommend the size to use for the hashmap, specify upper and lower bounds, etc.

Groovy is quite versatile in a number of ways; we just looked at one feature in this article. The dynamic nature of Groovy, its metaprogramming capabilities, convenience functions like the one we saw, and the ability to fine-tune between dynamic and static typing, all make it quite an interesting and powerful language for programming on the JVM. Groovy 2.0 was released recently and it’s gearing up for the next release 2.1 this month. You can find out more about the language at groovy.codehaus.org. If you’d like to indulge in the latest updates of this language, you might check out the recently released Programming Groovy 2nd Edition, updated to the latest version of Groovy.

Dr. Venkat Subramaniam is an award-winning author, founder of Agile Developer, Inc., and an adjunct faculty at the University of Houston.

He has trained and mentored thousands of software developers in the US, Canada, Europe, and Asia, and is a regularly-invited speaker at several international conferences. Venkat helps his clients effectively apply and succeed with agile practices on their software projects.

Venkat is the author of .NET Gotchas, the coauthor of 2007 Jolt Productivity Award winning Practices of an Agile Developer, the author of Programming Groovy 2nd Edition, Programming Scala: Tackle Multi-Core Complexity on the Java Virtual Machine, and Programming Concurrency on the JVM: Mastering synchronization, STM, and Actors.

Send the author your feedback or discuss the article in the magazine forum.

Mac Kung Fu: Breaking the Magician’s Code

Uncovering “Default Write” Commands

by Keir Thomas

Keir breaks the magician’s code and reveals useful secrets that Apple has hidden in its code.

Authors get asked the same questions time and again. Mine? People ask where I find the hints, tips, tricks, and hacks that fill Mac Kung Fu, the second edition of which was published last month. I’m tempted to steal a joke from Douglas Adams who, when asked where he got the zany ideas for his Hitchhiker’s books, said it was via mail order from a firm in Indianapolis.

I wish finding the tips were that easy. The trick is to develop methodologies for uncovering interesting new things. All involve exploration, trial and error, and a healthy helping of intuitive guesswork. Some methodologies are complicated. Some are simple. As just one example of how simple they can be, it’s genuinely surprising how many hidden features in OS X can be uncovered simply by clicking things on screen while holding down the Option key (a.k.a. the Alt key for international users). Hold down Option while clicking and dragging to resize the Dock and it’ll snap to preset sizes. Hold down Option while clicking the Notification Area icon in Mountain Lion and you’ll turn off notifications until tomorrow.

I click everything with Option held down so—to borrow a phrase—you don’t have to! One thing’s certain: The “Wow!” moment of discovery can be addictive.

Default Choices

An example of a more complicated methodology is uncovering the “default write” commands that form part of the book, and part of Mac power-user culture. These are commands typed into a Terminal window that write secret and/or unofficial settings and thereby unlock hidden features within apps. Default write hacks can transform the user experience by fixing annoyances and introducing new functionality. For some reason, many built-in OS X apps feature such hidden gems.

But perhaps some explanation is necessary.

Whenever a Mac program starts, it reads its settings from one or more property list files. These are similar to config/.ini files, as found on lesser operating systems, although within OS X they’re binary files that can be decoded to and from human-readable XML. Apple prefers things to be neat and tidy.

Any user selections made within an application’s Preferences dialog box are stored in one of these files, which have a .plist extension. They live in the /Library/Preferences, or ~/Library/Preferences folders. However, with most of OS X’s built-in applications—such as Finder or the Dock—there are quite a few secret preference “keys” you can write to the .plist files that activate features or change settings you can’t access via Preferences dialog boxes, or System Preferences.

These secret preference keys might unlock experimental or abandoned features, or features that for better or worse simply didn’t make it into the official Preferences panel for that app. This might be because they were considered too complicated, or perhaps too much choice was being offered to the user (which is a Bad Thing in the world of OS X).

A good example of this is the “suck” minimize windows visual effect, which has been featured in OS X since the very early days, yet isn’t accessible without using defaults write to write a secret preferences key. In System Preferences you’ll find just two visual effect choices: Genie, and Scale. Somebody at Apple clearly thought that offering a third choice would bamboozle the user.

Users can write their own preference keys into a .plist file by dropping down to a Terminal window and using the defaults write command. They must specify the preferences domain for the app in question (usually in form of something like com.apple.Finder), then the key, followed by a value. This value can be a string (that is, a keyword although in rare examples it’s a complete sentence), a Boolean value (specified as 1/0 or TRUE/FALSE), or a decimal integer (usually 8-bit). You can also specify floating point values, data arrays and calendar dates, but these are much less common in the world of defaults write hacking. Generally speaking, things are turned on or off via a Boolean 1/0.

Behind the Magic

I know what you’re thinking. How do people find the hidden preference keys? Therein lies the magic. Lucky for you, I don’t mind playing Penn and Teller.

Back in the old days finding hidden preferences was easier and involved using the strings command to search the app’s binary file for anything that was (a) human-readable, and (b) looked interesting. It was akin to fumbling around in the dark for the light switch, and it required dedication bordering on obsession to read through the output, but the rewards made it worthwhile.

Sadly, Apple began encrypting OS X binaries as part of OS X security measures, so this route was closed off. Decrypting the binary and then examining it in this way didn’t work.

A solution to this crisis was created by a developer called Arcticmac, who figured out a way of attaching the GNU debugger (gdb) to apps while they’re running in order to examine them for anything that took the format of a preference key. gdb comes as part of the Xcode command-line tools, which in Mountain Lion can be installed via Xcode’s Preferences dialog box.

I modified the technique slightly to attach gdb to already-running app processes rather than starting them from scratch. This seems to uncover a slightly different set of results.

However, even this wonderful technique is under threat: in the most recent release of Mountain Lion, Apple appears not to digitally sign gdb, limiting its access to memory locations. You can sign it yourself using Keychain Access (See these instructions, although skip the part about compiling gdb from source), but one can’t help feeling the walls are closing in once again. It may well be that the move towards sandboxing apps in current and future releases of OS X will rule out even the gdb technique for finding new defaults write strings.

Just Getting Started

The work has only just begun once the gdb script dumps a list of what it thinks are preference keys. Every one of them—and there will be hundreds—must be examined for anything that looks interesting. Suddenly you’re not just feeling around in the dark for the light switch. Now you’re wearing boxing gloves too.

Finding interesting stuff takes lots of time, plus pure intuition and common sense. It can help to know Apple’s codenames for certain features: MeetingRoom appears to refer to AirDrop, for example (and is a better name if you ask me). Ubiquity refers to iCloud. Springboard refers to Launchpad. For what it’s worth I figured out these by guesswork, plus trial and error. Once you identify a key that looks interesting, you then have to work out whether it takes a string value, an integer, or (most likely of all) a Boolean value. This can be done by testing it out on a working system using the defaults write command. At this point you’ll also see what effect—if anything—applying the new key actually has. Does it do anything at all? Perhaps its effect is so subtle that it’s nearly impossible to spot, or maybe it relates to the smallest functionality of an already very minor function. The only way to find out is to play with the app in question—click this, then that, and set this function working, and stop that other function.

Did I mention that this can be mind-bendingly difficult work? But on the other hand, finding something awesome makes it all worthwhile. Keys can be removed from the preferences plist file using the defaults delete command.

Hack Categories

Experience has shown me that keys uncovered by gdb fall into one of a few categories. Firstly, they might be keys that you can already set via the Preferences dialog. gdb finds these too, and this can catch out the unwary hacker who might believe he’s stumbled upon a gem when, in fact, that same gem is just a few clicks away for every single user.

Secondly, the preference key might simply be a dud. It doesn’t do anything. This might be because either gdb has returned something that isn’t actually a preference key, or the key is an orphan—it was connected to something once upon a time, but not any longer. There do seem to be many preference keys from very early versions of OS X that simply haven’t been removed when the functionality moved on. Some might even date from the NeXTSTEP days, from where OS X inherited its preferences system.

Thirdly, the preference key does something but it’s not very interesting or useful. A lot of keys relate to diagnostics and testing of the app in question, or are simply related to the internal plumbing of OS X (configuring the app to work with a particular API, for example). Recently I discovered a key that displayed within Finder the frame rate of Cover Flow animations within Finder (which I wrote up for the OS X Hints site). It’s hard to imagine how this could ever be useful to anybody other than somebody working at Apple, but it was pretty cool nonetheless.

Fourthly, and lastly, there are the gems—the hidden features that are both useful and cool. Perhaps my greatest find was a preferences key that turned on notification pop-outs in the Dock showing what track or movie was currently being played in iTunes. This astonishingly useful feature was included in OS X, yet was entirely hidden. Sadly, it’s been removed from Mountain Lion. However, for the second edition of Mac Kung Fu I’ve discovered gems that include preference keys that let you resize icons in Launchpad, turn off the Dock’s new smoked-glass visual effect, and activate a global setting that automatically unhides the scrollbars when the mouse cursor hovers over them (fixing one of the major irritations with OS X Mountain Lion when using trackpads).

There are others, and many of them I genuinely couldn’t manage without during everyday computer use, such as a hidden drag and drop mode in Finder that makes scrolling the window by dragging a file to its edges significantly more intuitive and useful. And I would go insane without the hack that turns off dialog boxes zooming out from the middle of the screen.

Fallacies

Some less-knowledgeable individuals have the impression that there are billions of defaults write options out there—if only somebody could figure them out! I’ve even seen people attempt to guess preference keys based on things that sound likely!

The bare fact is that while hidden defaults write options and their associated secrets come about for various reasons, many of the best from our perspective are the detritus of Apple’s programmers. Metaphorically speaking, we’re rummaging around in the dumpster around the back of Apple HQ. Because Apple is Apple, even the stuff they throw away can be pretty exceptional.

I suspect although can’t prove that Apple’s developers are aware of how useful many of us find these hidden features. Although some get removed with each new release of OS X (such as the iTunes notifications I mentioned earlier), a surprising number of them stick around once they’ve been discovered.

My theory? Folks at Apple find these secret settings useful too. After all, Apple’s corporate offices contain the world’s biggest contiguous group of Mac users.

If you decide to became a defaults write hacker, and uncover something interesting, be sure to share it in the Mac Kung Fu forums. If I use it in a future edition of the book I’ll give you a credit!

Keir Thomas is the author of Ubuntu Kung Fu and two editions Mac Kung Fu for Pragmatic Bookshelf. He recently signed-on to write a fourth book due next year. He describes himself as a 100% unofficial Mac and Linux "genius" who takes pride in writing computing books everyday users need and value — entertaining, readable, and full of useful knowledge. A former computer magazine editor and professional blogger, he switched to full-time authoring in 2005. He lives in the UK, is vegan, and his website is keirthomas.com.

Send the author your feedback or discuss the article in the magazine forum.

Launching a Gonzo Engineering Project

I: What Is Gonzo Engineering?

by Steven K. Roberts

The infamous 580-pound, 105-speed BEHEMOTH, with Mac, SPARC, and DOS environments as well as satellite datacomm, HF/VHF/UHF ham radio, heads-up display, head mouse, handlebar keyboard, 6-level security system, speech synthesis, 72 watt solar array, and deployable landing gear to keep the monster upright on killer hills. The bike now resides in The Computer History Museum.

With this issue we begin a series of articles unlike anything we’ve ever published. Steven K. Roberts has figured out how to live passionately, pursuing crazy dreams and building fantastic machines and going on amazing adventures. He calls what he does Gonzo Engineering, and in this series he tells you everything you need to know in order to pursue your own crazy gonzo engineering dream.

A Grand Vision is only the beginning. No matter how much passion you bring to bear on the project of your dreams, the odds of actually escaping the “gravity well” are low... unless you find a way to leverage larger forces. This series, derived from 25 years of audacious feats of gonzo engineering, presents the keys to six tools that are essential to a large-scale project:

A Business Angle

Your Own Education

Corporate Sponsorship

Media Coverage

A Public Presence

The Team of Volunteers

I contemplated publishing a book on this subject for years, and in 2009 I finally did, publishing the “trade secrets” that have made my adventures possible... the art of working with sponsors, media, and volunteers to get an insanely ambitious project off the ground and keep it moving on its own momentum.

This series of articles is a refactoring of that material for a new audience. But fundamentally the intended audience is still the same: people who are attempting to “reach escape velocity” with a massive feat of engineering. It is not about hardware or software technical methods; it is about the meta-hack of developing enough support and buzz to get your project to take on a life of its own. Large corporations can do this with brute-force methods (unlimited money and people), but individuals face daunting hurdles when competing for mindshare and resources. Without the ability to leverage larger forces as a sort of “martial art,” it is exceedingly difficult for a lone geek to escape the gravity well.

My own strange career has been the proving ground for the techniques revealed here. In 1983 I took off across the US on a computerized recumbent bicycle, freelance writing and consulting while underway. This was bizarre at the time, though it would now be unremarkable. But the project took on a life of its own, and I eventually covered 17,000 miles on three versions of this increasingly geeky machine. With a handlebar chord keyboard, heads-up display and head mouse, console Macintosh, and 24/7 satellite net connection (in 1990) it was a sort of geek extravaganza... and the project had about 150 corporate sponsors, 45 volunteers, and almost continuous mainstream media coverage. It became self-supporting, then segued through the ’90s into the Microship project—an amphibian pedal/solar/sail micro-trimaran with similar technological overlays.

This series of articles is not about any of that. What I intend to do here is to share the methods I used to pull the whole thing off without having a job or money in the bank. In these days of faltering economy, such techniques are more relevant than ever.

Before we get into the nuts and bolts of corporate sponsorships and media manipulation, though, I want to give you a brief introduction to Gonzo Engineering, exploring the kind of lean, intuitive, art-and-engineering thinking that is necessary to proceed rapidly despite scarce resources.

As I said, this kind of adventure is not for everyone—mainstream R&D has little room for such social engineering, and if you’re just carving out a niche as a freelancer, most of this is a bit over the top. The readers who will benefit from this information are those who are trying to pull off the impossible with an insanely audacious project, do an end run around traditional linear approaches to product design, or explore that strange territory best summarized by my old friend David Berkstresser in his immortal observation: There’s glory in using inappropriate tools. You can tell you’re pushing a new frontier when all available tools are inappropriate.

If any of that sounds deeply familiar, the experience I’m going to share with you will pay for itself very quickly.

The Microship, the result of an 8-year development project involving extensive sponsorship, students, and volunteer teams. This is an amphibian pedal/solar/sail micro-trimaran with retractible wheels, hydraulic systems, 480 watts of peak-power-tracked solar panels, and zippy performance under sail. BEHEMOTH, the Microship, and the later Nomadness project are all documented at microship.com.

What Is Gonzo Engineering?

If you probe the interstices of an industry increasingly dominated by Big Business, you’ll discover a microculture of hackers motivated by the mad bliss of invention, surviving on the sweet contagion of creative energy. Employment bonuses mean nothing here; fancy packaging and market share are viewed with contempt if a product lacks art. Beauty, now that’s the thing—the beauty of elegant code, of a robust network, of a balanced design that “just works” without duct tape and feature bloat.

It is from this culture that the Internet emerged, as well as the Open Source movement. Less obviously, it’s also a diverse community of home-shop machinists, Arduino artists, guerilla solar experimenters, human-powered vehicle designers, robotics hobbyists, amateur radio satellite builders, and countless other independent developers. If you want to see passionate invention without the sloppy overhead of a big R&D budget or the weird constraints of maximizing shareholder value, go find a hacker... someone who gets a thrill from circumventing limitations and who knows how to get things done.

This has been my world for 30 years—a world where fun is the bottom line and livings are made on the opportunistic spinoffs of creativity, not by selling one’s life for a salary. We subsist in the dark matter between industries, trolling flea markets and dumpsters for Obtainium, mail-ordering goodies, making holy pilgrimages to the surplus Mecca of Silicon Valley, repurposing the detritus of corporate America to our own obsessive ends. Scattered among us are conjurers, alchemists, wizards, lone-wolf inventors, quirky entrepreneurs, larger-than-life writers, and the origins of more than a few disturbing geek stereotypes.

In this parallel universe, the motivation for creating is highly personal. In industry, you can bet that any massive development effort is associated with a business plan—there’s no room for slack in a bottom-line world, and seldom are things done for fun. But here, you’ll find entire lifetimes given over to chasing quixotic dreams; you’ll see personal fortunes whittled down to marginal subsistence in the name of invention and reputation. Occasionally there’s an imagined pot o’ gold, to be sure, but most likely it’s just a reassuring fiction to keep the spousal unit calm in the face of demonic focus, Every Goddamn Night Out There in the Shed. No, our motives are usually as guileless as passion itself: chasing daydreams, building tools, realizing obsessions, shattering limits, publishing, earning grins of appreciation from the cognoscenti and accolades from neophytes.

These are things that touch the soul more than the bank account, and there’s definitely a conceit about it—our sense of security lies more in our toolsets than our 401Ks. We feel sorry for vested employees with their BMWs and well-appointed houses, even as we decorate our labs with rusted hand-me-down office furniture and pay for system upgrades by mining our hardware boneyards through eBay. But money is not the point. It’s the exhilaration of surfing the knee of the learning curve, the almost erotic bliss of a machine flickering to life—catching the spark and glowing while the rest of the world sleeps.

Of course, getting to that point can involve a ludicrous amount of work.

The First Steps

This is an almost embarrassingly intimate look at how crazy unbalanced people can take an ambitious dream and pull together the resources to make it come true (and then go out and play). You’ll never get a corporate middle manager to admit it, but such lunacy, driven by emotion and other unquantifiable wild cards of the psyche, lies at the very heart of the design process. You can formalize tools and implement procedures all you like, but you can’t fit passion on a PERT chart; trying to do so will repel the very people you need most.

The first step is one of the most fun: indulging in a fantasy rich enough to trigger secret grins of hard-core technolust. That’s the stuff that makes otherwise sensible engineers willing to devote years, if that’s what it takes, to getting it right.

One of the great secrets I’ve discovered is that even someone with stupendously bad work habits (like me) can get a prodigious amount accomplished by applying one simple and obvious technique: keep moving in the same direction for a long time. Unfortunately, that can lead one down the path of specialization—an essential part of the great symbiosis between those who dream and those who produce. Specialization along with its concomitant skills is obviously necessary to get real work done, but if you’re not careful it can also become a filter through which you see the world, attenuating everything that is not somehow related to your primary focus. Over time, this can cause severe perceptual distortion from which it can be difficult to recover (especially if said specialty ends up, not necessarily through any fault of your own, becoming an evolutionary dead end in a rapidly changing industry).

That’s an easy platitude for a self-proclaimed generalist to spout, but how do we resolve the problem? How do we hold on to a central design objective for a decade or more without becoming like one of those single-issue political or religious zealots who lose the broader context entirely and descend into extremism? It’s much easier to end up there than you might think, especially when you audaciously choose to chase a personal obsession rather than sell 40-hour weeks while hanging onto the remainder for your own sanity-preserving pursuits.

The trick is at once simple and fiendishly tricky: all it takes is caring so passionately about the project that it fills your daydreams, turns trade journals into treasure hunts, induces you to recruit your friends, inspires doodles, and overlays a sense of purpose onto every foray into the backwaters of the web. This is a lot to ask of a job that’s been dumped on you by management, and one of our central messages here is that if this crazy-talk of passion gets you all fired up and chafing at the bonds of a career that isn’t letting you play enough, then maybe some restructuring is in order. For there is simply no way that crank-turning, even by a well-oiled department full of Really Smart People, is going to give you a sustained rush of intense creative obsession; doing that requires a suite of characteristics that are generally regarded as pathological in a corporate environment:

Enough chutzpah to believe that you are doing something original and important, but the humility to steal shamelessly from the work of those who have preceded you

Enough schmoozing ability to induce others to buy into the dream, but the stubbornness to continue believing in your mad quest when associates have given up on you

Enough optimistic naiveté to interpret catastrophic failures as steps along a continuous path, but the sensitivity to recognize the real gotchas (like your own change of heart) when they subtly appear

Enough arrogance to ignore the warnings and skepticism of people with far more experience, but the wisdom to shut up and listen quietly to the advice of practitioners in a completely unrelated field

People who behave this way are often described as having attitude problems, difficulty working well with others, and a tendency to jump around and not finish assignments. These are not the things managers look for in employees.

What I’m trying to tell you here is that if you are one of these troublesome folks, you need to shape your environment to support your passions: nothing is more important than removing the barriers that our culture erects around creative madmen, and few companies are willing to customize a job description to allow your brain to go berserk in its own juices. In severe cases, you might even need to jump ship and accept the insecurities that accompany working alone. (On the other hand, if you are in management and are trying to pull off the impossible, then you need to recognize and encourage the hackers in your midst, giving them the freedom to be profoundly annoying and unpredictable.)

All this is simply a contextual backdrop for the real point here, which is that massively audacious feats of creativity fall out of a way of thinking that is much more a lifestyle than a toolset. I find myself smirking at books about management and team-building, when virtually every world-changing cusp in the fabric of technology can be at least partly attributed to the obsessive-compulsive behavior of some intense character who broke the rules, dropped out of school, irritated colleagues, jumped between careers, got in trouble, or, as the schoolbooks used to say about the inventors I tended to identify with, “died alone in poverty, an embittered man.”

It seems we keep returning to this theme: a lifestyle of dedication to a mad dream, with everything else shoved aside as necessary to make room for equipment, learning curves, relationships with gurus and assistants, testing phases, and the endless quest for support. It’s not necessarily profitable, nor is it particularly fun (in the amusing sense), but there is something blissful about having a raison d’etre, a central passion, an unwavering navigational objective that allows every instant of your life to be tagged unambiguously with Distance To Go, Cross-Track Error, Estimated Time of Arrival, and Speed Over Ground. Such clarity may be illusory, but it beats floundering around every day, changing direction on a whim, and questioning your purpose even while working your butt off and looking forward mostly to evenings, weekends, vacations, and retirement.

It’s also no guarantee of success. But even going spectacularly down the tubes feels kind of noble when it’s part of your life’s enduring quest.

Still, I keep wanting to overlay some kind of formality on this. Aren’t there a few rules we can apply that are a bit more useful than saying “just dream it,” like some incongruously successful relic of the 60s who became a crystal-sucker in the New Age fringes of Silicon Valley before stumbling into a founder’s pool during the can’t-fail dot-com boom? Like, it’s all about the fundamental vibrations of your creative energy, man...

Well, um, yes. But if this level of design is indeed a lifestyle, then the closest we can get to “formal tools” is a body of behaviors and attitudes. Let’s put on an engineering hat and attempt to consider the problem in that light.

Next up: formal tools, engineering, a sense of urgency, and economics.

Steven Roberts was the original “technomad,” covering 17,000 miles around the US on a computerized recumbent bicycle from 1983-1991 while publishing tales via CompuServe and GEnie, then extending the same design objectives to water with an amphibian pedal/solar/sail micro-trimaran that consumed all available resources until 2002. As is typical of homebuilt boat projects, however, by the time it was finished he didn’t really want to do that anymore... so he has since made the transition to a full-time life aboard a 44-foot steel pilothouse sailboat, and is now based in the San Juan Islands north of Seattle.

The ship is extensively networked with embedded data collection and control systems, streaming telemetry, and a user-interface layer reminiscent of the Enterprise... with a wrap-around console that includes communications, R&D lab, audio production studio, and a piano. Roberts has published 6 books ranging from travel and adventure to microprocessor design, and prior to becoming a technomad spent a decade developing custom industrial control systems, early home computers, and other paleo-geekery. More on his technomadic projects can be found at microship.com (with the new boat at nomadness.com). He is publishing the ongoing technical narrative of the new project as a monthly PDF “Nomadness Report,” as well as a series of Boat Hacking design packages detailing the subsystems.

Send the author your feedback or discuss the article in the magazine forum.

The Quiz

Quotable Coders

by Michael Swaine

Here are seven quotes from famous programmers. Your first challenge is to name the author of each quote. Caution: there are not necessarily seven different programmers represented. Your second challenge is to say what these programmers have in common.

Solution right here next issue. Or on Twitter right now, no doubt.

Solution to Last Issue’s Quiz

Last month we presented another Sudoku puzzle with a couple of twists—it used letters instead of digits, and there was a hidden message. In one of the rows or columns of the solved Sudoku, we said, you will find someone who was in the news recently.

W
Z
G

O
K
N

A
S
I

N
O
S

I
G
A

Z
K
W

A
K
I

W
Z
S

O
N
G

G
W
Z

N
I
S

K
A
O

K
S
O

A
W
Z

G
N
I

N
I
A

K
G
O

S
W
Z

I
N
K

Z
O
A

S
G
W

S
A
G

W
I
K

O
Z
N

Z
O
W

G
S
N

I
A
K

As you’ve surely noticed, the last line is the name of a famous programmer: S(tephen) G(ary) Wozniak, who was in the news for his announced decision to move to Austraila. Or possibly New Zealand.

Calendar

Want to meet one of our authors face-to-face? Here’s where they’ll be in the coming months.

Here’s what our authors will be up to in the coming months.

2013-01-08, Triangle.rb

The Rails View: The Junk Drawer Grows Up

John M Athayde (author of The Rails View)

2013-01-08, Boulder JUG

TBD

Venkat Subramaniam (author of Practices of an Agile Developer, Programming Groovy, Programming Scala, Programming Concurrency on the JVM, and Programming Groovy (2nd edition))

2013-01-09, Denver JUG

TBD

Venkat Subramaniam

2013-01-09, CodeMash, Sandusky, OH

Building Productive iOS 6 Apps (half-day tutorial)

Chris Adamson (author of iOS SDK Development)

2013-01-10, CodeMash, Sandusky, OH

Objective-C Is Not Java

Chris Adamson

2013-01-10, CodeMash 2013

RubyMotion - Ruby and Cocoa Touch Sitting In A Tree…

Jonathan Penn

2013-01-15, Atlanta JUG

TBD

Venkat Subramaniam

2013-01-18, AIPMM Webinar

Effective Project Portfolio Management

Johanna Rothman (author of Behind Closed Doors, Manage It!, and Manage Your Project Portfolio)

2013-01-24, T4G Limited Thinkers Big Data Congress 2013 Saint John, New Brunswick, CA

The Leading Thinker/Doer on our panel at the Big Data Conference event.

Jesse Anderson (author of The Cloud and Amazon Web Services)

2013-02-02, FOSDEM 2013

Live pair programming with Julien Biezemans, lead developer of Cucumber-JS

Matt Wynne (author of The Cucumber Book and Cucumber Recipes)

2013-02-04, BDD Kickstart, Brussels

Get a flying start with BDD, the collaborative process that's changing the face of software development. (with Julien Biezemans)

Matt Wynne

2013-02-04, JFokus - Sweden

Workshops and talks on JVM related languages.

Venkat Subramaniam

2013-02-18, DevNexus - Atlanta

Talks on JVM related languages.

Venkat Subramaniam

2013-02-20, RubyConf Australia

Full day Advanced Ruby tutorial

Dave Thomas (author of Programming Ruby (2nd edition), Programming Ruby 1.9 (3rd edition), Agile Web Development with Rails (3rd edition), The Ruby Object Model and Metaprogramming, and Agile Web Development with Rails (4th edition))

2013-02-21, RubyConf Australia

Keynote

Dave Thomas

2013-02-26, Agile India - Bangalore

Workshop and talks on Agile technical practices.

Venkat Subramaniam

2013-03-01, The Joy of Coding - Rotterdam, the Netherlands

Keynote at a conference focused on developer joy

Chad Fowler (author of The Passionate Programmer (2nd edition) and Rails Recipes)

2013-03-04, SpeakerConf - Aruba

An opportunity to handout with fellow speakers.

Venkat Subramaniam

2013-03-07, AgileIndy Conference

Tutorial: Agile Project Management: Do You Have What it Takes?

Johanna Rothman

2013-03-08, AgileIndy Conference

Keynote: Agile Management

Johanna Rothman

2013-03-13, 33degree - Warsaw, Poland

Various JVM languages related topics.

Venkat Subramaniam

2013-03-28, #inspect, Brussels Belgium

Wrapping RubyMotion - Discussing techniques for making Objective-C APIs more pleasant in Ruby

Clay Allsopp (author of RubyMotion)

2013-04-21, Problem Solving Leadership Workshop, Albuquerque, NM

5.5 days of leadership training for people who want to learn to be problem solving leaders.

Johanna Rothman

2013-04-22, Railsberry - Krakow, Poland

A talk on something other than Rails. This is a Rails conference about everything except for Rails.

Chad Fowler

2013-04-22, Percona Live MySQL Conference and Expo 2013

"How to Design Indexes, Really" and "Extensible Data Modeling with MySQL"

Bill Karwin (author of SQL Antipatterns)

2013-05-20, Let's Test Conference, Sweden

Tutorial: Hiring for Your Team: Culture Trumps Skills

Johanna Rothman

2013-05-21, Let's Test Conference, Sweden

Keynote: Becoming a Kick-Ass Test Manager

Johanna Rothman

2013-09-11, BDD Kickstart, Barcelona

Get a flying start with BDD, the collaborative process that's changing the face of software development.

Matt Wynne

2013-09-14, Barcelona Ruby Conference

Hexagonal rails

Matt Wynne

O’Reilly Events

Upcoming events from our friends at O’Reilly.

2013-01-22, online

Strata Online Conference: Data Warfare: “Join us in this free online event as we explore the downside of data dependency, and the coming Data War.”

2013-02-12, TOC New York, New York, NY

Tools of Change New York:“For six years, the O’Reilly Tools of Change for Publishing Conference in New York has been the seminal event for professionals and companies engaged with the challenges and opportunities of new publishing technologies and business models.”

2013-02-26, Strata 2013, Santa Clara, CA

O’Reilly Strata Conference: “Tap into the collective intelligence of more than 150 data experts at Strata 2013. Strata gives you the skills, tools, and technologies you need to make data work today.”

2013-03-24, TOC Bologna, Bologna, Italy

Tools of Change Bologna: “Happening in conjunction with the Bologna Children's Book Fair, TOC Bologna is where the children's publishing and technology industries come together to discuss, learn, and share ideas around the art, craft, and business of storytelling in the digital age.”

USENIX Events

What’s coming from our USENIX friends.

2913-02-12, FAST ’13, San Jose, CA

11th USENIX Conference on File and Storage Technologies: “FAST '13 brings together storage-system researchers and practitioners to explore new directions in the design, implementation, evaluation, and deployment of storage systems.”

Emancipation

As we begin 2013, here’s an ebook you might want to look at.

2013 is the 150th anniversary of the Emancipation Proclamation. If the movie Lincoln whetted your appetite for information about emancipation, the National Archives is putting on events related to the Proclamation all year, and they have released a free ebook, The Meaning and Making of Emancipation, that is worth a download.

Shady Illuminations

Instagram’s Image Problem

by John Shade

By reading this column you agree to abide by incomprehensible terms of use that John reserves the right to change at any time.

Marvel Comics just killed off Peter Parker, its most enduringing and beloved character. I was surprised. I didn’t realize that Marvel had become a subsidiary of Facebook.

Instagram, which did become a subsidiary of Facebook, seriously angered its fans late last year when someone actually read its terms of use. You’d have thought Instagram had killed off its beloved mascot.

And the funny thing is, Instagram really didn’t do anything out of the ordinary.

The boilerplate legalese in Instagram’s terms of use made the usual sweeping assertions regarding its claimed rights in your intellectual property, the kind of stuff lawyers have been polishing to a dazzling opacity since Medieval times.

Only this time there was a reaction. Word went out on, heh-heh, Facebook that you should get your pictures off Instagram because it was going to do nasty obscene things with them. Like license them to advertisers.

Instagram suddenly lost millions of users and was hit with a class-action suit, while Facebook’s stock price dropped a couple percent, although to be fair, Facebook is capable of losing stock points without any help from Instagram.

Instagram backtracked, offered assurances, even tweaked its terms of service. Was there ever really a problem? Yeah, there probably was. “Facebook... has a long history of gradually changing its contractual terms and business practices to obtain greater leverage over user-generated content,” one copyright lawyer warned. Like Saruman wanted leverage over Middle Earth.

So just what was Instagram claiming in its terms of use?

Chances are, Instagram management didn’t even know. And it’s a lead-pipe cinch that the users who brought out the torches and pitchforks didn’t know what they were up in arms about.

It looked like Instagram was claiming the right to license the use of your photographs to a third party. That doesn’t sound good, and it’s not hard to sketch out a scenario in which this could cost you real money. Say you take an awesome picture of Times Square on New Year’s Eve from the window of your hotel. Since you’re such an accomplished photographer, the hotel is willing to pay you to use that picture in its advertising. But wait a minute—if they can license it from Instagram, why would they bother to talk to you? They wouldn’t. Sorry shutterbug, Instagram just stole your sale.

Of course that’s just a made-up scenario based on a guesswork interpretation of the terms of use. Instagram insists that its motives were pure, but even if we buy that, and gosh, why wouldn’t we, intent can change. The only message from Instagram that counts is the contractual language in its terms of use. So what does that document actually mean?

Well, that’s the problem. The outraged users don’t really know. Instagram management doesn’t really know. Only the lawyers know for sure.

Right, the lawyers. Here’s the thing about lawyers: they don’t write to communicate. Lawyers are like programmers, their code doesn’t have to be human-readable. It may even be deliberately obfuscated. It just has to run on the target system and produce the desired outcome. In the case of terms-of-use language, the target system is the courts, and the desired outcome is income.

Instagram has spent a lot of money creating a place for you to display your pictures—which presumably have some value or you wouldn’t bother to post them. And just as a bank instinctively acts to extract all the value it can from your money as it passes through its corporate fingers, Instagram wants to squeeze what value it can out of your pictures while they’re taking up space on its servers. It can’t help itself. It’s a social media corporation, it was made that way.

Unfortunately, Instagram didn’t sufficiently obfuscate its lust. This hurt the company in the short term, but there are signs that all has been forgiven and the Amazing Instagram is bouncing back as the Superior Instagram. Because the only thing shorter than an outraged user’s fuse is an outraged user’s attention span.

John Shade was born under a cloud in Montreux, Switzerland, in 1962. Subsequent internment in a series of obscure institutions of ostensibly higher learning did nothing to brighten his outlook. He says he doesn’t post his pictures online because he doesn’t photograph well. Follow John on Twitter, send him your feedback, or discuss the article in the magazine forum.

But Wait, There’s More...

Coming Attractions

This may be the last page, but that doesn’t mean it’s the end of the conversation. Keep up with what’s going on in the Pragmatic universe by subscribing to our newsletter and drop by the Pub again next month for more Pragmatism. We’ll keep a seat warm and a mug chilled and a candle burning in the window.

Coming Soon in PragPub

Rubyists should be pleased. We’ve got a couple really nice Ruby-related articles in the pipeline. They might just both show up next month. And Steven Roberts will continue telling you how to pursue your dreams as a gonzo engineer. Paul Callaghan will continue his exploration of using the Haskell language for Web development and Matthias Günther will take you on to the next step in mastering the Pomodoro Technique. And John Shade and our other regular features will be here as well.

Recent Releases, Betas, and Upcoming Books

Programming Groovy: Dynamic Productivity for the Java Developer is now in beta. The Sublime Text 2 screencast is out. And we have our first audio book, Pomodoro Technique Illustrated. You can visit our site for even newer news.  

Right. But to really be in the know, you need to subscribe to our weekly newsletter. It’ll keep you in the loop, it’s a fun read, and it’s free. All you need to do is create an account on pragprog.com (email address and password is all it takes) and select the checkbox to receive newsletters.

While you’re waiting for the next issue of the newsletter or of PragPub to come out, you can follow us on twitter at @pragpub, @PragmaticAndy, or @pragdave. Or on Facebook at facebook.com/PragmaticBookshelf. Or email us at michael@pragprog.com.