The Itinerant Engineer

“Getting real is a system that’s worked terrifically for us. That said, the ideas in this book won’t apply to every project under the sun. If you are building a weapons system, a nuclear control plant, a banking system for millions of customers, or some other life/finance-critical system, you’re going to balk at some of our laissez-faire attitude. Go ahead and take additional precautions.” – Getting Real, a book by 37signals

I’d been having some trouble with my second-generation iPod shuffle, so I took a trip to the Apple store. Visiting the store reminded me again how strongly the culture of an engineering organization shapes the way it designs and sells products.

Read the rest of this entry »

“So my view of world-class software doesn’t have all that much to do, well in some sense, doesn’t have as much to do with software, as you might think. It has to do with company success; do the software development practices support the business directives? How low is the resistance between the software organization, and the business needs of the organization?”

Steve McConnell (Code Complete, Rapid Development, Software Estimation) spoke Thursday night at Bellevue College. Steve operates a consulting company, Construx, which advises clients around the world on how to develop software. Steve’s talk, entitled “Secrets of World-Class Software Organizations”, presented some distilled thinking on what makes world-class software organizations great.

Read the rest of this entry »

“Aesthetics” is the study of beauty. The connotation is art: painting, sculpture, dance, etc.

Science and engineering has aesthetics, too, but it hides behind the term “elegance”, not “beauty”. What could possibly be more elegant than Euler’s identity, that single equation relating e, i, and π?

Recently, I’ve seen more acceptance of the idea that science can partner with art to create very beautiful things. Electronic music creators — “artists”, use sophisticated digital hardware to create beautiful music. Brooklyn-based NYC Resistor is using LEDs and microcontrollers to create stunning visual experiences. The MakerBot allows sharing and replication of physical artifacts, the same way open-source authors share source code.

Walking on Broadway last Sunday, I came across the above display. Using laser projectors aimed at a grid of semi-reflective rods, artist Don Corson projected the laser beam into three dimensions. The end result looked SO COOL.

At Microsoft, there are three major engineering disciplines: development, test, and program management.  Program managers define what the product should do by authoring scenarios and specifications, developers write the code, and testers create automated test frameworks to ensure quality.

There’s a fun create tension between the disciplines.  The stereotypical tester is an obsessive-compulsive geek, who always worries about what could go wrong.  To a tester, a program manager is a pie-in-the-sky idealist with no clue what’s going on, but program managers and testers agree that the product would be infinitely better, if only the developers weren’t so, ahem, incompetent.

Some teams adhere strictly to disciplinary roles, (“I’m a tester, I don’t make those kinds of decisions”), while others are more fluid — everyone contributes as best they can, to get the work done.  I’m a big believer in the second model, where everyone contributes what they can.  Developers are not even remotely interchangeable, Joel Spolsky insists, and I think he’s right. We’re all good at different things, so it makes a lot of sense to parcel the work out based on who’s most fit to handle it.

I had a good opportunity to bridge a disciplinary boundary this week.  Shi, one of our developers, is facing a big backlog of bugs, and my manager has been busy trying to get our internal partners onboard with a big project we’re doing.  Faced with the choice of (a) working on some bugs or (b) doing nothing, I decided to take the gloves off and hack some C.

A few years back, Rob Mensching, one of the old hats, developed a tool called ESD Box (short for “Electronic Software Distribtion”), which we use to create self-extracting binaries.  ”Boxtool”, part of ESD Box, consumes an XML manifest describing how the self-extractor should work (which files it should contain, what it should launch post-extraction, etc.)  Boxtool creates a cabinet containing all the files we’re distributing, and then glues the cabinet onto the end of a boilerplate binary (“boxstub”) to create the redistributable package.

ESD Box works well, except for one annoying “feature”: it extracts all its files into the working directory, not a temporary folder, as is customary with most self-extractors.

Over the past two days, I waded through a few hundred lines of gory C code, extending ESD Box to allow selective extraction into a temporary folder.  In the process, I learned a lot about the Windows dialect of C (LPCWSTR, DWORD, lstrcmpW(), etc.), but moreso, I was reminded of how spoiled we are, up in .NET/Java/Python/Ruby managed-land.  Something as simple as copying a string (copying a string!) requires several function calls, because as we all know, strings are not a first-class primitive in C.  The entire experience felt like visiting an old friend, as the memories of nights spent hacking Linux Kernel code rushed back: string allocation by double-indirection, sizeof(), null dereference checks, the -> operator…

There’s something romantic about programming on the metal.  But I feel like it would get old, if I did it every day.

I think another fun story about scalability is our usernames launch…So, this was a product launch like many at Facebook, that was quite controversial, as we talked about doing it.  So we decided that, you know, kind of a lot of sites allow you to have a short name, and a lot of people have their association with Facebook as their personal identity, so it seems to make sense that instead of being facebook.com/uid, you know, blah blah blah, I could actually have a nice, easy-to-read username. So, this was all well and good, except that when most people do this, they have zero million users — and at the time, we had a little over 200 million users. So the problem was, how exactly do you get 200 million users to pick a username, and do this in a fair, and kind of balanced manner? So pretty much every way of allocating a scarce resource was debated, standard auctions like eBay, VCG auctions, for you auction nerds out there, which give you optimal results, but are confusing as hell to explain to someone what’s going on — we actually user-tested that one and people had no idea what they were doing — so, it’s like, telling them that it was the fairest way to allocate wasn’t going to make it any better — so we finally said, look, let’s just do this the simple way, everyone’s used to standing in line, first-come, first-served.

The problem with standing in line first-come first served, is that it basically means we’re asking 200 million people to show up at our website at exactly the same second on one evening.  When that happens, most people call that something like a denial of service attack.  For us, it’s like, a product launch. — Mike Schroepfer

Last Wednesday, Facebook brought their Seattle Engineering Roadshow to EMP. The event featured a presentation by Mike Schroepfer, Facebook’s Vice President of Engineering, on the challenges of scale.

Read the rest of this entry »

I got a knock on my door at work last week from some special visitors. I didn’t have any treats for them, so I made sure to watch out for tricks.