Do You Know How Long It Took to Develop Word for Windows 1.0?

Those of us who work in software live in a world of lean practices. We are encouraged to prototype, fail fast and iterate. We have seen software-based companies go from an idea in a kid’s brain to a billion dollar business in the time it would have taken him to get a college degree. Of course, those are the outliers on the positive side. For every extremely successful software project there are a large number of failed ones that we’ll never hear about.

There are also projects that managed to survive only because they had the backing of an already successful company, flush with cash. Microsoft Word for Windows could be a good example. Not many people know the story of what would become one of the most widely used software products ever. I learned about it from a case study for my Software Project Management class in 1997, what follows is my summary (original here, or Google it).

In September of 1984 Bill Gates wanted a word processor that would run on Windows (being developed at the time). He gave the project to John Hunt, Andrew Hermann and Lee Arthurs. Greg Slyngstad (who would later become the program manager for the project) said:

Gates put three real hotshots on the project. Hunt, who had single-handedly written the first version of PC Word, became the project manager. Arthurs, who had a Ph.D. in psychology, was responsible for user interface and documentation. Hermann had been at Wang and was supposed to know the word proces­sor business inside and out.

The project was codenamed Cashmere, and it was scheduled to take a year or less. At this point you can probably guess that this did not happen. The scope for the project turned out to be too ambitious. At the end of the first year, all the team had to show for their efforts were some design papers. The design included features such as email, spreadsheet capabilities, document protection, and much more. Hunt left the project in the summer of ’86. Slyngstad came on board, along with Doug Kurtz (lead developer for the DOS version of Word), and other MS stars. The new team decided to throw away all the work done to date, and start from the Word for Macintosh codebase. The project was renamed to Opus, and was staffed mostly with new hires.

The next year went by as the team wrote a new spec for the product. Soon it was 1988, and the team was under extreme pressure to deliver. One development manager put it this way:

Opus got into a mode that I call “Infinite Defects,” When you put a lot of schedule pressure on developers, they tend to do the minimum amount of work necessary on a feature. When it works well enough to demonstrate, they consider it done and the feature is checked off on the schedule. The inevitable bugs months later are seen as unrelated. Even worse, by the time the bugs are discovered, the developers can’t remember their code so it takes them a lot longer to fix. Furthermore, the schedule is thrown off because that feature was supposed to be finished.

To make a long story short, the project was deemed “code complete” in late ’88, but it was full of bugs. It took one full year to stabilize it enough for release. The release date was November 30, 1989. This is more than five years after the start date, and four years behind schedule. Needless to say, this project taught Microsoft a number of valuable lessons. Their software development process improved tremendously over the next decade. A blue screen of death here and there is nothing compared to the above 🙂

You’d think the software industry as a whole has improved enough that schedules no longer stretch by 5x. If you work in software, you know it still happens. Why? Hint: not because of the explanation given in what was the Quora Answer of the Day a few weeks ago. If only it were that simple. A much better argument was made (in 1986!) by Fred Brooks: No Silver Bullet – Essence and Accidents of Software Engineering.”

Open Source and Big Data: Two Things I Love About LinkedIn.

Today is InDay at LinkedIn. We are encouraged to spend one day each month learning, inspiring others, and sharing knowledge with the community. I’ve been at LinkedIn for four months now, and this day gives me the perfect excuse to write a post about two things I love about this place: open source code and big data.

Open source: I think LinkedIn has contributed some great open-source products to the world. My favorites so far are:

  • Apache Kafka: a distributed publish-subscribe messaging system. The design document is a great read, I believe anyone working with distributed systems should check it out. [btw, congratulations Jay on the new baby!]
  • Krati: a simple persistent data store with very low latency and high throughput. It’s similar to BDB but written natively in Java. It’s hash-based, which gives it better random access performance than BDB’s BTree structure. Jingwei Wu is one of the best systems programmers I’ve met, and I’m so happy that he’s on our team.
  • Project Voldemort: a Java implementation of the Amazon Dynamo paper. It’s some of the cleanest code I’ve seen, and a great place to start if you want to learn about distributed key-value stores. Also, you should follow Alex Feinberg on Twitter and Quora.
  • IndexTank, of course 🙂

There are too many to count at this point (SenseiDB, Zoie, Bobo, etc.). Take a look at our group’s page.

Big Data: we have a treasure chest of people profiles and activity, and a beefy Hadoop cluster to mine what we can out of it. More importantly, we have stellar data scientists like Daniel Tunkelang and his team. They perform daily acts of wizardry to improve our products, as well find out unexpected insights about the professional world. When I got to play with the Hadoop cluster a couple of months ago and try out a little idea, it was nerd Nirvana for me. Of course I have my hands full with search, but one of my goals for the next few months is to write a few more Pig scripts.

Of course LinkedIn is hiring like there’s no tomorrow; we need to understand that data even better, and we have a laundry list of apps we want to build on top of it. Check out some jobs.

BTW, this is what we had for lunch yesterday 🙂

Sushi


Starting a Company Is Not Risky

In Silicon Valley we have this notion of the entrepreneur as the lone hero. A risk taker who shoots for the moon and aims to knock the ball out of the park. A visionary, a dreamer, a jack-of-all-trades. In another time he would have been a discoverer of new continents, a warrior, an inventor, a heretic. I could keep the clichés going but I’ll save some for another post.

Obviously such one-in-a-million individuals need all the help they can get. They are misunderstood by the world, ignored, ostracized, ridiculed. They need inspirational quotes and self-help articles about the importance of sleep, how to overcome “schlep blindness“, or how to survive fundraising (tip: always have your light saber handy during VC meetings; they are often traps).

Of course this is organic fertilizer of the bovine kind. I’m sure I’m repeating myself, but I’m not a fan of the self-help trope that pervades startup blogs and communities. First off, I’d like to end the myth that starting a company in the Silicon Valley is a risky endeavor. Risky compared to what? What could happen to you? Fail and go back to working for the man. Boo-hoo. Welcome back to the 99%.

What is risky? Working in underground mines (in lots of ways, not just odds of dying). Climbing Everest. Being an astronaut. Touching an electric train line [warning: disturbing].

The risk of starting a company as opposed to working for someone else is some opportunity cost, and maybe a bruised ego. Sure, you can work yourself to death and destroy your relationships in the process. Only that’s not because of running a startup. That’s just being a workaholic.

The next time I hear an entrepreneur bragging about being a huge risk-taker I’ll invite him to play Russian Roulette against me. With a nerf gun, of course.

Rant mode off, back to work (blogging is too risky).

Why Software Development Estimations Are Regularly Off

Some people believe that carrying out a large software project is like building a bridge. You look at your past projects and use the data to estimate the time and resources needed. This view has been debunked decades ago; this analogy was frequently seen as an aspiration for the future when I did my Master in Software Engineering at Carnegie Mellon in the late nineties.

In reality, most projects worth doing are not repetitions of previous things. If you need a bridge, you gem install bridge or extend bridge4j. A new software project is more like this:

– You are an inventor. You’ve invented a solar-powered microwave oven, an engine that runs on dead bugs, and a laser-powered weapon to kill mosquitoes. Someone comes to you and says:

“Hey inventor, I need a drone that will pick up mice (but not other animals), locate my ex-girlfriend and drop them on her head. Give me a budget and a time estimate.”

Obviously you have no idea where to start. You need to understand the problem. This hasn’t been done before, but it’s not all new technology. Drones exist, location technology exists too. How about reliably identifying mice? How much do you really need to invent? Can you buy a DIY drone and modify it enough? Can the client slip a tracking device into his ex-girlfriend’s purse?

Software is not about repeating, it’s about inventing. This colorful post on Quora that’s been making the rounds on Twitter is completely off the mark. The analogy of hikers going from San Francisco to L.A. doesn’t apply. Long journeys by foot are something people have done for millennia, and all the knowledge you need is one Google search away. A luddite hiker will do the journey once and learn. When asked to hike between New York and Chicago, he will be much more accurate. After a few inter-city hikes he’ll know enough to produce decent estimates. If I asked you to tell me how long it will take to drive from LA to SF, your estimate may be off by a couple of hours depending on traffic.

On the other hand, an experienced software engineer asked to build a control system for a car to drive itself from SF to LA is facing a completely different problem. Real software development is about doing something that you’ve never done before. That’s why all stupid analogies about routine real-life stuff break.

Markets for Ideas and Patent Trolls

In Silicon Valley many people treat the concept that “ideas are worthless, it’s all about execution” as gospel. This thought is supported, among others, by Paul Graham [sorry, he’s an easy target because his essays have so much low-hanging fruit]. He says:

Actually, startup ideas are not million dollar ideas, and here’s an experiment you can try to prove it: just try to sell one. Nothing evolves faster than markets. The fact that there’s no market for startup ideas suggests there’s no demand. Which means, in the narrow sense of the word, that startup ideas are worthless.

 

This has always sounded illogical to me. It doesn’t follow that there is no demand. Some things just cannot be sold; there are no “previews” for ideas. To show how absurd the above is, replace startup ideas by brains, cities, reputation. All things that are not worthless but cannot be sold. You cannot sell your life, but you can insure it for a lot of money. Is love worthless because money can’t buy me love?

 

Wait a minute. There’s actually a preview for ideas, and ideas can be sold. Just listen to an episode of This American Life from last year, When Patents Attack. It’s about the US patent system and how it’s broken to an extent that’s ridiculous. That’s why Nathan Myhrvold, former Microsoft CTO and chef extraordinaire, was able to raise $5B to create a repository of ideas. The episode tracks one patent bought by his company, Intellectual Ventures.

 

Patent number 5771354. He got it in 1998, back in the relatively early days of the internet. And the way IV explained the patent to us, Chris Crawford invented something that we all do all the time now. He figured out a way to upgrade the software on your home computer over the internet. So in other words, when you turn on your computer and a little box pops up and says, click here to upgrade to the newest version of iTunes, that was Chris Crawford’s idea.

 

The show goes on to explain that the idea is so generic that 5,303 other patents that were granted by the USPTO cover the same material. It gets worse:

 

 

 Intellectual Ventures goes around to companies and says, hey, you want to protect yourself from lawsuits? We own tons of patents. Make a deal with us. Our patents will not only cover everything you’re doing in your business, no one will dare sue you.

So not only ideas are not worthless; they are indeed bought and sold (Sorry Paul). The way in which they are sold reminds Chris Sacca of: 

 

 A Mafia-style shakedown, where somebody comes in the front door of your building and says, it’d be a shame if this place burned down. I know the neighborhood really well and I can make sure that doesn’t happen. And saying, pay us up.

 

People like Myhrvold and companies like Intellectual Ventures are often referred to as Patent trolls. They make money by buying ideas (many of them obvious), only to sue people who are doing something with them. If you build a web application, it’s almost certain that you are violating many patents. If you become big enough, you will be sued eventually. This is why Google and Apple got into a bidding war for a collection of patents a few months ago, and Apple ended up buying them for 4.5 billion dollars.

 

Conclusion: ideas can be free, cheap, expensive, or worthless. Just like books, essays or blog posts 🙂

 

Apple Doesn’t Need a Twitter Account but You Do

I’ve seen variations of this “insightful thought” all over Twitter in the past 24 hours:

 

Apple

 

This seems to imply that having a Twitter feed or a Facebook fan page is useless marketing. The author is some sort of “marketing guru.”

 

Now, Apple is the company that spent a fortune on the 1984 commercial and aired it on the SuperBowl. According to Wikipedia, they have 361 global stores. They just announced profits of 13 billion dollars for last quarter. Their marketing dollars must be the corporate world equivalent of the US military budget.

 

A Twitter account is cheap; it’s like the Swiss Army Knife of marketing. Of course every puny startup must get one, because why not. No negatives, maybe you can turn it into a modest megaphone. On the other hand, Apple doesn’t need one. It could buy the factory any day. If Apple gets a Twitter account it adds value to Twitter, not the other way around.

 

What’s the next great marketing insight? Larry Page doesn’t fly commercial airlines like you and I. Interesting…

Kill Hollywood Remixed

This makes sense (barely) if you read the original first. I don’t expect to see this post on Hacker News 🙂

[Picture Morpheus reading this to an army of digital Neos, or maybe Bruce Willis as a general]

Gentlemen: Hollywood appears to have peaked. If it were an ordinary industry (adult vhs tapes, say, or thirst mutilators without electrolytes), it could look forward to locking itself in a hotel room and dying on the can like Elvis. But make no mistake: this is no ordinary industry. The people who run it are not people. They are evil androids who turned good people like Walt Disney into soylent green, and then fed it to Tom Cruise. These machines are so mean and so politically wired that they could do a lot of damage to the Federation. It would therefore be a good thing if rebels hastened their demise. Hacker, you are our only hope.

That’s one reason we want to train elite troops that will burn down the Los Angeles area, but not the main reason. The main reason we want to train such armies is not to protect the world from more Death Stars, but because Beverly Hills Chihuahua brought it to our attention that Hollywood is dying. They must be dying if they’re resorting to such utter pieces of dog shit. If the crops of human bodies in the Matrix were growing rapidly, that growth would take up all their attention. When a striker is fouled in the penalty area, he takes an acrobatic dive while his face winces in fake pain. He rolls on the floor as if he had been kicked senseless by twenty ninjas, a baby and Uma Thurman. Soccer shows Hollywood is beaten. And yet the audiences controlled by the androids’ brain-infecting nanodevices are still huge. There is a lot of potential energy to be liberated within the Matrix.

How do you kill the movie and TV industries dead? Well, the venom of a black mamba can kill a human in four hours. But these are androids! Anybody home? Huh? Think, McFly. Think! You cannot kill them with fire, even though they mostly come at night… mostly. What’s going to kill movies and TV is what’s already killing them: aliens, and sharks with frickin’ lasers. And Michael Bay. So the best way to approach this problem is to ask yourself not what Hollywood can do for you, but what would A Beautiful Mind do?

There will be several answers, and we will see dead people. We can have game shows where convicted felons are given the chance to run to freedom but have to elude the stalkers. Some of the best ideas may initially look like they’re serving M. Night Shyamalan, but there will be a twist. There will be blood. Dave seemed like a friend to HAL before it went crazy, but is this madness. Madness? This is Sparta.

It would be great if what people did instead of watching shows was exercise their right to remain silent. Maybe they will. All other things being equal, we’d prefer to hear only ourselves talk. Or listen to the cast of Glee murdering our childhood memories. But all other things are decidedly not equal. Whatever people are going to do for fun in 500 years is probably predetermined: watch Ow my balls! Winning is more a matter of accepting it than making it happen. In this respect at least, you can’t push history off a cliff, or even drop it and watch it scream in slo-mo. You can, however, go to Google and feel lucky.

Luck, Be A Lady While Our Funding Lasts

You are in a room full of roulette tables. Someone gives you one chip and tells you to bet it on a number. Here’s the twist: all tables pay 36 to 1, but not all wheels have the same number of pockets. Some are standard, with the typical 0 to 36 numbering. You see one that has 500 numbers. Screw that one. One has 30 and a sign disclosing that it’s biased, only they won’t tell you how. Hmmm.

 

All wheels start spinning simultaneously. You can change your bet as often as you like, although it’s a big room and you’ll have to run around. You know that all wheels will stop spinning roughly at the same time. What to do?

 

Roulette_wheel
Image from here.

 

This is a very rough analogy for what a startup is like. There are obvious ways to decrease your chances of success. Still, if you play the game optimally your odds are still low. There is much more randomness involved than most people are willing to admit. For this reason, most of the advice for startups is in the negative: there are tried-and-true paths that correlate with failure more than others (e.g. starting a company with people who don’t get along, outsourcing your core competency, placing your bet on the wheel with 500 numbers).

 

The truth about a startup is that we are performing an experiment as we go. We’re doing something that nobody has done before. Our combination of variables such as product, timing, market, team, competition, etc. is unique. We can find comparable stories, but we have no idea how the difference of a single variable affected the whole outcome.

 

Of course, this doesn’t stop very smart people from trying to find patterns. Because there are more failed startups than successful ones, we see articles like:

 

[Insert obligatory rant about correlation and causation here]

 

If you read these articles, you will find things that are common among startups that failed. Of course, who knows if those things are what made them fail. Some of them are pure hindsight. For example, Paul Graham has two bullet points that seem to contradict each other: slowness to launch and launching too early. What is the right time to launch then? Easy, look at successful startups. Those obviously launched at the right time! That advice amounts to “be lucky with timing.” He does the same thing with “raising too little money” and “raising too much.” How do you know how much to raise then? Here’s a hint: if you look at all successful startups, the amount of money raised ranges from zero to a lot.

 

Let’s pick on Paul Graham some more because it’s fun. An obvious question is how many startups that made many of those mistakes succeeded. I for one made mistake #1: being a single founder. It wasn’t a walk in the park of course, but I disagree with pretty much everything he says in bullet #1. I have lots of friends, but IndexTank just happened before I had a chance to get anyone else involved as a founder. I found great investors and advisors to brainstorm with. It was a resounding success in my book. Did I get lucky? Sure, that’s my whole point.

 

The other article is even worse because it has the words “reasons” in the title. If you take a look you will find gems such as #2, building a solution looking for a problem. This sounds a lot like Twitter, doesn’t it? Of course they could argue that Twitter is an outlier. I think we have a theme going on.

 

The authors enumerate a list of things that happen to startups that fail, and call them reasons. A few centuries ago they might have argued that gravity is caused by the large number of objects falling to the ground every day. I would sentence them to five years of studying the scientific method in a prestigious university, and then spend the rest of their lives trying to disprove evolution or something.

 

There are more balanced views out there of course. I found this presentation called  Contradictory Startup Advice… and how to navigate it. The author makes fun of unsound advice from gurus with strong opinions, and then goes on to give some relatively tame advice of her own (e.g. avoid the roulette wheel with 500 numbers). It boils down to “reduce your obvious chances of failure and then hope to get lucky.” What she says makes sense, but I would argue that someone who needs that advice has no business starting a business.

 

With so much luck involved, all someone can do is bet as many times as possible with the best odds you can find. Essentially, not giving up and not going broke. Successful startups are survivors, and that’s why there is so much survivor bias in the advice you’d get from a successful founder. Don’t expect any advice from me other than “be lucky.”

 

Wait, perhaps I’m on to something with the “not giving up” part. Can you really advise someone to be relentless? I’m not sure. Speaking for myself, I have been relentless (very annoying to my parents) for as long as I can remember. I tried really hard to change some of my personality traits for decades. I succeeded at some and failed at others, and it wasn’t for lack of trying. If someone said “learn to be relentless” my answer would be “and you learn to be a dolphin.” Or a towel. Don’t forget to bring a towel. That is good advice. I always know where my towel is.

 

Hit it, Frank.

 

 

 

 

 

 

 

 

 

 

 

 

 

PandoDaily: The Next Huffington Post?

I like Sarah Lacy’s writing. She’s funny, entertaining, and also a hard worker. Her pieces usually make it clear that she spends ample time researching the topics at hand. I wouldn’t go as far to say she’s my favorite tech reporter (today that distinction would go to Kara Swisher), but I was very happy this morning to learn of the launch of her new site, PandoDaily.

It’s interesting that a tech news site raised venture funding from a very impressive list of investors. To me, this means that it must be an extremely ambitious enterprise. They are aiming to build something much bigger than Techcrunch. Some of the funds and investors on the list won’t write you a check if you’re not swinging for the fences. This makes sense; If I were in Sarah Lacy and Michael Arrington’s place, I’d be out to create a larger megaphone than the Huffington Post just to one-up Arianna Huffington. Given their recent history that could be a powerful motivation, and it will be fun to watch how this develops.

There is one problem with taking money from an array of investors who collectively have a stake in a significant chunk of the Silicon Valley ecosystem: the obvious conflicts of interest that will arise. Gawker shares their opinion on the difference between PandoDaily and what Techcrunch used to be:

Lacy, in contrast, will spend a lot more time covering her owners. At least she’s “unashamed” about that, as she put it. And points to the longtime Valley reporter for knowing how to appeal to tech titans’ vanity. Ever since the changing of the guard at TechCrunch, they’ve longed for a more deeply intertwined publication that will cover the Valley with the fervor of a supplicant. Lacy just sold them one. It’s not clear if the seventeen different stakeholders will be able to effectively block stories they don’t like. But then it’s not clear they’ll even need to.

To which Lacy responds:

We’re being up front about who invested in us; it’s up to the readers to decide how much that conflict bothers them. 

Right off the bat, critical readers will see that one of their first articles is a PR piece for a sister portfolio company, Wealthfront. This startup provides wealth management for people who recently made some money and don’t know much about investing. I checked out the service and it looks useful. However, I do believe that in the interest of reporting fairness Lacy should have mentioned other options besides Wealthfront. There must be some competitors out there, right? A quick Google search brings up Covestor and Collective2.

I strongly believe that the first thing anyone who makes some money should do is educate him / herself. When I first sold some Inktomi stock in 1999, I immediately bought a copy of Burton Malkiel’s A Random Walk Down Wall Street. I read it as fast as I could to prepare for the incoming zombie invasion. I wouldn’t rule out using the services of a firm like Wealthfront, but only after having learned the basics. Paul Buchheit explains all this very eloquently in a from post almost two years ago, titled “What do do with your millions.”

Reading PandoDaily will be an exercise in critical thinking and agenda detection. This is a good training ground for a critical reader, because connecting the dots should be relatively easy. Everyone has an agenda but not everyone discloses it… and that includes me. Hint: brains.

Stop the Presses: Essay Shows That Hard Work Can Be Unpleasant

I just finished reading Paul Graham’s latest essay, Schlep Blindness. I could say many things about it, but I’ll get straight to the point. Graham talks about having the realization that creating a business involves lots of unpleasant and tedious work. He makes it sound like he just invented faster-than-light travel, even though every person who ever ran a successful business can tell you this. You don’t even need to have started a company. My grandmother burned this into my brain when I was five years old. Chores are necessary but not fun. If it were easy everyone would do it. Yawn.

 

Is it possible that Paul Graham’s audience (in his mind at least) is composed of privileged, spoiled kids who don’t know about blood, sweat and tears? I guess. No harm in telling suburban kids from Ivy League schools about the harsh reality of the entrepreneur, the unsung hero of modern society. My real issue with this piece is that it’s extremely disingenuous. Let me explain why.

 

Paul Graham is an investor. He invests in hundreds of companies. He is looking for the next Facebook or Google. Creating a company like that requires founders who won’t take an early exit. It’s easier for this to happen if those founders are a bit irrational and inexperienced. Here’s the most dangerous paragraph in the whole article:

 

How do you overcome schlep blindness? Frankly, the most valuable antidote to schlep blindness is probably ignorance. Most successful founders would probably say that if they’d known when they were starting their company about the obstacles they’d have to overcome, they might never have started it. Maybe that’s one reason the most successful startups of all so often have young founders.

 

This is pure speculation, and a rationalization to justify Paul’s preference for younger founders. Another possibility is that older founders most likely don’t align themselves with Paul’s investment objectives.

 

I’m 42 years old. I started IndexTank not that long ago, and I could say what Paul says above: if I had known how hard it was going to be, blah blah. That adage is clichĂ©, ignorance is bliss. That doesn’t mean anything though, because I know I will do other things in life that will have similar obstacles. I may start another company some day, and hopefully I won’t face the same ones. I will face unexpected hardships and it will be rough at times, because that’s life. Nobody knows what they are getting into when they start something new, old or young. That’s just because life is unpredictable, like a motherfucking box of chocolates. Don’t be fooled, what Paul Graham is doing here is thinking like the VC that he is.

 

And Paul, please stop trying to make “schlep” happen. It’s not going to happen.