Why Software Development Estimations Are Regularly Off

Some people believe that carrying out a large software project is like building a bridge. You look at your past projects and use the data to estimate the time and resources needed. This view has been debunked decades ago; this analogy was frequently seen as an aspiration for the future when I did my Master in Software Engineering at Carnegie Mellon in the late nineties.

In reality, most projects worth doing are not repetitions of previous things. If you need a bridge, you gem install bridge or extend bridge4j. A new software project is more like this:

– You are an inventor. You’ve invented a solar-powered microwave oven, an engine that runs on dead bugs, and a laser-powered weapon to kill mosquitoes. Someone comes to you and says:

“Hey inventor, I need a drone that will pick up mice (but not other animals), locate my ex-girlfriend and drop them on her head. Give me a budget and a time estimate.”

Obviously you have no idea where to start. You need to understand the problem. This hasn’t been done before, but it’s not all new technology. Drones exist, location technology exists too. How about reliably identifying mice? How much do you really need to invent? Can you buy a DIY drone and modify it enough? Can the client slip a tracking device into his ex-girlfriend’s purse?

Software is not about repeating, it’s about inventing. This colorful post on Quora that’s been making the rounds on Twitter is completely off the mark. The analogy of hikers going from San Francisco to L.A. doesn’t apply. Long journeys by foot are something people have done for millennia, and all the knowledge you need is one Google search away. A luddite hiker will do the journey once and learn. When asked to hike between New York and Chicago, he will be much more accurate. After a few inter-city hikes he’ll know enough to produce decent estimates. If I asked you to tell me how long it will take to drive from LA to SF, your estimate may be off by a couple of hours depending on traffic.

On the other hand, an experienced software engineer asked to build a control system for a car to drive itself from SF to LA is facing a completely different problem. Real software development is about doing something that you’ve never done before. That’s why all stupid analogies about routine real-life stuff break.

12 Replies to “Why Software Development Estimations Are Regularly Off”

  1. Invention, like any other work, involves research & discovery, analysis & interpretation, modeling, testing, prototyping and milestones. Though more complex by a order of magnitude than putting one foot in front of the other, it is still a pathway and can be properly predicted by allowing for some flexibility. As an operations & marketing person, the inability of devs to even consider tackling estimates is not only frustrating, but counter-intuitive and often destructive to business models. This is a divide that must be diminished, if not eliminated. I know I tend to recommend devs who have a “get it done” attitude over those who start out with “Well, I have to build a complete toolkit…”

  2. Of course it can be predicted; the more complex and innovative the task is, the more variance you will have in your estimates. The rest of your response is a non sequitur.

  3. Get it done attitudes and accurate estimates have nothing to do with one another.The post is accurate. A lot of development involves building your solutions from scratch (and then fixing the problems you create). Makes it hard to be dead on accurate.Also let’s not forget about how much scope tends to change right after project specs are locked in…

  4. Technical person breaks bad news, marketing person gets mad at the tech because reality isn’t the way he wanted it. Looks like everyone’s initial impressions were confirmed.

  5. @Don At my work (i develop software for a drive by wire system) a lot of times the target is exactly nown. As an example the brake starts to swing in some special state but how to get there is not known. The problem is not that the path is complex but that the path is not known. (We had several ideas how to improve that, i had to think how to implement this idea (hard work), implement and test them (easy work)) What worked pretty god for me was a system like liquid planer where you say a range (this task will need between 1 week and 4 weeks) and not a time (this task will need 2 weeks).

  6. Estimates are not the problem. It’s easy to create an accurate estimate given a reasonably complete specification: just double the total estimated engineering time. The problem is that projects take longer than estimated, but this is not the estimator’s fault. The culprit is a lack of accounting and accountability. It’s critical to track your time on a task-by-task basis, then analyze, iterate, and improve on your time management. This is easy in an hourly billing environment, because you simply do not bill for time not spent on a particular task. In a brick and mortar development house, it’s more difficult to keep up with estimates. I’ve recently transitioned from the hourly paradigm to a regular office environment, and the difficulty of simply tracking time is striking.

  7. Great post. You’ve hit upon the key insight: we are *inventing* not repeating a known mechanical process. It’s impossible to give a precise estimate when we’re dealing with the unknowns. At id8.com, we’ve helped bridge that gap between idea and execution by using digital prototyping. This allows us to easily “invent” (design-iterate-optimize) a realistic simulation that clears up an ambiguity. To follow the road-trip metaphor, it allows you to do a virtual walk-thru before you take your first step.

  8. I think you’ve missed the point of the Quora post. The purpose of the analogy is to reveal one key point: Most software estimates are made without understanding the problem at hand completely, and the reality on the ground is always more complex than initial estimate.You said it yourself: “Obviously you have no idea where to start. You need to understand the problem. This hasn’t been done before, but it’s not all new technology.”Making an estimate at this point is necessarily going to be fraught with unspoken assumptions and misapprehending the scope of a problem. That’s precisely what the two hikers from the analogy did; they made an estimate based on early discovery that ended up falling apart when it contacted reality.We as developers are never going to be able to do all the research necessary to accurately estimate a project, for the simple reason that most of the work *is* research. And trial and error. And detours. And having things explode spectacularly. I think the two hikers covered that experience perfectly.

  9. I got the point of the Quora post perfectly well. However, I attempted to point out the horrible flaws in the analogy. Particularly, how dangerous it would be for non-developers to believe that developers are so happy-go-lucky about making estimates. The hikers in the story would be the bottom of the barrel of all developers; the worst of the worst. They don't even do the most basic research. No attempt to see what others have done before, or that the problem was solved before. It would be like a college graduate deciding to write a database because he doesn't know that that they have existed for decades.

  10. I agree technical person breaks bad news, marketing person gets mad at the tech because reality isn’t the way he wanted it. Looks like everyone’s initial opinions were verified.

  11. The estimations are regularly off is because people don’t understand what estimations are and what are they for.
    The estimations are not numbers, they are guidelines, mostly for the managers. Developers just guess the “size” of the story in order to help management to come up with some release dates that business expects to be in place.

    Before totally loosing believes in estimations, i will recommend to use T-Shirt sizes at your Sprint Planning ceremonies. You will see that the story estimations are to identify the complexity of the story, it is not directly linked to the development time or budget.
    Eventually a manager will “attempt” to “convert” complexity to hours/days/money but this would be the managers worry.

Leave a Reply

Your email address will not be published. Required fields are marked *