How Sunk Cost Bias Kills Digital Transformation (And How to Stop It)

Sunk cost bias is one of the most pernicious forms of cognitive bias in tech. Sunk cost bias happens when companies continue investing in a decision or project, because of prior investment, even when continued investment is objectively dumb. 

Sunk cost bias is extraordinarily difficult to overcome precisely because it is an emotional response, and therefore immune to reason or logic. In my experience, sunk cost bias is at the root of much of the psychodrama that you see in business.

Unfortunately sunk cost bias is more than an annoying distraction. By its nature, sunk cost bias inflicts opportunity cost. If you’re intent on throwing good money after bad, you will never be able to see a better opportunity staring you in the face. This is why Blockbuster passed up a chance to acquire Netflix for $50M. 

Oops.

Cattle, not pets

You should be very uneasy if your teams spends weeks or months on a solution. If your team works on something for more than a week they’ll start to get emotionally attached. After a month, the work becomes their pet. But innovation doesn’t work that way. Innovation happens when you treat ideas like cattle, not pets.

I struggled with this as a new leader. Early in my career, if I saw a team spinning on work, I would yank them off the project, unceremoniously killing their pet. That might be the right thing to do for the company, but it’s kind of hard to be an effective leader if you go around killing your teams’ pets.

I have found that a better approach is to avoid attachment and sunk cost bias altogether, by making it really easy for my teams to walk away from bad decisions. 

There are lots of ways to do this, but the simplest approach is to select technology and processes with the lowest change penalty. Examples follow.

Trunk based development

If you use Gitflow, you end up with numerous, long-lived branches, each representing a significant investment of labor and time. Commits in a Gitflow model tend to be large and infrequent, which indicates big batches of work. It’s hard to walk away from something you put a lot of work into, even if walking away is the optimal choice. Perhaps this explains why Gitflow users defend it so vigorously. Emotional attachment inflates the change penalty.

If you want to avoid sunk cost bias, your teams should be doing trunk-based development. In trunk-based development, branches are small and short-lived. A focused engineer with good tooling should be able to rip out several pull requests a day. In trunk-based development, a branch is very low-commitment, and the change penalty is minimal. If something doesn’t work out, just nuke the branch and try another approach. 

Document databases

Relational databases have a very high change penalty. Even simple tasks, like adding a column, usually require migrations, which creates additional toil and risk for your team. Since nobody can predict the future, you can be certain that you are going to be fiddling with your database a lot. After a year, your relational database will represent a significant amount of work and investment. The cost of maintenance can easily distort the economic profile of your app, especially if your app doesn’t directly produce growth or revenue. 

If you don’t want your team to be held hostage by a relational database, a great alternative is a document database. Document databases have very low change penalty. Because they are schema-less, document databases promote evolutionary design. They generally require a lot less optimization and tuning. Additionally, backups tend to be much simpler for document databases. If you really need advanced querying, you can just create triggers to stream document changes into a separate relational store.

Step functions

Most apps should actually be step functions. Infrastructure, API’s, and “architecture” makes sense if you have a bunch of paying customers. If you don’t have a bunch of paying customers, then you should iterate quickly on many different experiments, until you find product-market-fit. When you have achieved product-market-fit, paying customers will follow. If you have so many paying customers that operational expenses are eating into profit, only then is it economically rational to create infrastructure, API’s and the like.

People who resist lambda and other forms of serverless computing usually cite fears related to cost. But if you don’t have millions of customers hammering your system, lambda won’t cost you anything. Furthermore, lambdas carry very low change penalty; because you haven’t poured a bunch of labor and investment into infrastructure, it’s easy to throw them away, or pivot to a server-bound architecture when appropriate.

Conclusion

If you are an engineering manager or product manager, you want a delivery system that optimizes for at-bats. Every once in a blue moon your team will hit a home run. But you should be looking for easy singles and doubles. Most of the time your teams will whiff. That’s okay. Your job is to coach them through it, and make sure their heads are straight for the next at-bat. This is why it’s so important to prevent sunk cost bias. Boondoggles happen when managers allow their teams to become emotionally attached to a “home run” that never materializes.