The case against experimentation
"We should test it!" = 💀
Since The Lean Startup came out, founders have LARPed as scientists.
“We should test it out!” “We need to experiment more!”
These sentences are unserious, and the underlying theory is fatal to startups.
Here’s why:
To figure out anything that’s worth knowing, you have to grind your soul down. The answer you’re looking for is on the other side of pain. Experimentation makes you think you’ll just get lucky and get signal immediately. Nope. Figuring your business out is like running a marathon - the answer emerges somewhere between mile 18 and mile 22, when you’ve put your soul into making it happen, and only if you’re paying attention.
Experimentation is more difficult than just doing the obvious thing. You’re telling me you can design an experiment that you can really learn something from? Or that you can know exactly what you’re looking for in advance, and design an experiment that can give you conclusive results? This is more difficult than the simple process of “selling & delivering to figure out what you’re selling & delivering.”
Experimentation is a reductive trap. More fundamental… you assume that you can split your business into parts, test the most “risky” parts, and the whole will work? This might sound right at first glance… but it ignores literally everything we know about how complex systems emerge & function.
In theory, experimentation makes sense. In practice, experimentation is functionally a LARP - an excuse to have no conviction and waste time instead of committing to figuring the thing out.
But who knows, I could be wrong. There are almost certainly some scenarios in which experimentation makes sense. Let’s test it. :)
—
PS - As I build Reframe & my next software business, I spend zero time thinking about experiments.
For Reframe, I am committed to growth via content. I am suffering through building an audience on LinkedIn, and have written this newsletter for like 4 years. With nearly zero growth on either… up until this fucking month. Conviction > experimentation.
For my next startup, there are a million GTM channels that could work. I am going all in on one, and I am going to make it work.
—
PS, v2 - I recorded a free, but very ugly, video on the path to 10 customers. Check it out here.
—
PS, v3 - If your team’s going to be deploying on AWS (or switching to AWS), and want some free CloudFormation templates that will save you weeks, prevent you from needing to become an AWS expert, and set you up to be SOC audit-ready from the start… email me at rob@reframeb2b.com. Happy to set you up!
Pretty controversial take I have to say! I know a few founders who would read this and think: great, let's dive back into the editor, we've got (blind) conviction. I've been on the other side though, experimenting our heads off where in the end we lost all conviction. I now as a structure like to work with experiment cycles because it helps to focus on one thing at the time, and instead of build blindly, you're focussing more on the outcome you want to achieve with it (and how to do that in the smallest way). If you know as much as you do about startups, your conviction isn't blind and probably pretty informed and customer centric.
"Since The Lean Startup came out, founders have LARPed as scientists."
Epic first line... and great post!!