"Be data-driven" is universal advice, and it sounds unimpeachable. But there's a trap inside it that most teams fall into: they end up driven by the data that's easy to collect, not the data that actually answers their questions. The convenient numbers — the ones already sitting in a dashboard — quietly become the basis for decisions, simply because they're available. And convenient data is rarely the same as useful data. Being data-driven by the wrong data is worse than using judgment, because it dresses a guess up as a fact.
Here's why convenience distorts decisions, and how to start from the question instead.
Most "data-driven" teams are driven by convenient data, not useful data — and the two are rarely the same.
The trap:
Don't ask "what does the data say?" Ask "what do I need to know, and what data would tell me?"
The problem with being driven by available data is that availability has nothing to do with relevance. The data that's easy to collect — already instrumented, sitting in a dashboard, exportable in one click — got there because it was easy to capture, not because it answers your most important questions. So when you reach for "what data do we have?" and let that shape decisions, you're letting the accident of what's convenient determine what you optimize. The questions that actually matter often have no convenient data, so they get quietly ignored in favor of the ones that do.
This distorts decisions in a subtle, dangerous way: it looks rigorous. You're using data, citing numbers, being "data-driven" — but the numbers answer a question adjacent to the one you actually care about, because that's the question the available data happens to address. A decision based on convenient-but-irrelevant data is a guess wearing the costume of evidence, which is more dangerous than an honest guess because it carries unearned confidence. The convenience of the data, not its relevance, drove the decision — and convenience is a terrible decision-maker.
The reason this matters so much is that organizations optimize what they measure — so measuring the wrong thing actively steers you wrong, not just incompletely:
| Driven by convenient data | Driven by useful data |
|---|---|
| Optimizes what's easy to track | Optimizes what actually matters |
| Question fits the available data | Data fits the actual question |
| Looks rigorous, may mislead | Genuinely informs the decision |
| Ignores hard-to-measure questions | Pursues the questions that count |
Once a metric is on the dashboard, it becomes a target, and people work to move it. If that metric is convenient rather than relevant, you've now mobilized the whole team to optimize the wrong thing — efficiently marching in a direction the data happened to point, not the one your goals require. This is how teams end up with great-looking numbers and disappointing outcomes: they optimized the measurable proxy instead of the thing that mattered. It's the vanity-metrics problem at the level of data strategy: the easy number isn't just useless, it's actively misleading once you start optimizing it. What you choose to measure becomes what you chase, so choosing by convenience means chasing by accident.
The fix is to invert the order: start from the question, then find the data that answers it — rather than starting from the available data and asking what question it can address. Before looking at any dashboard, ask "what do I actually need to know to make this decision?" Only then ask "what data would tell me that?" This keeps the question in charge of the data, instead of letting the data dictate which questions you're allowed to ask.
Often the data that genuinely answers your question is harder to get than the convenient stuff sitting around — it needs new instrumentation, a survey, a manual analysis, or a deliberate experiment. That effort is frequently worth it, because useful-but-hard data beats convenient-but-irrelevant data for actually making the decision well. And when the right data truly can't be obtained, the honest move is to say so and use clearly-labeled judgment — an explicit estimate is more trustworthy than a convenient number masquerading as the answer. Sometimes the most data-driven thing you can do is admit you don't have the data you need and reason carefully without it, rather than grab the nearest available metric and pretend it fits. This is the same discipline behind measuring what actually matters in email: the easy metric and the meaningful one are different, and the meaningful one is the one to chase. Let the question lead; make the data serve it.
To be driven by useful data rather than convenient data:
The throughline: being "data-driven" by convenient data is worse than honest judgment, because it dresses an accident of availability up as evidence and then optimizes the wrong thing. Start from the question, find the data that genuinely answers it — even when that's harder — and when the right data can't be had, use clearly-labeled judgment rather than the nearest convenient number. Let the question drive the data, not the data drive the question.
Q: What's wrong with using the data we already have? Nothing, if it actually answers your question — but available data got collected because it was easy to capture, not because it's relevant to your most important decisions. Letting "what data do we have?" shape decisions means the accident of convenience determines what you optimize, while the questions that matter most (which often have no convenient data) get ignored. Worse, it looks rigorous, so a convenient-but-irrelevant number becomes a guess wearing the costume of evidence — more dangerous than an honest guess.
Q: Why does measuring the wrong thing actively hurt rather than just being incomplete? Because organizations optimize what they measure. Once a metric is on the dashboard it becomes a target, and people work to move it. If that metric is convenient rather than relevant, you've mobilized the whole team to efficiently chase the wrong thing — marching in the direction the data happened to point, not the one your goals require. That's how teams end up with great-looking numbers and disappointing outcomes. What you choose to measure becomes what you chase, so a convenient choice means an accidental direction.
Q: What if the data I actually need is hard or impossible to get? If it's hard, it's often worth the effort — new instrumentation, a survey, a manual analysis, or an experiment — because useful-but-hard data beats convenient-but-irrelevant data for making the decision well. If the right data genuinely can't be obtained, the honest move is to say so and use clearly-labeled judgment; an explicit estimate is more trustworthy than a convenient number pretending to be the answer. Sometimes the most data-driven thing is admitting you lack the data and reasoning carefully without it.
The data you have and the data you need are rarely the same thing. Most "data-driven" teams are driven by convenient data — the numbers already sitting in a dashboard — not by the data that actually answers their questions. And being data-driven by the wrong data is worse than honest judgment, because it dresses an accident of availability up as evidence and carries unearned confidence.
It compounds because you optimize what you measure: a convenient-but-irrelevant metric, once it's a target, steers the whole team to chase the wrong thing efficiently. The fix is to invert the order — start from the question, then find the data that genuinely answers it, even when that data is harder to get. And when the right data truly can't be had, use clearly-labeled judgment rather than grabbing the nearest number. Let the question drive the data, not the other way around.
No following, no network, no luck. Just an unglamorous system I ran for eighteen months. Here's exactly what I did.
I went from 200 to 11,000 subscribers without hiring anyone. AI didn't write my newsletter — it did everything around it.
One person, output that looks like five. It isn't about working more hours — it's about a kind of leverage teams rarely have.
Comments
Sign in to join the conversation
No comments yet. Be the first to share your thoughts!