The Data You Have vs. the Data You Need

The Data You Have vs. the Data You Need

"Be data-driven" is universal advice, and it sounds unimpeachable. But there's a trap inside it that most teams fall into: they end up driven by the data that's easy to collect, not the data that actually answers their questions. The convenient numbers — the ones already sitting in a dashboard — quietly become the basis for decisions, simply because they're available. And convenient data is rarely the same as useful data. Being data-driven by the wrong data is worse than using judgment, because it dresses a guess up as a fact.

Here's why convenience distorts decisions, and how to start from the question instead.

Quick Answer

Most "data-driven" teams are driven by convenient data, not useful data — and the two are rarely the same.

The trap:

• Easy-to-collect data isn't the same as decision-relevant data — availability isn't usefulness.
• You optimize what you measure — so measuring the wrong thing steers you wrong.
• Start from the question, then find the data that answers it — not the reverse.
• Sometimes the useful data is hard to get — and worth the effort, or worth an honest estimate.

Don't ask "what does the data say?" Ask "what do I need to know, and what data would tell me?"

Photo by Luke Chesser on Unsplash

Why convenient data distorts decisions

The problem with being driven by available data is that availability has nothing to do with relevance. The data that's easy to collect — already instrumented, sitting in a dashboard, exportable in one click — got there because it was easy to capture, not because it answers your most important questions. So when you reach for "what data do we have?" and let that shape decisions, you're letting the accident of what's convenient determine what you optimize. The questions that actually matter often have no convenient data, so they get quietly ignored in favor of the ones that do.

This distorts decisions in a subtle, dangerous way: it looks rigorous. You're using data, citing numbers, being "data-driven" — but the numbers answer a question adjacent to the one you actually care about, because that's the question the available data happens to address. A decision based on convenient-but-irrelevant data is a guess wearing the costume of evidence, which is more dangerous than an honest guess because it carries unearned confidence. The convenience of the data, not its relevance, drove the decision — and convenience is a terrible decision-maker.

You optimize what you measure

The reason this matters so much is that organizations optimize what they measure — so measuring the wrong thing actively steers you wrong, not just incompletely:

Driven by convenient data	Driven by useful data
Optimizes what's easy to track	Optimizes what actually matters
Question fits the available data	Data fits the actual question
Looks rigorous, may mislead	Genuinely informs the decision
Ignores hard-to-measure questions	Pursues the questions that count

Once a metric is on the dashboard, it becomes a target, and people work to move it. If that metric is convenient rather than relevant, you've now mobilized the whole team to optimize the wrong thing — efficiently marching in a direction the data happened to point, not the one your goals require. This is how teams end up with great-looking numbers and disappointing outcomes: they optimized the measurable proxy instead of the thing that mattered. It's the vanity-metrics problem at the level of data strategy: the easy number isn't just useless, it's actively misleading once you start optimizing it. What you choose to measure becomes what you chase, so choosing by convenience means chasing by accident.

Start from the question, not the data

The fix is to invert the order: start from the question, then find the data that answers it — rather than starting from the available data and asking what question it can address. Before looking at any dashboard, ask "what do I actually need to know to make this decision?" Only then ask "what data would tell me that?" This keeps the question in charge of the data, instead of letting the data dictate which questions you're allowed to ask.

Often the data that genuinely answers your question is harder to get than the convenient stuff sitting around — it needs new instrumentation, a survey, a manual analysis, or a deliberate experiment. That effort is frequently worth it, because useful-but-hard data beats convenient-but-irrelevant data for actually making the decision well. And when the right data truly can't be obtained, the honest move is to say so and use clearly-labeled judgment — an explicit estimate is more trustworthy than a convenient number masquerading as the answer. Sometimes the most data-driven thing you can do is admit you don't have the data you need and reason carefully without it, rather than grab the nearest available metric and pretend it fits. This is the same discipline behind measuring what actually matters in email: the easy metric and the meaningful one are different, and the meaningful one is the one to chase. Let the question lead; make the data serve it.

How to be genuinely data-driven

To be driven by useful data rather than convenient data:

1. Start from the question. Decide what you need to know before looking at any data.
2. Find the data that answers it. Let the question dictate the data, not the reverse.
3. Distrust convenience. Easy availability is not evidence of relevance.
4. Invest in hard-but-useful data. New instrumentation often beats the dashboard you already have.
5. Label honest estimates as estimates. When the right data is unavailable, reason openly — don't fake it.

The throughline: being "data-driven" by convenient data is worse than honest judgment, because it dresses an accident of availability up as evidence and then optimizes the wrong thing. Start from the question, find the data that genuinely answers it — even when that's harder — and when the right data can't be had, use clearly-labeled judgment rather than the nearest convenient number. Let the question drive the data, not the data drive the question.

The bottom line

The data you have and the data you need are rarely the same thing. Most "data-driven" teams are driven by convenient data — the numbers already sitting in a dashboard — not by the data that actually answers their questions. And being data-driven by the wrong data is worse than honest judgment, because it dresses an accident of availability up as evidence and carries unearned confidence.

It compounds because you optimize what you measure: a convenient-but-irrelevant metric, once it's a target, steers the whole team to chase the wrong thing efficiently. The fix is to invert the order — start from the question, then find the data that genuinely answers it, even when that data is harder to get. And when the right data truly can't be had, use clearly-labeled judgment rather than grabbing the nearest number. Let the question drive the data, not the other way around.

Key Takeaways

• Start every decision by asking 'what do I actually need to know?'—then seek the data that answers it, not the data that’s easiest to access.
• Convenient data distorts decisions by making irrelevant metrics look rigorous; availability ≠ relevance—optimizing the wrong thing is worse than no data at all.
• Once a metric is on a dashboard, teams optimize it—so measure what truly matters, not what’s easy to track, or you’ll efficiently chase the wrong outcomes.
• Hard-to-get data is often worth the effort (new instrumentation, surveys, experiments) because useful-but-difficult data beats convenient-but-irrelevant data.
• When the right data isn’t available, use clearly-labeled estimates—honest judgment is more trustworthy than a convenient number masquerading as evidence.
• Invert the workflow: let the question drive the data, not the data drive the question—otherwise, you’re solving for what’s measurable, not what’s meaningful.