Before you forward that number: a data check for managers, marketers and founders

A founder gets a screenshot from ChatGPT: “you have 48,000 users”. The product has 11,000 paying customers. Both numbers can come from the same database — one counts every account ever created, including test accounts and people who never bought; the other counts customers who paid this year. The mistake isn’t in the arithmetic. It’s in forwarding a number without knowing what it counts.

If you manage a product, a budget or a company, you will increasingly get numbers from AI assistants that sound as sure as a senior analyst. You don’t need to write SQL to judge them. You need a few habits.

Short version: ask what exactly was counted, compare with a number you already trust, check the order of magnitude, look at the period, and ask what would change the conclusion. If any answer is “I don’t know”, the number isn’t ready for a deck.

Five habits

1. Ask for the definition in one sentence

“Users” — registered, active, paying? “Revenue” — before discounts, after refunds, or the platform’s commission? On one marketplace dataset, the word “revenue” gets used for four different money measures between €67,000 and €553,000 (details). If the person or the AI can’t say which one it is, don’t use it yet.

2. Keep three reference numbers in your head

Orders per week, active customers, monthly revenue. Any new number that disagrees with them by a factor of two or more deserves a definition, freshness and calculation check before interpretation. It could also be a real change.

3. Check the order of magnitude

Divide the big number by something you know. “€2 million from the promo” with 3,000 promo orders means €667 per order — is that your average basket? A quick ratio check can expose implausible totals; it does not rule out smaller errors.

4. Look at the period — especially the last one

Half a week compared with a full week looks like a collapse; a promo week compared with a normal week looks like a trend. Ask when the data ends and whether the periods are comparable.

5. Ask what would change the conclusion

“We should extend the promo” — what if half those orders would have happened anyway? What if refunds are still coming in? A good analyst — human or AI — can name the risk. If nobody can, the recommendation isn’t finished.

The one-minute check before a number goes into a deck

1. What exactly does it count? (one sentence)
2. Which period, and is it complete?
3. Does it agree with a number I already trust?
4. Is the order of magnitude plausible?
5. What would make it wrong?

FAQ

How can a non-technical manager check data from an AI?

Ask for the definition, the period and the filters in plain words; compare the result with a known total; and do a quick division to check the order of magnitude. None of this needs code.

Why does ChatGPT give me a different number from my analyst?

Often because it counted something different: all accounts instead of paying customers, a different period, or cancelled orders included. Ask both for their definition in one sentence; then compare the data snapshot and calculation too.

Do managers need to learn SQL?

Not necessarily. Judging a number — its definition, its period, its plausibility — matters more than writing the query. If you want to understand what analysts and AI tools actually do, the first days of Atterna Start show it hands-on without prior knowledge.

Is there a quick way to train this?

Play Wrong Number: three questions, no SQL, no account. Three colleagues ask for “the number of customers” and one database gives three different right answers.