Product analyst case study interview: 5 practice cases with how to answer

In a product analyst case study interview, the first sentence of your answer matters more than the last query. Interviewers want to hear you check the number, break it into parts and find where it changed before you guess why.

Below are five prompts that come up in real interviews in one form or another. Four of them are also episodes of Atterna’s Meridian simulator, and their numbers come from the fictional Meridian marketplace data. Start with the prompts and choose your first check before reading the worked answers:

Formats vary by company; these are practice prompts, not a forecast of your interview.

One frame for every product case

1. Clarify the metric, the population, the period and the decision that depends on the answer. A metric definition template helps.
2. Check the number is real: complete data, same definition, no tracking change.
3. Build the metric tree: split the metric into factors that multiply or add up to it.
4. Segment where it moved: platform, version, channel, new vs returning, cohort.
5. Recommend, say how sure you are and what would change your mind.

Worked answers below — Meridian spoilers. These examples include findings from simulator episodes 1, 2, 3 and 8. If you want to investigate them without knowing the answers, use the prompts above first and return here afterwards.

Case 1. “Retention dropped last week. Why?”

Clarify first. Which retention: day 7, next month, any repeat purchase? Of whom: all users or new customers? Has the latest cohort had time to come back? A window that hasn’t closed looks like a drop.

Metric tree. Blended retention = Σ (share of new customers in each group × retention of that group). It can fall because some group got worse, or because the mix changed while every group stayed the same.

Checks. Window completeness; definition or tracking changes; split by channel, promo and platform; compare cohorts of the same age.

On Meridian. New-customer retention (a repeat purchase in the next calendar month) fell from 27% to 15%. The September cohort was 2.3 times a normal one: 567 new customers, 334 of them acquired through the AUTUMN20 promo. Regular customers came back at 27%, the same rate as before. Promo customers came back at 7%.

Strong answer. “The figures support a mix explanation: regular-customer retention stayed at 27%, while the larger promo group returned less often. That does not rule out every product issue. Next I’d assess whether the promo customers are worth their acquisition cost.”

Weak answer. Jumping to the product: “the new feature annoyed users, let’s run a survey”, before checking who the new customers were.

Case 2. “Conversion fell, but traffic is flat”

Clarify first. Which conversion: visit to order, visitor to buyer, checkout to payment? Flat in total, or also by source and platform? Is the latest period complete?

Metric tree. Orders = visits × add-to-cart rate × checkout rate × payment success. Flat visits can also hide a mix shift: fewer visits from a channel that converts well, more from one that doesn’t.

Checks. The funnel step by step, then by platform, app version and source; payment errors; price or stock changes; tracking changes that inflate visits.

Strong answer. Names the step and the segment that moved, the evidence that would confirm the cause, and its size in orders or money.

Weak answer. “It’s seasonality” or “let’s run an A/B test” without having looked at the funnel.

Case 3. “A metric rose after the launch. Good or bad?”

Clarify first. What was the launch supposed to achieve? Which metric is primary, which are guardrails? Is the rise real, and is it extra demand or demand pulled forward from next week?

Metric tree. For a marketplace: platform’s money = GMV × take rate − the platform’s own costs, such as discounts it funds.

On Meridian. In the promo week GMV rose 54% over the average of the two weeks before (€230,967 vs about €150,157), on 22% more orders. The dashboard showed +75%, because a join counted orders more than once. The take rate fell from 12.6% to 11.5%: Electronics, which pays a 7% commission, rose from 35% to 48% of GMV. Commission grew 40%, but the platform also paid €21,516 in discounts that week, against about €900 in a normal week. Commission minus those discounts: €4,961, against roughly €18,000 in a normal week.

Practice · check 1 of 2

AI

An AI assistant

“Paid orders totalled €552,645 in gross merchandise value (GMV).”

Can you infer the platform’s own revenue from that number?

Preparing the practice…

Yes — all €552,645 belongs to the platformYes — subtract customer refunds from GMVNo — I need its commission revenue or take rate

Strong answer. “GMV went up, the platform’s money went down. By the metric the business keeps, the promo week was worse than a normal one. Before repeating it, I’d check the weeks after for a dip.”

Weak answer. “GMV +54%, it worked”, or quoting the dashboard’s +75% unchecked.

Case 4. “The A/B test says the badge won. Do you ship?”

Clarify first. What was the planned split and the unit of randomisation? Which metric was primary, and was the duration fixed in advance?

Checks, in order. The split, then the result. A sample ratio mismatch (SRM) alert means the counts are unusual under the planned split. Investigate assignment and logging before interpreting the metric; the alert alone does not prove a bug or its cause.

On Meridian. Product’s report said conversion was up 17%. The 50/50 test had 5,978 users in control and 5,663 with the badge: a chi-square of 8.5, p ≈ 0.004. Split by app version, the imbalance sits in app 5.2 (1,247 badge vs 1,592 control), where the badge crashed the app on older iPhones. On the parts with a fair split (web and apps 5.1 and 5.3), conversion was 12.2% with the badge against 10.6% without: +1.5 percentage points, 95% interval +0.2 to +2.9.

Practice · check 2 of 2

A planned 50/50 test recorded 5,978 users in control and 5,663 with the badge. The split check gives p ≈ 0.004; the SRM alert threshold was set at 0.01 before the test.

The dashboard says the badge won. What should you do first?

Preparing the practice…

Accept the result — a 5% size gap is harmlessInvestigate the unexpected split before interpreting the winAccept it if the badge’s win is statistically significant

Strong answer. “I wouldn’t quote +17%. Fix 5.2 first. On the fair data the effect is probably positive, but anywhere from tiny to meaningful, so the decision depends on cost and risk.” More on duration and early stopping in how long to run an A/B test.

Weak answer. “p < 0.05, ship it.” Or the opposite: throwing away the whole test instead of finding the broken segment.

Case 5. “Weekly revenue is down 15%. Find the broken step”

Clarify first. Which revenue: GMV, net of refunds, the platform’s commission? Compared with what: last week, the same week last year? Is the current week complete?

Metric tree. GMV = visits × add-to-cart rate × checkout rate × payment success × average order value. Then split each factor by platform and version.

On Meridian. GMV fell 15.1%, from €74,472 to €63,253, when normal week-to-week moves had been up to 5.3%. Visits fell 7.9%, but the add-to-cart rate actually rose, from 9.5% to 10.2%. The break was at payment: the share of checkouts ending in an order fell from 96.2% to 84.3%. By version, the break is Android 6.4, released that week with a new payment module: 22.7% of its checkouts became orders, against 95.3% on Android 6.3 and 93.7% on iOS 6.4 with the same module. That bug explains about €8,000 of the €11,219 drop. A cut in paid search explains roughly €2,250 more.

Strong answer. “About 70% of the drop is a payment failure in Android 6.4: roll back or hotfix today. The ad cut is real but smaller, and whether it paid off is a separate question.”

Weak answer. Picking the first story in the room, “it’s the ad cuts”, without sizing it against the others.

Practise on data, not just out loud

Model answers teach structure, not noticing the problem in real rows. For broader prompts, such as defining an active user, see the data analyst case interview guide. Case 1, Bigger baskets?, is free with no account: marketing has a real 2.5× number and a claim it doesn’t support. Four of the scenarios above are Meridian simulator episodes, where your queries are checked on the data; Meridian is part of Atterna Analyst, €59 one-time.

FAQ

What is a product analyst case study interview?

A round where you get an open product question, such as a metric that changed, a launch or an experiment result, and talk through how you would investigate it. It tests structure and judgement more than SQL syntax.

How do I structure an answer to a product analytics case study?

Clarify the metric and the decision, check the change is real, split the metric into factors, segment to find where it moved, then recommend an action and say how sure you are.

What metrics should a product analyst know for case interviews?

Funnel conversion by step, cohort retention, average order value, revenue or GMV and, for marketplaces, take rate. More important than the list: defining each one precisely and splitting it into factors.

How do I answer “the A/B test won, should we ship?”

Check the split before the result: compare a statistical split check with the alert threshold fixed in advance, then investigate any alert. Then read the confidence interval, not only the p-value, check the planned duration was respected, and weigh the effect against the cost of shipping.

How can I practise product analyst case studies?

Say the five-step frame out loud on the prompts above, then solve a few cases on data where your numbers are checked. Atterna’s case 1 is free; the level check suggests where to start.