Stop Running KPI Reviews on Folklore and Screenshots

Every weekly KPI review turns into a scavenger hunt because your product metrics live in one place, your revenue events live in another, and “the dashboard” is whatever someone last screenshotted in Slack. That’s not analytics. That’s folklore.

The fix isn’t buying a heavier BI tool. It’s owning the ingestion path and the definitions.

This playbook builds a single, queryable metrics layer and a weekly narrative report that ships itself using three tools: Supabase for the metrics store, n8n for orchestration, and Perplexity for fast, sourced context on anomalies (not vibes).

Supabase: Create a metrics schema you can defend. One table for raw events (timestamp, source, account_id, event_name, value, payload JSON), one table for derived weekly aggregates, one table for metric definitions (name, SQL snippet, owner, last_reviewed). The definitions table is the point. If nobody owns it, nothing matters.

n8n: Build two workflows. First, an hourly ingest job that pulls from Stripe, your app DB, and ad platforms, normalizes fields, and upserts into Supabase with idempotency keys. Second, a Monday 7am job that runs the aggregate SQL, compares week-over-week deltas, and flags outliers beyond your thresholds.

Perplexity: When n8n detects an outlier (CAC spike, conversion dip, churn jump), it calls Perplexity with a constrained prompt: “Suggest 3 plausible external factors and 3 internal instrumentation failure modes; cite sources for external factors.” You’re not asking it to explain your business. You’re using it to pressure-test the story before it ships.

Output: n8n posts a report to Slack and emails leadership, including: metric values, definition links, anomaly flags, and a short “what changed / what to verify” checklist.

You don’t need prettier charts. You need fewer arguments about what the number means.

Maya, the growth lead, gets the Monday 7:03am Slack ping while she’s still opening her laptop. “Weekly Metrics: ARR +2.1%, Trials -11.8%, CAC +27% (outlier).” Finally. One thread. Not six.

She clicks CAC. It links to the metric definition row in Supabase: name=cac_blended, owner=Maya, last_reviewed=2026-03-01, SQL snippet. She scrolls. Pause. The SQL uses spend from Meta and Google, but revenue events from Stripe are filtered to successful invoices. Good. Clean. Except… one line: WHERE account_id IS NOT NULL. Stripe doesn’t have account_id for one-time purchases. Those got dropped. Last week too? Who knows.

n8n’s Monday workflow ran like this: execute SQL to build weekly aggregates, compare to prior week in derived_weekly table, flag anything over threshold, then for each flag call Perplexity. Perplexity came back with “Meta CPM increases due to seasonal auction pressure” with citations and “possible attribution window change” with a source. Then it listed three internal failure modes: ad connector token expired, event_name mapping changed, currency conversion bug.

Maya pings DevOps. “Did the Meta node fail?” DevOps says no. It succeeded. That’s the problem. The failure wasn’t a crash. It was a silent schema drift.

On Friday, someone renamed event_name from trial_started to started_trial in the app. The ingest workflow happily upserted it. New string. No errors. The aggregate SQL still counted trial_started. Trials “dropped.” CAC “spiked” because denominators moved. Nobody noticed until Monday because the dashboard was a screenshot.

They hotfix the definitions table first. Not the dashboard. Update SQL to include both names, plus a guard: if unknown event_name volume rises above 2% week-to-date, n8n posts an alert immediately. Not Monday.

And then the messy question: do you lock event names with a registry and block deploys, or do you allow flexibility and rely on detection? There isn’t a clean answer. Only tradeoffs. But now the argument happens in one place. With the query. With the owner. With receipts.

Here’s the part most teams skip: making this survivable inside an org that ships code every day and hires people who weren’t in the room when you coined “trial_started.”

If you try to solve it with “everyone just be careful,” you’ll be back to folklore by next quarter. What works is treating metrics like an interface. Same vibe as an API contract: version it, test it, and assign a pager to it (even if the pager is just a Slack channel with adults in it).

Practically, we’d set up three guardrails.

First: an event registry that lives alongside the codebase, not in a doc. A simple YAML/JSON file is enough: event_name, required properties, types, and allowed enums. CI runs a check that compares the registry to what your app instrumentation is emitting in staging. If someone renames trial_started, the build doesn’t fail because “rules,” it fails because you just broke the company’s ability to count trials. That’s a real bug.

Second: schema drift detection at ingestion time, not reporting time. n8n shouldn’t just normalize and upsert; it should classify. Known event? Route to metrics_raw. Unknown event_name or missing required fields? Route to metrics_quarantine with a reason code, then notify the metric owner. Not engineering-on-call. The owner in the definitions table. You want the person who cares about the number to feel the blast radius immediately.

Third: metric definition review as an operational habit. last_reviewed can’t be decorative. Put a 30-day SLA on Tier-1 metrics (ARR, trials, CAC, churn). If it’s expired, the weekly report includes “definition stale” next to the value. That social pressure works better than policy docs.

None of this requires heavyweight tools. It requires accepting that the “metrics layer” is production software. When you treat it that way, the arguments get shorter, the fixes get faster, and Maya stops being the human diff between Slack screenshots and reality.