Product specs and success metrics

01The problem

Verification help at agents is at the top of the roadmap. The engineers ask: what exactly are we building? The designer asks: for whom, and what's the flow? Compliance asks: what are we changing about identity checks? The CEO asks: how will we know it worked?

A product spec (often called a PRD, product requirements document) answers all of them in one place, before work starts, so that everyone builds the same thing for the same reason.

02The concept

What a good spec contains

Baselines from data

A target without a baseline is a guess. Measure the current value first, then set a realistic target.

03Example

Baselines for verification help, from the signup data:

import pandas as pd

base = "https://academy.cloudtechanalytics.com/datasets/product/"
users = pd.read_csv(base + "users.csv")
feedback = pd.read_csv(base + "feedback.csv")

traders = users[(users["segment"] == "Market trader") & (users["phone_verified"] == 1)]
print(f"Traders who verified their phone: {len(traders)} in 8 weeks")
print(f"Their BVN verification rate: {traders['bvn_verified'].mean():.1%}")
print((traders.groupby("channel")["bvn_verified"].mean() * 100).round(1).to_string())
bvn = feedback[feedback["theme"] == "BVN verification problems"]
print(f"Verification feedback items: {len(bvn)}, of which app reviews average {bvn['rating'].mean():.2f} stars")

Traders who verified their phone: 2555 in 8 weeks
Their BVN verification rate: 57.3%
channel
Agent sign-up        69.1
Instagram ad         35.8
Play Store search    47.7
Referral             53.3
Verification feedback items: 249, of which app reviews average 1.86 stars

Traders who signed up with an agent already verify far more often than those who signed up alone. That's evidence that helping at the agent works, and it suggests a realistic target: bring other traders closer to the agent-assisted rate. A spec that only said "improve verification" would give the team nothing to aim at.

04Walkthrough

1. Run the cell. Set a target for trader BVN verification and justify it with the channel numbers.
2. What guardrail matters most for an identity feature? (Think about fraud and compliance.)
3. Decide the launch plan: which traders get it first, and what's the holdout?
4. Write the spec (the task below).

05Practice

06Check your understanding