What is a smoke test?
A smoke test is the smallest possible experiment to validate that a product idea has commercial demand — before any meaningful engineering investment. The pattern, popularised in lean-startup methodology, ranges from a landing page measuring email sign-ups to a fake-door button that records click intent. The signal sought is real money or real intent commitment from a real prospect, with the lowest possible cost.
Common smoke test patterns
- Landing page test: a single page describing the offer with a sign-up form. Measure conversion rate from traffic.
- Fake-door / Wizard of Oz: a button or feature in an existing flow that records intent but is not yet built; users see a “coming soon” or get the result delivered manually.
- Pre-sale: ask for payment before the product exists; refund if the threshold is not reached.
- Concierge MVP: deliver the value manually to a small number of customers; learn whether they would pay for the automated version.
- Ad-only test: run paid ads to a placeholder page; measure CPM to convert into estimated cost-of-acquisition.
What a smoke test is not
- Not an MVP: an MVP delivers actual product value; a smoke test measures intent before any value is delivered.
- Not a survey: survey answers about future behaviour are notoriously weak predictors. Smoke tests measure behaviour (clicks, sign-ups, payments) — not opinions.
- Not a substitute for usage: a smoke test validates demand; only real product usage validates retention.
Reading the results
- Signal: the click-to-sign-up or click-to-payment conversion rate, calibrated against industry benchmarks. Above benchmarks indicates pull; below indicates weak proposition.
- Cost per intent: what it costs to generate one sign-up — an early proxy for CAC.
- Qualitative follow-up: talk to the people who signed up; understand why they did.
Ethical considerations
If a smoke test commits people to a delivery they expect, honour the commitment or refund visibly. Promising delivery and silently abandoning is reputationally and legally risky — in EU/UK and Türkiye, consumer protection law applies once payment is taken or a commercial offer is made.
Do: design the smallest test that measures real behaviour, not opinion; document what would constitute a kill signal before running.
Don’t: read smoke-test results as a guarantee of retention — demand and retention are separate hypotheses that each require their own validation.