What statistical significance threshold do you use for A/B tests, and why does it matter?
We target 95% statistical confidence as a minimum before calling a test result. This means there is less than a 5% probability that the difference between variants is due to chance rather than the change we tested. Lower thresholds produce more false positives — changes that appear to improve conversion but do not. For high-traffic pages, we target 99% confidence. We also calculate minimum sample size before a test starts, so we never call a test early.
What should be tested first — headline copy, CTA button, form length, or page layout?
Test in order of expected impact: message match (headline and subheadline rewrite) first, then CTA copy and placement, then form field reduction, then layout and visual hierarchy, then page speed. This sequence prioritises the changes with the highest probability of moving conversion rate. Button colour and font size changes are last — they are the easiest to test but typically produce the smallest effect sizes.
How is landing page optimisation different for SaaS trial pages versus B2B demo request pages?
SaaS trial pages are optimised primarily for message match (ad-to-headline), friction reduction (form length, credit card requirement), and above-fold social proof (review scores, customer logos). B2B demo pages are optimised primarily for specificity of the CTA ("Book a 30-Minute Demo" vs. "Contact Us"), proof quality (named customer outcomes, not generic testimonials), and qualifying the intent of the visitor before they reach the form. The underlying framework is the same; the priority order of changes differs.
We are running multiple ad campaigns to the same landing page — should we have separate pages per campaign?
Yes, where traffic volume supports it. Message match is highest when the landing page headline mirrors the specific promise in the ad. A single generic page receiving traffic from multiple campaign themes will underperform against campaign-specific variant pages. We audit your current campaign-to-page mapping and recommend the minimum number of variants that would produce meaningful lift.
How long does it take to reach statistical significance on a test?
At a 2% baseline conversion rate, 10% expected lift, and 95% confidence target, you need approximately 12,000 visitors per variant — 24,000 total. At 5,000 visitors/month to the page, that is roughly 5 months. At 20,000 visitors/month, it is about 6 weeks. We calculate this before the test starts so you have a realistic timeline. For lower-traffic pages, we run directional tests — changes are made and conversion movement is tracked without formal significance testing.