A/B testing compares two versions of a feature to see which one users prefer. PostHog's Experiments feature removes the manual work—it assigns variants, tracks conversions, and tells you when results are statistically significant enough to act on.
How A/B Testing Works in PostHog
PostHog treats A/B testing as a feature flag experiment. You define two variants, PostHog assigns users to each, and you track which version drives better outcomes.
Check Which Variant Each User Gets
Use posthog.getFeatureFlag() to fetch the variant assigned to the current user. PostHog returns the variant name (e.g., 'control', 'test', or 'redesign'). Render different UI based on the variant.
// Get the variant assigned to this user
const variant = posthog.getFeatureFlag('checkout-flow');
if (variant === 'control') {
renderCurrentCheckout();
} else if (variant === 'simplified') {
renderSimplifiedCheckout();
}Track Conversion Events in Both Variants
Capture the same event (e.g., purchase_complete) for both variants. PostHog automatically groups these events by variant and compares conversion rates. Use posthog.capture() to log your success metric.
// Capture the conversion event—PostHog links it to the variant automatically
function handlePurchase(orderId, amount) {
posthog.capture('purchase_complete', {
order_id: orderId,
total: amount
});
}
// PostHog knows which variant the user saw via feature flags
// No need to manually add variant to every eventCreating and Running an Experiment
Setting up an A/B test means creating an experiment in PostHog, defining your variants, and choosing which metric proves success.
Create an Experiment in PostHog UI
Navigate to Experiments in the sidebar, click New Experiment, and name it (e.g., 'Checkout Redesign'). PostHog creates a linked feature flag. Your code will call this flag to get the variant.
// After creating the experiment named 'checkout-redesign' in PostHog,
// your app checks it like any feature flag
const variant = posthog.getFeatureFlag('checkout-redesign');
if (variant) {
console.log('User assigned to:', variant);
// Render the appropriate version
}Define Your Variants and Split
Set up two variants in the experiment settings (e.g., 'control' and 'redesign'). Assign the split percentage—50/50 is standard, but you can do 10/90 for a smaller rollout test. PostHog evenly distributes users across variants.
// Feature flag payloads let you configure variants dynamically
const payload = posthog.getFeatureFlagPayload('checkout-redesign');
if (payload?.color === 'blue') {
applyBlueButtonTheme();
} else if (payload?.color === 'green') {
applyGreenButtonTheme();
}
// Use payloads to test different configurations within the same variantSelect Your Success Metric
Pick the event PostHog should measure (e.g., purchase_complete, sign_up, page_view). PostHog calculates conversion rate per variant and reports statistical significance. Pick one metric before launch and don't change it mid-test.
// Make sure you're tracking the metric consistently
posthog.onFeatureFlags(() => {
// Called when feature flags load
const variant = posthog.getFeatureFlag('checkout-redesign');
trackPageView({
event: 'checkout_viewed',
experiment_variant: variant
});
});
// PostHog will calculate:
// Control: 8% conversion rate
// Test: 11% conversion rateReading and Acting on Results
PostHog displays conversion rates for each variant, sample size, and statistical confidence. When confidence hits 90%+, you have your answer.
Compare Variant Conversion Rates
In the experiment results, PostHog shows Conversion Rate for each variant side by side. Look for the Confidence percentage—if it's above 90%, the result is statistically reliable. A higher confidence means you can ship the winner.
// Once you ship the winning variant,
// release it to 100% of users via PostHog UI
// Then your code can simplify:
const variant = posthog.getFeatureFlag('checkout-redesign');
// After release, everyone gets 'redesign'
renderRedesignedCheckout();
// Or remove the flag entirely once you're confidentMonitor Sample Size and Duration
Check how many users entered each variant. Larger tests (10k+ users) reach statistical significance faster. Smaller tests (100 users) need to run 2+ weeks. Don't stop early—variance in early days leads to false winners.
// Use PostHog's API to programmatically check experiment status
fetch('https://api.posthog.com/api/experiments/', {
headers: {
'Authorization': `Bearer ${POSTHOG_API_KEY}`
}
}).then(res => res.json())
.then(data => {
const experiment = data.results.find(e => e.name === 'checkout-redesign');
console.log('Sample sizes:', experiment.count_per_variant);
});Common Pitfalls
- Calling
posthog.getFeatureFlag()before flags load returns null. Always useposthog.onFeatureFlags()to wait for the flag library, or check if the variant exists before rendering. - Changing your success metric mid-experiment invalidates the confidence score. Pick your metric before launch and commit to it—switching to a different event breaks statistical validity.
- Running too many overlapping experiments confuses variant assignment. PostHog supports this, but each adds noise. Use Experiment Conditions to target separate user groups.
- Stopping the test as soon as confidence hits 90% can introduce bias if traffic fluctuates. Let the test run for at least 1-2 weeks and 1000+ users per variant to catch seasonal variance.
Wrapping Up
A/B testing in PostHog is straightforward: create an experiment, assign variants with a feature flag, track your conversion metric, and PostHog calculates the winner. When you hit 90%+ confidence and the winner is meaningful, ship it to 100% of users. If you're running experiments across Amplitude, Stripe, and other tools and want results in one place, Product Analyst can help.