Feature flags let you control which users see which product experience. But flagging code in production doesn't tell you if that new checkout flow actually converts better. PostHog makes it straightforward to measure whether a flag variant moves your metrics—you just need to capture the right data and segment your insights by flag.
Capture Flag Evaluations as Events
Before you can measure impact, PostHog needs to know which users encountered which flag variant. The cleanest approach is to capture an event whenever a user gets exposed to your flag.
Check if flag is enabled and get its variant
Use posthog.isFeatureEnabled() to check if a user qualifies for the flag, then capture an event that records which variant they received. PostHog's flag evaluation is cached client-side, so this is instant.
// Check flag status
const hasNewCheckout = posthog.isFeatureEnabled('new-checkout-flow');
const variant = posthog.getFeatureFlagPayload('new-checkout-flow');
// Capture which variant the user saw
posthog.capture('checkout_viewed', {
flag_variant: variant,
flag_key: 'new-checkout-flow',
});
// Continue with your flow
if (hasNewCheckout) {
renderNewCheckout();
} else {
renderOldCheckout();
}Include flag context in business events
When a user completes a goal (purchase, signup, etc.), attach the flag variant as a property. This links your business metric directly to the flag they saw.
posthog.capture('purchase_completed', {
amount: cartTotal,
currency: 'USD',
flag_variant: posthog.getFeatureFlagPayload('new-checkout-flow'),
flag_key: 'new-checkout-flow',
});
posthog.capture('form_submitted', {
form_type: 'signup',
flag_variant: posthog.getFeatureFlagPayload('new-onboarding'),
flag_key: 'new-onboarding',
});Validate flag data is flowing
In PostHog, go to Live events and search for your flag events (e.g., checkout_viewed). Confirm that the flag_variant and flag_key properties appear. If they're missing, events won't segment correctly later.
// Debug: log flag state before capturing
console.log('Flag enabled:', posthog.isFeatureEnabled('new-checkout-flow'));
console.log('Flag payload:', posthog.getFeatureFlagPayload('new-checkout-flow'));
console.log('All flags:', posthog.getFeatureFlags());
// This helps confirm flags loaded correctly
posthog.onFeatureFlags(function() {
console.log('Feature flags loaded');
});flag_variant to compare cohorts. If you only capture the flag at page load and not at the goal event, you'll lose the connection.Measure Impact with Insights
Now that flag variant data is flowing into PostHog, build insights to compare metrics between users who saw different variants.
Create a trend insight filtered by flag variant
Go to Insights and create a new Trend. Select your conversion event (e.g., purchase_completed). Under Add filter, select flag_variant and choose the specific variant or variants you want to compare. PostHog will show you conversion rate (or event count) for that cohort.
// Example: tracking purchase_completed event filtered by new-checkout-flow variant
// In PostHog UI:
// 1. Create Trend insight
// 2. Event: purchase_completed
// 3. Add filter: flag_variant equals "new-checkout-flow"
// 4. Optionally add breakdown by flag_variant to see both variants side-by-side
// This is what the backend sees in your event stream:
posthog.capture('purchase_completed', {
amount: 129.99,
flag_variant: 'new-checkout-flow',
flag_key: 'new-checkout-flow',
// ... other properties
});Breakdown by flag variant to compare cohorts
In the same insight, use Breakdown by and select flag_variant. PostHog will now show two lines (or bars): conversion for each variant. This is your direct A/B comparison. Watch for at least 100 users per variant before trusting the difference.
// Example capture with both control and treatment variants
// Control variant (no flag):
posthog.capture('purchase_completed', {
amount: 79.99,
flag_variant: 'control',
flag_key: 'new-checkout-flow',
});
// Treatment variant (flag enabled):
posthog.capture('purchase_completed', {
amount: 89.99,
flag_variant: 'new-checkout-flow',
flag_key: 'new-checkout-flow',
});Check funnels to catch drop-off by variant
Go to Insights > Funnel and add your conversion steps (e.g., checkout_viewed → payment_submitted → purchase_completed). Add a breakdown by flag_variant. If one variant has higher drop-off at a specific step, you've found a usability issue.
// Capture funnel events with flag context
// Step 1: View checkout
posthog.capture('checkout_viewed', {
flag_variant: posthog.getFeatureFlagPayload('new-checkout-flow'),
});
// Step 2: Submit payment
posthog.capture('payment_submitted', {
flag_variant: posthog.getFeatureFlagPayload('new-checkout-flow'),
payment_method: 'card',
});
// Step 3: Purchase confirmed
posthog.capture('purchase_completed', {
flag_variant: posthog.getFeatureFlagPayload('new-checkout-flow'),
amount: 99.99,
});flag_key (not flag_variant), you're including all users who got any variant of that flag. Always use flag_variant to isolate a single variant for measurement.Query Flag Impact with SQL
For deeper analysis, use PostHog's SQL mode to calculate exact conversion lift and statistical significance across flag variants.
Write a SQL query to compare conversion rates
Go to Insights > SQL and write a query that groups events by properties['flag_variant'] and calculates conversion rate. This gives you precise numbers instead of relying on the UI.
// SQL query (in PostHog SQL editor):
SELECT
properties->'flag_variant' as variant,
COUNT(DISTINCT person_id) as users,
COUNT(*) FILTER (WHERE event = 'purchase_completed') as purchases,
ROUND(COUNT(*) FILTER (WHERE event = 'purchase_completed')::float / COUNT(DISTINCT person_id), 4) as conversion_rate
FROM events
WHERE event IN ('checkout_viewed', 'purchase_completed')
AND timestamp > now() - interval '7 days'
AND properties->>'flag_key' = 'new-checkout-flow'
GROUP BY properties->'flag_variant'
ORDER BY conversion_rate DESC;
// This returns:
// variant | users | purchases | conversion_rate
// control | 1000 | 180 | 0.1800
// new-checkout-flow | 1050 | 220 | 0.2095Calculate lift and count minimum sample size
With your conversion rates, calculate lift: (treatment - control) / control * 100. For significance, aim for at least 100-200 conversions per variant. PostHog doesn't do built-in stats tests, so use a separate tool (Statsig, Eppo, or a Python notebook) if you need p-values.
// Manual lift calculation from your SQL results:
const controlConversionRate = 0.1800;
const treatmentConversionRate = 0.2095;
const lift = ((treatmentConversionRate - controlConversionRate) / controlConversionRate) * 100;
console.log(`Lift: ${lift.toFixed(2)}%`); // Output: Lift: 16.39%
// Check sample size adequacy:
const controlConversions = 180;
const treatmentConversions = 220;
const minSamplePerVariant = 100;
if (controlConversions >= minSamplePerVariant && treatmentConversions >= minSamplePerVariant) {
console.log('Sample size is adequate');
} else {
console.log('Run longer to reach statistical power');
}Common Pitfalls
- Forgetting to include flag variant in your goal events—if you only capture the flag at page load, you'll lose the link to conversion later.
- Mixing flag variants in a single event property instead of using a consistent
flag_variantfield—this makes filtering and breakdown inconsistent. - Running flags for too short a window (hours, not days)—feature flag impact often needs 100+ users per variant before you see real differences.
- Assuming PostHog will auto-track flag evaluations—you must manually capture flag state in your events for segmentation to work.
Wrapping Up
Tracking feature flag impact in PostHog boils down to three steps: capture which variant each user sees, segment your conversion metrics by variant, and compare the results. You now have a direct way to measure whether your flag experiment moves the needle. If you want to automate this across all your tools and flags, Product Analyst can help.