6 min read

How to Monitor Feature Flag Impact in PostHog

Feature flags control how users experience your product, but you can't improve what you don't measure. PostHog lets you capture events tied to flag variants and compare metrics between groups to see what actually changed. We'll show you how to set up impact tracking so you know whether your flags are helping or hurting.

Capture Events When Flags Are Evaluated

Start by instrumenting your code to record when users encounter a flag and which variant they got. This becomes your baseline for all impact analysis.

Initialize PostHog and get the flag value

Load PostHog in your app and call getFeatureFlag() to check which variant a user is assigned to. Store the variant so you can tag events with it.

javascript
import posthog from 'posthog-js';

posthog.init('phc_YOUR_PROJECT_KEY', {
  api_host: 'https://us.i.posthog.com',
});

const checkoutVariant = posthog.getFeatureFlag('new-checkout-flow');
// Returns 'control', 'variant-a', 'variant-b', or false if flag is off
Check the flag value. This tells you which experience the user is in.

Capture events and tag them with the flag variant

When users take key actions (form submission, purchase, signup), call capture() and include the flag variant as a property. This links the action directly to the flag.

javascript
posthog.capture('checkout_started', {
  revenue_impact: order_total,
  checkout_variant: checkoutVariant,
  cart_size: items.length,
});

// Later, on purchase:
posthog.capture('order_completed', {
  order_value: finalTotal,
  checkout_variant: checkoutVariant,
  time_on_page: timeSpent,
});
Include the variant as a property in every relevant event. This is how PostHog links behavior to flags.

Set user properties with flag assignments

Use people.set() to tag users with their flag assignments. This persists across sessions so you can segment by flag later, even if the flag changes mid-visit.

javascript
posthog.people.set({
  'checkout_variant': checkoutVariant,
  'flag_assigned_at': new Date().toISOString(),
});

// All subsequent events from this user inherit the flag variant automatically
User properties stick around, so downstream events are tagged even if you don't pass the variant explicitly.
Watch out: Avoid capturing the flag variant on every single event—it creates noise. Tag only on actions that matter for decision-making (signups, purchases, feature usage). Page view events don't need it.

Analyze Impact Using Insights and Segmentation

Once you're capturing flag-tagged events, use PostHog's Insights tool to compare metrics between variants.

Create a trend insight to compare conversion rates

Go to Insights > New insight and create a trend. Select order_completed events, then add a Breakdown by the checkout_variant property. This shows conversion split by flag variant.

javascript
// Events are captured like this; PostHog UI handles the aggregation:
posthog.capture('order_completed', {
  checkout_variant: checkoutVariant,
  conversion_value: amount,
});

// In PostHog UI:
// Insights > Events > Select 'order_completed'
// Breakdown > Property 'checkout_variant'
// You now see conversion count/rate for control vs variant-a vs variant-b
The UI aggregates your events automatically. Each variant's performance appears as a separate line.

Build a funnel to track drop-off by variant

Go to Insights > Funnel and add events in sequence (checkout_started → payment_form_loaded → order_completed). Segment by checkout_variant to see where each variant loses users.

javascript
// Emit events in order with the flag variant:
posthog.capture('checkout_started', { checkout_variant: variant });
posthog.capture('payment_form_loaded', { checkout_variant: variant });
posthog.capture('payment_submitted', { checkout_variant: variant });
posthog.capture('order_completed', { checkout_variant: variant });

// PostHog calculates drop-off and duration between steps per variant
Funnels show not just conversion, but where users drop off and how long each step takes, split by flag.

Use cohorts to isolate and control for confounds

Create a Cohort for each flag variant using the property checkout_variant. Filter insights by cohort to control for geography, device, or other factors that might skew results.

javascript
// In PostHog UI:
// Cohorts > Create cohort > Criteria: Property 'checkout_variant' equals 'variant-a'
// Then in any insight, add filter: "User is in cohort 'variant-a'"

// Example: See mobile conversion rate only for variant-a
// Insights > order_completed
// Filter: "User is in cohort 'variant-a' AND property 'device_type' equals 'mobile'"
Cohorts make filtering reusable and ensure you're comparing like-for-like.
Tip: Use Session Replay to watch actual users in each variant. Numbers say 'variant-b is better' but replay shows users are confused by the UI. Combine quantitative and qualitative.

Monitor Continuously with Dashboards and Alerts

One-off reports are useful for decisions, but flag monitoring works best as a live dashboard you check during rollout and after.

Create a monitoring dashboard

Go to Dashboards > New dashboard and add your key insights: conversion by variant, funnel drop-off by variant, revenue per user. Name it something searchable like 'New Checkout Flag Rollout'.

javascript
// Build your insights first (trend, funnel, etc.)
// Then:
// Dashboards > Create dashboard > 'New Checkout Flag Rollout'
// Click 'Add insight' and select your variant-breakdown insights
// Set refresh to 5 minutes for live monitoring

// Share the dashboard link with your team so everyone watches the same metrics
Dashboards auto-refresh, so you see real-time data as the flag rolls out without manual checks.

Set alerts for metric degradation

On any insight, click Alert to notify Slack or email if a metric drops below a threshold. This catches bad flags before they damage the product.

javascript
// In PostHog:
// Insights > Your conversion insight > (three dots) > Create alert
// Alert when: conversion_rate < 4% (your baseline)
// Notify: #product-alerts Slack channel
// Threshold compares current period to previous period automatically

// PostHog sends a Slack message with the metric and a link to investigate
Alerts act as an early warning. You can roll back the flag immediately instead of waiting for the weekly report.
Watch out: Alert on percentages or ratios (conversion rate, revenue per user), not raw event counts. Raw counts fluctuate with traffic; rates tell the true story.

Common Pitfalls

  • Forgetting to tag events with the flag variant before rollout—then you can't compare variants because the data isn't there. Instrument early, before you turn the flag on.
  • Checking the flag once client-side then assuming it sticks—flag assignments can change mid-session. Get the variant value once and store it in state so events are tagged consistently.
  • Not waiting for statistical significance. With low traffic, one variant might look better by pure chance. PostHog Insights shows confidence intervals; wait for 95% confidence before claiming victory.
  • Monitoring only top-level metrics (DAU, total revenue). Flags often affect niche flows (checkout, onboarding). Instrument specific user actions so you see the real impact.

Wrapping Up

Feature flags only drive improvement if you measure their impact. With PostHog, the workflow is straightforward: tag events with flag variants, segment insights by those variants, and build a live dashboard to track changes in real time. This turns flags from guesses into data-driven experiments. If you want to track this automatically across tools, Product Analyst can help.

Track these metrics automatically

Product Analyst connects to your stack and surfaces the insights that matter.

Try Product Analyst — Free