5 min read

How to Track Impact Analysis in Mixpanel

When you ship a feature or run an experiment, you need to know if it actually moves the needle. Mixpanel's Impact Analysis lets you compare your key metrics before and after a specific event, cutting through the noise to show what changed. To make it work, you need to track the triggering event alongside your metric events.

Track the Change Event

First, create an event that marks when your change happened — a release, experiment start, or marketing campaign launch.

Send the change event from your app

Call mixpanel.track() with a clear event name and properties that identify what changed. Include a timestamp to match against metric data.

javascript
mixpanel.track('Feature Released', {
  feature_name: 'dark_mode',
  environment: 'production',
  variant: 'roll_out',
  released_by: 'product_team',
  timestamp: new Date().toISOString()
});
Track a feature release as the trigger event for Impact Analysis

Use consistent property names

Make sure the properties you send (like feature_name) match what you'll filter by later. Mixpanel is case-sensitive, so feature_name and featureName are different.

javascript
// ❌ Don't mix naming conventions
mixpanel.track('Experiment Started', { ExperimentID: '123' });

// ✅ Use snake_case consistently
mixpanel.track('Experiment Started', { experiment_id: '123' });
Keep property naming consistent across your codebase
Tip: Always include a variant or cohort property if you're running an A/B test. Impact Analysis can segment the results by variant.

Instrument Your Key Metrics

Impact Analysis only works if you're already tracking the events you want to measure. Focus on events that represent real business outcomes.

Track outcome events with numeric properties

Events like Purchase Completed, Signup, or Subscription Upgraded should include numeric values that Impact Analysis can aggregate (revenue, quantity, time spent).

javascript
mixpanel.track('Purchase Completed', {
  amount: 99.99,
  currency: 'USD',
  product_id: 'prod_dark_mode',
  user_plan: 'pro',
  items_count: 3
});
Include numeric properties that Impact Analysis can sum or average

Set user properties for segmentation

Use mixpanel.people.set() to tag users with properties you want to filter by in Impact Analysis results, like plan tier or feature flags.

javascript
mixpanel.people.set({
  'dark_mode_enabled': true,
  'plan_tier': 'pro',
  'signup_date': '2025-01-15',
  'experiments_enrolled': ['dark_mode_v1', 'onboarding_v2']
});
Set properties for segmenting Impact Analysis by user cohort
Watch out: Measuring vanity metrics like Page Viewed won't help. Focus on events that directly reflect user intent or business value.

Run Impact Analysis in Mixpanel

With your change event and metrics tracked, you can now measure impact using Mixpanel's Impact Analysis tool.

Navigate to Impact Analysis

In your Mixpanel project, select Impact from the left sidebar. Mixpanel will show you options to select a change event and a metric.

Select your change event and date range

Pick the event you sent earlier (e.g., Feature Released) and filter by the property that identifies it (e.g., feature_name = 'dark_mode'). Set the date range around when the change occurred.

Choose the metric to measure

Select one of your tracked events (e.g., Purchase Completed) and choose how to aggregate it. You can measure total event count, sum of a numeric property, or distinct user count.

javascript
// Your metric event (sent from your app)
mixpanel.track('Purchase Completed', {
  amount: 75.50,
  currency: 'USD'
});

// Impact Analysis sums the 'amount' field
// Result: "Revenue increased 23% after Feature Released"
Impact Analysis aggregates numeric properties to show metric changes

Interpret statistical significance

Mixpanel displays the before/after comparison and a p-value. If p < 0.05, the change is statistically significant. Higher p-values indicate the difference might be random variation.

javascript
// Don't rely on a single metric
const impactData = {
  metric: 'Purchase Completed',
  before: { count: 342, avg: 67.50 },
  after: { count: 420, avg: 72.30 },
  pvalue: 0.032,
  confidence: 'statistically significant'
};

// High p-value means the metric change could be random
if (impactData.pvalue > 0.05) {
  console.log('Result inconclusive — need more data');
}
Evaluate p-value to determine if impact is statistically significant
Tip: Use Breakdown in Impact Analysis to segment results by user properties. You might find that dark mode helps mobile users but hurts desktop users — that matters.

Common Pitfalls

  • Tracking the change event too late — if you send 'Feature Released' hours after deployment, Impact Analysis misses the early spike.
  • Measuring vanity metrics instead of outcomes — 'Feature Viewed' tells you nothing; measure conversions, revenue, or retention instead.
  • Forgetting to set user properties — you'll see overall impact but can't drill into which cohorts benefited.
  • Confusing change events with metric events — the change event ('Feature Released') triggers the analysis; the metric event ('Purchase Completed') is what gets measured.

Wrapping Up

You're now tracking changes with clear event definitions and measuring their impact on your core metrics. This tells you whether your shipping decisions actually work. If you want to track impact automatically across all your analytics tools — Mixpanel, Amplitude, PostHog — and get unified insights, Product Analyst can help.

Track these metrics automatically

Product Analyst connects to your stack and surfaces the insights that matter.

Try Product Analyst — Free