AB testing

Feature Flags

The feature flag system provides both server-side and client-side feature flag resolution to eliminate loading states on critical UI components while maintaining flexibility for less critical features.

Overview

The system offers three main approaches:

  1. ServerFeatureFlagResolver - Complete server-side resolution (no client-side loading)
  2. HybridFeatureFlagProvider - Fetches critical flags server-side, passes to client provider
  3. FeatureFlagProvider - Enhanced client provider that accepts server flags

Architecture

Server-Side Feature Flags

ServerFeatureFlagResolver (Recommended for Critical Components)

Use this approach for components where loading states are unacceptable, such as hero sections or above-the-fold content. The resolver automatically handles user exposure tracking on the client side.

Import the resolver

import { ServerFeatureFlagResolver } from "core/libs/featureFlags"

Use in server components

page.tsx
import { ExperimentalComponent } from "./components/ExperimentalComponent"
import { StandardComponent } from "./components/StandardComponent"
 
import { ServerFeatureFlagResolver } from "core/libs/featureFlags"
 
export default async function Page() {
  return (
    <ServerFeatureFlagResolver flag="my-experiment">
      <StandardComponent />
      <ExperimentalComponent />
    </ServerFeatureFlagResolver>
  )
}

Benefits

  • ✅ Zero loading states - No flickering or spinners
  • ✅ SEO-friendly - Correct variant rendered server-side
  • ✅ Better UX - Eliminates layout shifts
  • ✅ Performance - Faster initial page load
  • ✅ Automatic tracking - Exposure tracking happens client-side automatically

When to use

  • Hero sections and above-the-fold content
  • Components that affect layout significantly
  • Critical user flows where loading states hurt UX

How tracking works

The ServerFeatureFlagResolver automatically wraps the resolved component with ServerFeatureFlagWithTracking, which:

  1. Renders server-side - Component is resolved and rendered on the server
  2. Tracks client-side - Exposure tracking happens after hydration
  3. No additional setup - Works automatically with existing analytics infrastructure
📊

Automatic Analytics

No manual trackUserExposure calls needed! The resolver handles this automatically while maintaining zero loading states.

ServerFeatureFlagProvider

For more complex server-side scenarios where you need access to multiple flags:

server-component.tsx
import { ServerFeatureFlagProvider } from "core/libs/featureFlags"
 
export default async function ServerComponent() {
  return (
    <ServerFeatureFlagProvider
      criticalFlags={["navigation-redesign", "checkout-flow"]}
    >
      {({ serverFeatureFlags }) => (
        <div>
          {serverFeatureFlags?.["navigation-redesign"] === "b" && (
            <NewNavigation />
          )}
          {serverFeatureFlags?.["checkout-flow"] === "a" && (
            <StandardCheckout />
          )}
        </div>
      )}
    </ServerFeatureFlagProvider>
  )
}

Hybrid Approach

HybridFeatureFlagProvider

Combines server-side fetching for critical flags with client-side flexibility:

layout.tsx
import {
  FeatureFlagProvider,
  HybridFeatureFlagProvider
} from "core/libs/featureFlags"
 
export default function Layout({ children }) {
  return (
    <HybridFeatureFlagProvider
      criticalFlags={["navigation-redesign", "checkout-flow"]}
      trackingId="user-tracking-id"
    >
      {({ serverFeatureFlags }) => (
        <FeatureFlagProvider serverFeatureFlags={serverFeatureFlags}>
          {children}
        </FeatureFlagProvider>
      )}
    </HybridFeatureFlagProvider>
  )
}

Benefits

  • ✅ Best of both worlds - Critical flags served instantly, others load as needed
  • ✅ Backward compatible - Works with existing client components
  • ✅ Flexible - Easy to move flags between server and client as needed

Client-Side Feature Flags

Enhanced FeatureFlagProvider

The client provider now accepts server-provided flags and merges them seamlessly:

client-component.tsx
"use client"
 
import { useFeatureFlag } from "core/libs/featureFlags"
 
export function MyComponent() {
  const { featureFlags, initialised, trackUserExposure } = useFeatureFlag()
 
  // Server flags are available immediately, no loading state needed
  const experimentVariant = featureFlags?.["my-experiment"]
 
  // Track user exposure for analytics
  if (experimentVariant && initialised) {
    trackUserExposure("my-experiment")
  }
 
  return (
    <div>
      {experimentVariant === "b" ? (
        <ExperimentalComponent />
      ) : (
        <StandardComponent />
      )}
    </div>
  )
}

Enhanced Features

  • Server flag integration - Accepts server-provided flags
  • Automatic merging - Combines server and client flags intelligently
  • Debug logging - Development-time logging shows flag sources
  • Backwards compatible - Existing components work unchanged

Best Practices

Flag Placement Strategy

📋

Use this decision tree to choose the right approach:

  1. Critical for UX (hero, above-fold) → ServerFeatureFlagResolver
  2. Multiple critical flags → HybridFeatureFlagProvider
  3. Non-critical, interactive → Client FeatureFlagProvider
  4. Legacy/existing → Enhanced client provider (no changes needed)

Performance Considerations

// ✅ Good - Server flags for critical components
<ServerFeatureFlagResolver flag="checkout-flow-experiment">
  <StandardCheckout />
  <SimplifiedCheckout />
</ServerFeatureFlagResolver>
 
// ✅ Good - Hybrid for mixed scenarios
<HybridFeatureFlagProvider criticalFlags={["navigation", "checkout"]}>
  {({ serverFeatureFlags }) => (
    <FeatureFlagProvider serverFeatureFlags={serverFeatureFlags}>
      <App />
    </FeatureFlagProvider>
  )}
</HybridFeatureFlagProvider>
 
// ❌ Avoid - Client-only for critical above-fold content
<FeatureFlagProvider>
  <CriticalComponent /> {/* This will show loading spinner */}
</FeatureFlagProvider>

Flag Naming Convention

Follow consistent naming for better organization:

// ✅ Good - Descriptive and categorized
const flags = [
  "checkout-flow-simplified",
  "navigation-redesign-mobile",
  "pricing-table-experiment"
]
 
// ❌ Avoid - Vague or unclear
const flags = [
  "test-a",
  "experiment-1",
  "flag-new"
]

Analytics Integration

Always track user exposure for analytics:

const { featureFlags, trackUserExposure } = useFeatureFlag()
 
useEffect(() => {
  if (featureFlags?.["my-experiment"] && initialised) {
    trackUserExposure("my-experiment")
  }
}, [featureFlags, initialised, trackUserExposure])

Migration Guide

From Client-Only to Hybrid

Identify critical flags

Review your application and identify flags that affect above-the-fold content or cause layout shifts.

Update layout

Add the HybridFeatureFlagProvider to your layout:

layout.tsx
// Before
<FeatureFlagProvider>
  <App />
</FeatureFlagProvider>
 
// After
<HybridFeatureFlagProvider criticalFlags={["homepage-hero", "navigation"]}>
  {({ serverFeatureFlags }) => (
    <FeatureFlagProvider serverFeatureFlags={serverFeatureFlags}>
      <App />
    </FeatureFlagProvider>
  )}
</HybridFeatureFlagProvider>

Test thoroughly

Verify that:

  • Critical flags load without loading states
  • Non-critical flags still work as expected
  • Analytics tracking continues to work
  • No performance regressions

Troubleshooting

Common Issues

⚠️

Server/Client Flag Mismatches

If you see hydration errors, ensure server and client flags are consistent. The hybrid provider automatically handles this by prioritizing client flags for updates.

Debug Mode

  • Automatically enabled in development
  • Check browser console for flag sources and values

API Reference

ServerFeatureFlagResolver

type Props = {
  flag: string
  criticalFlags?: string[]
  children: [ReactNode, ReactNode] // [VariantA, VariantB]
}

ServerFeatureFlagWithTracking

type Props = {
  flag: string
  variant: "a" | "b" | "c"
  children: React.ReactNode
}
ℹ️

Advanced Usage

While ServerFeatureFlagResolver handles tracking automatically, you can use ServerFeatureFlagWithTracking directly for custom server-side implementations.

HybridFeatureFlagProvider

type Props = {
  children: (props: {
    serverFeatureFlags: Partial<Record<string, FeatureFlagValues>> | null
  }) => ReactNode
  criticalFlags?: string[]
  trackingId: string
}

FeatureFlagProvider

type Props = {
  children: ReactNode
  serverFeatureFlags?: Partial<Record<string, FeatureFlagValues>> | null
}

useFeatureFlag Hook

type Return = {
  initialised: boolean
  featureFlags: Partial<Record<string, FeatureFlagValues>> | null
  trackUserExposure: (featureFlag: string) => void
}

CatsAPI Setup for Local Development

When creating a new feature flag or feature experiment, follow these steps to set up proper local development with the CatsAPI:

Set up the feature experiment in Amplitude

  1. Create the experiment in the ‘Marro Cat - Development’ project in Amplitude
  2. Connect the experiment to the ‘rails-server-development’ deployment
  3. Set evaluation mode to ‘Local’ (matches the local evaluation Amplitude client in CatsAPI)
  4. Configure users to be bucketed by User ID

Later on, once satisfied with the configuration, you can copy the experiment to the ‘Marro Cat’ production project and connect it to the ‘rails-server’ deployment.

Configure environment variables in CatsAPI

In your .env file in CatsAPI, set:

AMPLITUDE_EXPERIMENT_STUB_EVALUATION=false
STUB_ANALYTICS_USER_ID=false
AMPLITUDE_EXPERIMENT_DEBUG_MODE=true  # Optional: for extra debugging logs
⚠️

Remember to remove these or update to the opposite value when finished with local development, to avoid contributing to the MTU quota in Amplitude and Segment.

Enable Segment destination for local development

To see exposure events in Amplitude during local development:

  1. Enable the ‘API Development’ source in Segment, if not already enabled
  2. Enable the ‘Amplitude cats - development’ destination in Segment
⚠️

Remember to disable the destination when finished with local development, to avoid contributing to the MTU quota in Amplitude.

As exposure events are tracked asynchronously by the API, you will need to run sidekiq locally if you want to see exposure events send to Segment and therefore Amplitude. bundle exec sidekiq

Understanding event tracking

GraphQL Queries and Event Types

featureFlagsVariantValues Query:

  • Purpose: Fetches all available feature flags for a user
  • Triggers: Assignment events automatically (server-side via Amplitude)
  • When to use: Initial flag resolution and when user ID changes

variantValueWithExposure Query:

  • Purpose: Fetches the variant for a particular feature flag and user, and records that the user has been exposed to the assigned variant
  • Triggers: Exposure events via Segment in CatsAPI (sent on to Amplitude)
  • When to use: When user actually sees a variant

Event Types Explained

  • Assignment events: Sent automatically by Amplitude server-side when a flag is evaluated (when featureFlagsVariantValues or variantValueWithExposure are queried). Indicates a user has been assigned a variant but may not have seen it yet.

  • Exposure events: Not automatically sent by Amplitude’s server-side SDK. We send these via Segment when variantValueWithExposure is queried. Indicates a user has actually seen the variant they were assigned.

QA-ing with multiple events

Amplitude automatically deduplicates events based on tracking ID (user ID in Amplitude). When QA-ing an experiment locally, you can send more than one event from local development within the same session by updating the analytics_user_id cookie value to a new value.

Examples

Homepage Implementation

homepage/page.tsx
import { Hero } from "./Hero"
import { HeroVariant } from "./HeroVariant"
 
import { ServerFeatureFlagResolver } from "core/libs/featureFlags"
 
export default async function Homepage() {
  return (
    <main>
      <ServerFeatureFlagResolver flag="homepage-hero-april">
        <Hero />
        <HeroVariant />
      </ServerFeatureFlagResolver>
 
      {/* Other components can use client flags */}
      <Benefits />
      <Testimonials />
    </main>
  )
}

Layout with Hybrid Provider

layout.tsx
import { HybridFeatureFlagProvider, FeatureFlagProvider } from "core/libs/featureFlags"
 
export default function RootLayout({ children }) {
  return (
    <HybridFeatureFlagProvider
      criticalFlags={["homepage-hero", "navigation-redesign"]}
      trackingId={await getUserTrackingId()}
    >
      {({ serverFeatureFlags }) => (
        <FeatureFlagProvider serverFeatureFlags={serverFeatureFlags}>
          <Header />
          {children}
          <Footer />
        </FeatureFlagProvider>
      )}
    </HybridFeatureFlagProvider>
  )
}

Client Component Usage

components/InteractiveFeature.tsx
"use client"
 
import { useFeatureFlag } from "core/libs/featureFlags"
 
export function InteractiveFeature() {
  const { featureFlags, initialised, trackUserExposure } = useFeatureFlag()
 
  const showNewUI = featureFlags?.["interactive-redesign"] === "b"
 
  useEffect(() => {
    if (showNewUI && initialised) {
      trackUserExposure("interactive-redesign")
    }
  }, [showNewUI, initialised, trackUserExposure])
 
  if (!initialised) {
    return <Skeleton /> // Only for non-critical components
  }
 
  return showNewUI ? <NewInteractiveUI /> : <StandardInteractiveUI />
}
Getting startedAnalytics