The Problem: Reactive Debugging

Operating a React Native app without proper observability meant:

User-Reported Issues:

• "The app crashes sometimes" - no stack trace, no context
• "It's slow" - which screen? which interaction?
• "I can't log in" - works fine in testing

Development Pain:

• Hours spent reproducing bugs locally
• No visibility into production behavior
• Guessing at performance bottlenecks
• Learning about crashes days after they happened

Business Impact:

• Users churning before we knew they had problems
• No data to prioritize fixes
• Slow incident response times

The Solution: Comprehensive Observability

Stack Components

1. Sentry: Error tracking, crash reporting, performance monitoring
2. Prometheus: Time-series metrics collection
3. Grafana: Visualization, dashboards, and correlation
4. Loki: Log aggregation (bonus addition)

Why This Stack?

• Sentry: Best-in-class React Native integration, excellent UX
• Prometheus: Industry standard for metrics, powerful query language
• Grafana: Combines metrics + logs + traces in one view
• Total cost: <$200/month for early-stage startup

Implementation

Phase 1: Sentry for Error Tracking

Installation

npm install --save @sentry/react-native
npx @sentry/wizard -i reactNative -p ios android

Configuration

// src/config/sentry.ts
import * as Sentry from '@sentry/react-native';
import { CaptureConsole } from '@sentry/integrations';

Sentry.init({
  dsn: process.env.SENTRY_DSN,
  
  // Environments
  environment: __DEV__ ? 'development' : 'production',
  
  // Release tracking
  release: `${APP_NAME}@${VERSION}`,
  dist: BUILD_NUMBER,
  
  // Performance monitoring
  tracesSampleRate: __DEV__ ? 1.0 : 0.2,
  
  // Session tracking
  enableAutoSessionTracking: true,
  sessionTrackingIntervalMillis: 30000,
  
  // Integrations
  integrations: [
    new CaptureConsole({
      levels: ['error', 'warn']
    }),
    new Sentry.ReactNativeTracing({
      routingInstrumentation: new Sentry.ReactNavigationInstrumentation(),
    }),
  ],
  
  // Filter sensitive data
  beforeSend(event, hint) {
    // Remove PII
    if (event.request) {
      delete event.request.cookies;
      delete event.request.headers?.Authorization;
    }
    return event;
  },
});

React Navigation Integration

// src/navigation/index.tsx
import { NavigationContainer } from '@react-navigation/native';
import * as Sentry from '@sentry/react-native';

const navigationRef = React.createRef();

function RootNavigator() {
  return (
    <NavigationContainer
      ref={navigationRef}
      onReady={()=> {
        Sentry.ReactNavigationInstrumentation.registerNavigationContainer(
          navigationRef
        );
      }}
    >
      <Stack.Navigator>{/* routes */}</Stack.Navigator>
    </NavigationContainer>
  );
}

Custom Context

// Attach user context
Sentry.setUser({
  id: user.id,
  email: user.email,
  username: user.username,
});

// Attach tags for filtering
Sentry.setTag('user_role', user.role);
Sentry.setTag('subscription_tier', user.tier);

// Breadcrumbs for debugging
Sentry.addBreadcrumb({
  category: 'user_action',
  message: 'User tapped checkout button',
  level: 'info',
  data: {
    screen: 'ProductDetail',
    product_id: productId,
  },
});

Performance Monitoring

// Measure critical operations
function processPayment(data: PaymentData) {
  const transaction = Sentry.startTransaction({
    name: 'payment_processing',
    op: 'payment',
  });

  try {
    const span1 = transaction.startChild({
      op: 'validate',
      description: 'Validate payment data',
    });
    validatePayment(data);
    span1.finish();

    const span2 = transaction.startChild({
      op: 'api_call',
      description: 'Call payment gateway',
    });
    await paymentGateway.charge(data);
    span2.finish();

    transaction.setStatus('ok');
  } catch (error) {
    transaction.setStatus('error');
    throw error;
  } finally {
    transaction.finish();
  }
}

Phase 2: Prometheus for Metrics

Setup with Expo (or bare React Native)

// src/lib/metrics.ts
import { Platform } from 'react-native';

class MetricsCollector {
  private endpoint = 'https://your-prometheus-pushgateway.com';
  
  private metrics: Map<string, number> = new Map();
  
  increment(metric: string, value: number = 1, labels: Record<string, string> = {}) {
    const key = this.generateKey(metric, labels);
    this.metrics.set(key, (this.metrics.get(key) || 0) + value);
  }
  
  gauge(metric: string, value: number, labels: Record<string, string> = {}) {
    const key = this.generateKey(metric, labels);
    this.metrics.set(key, value);
  }
  
  async flush() {
    const data = this.formatPrometheus();
    
    await fetch(`${this.endpoint}/metrics/job/${Platform.OS}`, {
      method: 'POST',
      headers: { 'Content-Type': 'text/plain' },
      body: data,
    });
    
    this.metrics.clear();
  }
  
  private generateKey(metric: string, labels: Record<string, string>): string {
    const labelStr = Object.entries(labels)
      .map(([k, v]) => `${k}="${v}"`)
      .join(',');
    return `${metric}{${labelStr}}`;
  }
  
  private formatPrometheus(): string {
    return Array.from(this.metrics.entries())
      .map(([key, value]) => `${key} ${value}`)
      .join('\n');
  }
}

export const metrics = new MetricsCollector();

// Flush every 30 seconds
setInterval(() => metrics.flush(), 30000);

Instrumenting the App

// Track screen views
function useScreenTracking(screenName: string) {
  useEffect(() => {
    const startTime = Date.now();
    
    metrics.increment('screen_view_total', 1, {
      screen: screenName,
      platform: Platform.OS,
    });
    
    return () => {
      const duration = Date.now() - startTime;
      metrics.gauge('screen_duration_ms', duration, {
        screen: screenName,
      });
    };
  }, [screenName]);
}

// Track API calls
async function apiCall(endpoint: string, options: RequestInit) {
  const startTime = Date.now();
  
  try {
    const response = await fetch(endpoint, options);
    const duration = Date.now() - startTime;
    
    metrics.gauge('api_response_time_ms', duration, {
      endpoint,
      status: String(response.status),
    });
    
    metrics.increment('api_call_total', 1, {
      endpoint,
      status: String(response.status),
      method: options.method || 'GET',
    });
    
    return response;
  } catch (error) {
    metrics.increment('api_call_errors_total', 1, {
      endpoint,
      error: error.message,
    });
    throw error;
  }
}

// Track business metrics
function trackCheckout(amount: number, items: number) {
  metrics.increment('checkout_total', 1);
  metrics.gauge('checkout_amount_usd', amount);
  metrics.gauge('checkout_items_count', items);
}

Phase 3: Grafana Dashboards

Dashboard 1: App Health

# Grafana dashboard config (JSON simplified as YAML for readability)
panels:
  - title: "Error Rate"
    type: graph
    targets:
      - expr: rate(app_errors_total[5m])
    
  - title: "Crash-Free Users"
    type: stat
    targets:
      - expr: (sum(users_total) - sum(users_crashed)) / sum(users_total) * 100
    
  - title: "Active Users"
    type: graph
    targets:
      - expr: sum(screen_view_total) by (platform)

Dashboard 2: Performance

panels:
  - title: "API Response Time (P95)"
    type: graph
    targets:
      - expr: histogram_quantile(0.95, rate(api_response_time_ms[5m]))
    
  - title: "Screen Load Time"
    type: heatmap
    targets:
      - expr: screen_duration_ms
    
  - title: "Memory Usage"
    type: graph
    targets:
      - expr: app_memory_usage_mb

Dashboard 3: Business Metrics

panels:
  - title: "Conversion Funnel"
    type: table
    targets:
      - expr: |
          sum(screen_view_total{screen="ProductList"}) /
          sum(screen_view_total{screen="Checkout"}) /
          sum(checkout_total)
    
  - title: "Revenue"
    type: stat
    targets:
      - expr: sum(checkout_amount_usd)

Phase 4: Alerting

Sentry Alerts

# Configured in Sentry UI
alerts:
  - name: "High Error Rate"
    condition: "errors > 10 in 5 minutes"
    actions:
      - type: slack
        channel: "#alerts-critical"
      - type: pagerduty
        
  - name: "New Error Type"
    condition: "first seen"
    actions:
      - type: slack
        channel: "#alerts-new-issues"

Prometheus Alert Rules

# prometheus/alerts.yml
groups:
  - name: app_alerts
    interval: 30s
    rules:
      - alert: HighErrorRate
        expr: rate(app_errors_total[5m]) > 0.05
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "High error rate detected"
          description: "Error rate is {{ $value }} errors/second"
      
      - alert: SlowAPIResponses
        expr: histogram_quantile(0.95, rate(api_response_time_ms[5m])) > 2000
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "API responses are slow"
          description: "P95 response time is {{ $value }}ms"
      
      - alert: LowCrashFreeRate
        expr: (sum(users_total) - sum(users_crashed)) / sum(users_total) < 0.98
        for: 1h
        labels:
          severity: critical
        annotations:
          summary: "Crash-free rate below 98%"

Results

Crash Diagnosis Time

• Before: 2-4 hours per crash
• After: 20-40 minutes per crash
• Improvement: 70% reduction

Mean Time to Resolution

• Before: 3-5 days
• After: <24 hours
• Improvement: 80% reduction

Proactive Bug Detection

• Before: 0 bugs caught before user reports
• After: 60% of bugs caught via alerts
• Impact: Better user experience

Business Impact

• Crash-free users: 92% → 99.2%
• User retention: +15%
• App store rating: 3.8 → 4.6

Critical Metrics to Track

1. Stability Metrics

metrics.gauge('crash_free_sessions_percent', calculateCrashFree());
metrics.increment('app_crashes_total', 1, { screen, error_type });
metrics.gauge('anr_rate', calculateANRRate()); // Android

2. Performance Metrics

metrics.gauge('app_start_time_ms', startupDuration);
metrics.gauge('js_bundle_load_time_ms', bundleLoadTime);
metrics.gauge('api_response_time_ms', responseTime, { endpoint });
metrics.gauge('frame_drop_count', frameDrops, { screen });

3. User Engagement

metrics.increment('daily_active_users', 1, { date });
metrics.gauge('session_duration_ms', sessionLength);
metrics.increment('feature_usage_total', 1, { feature_name });

4. Business Metrics

metrics.increment('signup_total', 1, { source });
metrics.increment('conversion_total', 1, { funnel_step });
metrics.gauge('revenue_usd', amount, { product_type });

Alerting Strategy

Avoid Alert Fatigue

1. Severity Levels:

   • critical: Page on-call engineer (crash rate spike)
   • warning: Slack notification (slow responses)
   • info: Dashboard only (minor anomalies)

2. Alert Thresholds:

   • Use percentiles, not averages (P95, P99)
   • Set thresholds based on baseline + margin
   • Require sustained issues (e.g., "for 5 minutes")

3. Alert Grouping:

   • Group related errors (same root cause)
   • Rate limit notifications (max 1 per 30 min)

Example: Tiered Alerting

# Critical: Immediate action required
- alert: AppCrashSpike
  expr: rate(app_crashes_total[5m]) > 0.01
  severity: critical
  notify: pagerduty

# Warning: Investigate soon
- alert: SlowCheckout
  expr: histogram_quantile(0.95, checkout_duration_ms) > 5000
  severity: warning
  notify: slack

# Info: Monitor trend
- alert: IncreasingMemoryUsage
  expr: deriv(app_memory_usage_mb[1h]) > 5
  severity: info
  notify: dashboard

Cost Considerations

Startup Budget (<$200/month)

Sentry:

• Free tier: 5K errors/month
• Team plan: $26/month (50K errors)
• Business plan: $80/month (500K errors)

Prometheus + Grafana:

• Self-hosted on AWS t3.small: ~$15/month
• Grafana Cloud free tier: 10K metrics

Total: ~$100/month for early-stage app

Cost Optimization Tips

1. Sample Performance Transactions:

Sentry.init({
  tracesSampleRate: __DEV__ ? 1.0 : 0.1, // 10% in production
});

2. Filter Noisy Errors:

beforeSend(event) {
  // Ignore known third-party errors
  if (event.exception?.values?.[0]?.value?.includes('Network request failed')) {
    return null;
  }
  return event;
}

3. Aggregate Metrics Client-Side:

// Send aggregated metrics every 30s instead of individual events

Lessons Learned

1. Start with Observability on Day One

Adding observability to a mature app is painful. Instrument from the start.

2. Context is Everything

Stack traces alone don't solve bugs. Add user context, breadcrumbs, and tags.

3. Dashboards Should Tell a Story

Don't just show metrics. Create dashboards that answer questions: "Is the app healthy?" "Why are users churning?"

4. Alert on User Impact, Not Just Errors

An error affecting 1 user is different from one affecting 1000. Alert on impact.

5. Metrics Inform Product Decisions

Performance data showed users dropping off on slow screens, leading to UX improvements that increased conversion by 18%.

Next Steps

Phase 5 Additions:

• Distributed Tracing: Jaeger for full request flow
• Real User Monitoring: Track actual user experience
• Log Aggregation: Centralized logs with Loki
• ML-Powered Alerts: Anomaly detection for unusual patterns