Panoptic Estimator

NOTE

Web SDKs Only Realtime estimation is currently available for web-based SDKs only (JavaScript, React, Vue). It is not available for mobile SDKs (iOS, Android, Flutter, React Native).

Overview

The Panoptic Estimator is a medical-grade heart rate measurement system that uses advanced signal processing to analyze photoplethysmography (PPG) signals from facial video. It provides Panoptic-level accuracy without requiring external AI models, making it ideal for medical applications and instant initialization.

IMPORTANT

Proprietary Technology The Panoptic Estimator is proprietary software developed by PanopticAI. The core algorithm runs in WebAssembly for intellectual property protection.

Key Features

✅ Medical-Grade: Panoptic-level accuracy
✅ Instant Init: No model loading required
✅ Zero Dependencies: No external files needed
✅ Deterministic: Same input → same output
✅ Lightweight: < 200 KB memory footprint
✅ Adaptive: First estimates at ~8 seconds, optimal at ~15 seconds

Basic Usage

typescript

import { createVitalSignCamera, RealtimeEstimatorType } from 'ts-vital-sign-camera';

const camera = createVitalSignCamera({
  realtimeEstimationConfig: {
    estimatorType: RealtimeEstimatorType.Panoptic
  }
});

camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  if (!estimation) return;
  
  console.log(`HR: ${estimation.heartRate} BPM`);
  console.log(`Confidence: ${estimation.confidence}`);
  console.log(`SNR: ${estimation.snr}`);
});

TIP

Simplest Setup: Use enableRealtimeEstimation: true to activate the Panoptic estimator with default settings — no configuration needed.

Configuration

For Consumer Applications

typescript

{
  estimatorType: RealtimeEstimatorType.Panoptic,
  debug: false
}

For Medical Applications

typescript

{
  estimatorType: RealtimeEstimatorType.Panoptic,
  signalQualityDelay: 10,  // Wait longer before quality computation
  debug: false
}

SDK Behavior

Initialization

typescript

// Panoptic estimator initializes instantly (no async loading)
const camera = createVitalSignCamera({
  realtimeEstimationConfig: {
    estimatorType: RealtimeEstimatorType.Panoptic
  }
});

// Ready immediately - no waiting required
camera.startScan();

Processing Timeline

0s ────► 5s ────► 8s ────► 15s ────► 30s
│        │        │        │         │
│        │        │        │         └─ Scan complete
│        │        │        └─ Optimal accuracy (15s window)
│        │        └─ First estimate (warmup done, 3+ estimates)
│        └─ Signal quality delay (5s)
└─ Scan starts (instant init)

Warmup & Smoothing

The Panoptic estimator applies built-in warmup and smoothing to ensure stable, reliable readings.

Warmup Gate

No HR estimate is emitted until the estimator has collected 150 RGB samples (5s at 30fps) and accumulated at least 3 estimates. This prevents early garbage values from appearing while the buffer fills.

Typical first estimate: ~8 seconds (5s buffer + time for 3 estimate cycles at 1Hz)
During warmup, onVideoFrameProcessed events have realtimeEstimation === null.

1Hz Throttle

Signal processing runs at 1 Hz (once per second), not per frame. This ensures that each estimate is based on a meaningful time window and makes EWMA smoothing effectively regulate at a natural rate.

EWMA Smoothing

An Exponentially Weighted Moving Average smooths the HR output:

typescript

// EWMA formulation used by the estimator
HR_smooth[t] = 0.4 * HR_raw[t] + 0.6 * HR_smooth[t-1]

α = 0.4 — balances responsiveness with noise reduction (~2s time constant)
Seed value: median of all warmup-period estimates (resistant to outliers)
Stabilizes display without hiding real HR changes

Rate-of-Change Clamp

Maximum HR change between consecutive estimates: ±5 BPM/sec. This prevents sudden jumps from motion artifacts while still tracking real trends.

typescript

// Clamp prevents artifactual jumps
if (Math.abs(delta) > 5) {
  HR_output = HR_previous + sign(delta) * 5;
}

Adaptive Time Window

The Panoptic estimator uses an adaptive time window that grows as more data arrives:

Time Elapsed	Window Size	Accuracy	Status
< 5s	—	—	Warmup (no estimates yet)
8 seconds	8s	±5-8 BPM	First estimate
15+ seconds	15s	±2-3 BPM	Optimal ✓

typescript

camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  if (!estimation) {
    showMessage('Analyzing... Please stay still');
    return;
  }
  
  if (getCurrentScanTime() < 15) {
    showHeartRate(estimation.heartRate, 'Refining...');
  } else {
    showHeartRate(estimation.heartRate, 'Stable ✓');
  }
});

Performance Characteristics

Metric	Value
Initialization	Instant (< 10ms)
First Result	~8 seconds
Optimal Accuracy	~15 seconds
Processing Speed	5-10ms per frame
Memory Usage	< 200 KB
Bundle Size	~100 KB (WASM)
Typical Error	±2-3 BPM
Heart Rate Range	50-140 BPM

Quality Metrics

Signal-to-Noise Ratio (SNR)

The Panoptic estimator provides SNR values to indicate signal quality:

SNR	Confidence	Quality	Interpretation
10+	1.0	Excellent	Trust completely ✓
8-10	0.8-1.0	Very Good	Highly reliable ✓
6-8	0.6-0.8	Good	Reliable
4-6	0.4-0.6	Fair	Acceptable
2-4	0.2-0.4	Poor	Consider retrying
< 2	0.0-0.2	Very Poor	Retry required

typescript

camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  if (!estimation) return;
  
  if (estimation.snr && estimation.snr > 6) {
    showQualityIndicator('excellent');
  } else if (estimation.snr && estimation.snr > 4) {
    showQualityIndicator('good');
  } else {
    showQualityIndicator('poor');
    showTip('Improve lighting or reduce movement');
  }
});

Advantages

🏥 Medical-Grade Quality

Panoptic-level accuracy
Matches validated server implementation
Suitable for medical applications
Regulatory compliance friendly

📦 Zero Dependencies

No model files to download
Instant initialization
Works offline immediately
Smaller bundle size

⚡ Fast Initialization

No async loading required
Ready to use immediately
No waiting for models
Simpler error handling

🔒 IP Protection

Core algorithm in WebAssembly
Proprietary implementation protected
Binary format prevents reverse engineering

🎯 Deterministic

Same input → same output
No AI randomness
Reproducible results
Easier to debug

When to Use

✅ Best For

Medical and healthcare applications
Regulatory compliance requirements
Instant initialization needed
Minimal bundle size required
Offline-first applications
Embedded systems and IoT devices

❌ Avoid If

Users will be moving significantly
Lighting conditions are uncontrolled
Maximum motion tolerance needed
Open-source licensing required
AI-powered features preferred

Troubleshooting

Low Confidence Scores

Symptoms: Confidence < 0.6, unstable readings

Solutions:

typescript

// 1. Improve environmental conditions
const instructions = [
  "Use natural daylight",
  "Avoid backlighting",
  "Ensure even illumination",
  "Remove shadows from face"
];

// 2. Reduce motion
const motionTips = [
  "Stay completely still",
  "Don't talk during measurement",
  "Use phone stand if possible",
  "Maintain steady distance"
];

// 3. Check face detection - note: face detection events vary by SDK
// For web SDKs, check scanConditions in VideoFrameProcessedEvent
camera.onVideoFrameProcessed = (event) => {
  if (event.scanConditions && !event.scanConditions.centered) {
    showWarning('Keep face centered and stable');
  }
});

Unstable Readings

Symptoms: Heart rate jumps around

Solutions:

typescript

// Guide user
camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  if (!estimation) return;
  
  if (estimation.confidence < 0.6) {
    showTip('Please stay still and improve lighting');
  }
});

Best Practices

1. Progressive Feedback

typescript

camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  
  if (!estimation) {
    showMessage("Analyzing... Please stay still");
    return;
  }
  
  if (getCurrentScanTime() < 15) {
    if (estimation.confidence > 0.4) {
      showHeartRate(estimation.heartRate, "Refining...");
    } else {
      showMessage("Adjusting... Improve lighting if possible");
    }
  } else {
    if (estimation.isStable) {
      showHeartRate(estimation.heartRate, "Stable ✓");
    } else {
      showMessage("Low confidence. Try again with better conditions.");
    }
  }
});

2. Quality Indicators

typescript

function getQualityIndicator(estimation: RealtimeEstimation): string {
  if (estimation.confidence >= 0.8) return "Excellent ⭐⭐⭐⭐⭐";
  if (estimation.confidence >= 0.6) return "Very Good ⭐⭐⭐⭐";
  if (estimation.confidence >= 0.4) return "Good ⭐⭐⭐";
  if (estimation.confidence >= 0.2) return "Fair ⭐⭐";
  return "Poor ⭐";
}

camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  if (!estimation) return;
  
  updateQualityDisplay(getQualityIndicator(estimation));
});

3. Retry Logic

typescript

let retryCount = 0;
const MAX_RETRIES = 3;

camera.onVideoFrameProcessed = (event) => {
  const result = event.healthResult;
  if (!result) return;
  
  if (result.confidence < 0.6 && retryCount < MAX_RETRIES) {
    retryCount++;
    showMessage(`Low confidence. Retrying (${retryCount}/${MAX_RETRIES})...`);
    camera.reset();
    camera.startScan();
  } else {
    retryCount = 0;
    displayFinalResult(result);
  }
});

4. User Guidance

typescript

const guidanceMessages = {
  lowConfidence: "Improve lighting or stay still",
  noFace: "Position your face in the center",
  unstable: "Keep your face steady",
  success: "Measurement complete ✓"
};

camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  if (!estimation) return;
  
  if (estimation.confidence < 0.4) {
    showGuidance(guidanceMessages.lowConfidence);
  } else if (estimation.isStable) {
    showGuidance(guidanceMessages.success);
  }
});

Comparison with ME-rPPG Estimator

Feature	Panoptic	ME-rPPG
Accuracy	⭐⭐⭐⭐⭐ Medical-grade	⭐⭐⭐⭐⭐ State-of-the-art
Motion Tolerance	⭐⭐⭐⭐ Good	⭐⭐⭐⭐⭐ Excellent
Lighting Tolerance	⭐⭐⭐⭐ Good	⭐⭐⭐⭐⭐ Excellent
Initialization	⭐⭐⭐⭐⭐ Instant	⭐⭐⭐ (1-2s)
Bundle Size	⭐⭐⭐⭐⭐ ~100 KB	⭐⭐⭐ ~10 MB
Memory Usage	⭐⭐⭐⭐⭐ < 200 KB	⭐⭐⭐ ~4 MB
Licensing	Proprietary	Open-source
First Result	~8 seconds	~3 seconds
Best For	Medical apps, embedded	Consumer apps, challenging conditions

Example: Complete Implementation

typescript

import { createVitalSignCamera, RealtimeEstimatorType } from 'ts-vital-sign-camera';

// Create camera with Panoptic estimator
const camera = createVitalSignCamera({
  realtimeEstimationConfig: {
    estimatorType: RealtimeEstimatorType.Panoptic,
    debug: false
  }
});

// Panoptic is ready immediately (no async loading)
document.getElementById('start-btn').disabled = false;

// Handle realtime updates
camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  if (!estimation) return;
  
  updateHeartRateDisplay(estimation.heartRate);
  updateConfidenceBar(estimation.confidence);
  
  // Show SNR if available
  if (estimation.snr) {
    updateSNRDisplay(estimation.snr);
  }
  
  // Quality indicator
  if (estimation.isStable && estimation.confidence > 0.7) {
    showQualityIndicator('excellent');
  } else if (estimation.confidence > 0.5) {
    showQualityIndicator('good');
  } else {
    showQualityIndicator('poor');
    showTip('Improve lighting or reduce movement');
  }
});

// Handle scan completion
// Check for final results
let scanCompleted = false;
camera.onVideoFrameProcessed = (event) => {
  const results = event.healthResult;
  if (results && !scanCompleted) {
    scanCompleted = true;
    if (results.confidence > 0.7) {
      displayFinalResults(results);
    } else {
      showRetryOption();
    }
  }
});

// Start scan
document.getElementById('start-btn').onclick = () => {
  camera.startScan();
};

Example: Medical Application

typescript

// Medical-grade configuration
const camera = createVitalSignCamera({
  realtimeEstimationConfig: {
    estimatorType: RealtimeEstimatorType.Panoptic,
    signalQualityDelay: 10,  // Longer quality delay for medical use
    debug: false
  }
});

// Only record high-confidence measurements
camera.onVideoFrameProcessed = (event) => {
  const estimation = event.realtimeEstimation;
  if (!estimation) return;
  
  if (estimation.isStable && estimation.confidence > 0.7) {
    recordMeasurement({
      heartRate: estimation.heartRate,
      confidence: estimation.confidence,
      snr: estimation.snr,
      timestamp: new Date()
    });
  }
});

// Validate before saving
// Validate before saving
let validationCompleted = false;
camera.onVideoFrameProcessed = (event) => {
  const results = event.healthResult;
  if (results && !validationCompleted) {
    validationCompleted = true;
    if (results.confidence > 0.7 && results.snr > 5) {
      saveMedicalRecord(results);
    } else {
      requestRetry('Measurement quality below medical threshold');
    }
  }
});

Next Steps

ME-rPPG Estimator - Compare with AI approach
Selection Guide - Choose the right estimator
Signal Visualizer - Visualize PPG signal
API Reference - Complete API docs

Panoptic Estimator ​

Overview ​

Key Features ​

Basic Usage ​

Configuration ​

For Consumer Applications ​

For Medical Applications ​

SDK Behavior ​

Initialization ​

Processing Timeline ​

Warmup & Smoothing ​

Warmup Gate ​

1Hz Throttle ​

EWMA Smoothing ​

Rate-of-Change Clamp ​

Adaptive Time Window ​

Performance Characteristics ​

Quality Metrics ​

Signal-to-Noise Ratio (SNR) ​

Advantages ​

🏥 Medical-Grade Quality ​

📦 Zero Dependencies ​

⚡ Fast Initialization ​

🔒 IP Protection ​

🎯 Deterministic ​

When to Use ​

✅ Best For ​

❌ Avoid If ​

Troubleshooting ​

Low Confidence Scores ​

Unstable Readings ​

Best Practices ​

1. Progressive Feedback ​

2. Quality Indicators ​

3. Retry Logic ​

4. User Guidance ​

Comparison with ME-rPPG Estimator ​

Example: Complete Implementation ​

Example: Medical Application ​

Next Steps ​

Panoptic Estimator

Overview

Key Features

Basic Usage

Configuration

For Consumer Applications

For Medical Applications

SDK Behavior

Initialization

Processing Timeline

Warmup & Smoothing

Warmup Gate

1Hz Throttle

EWMA Smoothing

Rate-of-Change Clamp

Adaptive Time Window

Performance Characteristics

Quality Metrics

Signal-to-Noise Ratio (SNR)

Advantages

🏥 Medical-Grade Quality

📦 Zero Dependencies

⚡ Fast Initialization

🔒 IP Protection

🎯 Deterministic

When to Use

✅ Best For

❌ Avoid If

Troubleshooting

Low Confidence Scores

Unstable Readings

Best Practices

1. Progressive Feedback

2. Quality Indicators

3. Retry Logic

4. User Guidance

Comparison with ME-rPPG Estimator

Example: Complete Implementation

Example: Medical Application

Next Steps