Run report — vision_smoke / 30a843b2500b

model dummy_color_v1 · created 2026-08-14T04:30:44+00:00 · status completed · duration 0.09s

Metrics

namevalue
accuracy0.966667
mean_latency_ms1.39441
n_errors0
n_samples60
p95_latency_ms0.819868

Gate PASSED

checkverdictvaluethresholddetail
min_accuracy PASS 0.9667 0.9000 candidate accuracy 0.9667 vs floor 0.9000
accuracy_regression PASS 1.0000 0.0500 mcnemar_exact(b=0, c=0); baseline acc 0.9667, drop +0.0000, p=1.0000
max_accuracy_drop PASS 0.0000 0.0200 practical drop +0.0000
score_psi PASS 0.0000 0.2500 PSI(baseline scores -> candidate scores) = 0.0000
latency_regression INFO — — skipped: both p95s under 5 ms (0.83 -> 0.82); relative change is not meaningful at this scale
bootstrap_ci INFO — — accuracy delta 95% CI [-0.0667, +0.0667]

Latency

Latency distribution (ms)59000000000011–44–77–1010–1313–1717–2020–2323–2626–2929–3333–3636–39

Worst samples

sampleexpectedpredictedcorrectscore
s0036greenred 0 0.5464
s0011redblue 0 0.5690