Run report — vision_smoke / a395ae4e7cb2

model dummy_color_v1 · created 2026-08-14T04:30:49+00:00 · status completed · duration 0.09s

Metrics

namevalue
accuracy0.95
mean_latency_ms1.33402
n_errors0
n_samples60
p95_latency_ms0.807647

Gate FAILED

checkverdictvaluethresholddetail
min_accuracy PASS 0.9500 0.9000 candidate accuracy 0.9500 vs floor 0.9000
accuracy_regression PASS 1.0000 0.0500 mcnemar_exact(b=1, c=0); baseline acc 0.9667, drop +0.0167, p=1.0000
max_accuracy_drop PASS 0.0167 0.0200 practical drop +0.0167
score_psi FAIL 3.3078 0.2500 PSI(baseline scores -> candidate scores) = 3.3078
latency_regression INFO — — skipped: both p95s under 5 ms (0.83 -> 0.81); relative change is not meaningful at this scale
bootstrap_ci INFO — — accuracy delta 95% CI [-0.0833, +0.0500]

Latency

Latency distribution (ms)59000000000011–44–66–99–1212–1515–1818–2121–2424–2626–2929–3232–35

Worst samples

sampleexpectedpredictedcorrectscore
s0001greenred 0 0.3432
s0036greenred 0 0.3881
s0011redblue 0 0.4147