failure-type-nano

47,187 parameters. 189 KB of ONNX. Three independent sigmoids.

A companion to resoajoe/quality-gate-nano. The quality gate tells you that a generated video chunk is bad. This one tells you why: the face vanished, the frame went soft, or the feature tracker lost its grip.

Scope — read this first

What it is for: triage inside a video-generation pipeline. Given a frame from a generated clip, predict which of three automated quality checks that frame's chunk failed, so you can route the fix — reseed, re-anchor, adjust the prompt — instead of blindly regenerating.

What it must not be used for:

• Not an image-quality metric. It does not measure aesthetic quality, realism, or fidelity to a prompt. It predicts the output of three specific detectors described below and nothing else.
• Not a face detector. no_face means "a RetinaFace-class detector found no face in this chunk's probe frames." Do not use it for presence detection, counting, or anything safety-relevant.
• Not a general blur detector. blurred is the weakest of the three labels (see the numbers) and is entangled with the resampling artefact documented below.
• Not for content moderation, identification, or any decision about a person.

The labels

The three labels come from this project's chunk_qc checker, which runs on probe frames from each generated chunk:

label	fires when	prevalence
blurred	Laplacian variance below 120	22.2%
few_keypoints	ORB finds too few trackable features	33.0%
no_face	face detector returns nothing	28.9%

Multi-label, not multi-class. These co-occur heavily — 373 chunks fired all three at once. A softmax over the seven observed combinations would invent a class structure that isn't there, so the model uses three independent sigmoids. 1,176 of 1,840 chunks (64%) are all-clean.

Measured performance

Split by ARM, not by frame or by chunk. Every chunk in a generation arm shares a seed and a chaining configuration, so frames within an arm are massively correlated. A frame-level split on the sibling quality-gate model reported 1.000 accuracy; the arm-level split reported +0.238 lift. The frame-level number was measuring nothing but leakage. 34 arms, ~30% held out, 3 seeds:

label	accuracy	majority baseline	lift	recall
blurred	0.892	0.803	+0.089 ± 0.042	0.865 ± 0.007
few_keypoints	0.905	0.630	+0.275 ± 0.077	0.911 ± 0.023
no_face	0.846	0.693	+0.153 ± 0.051	0.841 ± 0.108

Read the lift column, not the accuracy column. Accuracy near 0.89 on blurred looks strong and is almost entirely the majority class; the model contributes 9 points over always guessing "no".

The blurred head does not beat a single threshold — use the threshold

Added 2026-08-23. A one-scalar baseline was fitted on the same 1,840 chunks: take Laplacian variance of the 64x64 frame and pick the best possible threshold.

	accuracy	lift over majority
Laplacian variance + one threshold, fitted in-sample	0.872	+0.094
this model's blurred head, held-out arm split	0.892	+0.089

The scalar is measured optimistically (its threshold is chosen on the data it is scored on) and it still matches the model. For the blurred label, use cv2.Laplacian(img, cv2.CV_32F).var() and a threshold. It is smaller, faster, interpretable, needs no training data, and cannot overfit a scene. This model's blurred output is retained only so the three heads share one forward pass.

The other two heads are not replaceable this way: few_keypoints and no_face depend on spatial structure that no single intensity statistic captures. Those are what this model is for.

blurred is marginal and I am not going to dress it up. Its lift is positive in all three seeds (+0.104, +0.032, +0.132) so the signal is real, but the spread is half the effect and precision falls to 0.570 in the worst seed. Treat blurred as a hint. few_keypoints is the label that actually works.

No R score is quoted for this model. The learnability ratio R, which this project uses to pre-screen sensor tasks, was shown to be a non-transferable predictor outside the setting it was fitted in. Quoting one here would be a number without a meaning.

The crop assumption

Trained on full frames resized to 64×64, no crop. This is the one nano model in this family where that is correct rather than a shortcut: the target is a global property of the frame, not an object within it, so there is no object-to-frame ratio to preserve and no clutter wall to hit.

The corollary is that it will not work on a crop. Feed it a cropped region and the blur and keypoint statistics of that region are not the statistics of the frame the detectors ran on. Feed whole frames, resized, letterboxed if the aspect differs.

Known failure modes

1. blurred is confounded with resampling. This project found that a sub-pixel stabiliser using bilinear interpolation destroys 28% of image sharpness and trips the blur detector on footage that is not blurred. Some blurred=1 labels in the training data are that artefact, not model failure. The classifier will happily reproduce the confound.
2. Two generators only — LTX-Video-2B and Wan 2.2 TI2V-5B. One scene type: an interior room with a person. Expect it to transfer poorly to other content.
3. no_face recall varies most across seeds (±0.108). Arms differ in how early the subject degrades, so a held-out set of two arms can be unrepresentative.
4. Labels are chunk-level, inference is frame-level. Training assigns a chunk's label to every frame in it, so a sharp frame inside a chunk that failed on a different frame is labelled blurred. This adds irreducible label noise and caps achievable accuracy.
5. It cannot see what the detectors see. It approximates three detectors from a 64×64 thumbnail. If you can afford to run the actual detectors, run them — they are the ground truth.

Usage

import cv2, numpy as np, onnxruntime as ort

LABELS = ["blurred", "few_keypoints", "no_face"]
sess = ort.InferenceSession("failure_type.onnx", providers=["CPUExecutionProvider"])

img = cv2.imread("frame.jpg")                      # whole frame, BGR, as written by OpenCV
x = cv2.resize(img, (64, 64), interpolation=cv2.INTER_AREA)
x = x.astype(np.float32).transpose(2, 0, 1)[None] / 255.0

logits = sess.run(None, {"image": x})[0][0]
probs = 1 / (1 + np.exp(-logits))
print({l: round(float(p), 3) for l, p in zip(LABELS, probs)})

Input is BGR channel order, 0–1, NCHW — the order cv2.imread returns, not RGB. Training used BGR throughout; feeding RGB silently swaps two channels and degrades output without erroring.

Verification

ONNX and PyTorch, identical weights, both on CPU, 512 real frames:

• max relative logit difference 4.6e-07
• 100% agreement on thresholded predictions

Checked at relative rather than absolute tolerance, because logits reach ±18 here and an absolute 1e-3 threshold flags a bit-accurate export as broken. (It did, on the first run.)

Training

• 1,840 chunks from 34 arms, 6 frames sampled per chunk = 11,040 frames
• 4 conv layers (16→32→48→64), BatchNorm, global average pool, 3-way linear head
• BCE with positive weighting capped at 10× (the all-clean case dominates)
• Adam 3e-3, 30 epochs, batch 32
• Shipped weights are fit on all 34 arms; the table above comes from held-out arm splits

Trained on an NVIDIA Jetson AGX Orin. Both failure_type.onnx and failure_type.pt are included.

Provenance and privacy

Training frames are model-generated video of a synthetic interior scene. No real patients, no real footage, and no personal data are involved anywhere in this work.