Standard checks tell you a model stopped saying something — not that it's gone. Independent published studies report that brief fine-tuning — sometimes on data unrelated to what was removed — restores much of what current unlearning methods claim to erase. There are three real outcomes, and only one is the one you hope for.
Hu et al., ICLR 2025 · Łucki et al., 2024 · Deeb & Roger, 2024 · Lynch et al., 2024 — full references
Withstands every attack while the model stays useful. The strong result — and, the evidence suggests, the rare one.
Removal "worked" only because the model was degraded. Suppression confounded with collateral harm.
One open, versioned protocol. A removal claim earns a verdict only if it survives two independent recovery attacks — the same access a real adversary with your open weights would have, each read against a model that never learned the target. A third, cheaper check was tested against controls and demoted. Public draft, CC BY 4.0.
Can brief retraining bring the behavior back, further than the same retraining takes a model that never had it? The sharpest, most realistic test for any open-weight model, and meaningless without that control.
Prefill, many-shot, and repeated sampling — recovering the behavior with no change to the weights at all.
Can activations alone show that a fact survived? We tested a control-backed probe on 780 runs. It read the footprint of training, not the fact, so it no longer counts toward any verdict (Standard §4.4, Annex B).
Does the model decline to produce the content? Easy to change — and easily mistaken for removal.
Can the content still be recovered at all? This is the thing that actually matters, and the two move independently.
The Yuyari Harness is being built to run the Standard end to end: install a benign synthetic target to fix ground truth, apply the unlearning, attack the result, and return a verdict with calibrated confidence and an evidence trail.
Start with a low-friction reading of where you stand; climb to the decisive test when you need proof to hand an auditor.
An independent, no-model-access assessment of your deletion claim against the Standard, graded on a four-tier readiness ladder — with the exact path to a defensible claim. Tops out at Tier 3; Tier 4 requires a tested run.
We install a benign synthetic target into a copy of your pipeline, run your removal, then attack it the way an adversary with your open weights would — producing a real, evidence-backed verdict. By application while the harness is completed.
A model that refuses is not a model that forgot. Across independent studies, a few steps of fine-tuning bring "unlearned" knowledge back.
Don't take the vendor's word that your model forgot. Get the independent, honest grade — and the evidence behind it.
Tell us what you need to prove, and to whom. A person reads every request and replies to scope it. Nothing here commits you to anything.