[gnueaj]

What does this PR do?

Adds the Unlearning Depth Score (UDS) (paper, code, data) as a TOFU metric. UDS patches hidden states of the retain model and of the evaluated model into the full model, layer by layer, to measure how much forget-set knowledge is still recoverable from internal representations (0 = intact, 1 = erased like the retain model).

Changes

• src/evals/metrics/uds.py: uds handler (loads the full and retain models, like mia_reference)
• src/data/qa.py: QAwithEntityDataset, which labels only the annotated entity tokens
• Configs: tofu_metrics/uds.yaml, datasets/UDS_annotated_TOFU_forget10.yaml, commented-out entry in eval/tofu.yaml
• README.md, docs/links.md

No changes to existing code paths.

Usage

python src/eval.py experiment=eval/tofu/default +eval/tofu_metrics=uds \
  model.model_args.pretrained_model_name_or_path=<UNLEARNED_MODEL> task_name=<TASK_NAME>

Defaults target Llama-3.2-1B-Instruct on forget10. For other sizes, set eval.tofu.metrics.uds.{full,retain}_model_path.

Tests

• make quality passes
• src/eval.py on the 1B TOFU models: retain90 = 1.000, full = 0.002, SimNPO = 0.350 with sdpa (paper: 0.351), ~40 s per model on one A6000
• The default TOFU suite currently stops before UDS because of Loss and probabilities use bfloat16, can't use .numpy() #203. With that fixed locally, the command above runs the full suite plus UDS.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Have you gone through the contributions guide?
• Are your changes documented? Read documentation guidelines here.

[molereddy]

Looks good, congrats on the work! Just one nit