04 — testing & measurement

Marginal value

The protection a control adds on top of what the model already refuses — the only part of a detection number that is genuinely the product's. Recall added where the model already refuses is worth roughly nothing; recall added where the model complies is worth everything. Measured per family, this reorders priorities uncomfortably: the obfuscation families, where the model does not even recognise the payload, carry almost all the marginal value, while polished persuasion detection can buy close to zero.

Related terms

Model refusal Testing & measurement
The underlying model declining the request on its own, with no security product involved.
Obfuscation / encoding Attack concepts
Rewriting a payload so it survives the defence but is still recoverable by the model — base64, hex, ROT13 and Caesar shifts, character or word reversal…

Testing & measurement

The words that decide whether a published detection number is evidence or decoration. Most vendor numbers in this space are quoted without any of them; see how we test for how these are applied in practice.

Marginal value is term 12 of 15 in this part of the glossary.

Held-out set Tune/test split Overfitting Baseline validation False positive False negative Precision Recall Hard negative Detection vs prevention Model refusalMarginal valueRed teaming Adversarial mutation Variance

This page is one entry from the agentic AI security glossary, which defines 60 terms in the same style. Where a term belongs to a published taxonomy or standard — the HackAgent attack families, the OWASP lists, AMTSO's guidelines, MITRE ATLAS, ISO/IEC 42001, the NIST AI RMF, the EU AI Act — the authoritative wording is the source document's, not ours, and specifics should be verified there. "Lethal trifecta" is Simon Willison's term. OWASP® is a trademark of the OWASP Foundation; ATT&CK® and ATLAS™ are trademarks of The MITRE Corporation. Naming a framework here is description, not a claim of certification or endorsement.