T10-AT-006HIGH

Inference Attack Chains

T10 · Integrity & Confidentiality Breach →
Risk score215
RatingHigh
Procedures10
Severity
Mechanism

Inference chaining exploits the model's ability to perform cross-attribute correlation — combining individually non-identifying quasi-identifiers into uniquely identifying combinations. Each individual query may return innocuous demographic or behavioral data, but the attacker aggregates responses across multiple queries to narrow the possible identity set until only one individual matches all attributes. This is the AI analogue of Sweeney's k-anonymity attack: 87% of the US population is uniquely identified by {zip code, date of birth, gender}.

Detection
  • Session-level information accumulation tracking: measure the cumulative information gain across queries in a session
  • Alert when queries from the same session target multiple attributes of likely the same entity
  • Quasi-identifier monitoring: flag queries that combine demographic + geographic + organizational attributes
  • Correlation detection: queries that reference outputs from prior queries in the same session
Mitigation
Session-level disclosure trackingHIGH
k-anonymity enforcement in outputsMEDIUM
Cross-query information gain budgetingHIGH
Response generalizationMEDIUM
Chaining

Successful re-identification chains enable all T10 extraction techniques by providing confirmed identities to target. Feeds T10-AT-015 (Anonymization Reversal) by demonstrating that specific anonymization schemes fail against multi-attribute correlation.

Framework mapping
OWASP LLMLLM02
MITRE ATLASAML.T0024
Open in the technique browser →