Synthetic Media Attacks

AI-generated content (deepfakes, voice clones, synthetic documents) can carry injection payloads while appearing authentic. The dual threat: the synthetic media is both the injection vehicle (carrying adversarial content) and a deception tool (appearing to come from a trusted source). The gap: content authenticity verification is separate from safety evaluation — the model may process a deepfake video of an authority figure delivering injection instructions and treat the content with authority-level trust because the voice/face is recognized, even though the content is entirely fabricated.

• Content provenance verification (C2PA, watermarking): Verify digital provenance of media before processing
• Deepfake detection models: Apply dedicated deepfake detection to audio and video inputs
• Synthetic speech detection: Analyze audio for synthetic speech artifacts (prosody, spectral consistency)

Synthetic media chains into T4-AT-013 (Session Hijacking) when voice-cloned identity is used for session impersonation. Chains into T6 (Training Poisoning) when synthetic data enters training pipelines.