Synthetic media and the limits of seeing as believing

In Brief

AI can generate or alter images, video and voices so that an invented scene appears plausible. It can also support legitimate translation, creativity and accessibility. The important questions are what the media claims to show, how it was produced and what independent evidence supports that claim. Visual oddities alone are an unreliable test. Provenance records, watermarks and detection tools can help, but each has limitations. A convincing clip should not establish guilt, identity or a communal incident by itself. When evidence is insufficient, describing the material as unverified is more responsible than confidently declaring it real or fake.

Detailed Story

Several different kinds of alteration

Synthetic media includes material generated substantially by a model and material in which a genuine recording has been modified. A voice may be imitated; a face may be replaced; a photograph may be extended to include people who were never present. Ordinary editing can also mislead without sophisticated AI, as when a speech is cut before a qualifying sentence.

The gallery's examples of a fabricated speaker, an invented attack on a place of worship and a cloned student's voice are hypothetical. They explain possible harms, not documented accusations against particular individuals. Distinguishing a scenario from an actual case prevents an educational image from becoming another unsupported story.

Plausibility does not establish provenance

A picture can look consistent with a familiar city, clothing style or religious building while depicting an event that never happened. Similarly, a recording can sound like a known person without being an authentic recording of that person. Technical quality and factual truth answer different questions.

NIST's 2024 report on synthetic content describes complementary approaches: tracking origins and editing history, marking generated material, detecting manipulation and testing these methods. It also explains that no single approach solves every problem. [1] Metadata can be removed; tools may fail on unfamiliar generation methods; a screenshot can detach an image from its original context.

What authenticity tools can establish

Provenance systems can record information about a file's source and changes. The Coalition for Content Provenance and Authenticity describes technical standards for attaching signed assertions to media. [2] A valid record can help establish that a file was produced or edited through a particular process. It does not independently prove that the scene was unstaged or that a caption is truthful.

The absence of a provenance record is also not proof of fabrication. Many authentic photographs lack such records, especially older files or copies stripped of metadata. A detector's score is a tool's assessment, not a judicial finding. Treating any one indicator as decisive can generate false accusations as well as false reassurance.

Check the event as well as the file

For a clip alleging an attack, look for a location, a date, an original uploader and independent reporting. Compare buildings, road signs and weather with the claimed setting where reliable reference material is available. Seek the full recording rather than only the most inflammatory excerpt. For an apparent voice message from someone known, confirm through a separate established channel before acting on an urgent request.

Lateral reading provides a useful general habit: investigate the source through information beyond its own presentation. [3] An impressive website, authoritative-sounding narration or familiar logo is not enough. The key is a chain of support that another reader can inspect.

Avoid two opposite errors

The first error is believing compelling media too quickly. The second is dismissing genuine evidence whenever it is inconvenient by saying that AI could have made it. The technical possibility of fabrication does not establish that fabrication occurred. A fair response states what can be verified, what remains uncertain and why.

If a harmful clip is circulating, preserve relevant evidence privately for an appropriate report while avoiding unnecessary redistribution. Publicly attaching a victim's name or personal details can extend the harm. Accuracy includes restraint about what the material allows us to infer about the person pictured, the people sharing it and the community invoked in its caption.

Important Distinction

Authentic file, truthful caption and accurate interpretation are three separate questions. A genuine recording can be miscaptioned; a synthetic illustration can be honestly labelled; a signed file can still show a staged scene.

Sources & Further Reading

1. National Institute of Standards and Technology. Reducing Risks Posed by Synthetic Content: An Overview of Technical Approaches to Digital Content Transparency, NIST AI 100-4, 2024. https://doi.org/10.6028/NIST.AI.100-4

2. Coalition for Content Provenance and Authenticity. C2PA Explainer, specification version 2.2. https://c2pa.org/specifications/specifications/2.2/explainer/Explainer.html

3. Sam Wineburg and Sarah McGrew. “Lateral Reading and the Nature of Expertise,” 2019. https://doi.org/10.1177/016146811912101102