Claim checks

Checking each sentence that cites a source against that source, and recording whether you agree.

Writing paperscitationsclaimsverification2 min read

A claim check reads each sentence of a sealed version that cites a source, and asks a model whether the source supports it.

Running one

Open a version and, under Claims against their sources, press Check claims. Before anything is sent, the workbench shows the plan, for example: 24 claims to check, each sentence that cites a source, once for each source it cites, read by llama3.1 on this computer. Press Check them.

While it works, it shows Checking 5 of 24…, and Stop ends it after the current claim. With a hosted model, sources labelled personal data or higher are not sent, and their claims say so.

Reading the findings

The summary counts each verdict, for example 2 not supported, 3 partly supported, 1 can't tell, 18 supported. Each finding shows:

The sentence, and the source and page it cites.
The verdict: supported, partly supported, not supported, can't tell, no such reference, source not in the vault, not sent or no answer.
The quotation the model gave from the source, with its page. If the quotation is not found in the source's text, treat the verdict with suspicion.
The model's reason.

Each verdict is the verdict of the model, not the author's, with a link to exactly what it was asked. Press I agree or I disagree (and optionally say why) to record your own judgement beside it. Earlier checks are kept.

Citations against the references

Without any model, each version also lists its citations against the references: citations that name no reference, and references never cited. For Word papers, the citations are matched to the paper's own reference list by first author and year.