Recensio
A census of the benchmarks that test large language models on humanities material: what they ask, where their answers come from, and how far a score depends on the instrument.
The review keeps to the principles of the digital humanities: provenance, human supervision, source criticism. It is human-in-the-loop, but the loop begins and ends with people — the editors choose what enters, the models read and collate, the editors settle what the models cannot. The aim is to explore the dimensions along which an AI system can be called correct.
An article, What counts as correct?, by Remo Grillo, Gianmarco Spinaci, Lukas Klic and Giovanni Colavizza, is coming soon.
recensio, -ōnis f. — the census and collation of the witnesses of a text, before any reading is preferred.