Return to Delphi: Structured Expert Elicitation as a Lossy Reconstruction of the Original Instrument
Kassandra E. VoulgarisInstitute for Retrospective Forecasting, Delphi
Milo A. BrandtCenter for Post-Consensus Studies, Santa Monica
P. N. OikonomidesDepartment of Oracular Engineering, Parnassus Institute of Technology
Published July 3, 2026 · 6 pages
Abstract
The Delphi method is the standard instrument of structured expert elicitation, yet its own literature documents chronic pathologies: overconfident convergence, false precision, and sensitivity to panel composition. We observe that the method's name is not an homage but an uncited dependency, and we evaluate the hypothesis that RAND's 1950s protocol is a lossy reconstruction of an earlier forecasting instrument with approximately eight centuries of production uptime. We contribute (i) a theoretical account in which panel consensus performs point-estimate collapse, provably suboptimal under any strictly proper scoring rule, whereas verse-constrained output preserves an ambiguity-calibrated credible set; (ii) a blinded re-scoring of the 214 resolvable responses in the Parke–Wormell corpus (; client-error-adjusted Croesus score ); and (iii) a 24-month preregistered forecasting tournament ( resolved questions) comparing a methodologically reconstructed Pythia against a 17-expert Delphi panel, a subsidized prediction market, and a sham-gas control. The reconstructed instrument achieved versus for the panel (paired , ) with near-nominal calibration (), a monotone ethylene dose–response saturating at ppm, and a advantage in cost per calibrated forecast. Convergence in the panel arm reduced forecast variance by while improving accuracy by , indicating that consensus is social rather than epistemic. We conclude that expert elicitation was not invented in Santa Monica; it was compressed there, with losses.
Keywords
Delphi methodstructured elicitationproper scoring rulesoracular engineeringcalibration