Published
When FEVER Meets Finance: A Controlled Benchmark Reveals Overlap-Driven Financial Claim Verification
FinNLP Workshop at EMNLP 2026. Presented online, 28 October 2026.
Research
Research
Every study here tests one part of the same path: where a claim came from, whether its evidence supports it, whether the reasoning can be followed, and which version of the record it was checked against. The tools are measured too: each pipeline reports its own errors, and its verdicts can be contested.
Active now: 4 projects, and Eekvul.
Published
Accepted
AcceptedWorkshop: 1 December 2026, Melbourne
Today's Number, Yesterday's Claim: How Tool Design Shapes LLM Checks of Revised Statistics
OE-Agent workshop at ACML 2026 (non-archival). Workshop: 1 December 2026, Melbourne.
Camera-ready submitted. An 8B open model given the figures as published beat a 1.6T open model without them (preregistered): 64.5% against 51.4%. A single-host rerun after review confirmed it (67.4% against 51.6%).
64.5%8B open model, with the right version of the record
51.4%1.6T open model, without it
Share correct on the 322 claims that were right when published and contradicted by the revised record. Preregistered comparison on 321 paired claims, Holm-corrected p = 0.0009. A single-host rerun after review confirmed it (67.4% against 51.6%). With only today's figures, models rejected accurate reporting more often, though for the smallest model this depended on where it was served. In an exploratory comparison, a closed frontier model without tools also scored well (77.6%).
Under review
Under review
Misinformation About Economic Numbers Is Rare; Mis-Verification Is Common
Research note, Harvard Kennedy School Misinformation Review.
Under review
Checked Against Which Vintage? Data Revisions and the Automated Verification of Claims about Official Statistics
Extended abstract, NTTS 2027 (Eurostat). Decision 30 November 2026.
Under review
A trace study of delegated AI agents
Under review.
In preparation
In preparationActive now
Checked Against Which Record? Time-Indexed Verification of Restated Financial Figures (working title)
Design and analysis plan complete; model runs pending.
In preparationActive now
Decision models as write-time gates for AI knowledge stores
With input from Baalateja Kataru. In progress; benchmark build from November 2026.
In preparationActive now
Bilingual hate speech detection: consistency across English and Telugu
With Harshith Patnala and Prof. Atul Negi (University of Hyderabad). In progress; paper in preparation.
In preparation
Auditing claim groundedness against primary records
Being reworked after its first submission.
Audit
The audit, on one page
I collected 4,669 claims from 334 news articles about 19 US statistical releases, typed each one for whether it could be checked, and linked it to the official series it cites. Every checkable claim was checked twice: against the record as it stood on the day it was published, and against the record today.
Outlets are reported by tier, never by name, so the audit doesn’t become a label-maker. Rates are computed only over claims that could be checked, so outlets that say less aren’t scored as more accurate.
Outright contradictions at publication were rare. But 17.8% of the claims the record supported when they were published are contradicted by it today, rising to 43.3% for monthly payroll. Outlet tier did not predict groundedness.
Next
In progress and planned
Active nowIndia. A dated archive of India’s headline statistics, and the public claims that quoted each version of the record. See the demo · the public archive.
Active nowEekvul. An analysis engine that computes every finding from stored public data and attaches what the finding rests on: the evidence, the method and its version, the data snapshot, a confidence label with written reasons, and the known gaps. There are four labels, CONFIRMED, INFERRED, UNCERTAIN and UNKNOWN, described at the level of the public standards they draw on: ICD 203, the Admiralty code and GRADE. The public data are snapshotted and sealed every night, and a recompute check holds publication if any number fails to reproduce, so anyone holding the archive can recompute every published figure. The first brief asks which of 17 open-source research tools for checking claims are actively maintained and adopted, and how far each of those findings can be trusted.
People. Does a true but selectively chosen number, labelled ‘accurate’, mislead more than the same number shown in full context? A preregistered online experiment that measures behaviour as well as belief. Planned.
Records. How often is the indicator a government cites the most favourable of those available at the time? Starting with India, where MoSPI’s 2026 base-year revisions are changing the record under claims about GDP and industrial production. Planned.
Models. When a model’s answer about an official statistic changes, did the model change or did the record? A monthly archive of what several models, including pinned and dated versions, say about the same fixed questions, each answer checked against the record as it stood that day. It can only be built going forward, because a retired model can’t be asked again. Planned.
Actions. Every action an AI agent takes rests on claims. When a premise is revised after the action was approved, was the action justified on what was known at the time? A formal model, an implementation with ARES acting and Eruka holding the premises, and an audit that can answer that from the log. Planned.
Sources. When a model states a fact, did it come from checked evidence it was given or from its training? The two should carry different confidence and different authority to act. Revised statistics make the test, because training holds the old value and the record holds the new one. Planned.
Boundaries. A position paper: you can’t govern what you can’t attribute, so AI governance has to live where provenance can be recorded: what enters a store, what it returns, and what an action is allowed to rest on. Held as a bet, with what would weaken it stated. Planned.
Framework. One framework for verification that carries the version of the record, covering LLM checkers, decision models and ordinary claims whose confidence should change when the record does. Planned.
Accounts. Can language models trace the figures in a listed company’s statements back to the activity they report, and show where the two diverge? Early question.
Shared knowledge. Eruka as a shared knowledge framework: a tree of claims in which each one would carry its state, its source and its time, with Eekvul’s analysis behind it, so anyone can see what a claim rests on and how sure the system is. A first test is how far public political speeches in the US and India hold up against the record. Early question.