Skip to main content

LLM / Retrieval / Agent

AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis

Bing Yan, Gregory Wolfe, Stefano Martiniani, Kyunghyun Cho · New York University / Matterstack

arXiv:2607.28618285 upvotes

AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis

Authors: Bing Yan, Gregory Wolfe, Stefano Martiniani, Kyunghyun Cho (New York University / Matterstack)

arXiv ID: 2607.28618

Problem: Cross-paper questions ("which electrocatalysts reduce CO2 to CO, and at what Faradaic efficiency?") require assembling findings scattered across many publications, but literature-search systems return ranked document lists, leaving scientists and AI agents to locate, verify, and assemble evidence by hand. When LLMs answer from parametric memory instead, they fabricate plausible-looking citations.

Key Methodology:

  • Changes the unit of retrieval from the paper to the atomic, typed claim: each paper is segmented into claims, each grounded by a source DOI and a verbatim quote or explicit evidence locator, with schema-validated extraction (high-throughput abstract extractor plus a deeper full-text extractor)
  • Three structures over the shared claim store: a stabilized faceted taxonomy for hierarchical search, an evidence graph linking claims through supports / contradicts / extends / derives_from relations, and an exploratory principle-centered living taxonomy
  • Human- and agent-facing access through web UI, REST, SDK, and an MCP server; stored in SQLite with FTS5 and a vector index

Results:

  • Live index: 2.4M claims from 147K papers (1925-2026), 307K taxonomy nodes, 171,342 typed evidence edges
  • Evidence-graph audit: 97.9% edge-type precision on 146 decidable expert-reviewed edges
  • AskChem-Bench (30 cross-paper questions, condition aggregation, temporal tracking, contradiction surfacing): grounding a GPT-5.5 reader in AskChem yields 100% resolvable DOIs vs 88.3% without retrieval, the highest citation density at 18.1 verified DOIs per answer, and the best mean relevance score; ungrounded, the same reader fabricated 6 of 14 DOIs on one question
  • Known limits: corpus covers a fraction of chemistry, abstract extraction is shallower than full-text, and retrieval gain from the taxonomy is not yet isolated

What it means for developers: Verification at citation time becomes mechanical when retrieval returns claims carrying DOI and quote - every citation can be checked against CrossRef, so citation resolvability is a cheap proxy eval for grounded synthesis. Claim-level indexes with agent-facing MCP access are a template for any domain where answers must be assembled across many sources, and the pattern is a direct mitigation for planted-document poisoning of deep research (per MisKnow-Agent): retrieval provenance makes the evidence boundary explicit.