Upgrade Storagecitefact – audit a manuscript against the full text of its Zotero sources (citations, quotes, claims
I built a small open-source CLI that might be useful to people here. Existing tools check that citations exist; citefact reads the full text of every cited source and checks that it supports what you wrote.
It runs three checks, cheapest first:
1. Citations – does every in-text citation exist in the bibliography, with a source PDF? (deterministic, free)
2. Quotes – is every direct quote really verbatim in its cited source? Shows word-level diffs for near-misses. (deterministic, free)
3. Claims – does the source actually support the claim attached to it? (optional, uses an LLM with your own API key; every verdict comes with a verbatim evidence quote so you judge, not the model)
The Zotero part: point it at a collection and it reads metadata and attached PDFs through Zotero's local API – no exports, no Better BibTeX needed:
The first two checks make zero network requests and need no key (
Example report (claims check with evidence):
https://s3.amazonaws.com/zotero.org/images/forums/u178395/21tp7zihv9i8oq8bbcwm.png
MIT licensed, Python 3.11+. Markdown manuscripts and author-year citation styles only for now (DOCX and numeric styles are on the roadmap).
GitHub: https://github.com/hearthresearch/citefact
I'm the author of ZotSeek (https://github.com/introfini/ZotSeek) and MCP Server Zotero Dev (https://github.com/introfini/mcp-server-zotero-dev) as well; this came out of the same PhD workflow. Feedback very welcome, especially on citation styles you'd want supported next.
It runs three checks, cheapest first:
1. Citations – does every in-text citation exist in the bibliography, with a source PDF? (deterministic, free)
2. Quotes – is every direct quote really verbatim in its cited source? Shows word-level diffs for near-misses. (deterministic, free)
3. Claims – does the source actually support the claim attached to it? (optional, uses an LLM with your own API key; every verdict comes with a verbatim evidence quote so you judge, not the model)
The Zotero part: point it at a collection and it reads metadata and attached PDFs through Zotero's local API – no exports, no Better BibTeX needed:
uvx citefact check manuscript.md --zotero-collection "PhD/Chapter 3"The first two checks make zero network requests and need no key (
--skip-claims). Output is a single self-contained HTML report plus JSON. Everything is cached, so re-runs after edits are fast and only re-verify what changed.Example report (claims check with evidence):
https://s3.amazonaws.com/zotero.org/images/forums/u178395/21tp7zihv9i8oq8bbcwm.png
MIT licensed, Python 3.11+. Markdown manuscripts and author-year citation styles only for now (DOCX and numeric styles are on the roadmap).
GitHub: https://github.com/hearthresearch/citefact
I'm the author of ZotSeek (https://github.com/introfini/ZotSeek) and MCP Server Zotero Dev (https://github.com/introfini/mcp-server-zotero-dev) as well; this came out of the same PhD workflow. Feedback very welcome, especially on citation styles you'd want supported next.
PDF conversion now goes through pagebound (https://pypi.org/project/pagebound/), a small package I split out for this. Every PDF is converted once, keyed on its content hash, into
.citefact/next to the manuscript; a shared cache in~/.cache/pageboundis read too, so PDFs converted by another tool are reused instead of reconverted. Scanned PDFs without a text layer are OCR'd instead of failing, and the report marks quotes matched against OCR text as weaker evidence.The claims check reads the whole paper. Previously each source was silently cut at 40,000 characters, about half a typical article, so a claim supported in the discussion section could come back as "not in paper" with high confidence. The budget is now 300,000 characters and any truncation is reported in the summary and on the affected verdicts.
Block quotes are checked. APA sets quotations of 40+ words as indented blocks without quotation marks, and the quotes check only saw text between marks, so the longest quotations in a manuscript were never verified. Markdown
>blocks are now compared whole against the source, with the closing (Author, year, p. n) treated as the attribution.Page locators are checked. (Smith, 2023, p. 47) now means something: every matched quote reports the PDF page it was found on (the HTML report links straight to it) and, when the PDF's printed page numbers can be read from its running headers or from the entry's page range, the printed page. A verbatim quote that sits on a different page than cited is flagged as a warning. When the numbering cannot be established, citefact reports the page it found and stays silent rather than guess.
--help was rewritten for agents. Every command states what it writes, its exit codes, and the exact shape of report.json, so an LLM agent can drive it without reading the README.
Install or upgrade:
uvx citefact@0.3.0 --help. Still Markdown manuscripts and author-year styles only; feedback on which styles matter to you is still the most useful thing you can send me.Changelog: https://github.com/hearthresearch/citefact/blob/main/CHANGELOG.md