Generate bib file from markdown doc that refers to betterbibtex keys

mtl_zack

I have a markdown file that references my zotero library via citekeys generated by betterbibtex. I now have to go through and update the references' metadata, and it would be much easier if I could collate them all into a collection automatically. I just found out that Reference Extractor (https://rintze.zelle.me/ref-extractor/) does this pretty well for word and libreoffice documents, but this will not work for documents that have been converted from markdown to word or libreoffice using pandoc since they do not have an inherent connection with zotero.

Is there a tool out there already that compiles the list of citekeys in a document, checks against the betterbibtex export, and generates a bib file based on that subset? This seems relatively straightforward, but I'm wondering if anyone has done this before, which would save me lots of time and effort to do this myself (with my minimal and shoddy scripting skills).

mtl_zack

One potential snag I anticipate is that if I update a field that is a component for building the citekey it might render the subset incomparable with what is already in the collection. Not sure how to get around this.

adamsmith

I'd ask on the BBT github. I'm pretty sure BBT can do this already. It definitely can for LaTeX docs, but I think it can do the same with markdown, or at least via a detour (there's a Lua filter that lets you create a LO document with life citations from md, which you can then run through refextractor).

emilianoeheyns

@mtl_zack you can prevent this by pinning the citekey, either manually by right-click or have it auto-pinned after a delay (the delay is there to allow initial editing).

The scanner function also supports markdown, you do need pandoc installed and findable through the PATH system variable.

xiaobinee

If you still keep the markdown file, you can try this lua-filter (https://retorque.re/zotero-better-bibtex/exporting/pandoc/). This filter generates embedded citations that can be recognized by Reference Extractor.

It's awesome!

bwiernik

If you just want to generate a BibTeX file based on the citekeys used in a document, BBT has a function that can do that. Check out the rbbt R package for an example https://github.com/paleolimbot/rbbt

(As an aside, if you are writing in markdown and using pandoc’s default citation engine, you should use CSL YAML or CSL JSON, not BibTeX, as your citation data format. BibTeX will tend to produce inaccurate citations for non-book/journal citations.)

emilianoeheyns

There's no need to go through the ref extractor for this case. The aux scanner accepts pandoc markdown.

mtl_zack

Thank you very much for all these suggestions. Updating the PATH worked like a charm. But it's also good to know that I can generate docs/odt files with zotero live citations and that theres an R implementation as well, which will help with future collaborations with people who regularly use both of those tools in their day-to-day work.