DOI from Extra when using "Export Items..."

jim

I have a number of items of non-journal types that have DOIs associated. I am using the technique that I have seen in several places on the forum to record DOIs in the Extra field.

If I select a group of items, right click and choose "Create Bibliography from Items..." this seems to work well: the Bibliography comes out well with the DOIs properly handled.

However, if I select the same group and choose "Export Items..." from the right-click menu I get a .bib (or .bibtex) file where the DOIs are included in "note = " fields, rather than "doi = " fields. The software I am using to read these exported bibliography files does not know to find the DOIs in these fields, so acts as if they are not there. (I am using R Markdown in R Studio, which I believe uses Pandoc in the background, though I'm unsure of the details.)

I also notice that the DOIs are encoded with extra curly brackets around various parts, which I believe is because they include capital letters. For example, I have the following line:

note = {\{:{DOI}: 10.17605/{OSF}.{IO}/E8QPW\}},

which I would like to be something like:

doi = {10.17605/OSF.IO/E8QPW},

Is there any way to get Zotero to move the DOI values from the Extra (notes) field to a DOI field at the point of exporting to bib / bibtex, in the same way as it seems to manage when I use the "Create bibliography" functionality? Or is this something that needs to be handled on the importer side. (If so, I'd naturally appreciate any pointers on how to do that, but of course understand it might be off-topic as a non-Zotero thing.)

Version 4.0.29.17, Zotero Standalone, Windows 10.

I have seen various formats recommended in the forum for using the Extra field to record DOIs and have tried them all - "key: value", "{:key:value}" and "{:key: value}" (no brackets, with brackets, and with brackets plus a space) - the "Create bibliography" tool seems to be relaxed and works with any of these; "Export items" works with none of them.

adamsmith

so you definitely want to use "key: value" -- that's the only format we'll make every effort to support.

I'm pretty sure better bibtex already handles this:
https://github.com/ZotPlus/zotero-better-bibtex
and I'd recommend you using that anyway (also, check out R-Studio/Zotero integration via Citr https://cran.r-project.org/package=citr which relies on better bibtex) so that may well solve your problem.

I'll look into this for our generic bibtex & biblatex export, but that may still take a little.

jim

Thanks for the quick response, @adamsmith. I understand that Better Bibtex is unlikely to support Zotero 5 particularly soon, and I was planning on upgrading to that once it is out of beta.

https://github.com/retorquere/zotero-better-bibtex/issues/555

The Bibliography for this project is small enough that I might be able to just run a small script to do the DOI extraction myself with some manual checking.

Good to know that "key: value" is the preferred format. I'll update my entries accordingly. Would it be considered good etiquette on this forum for me to post responses to the older messages that had been saying other formats directing them to this as the current preferred approach?

adamsmith

Would it be considered good etiquette on this forum for me to post responses to the older messages that had been saying other formats directing them to this as the current preferred approach?

no, please don't -- a number of people follow all forum activity and that makes things very messy, though I understand the idea.

If you're able to do some scripting, you can also consider just modifying your local copy of BibTeX.js, which is in the translators subdirectory of the zotero data folder
https://www.zotero.org/support/zotero_data

jim

Great - editing BibTeX.js worked a treat and only required a small edit. Thanks for the help @adamsmith.

For future users coming across this, the machinery is already all there in the .js, and already enabled for other identifiers like PMID. Near-ish the top (line 101 in my BibTeX.js) I found a variable definition:

var extraIdentifiers = {
lccn: 'LCCN',
mrnumber: 'MR',
zmnumber: 'Zbl',
pmid: 'PMID',
pmcid: 'PMCID'

(There's a closing }; after several comment lines.)

To this I just inserted:

doi: 'DOI',

Note that I included a trailing comma since I was inserting it at the top of the list.

I did a few manual tests to check that this doesn't break anything obvious. The other extraIdentifiers seem to still work (i.e., I could have both a DOI and PMID in my "Extra" field and they both seem to export into the .bib file). It also does not seem to cause any clashes with other items that have an actual DOI field. If you have an actual DOI field and also specify one in Extra then both seem to end up in the .bib file - I guess a fuller solution might try to prevent that.

One odd thing to report: the machinery to do this also seems to be present in the BibLaTex.js translator, but I could not get that to work. The variable definition is a slightly different name and format (UPPERCASE: 'lowercase' vs. lowercase: 'UPPERCASE'), but adjusting my BibLaTex.js to include DOI in that list as below did not work.

var revExtraIds = {
DOI: 'doi',
LCCN: 'lccn',
MR: 'mrnumber',
Zbl: 'zmnumber',
PMCID: 'pmcid',
PMID: 'pmid'
};

I could not find an obvious reason for this failing, since the BibLaTex export does seem to work for PMIDs included in 'Extra', so the code does seem to be functional.

Anyway, my problem is solved because I'm happy using BibTeX, but thought I'd flag that it might not work for some reason if modifying BibLaTex.js. Most likely I'm doing something wrong - I could read the js well enough to figure out what it was supposed to be doing, but it's not a language I actually know how to write!

adamsmith

The general idea of what you're doing is definitely right. I'd have to look at what's preventing this from working in BibLaTeX and I'd also want to double-check the BibTeX for anything problematic (e.g. it could affect import), but you should definitely be fine.
Remember that future updates to the BibTeX translator on our end will overwrite your changes, so you may have to do this again in the future.

LiborA

@adamsmith Is there any problem to add this hack in the "official" BibTex (and/or other translators)? Extra field is very often used for DOI, due to missing of DOI field for some types of document.

adamsmith

as I say above

I'll look into this for our generic bibtex & biblatex export, but that may still take a little.

so definitely yes in general, but obviously we need to check this more carefully for the official translator.