Importing bib file to zotero the fields are being mixed

Hi,
I'm implementing a script where the bib file is generated according to zotero standard fields, using python. The bib file is successfully generated, but while importing to zotero the fields are getting mixed. One common pattern which I witnessed is that the next field after "title" field in bib file is put into extras with a auto-prefix "Publication Title:"
Please help me understand where I'm heading wrong.

Sample bib:
"
@book{Ratsirarson_et_Goodman_2000_Recherches_pour_le_développement,
author = "Ratsirarson, J.
and Goodman, S. M.",
date = "2000",
title = "Monographie de la forêt d'Ambohitantely",
journal = "Recherche pour le Développement, Série Sciences Biologiques",
editor = "J. Ratsirarson & S. M. Goodman",
bookTitle = "Monographie de la forêt d’Ambohitantely",
publisher = "Centre d’Information et de Documentation Scientifique et Technique, Antananarivo",
volume = "16",
pages = "pp. 1-152",
extra = "Site 71
and Inforamtion on the protected area, biogeography, paleontology, habitat loss, amphibians, birds, tenrec, flora and vegetation",
}
"
«1
  • You have both 'title' and 'bookTitle', but the Book item type in Zotero doesn't have a Book Title field — only Book Section does. So when you import this, Zotero stores the Book Title in the Extra field instead of discarding it.

    If it were @incollection instead of @book, the bookTitle value would be placed in Book Title, which is a valid field for Book Section.
  • edited May 14, 2020
    Thanks, it was insightful!
    But a new issue is happening now, the extra field is not consumed in @incollection. Is there some documentation which could help me understand what is consumed and what is not by Zotero depending on the annotation used.
  • Imports with BBT would put all unrecognized fields in the extra field.
  • I have an "extra" field in my bib, wouldn't that be considered by the software while importing bib file?
  • BBT would make that tex.extra: Site 71
    and Inforamtion on the protected area, biogeography, paleontology, habitat loss, amphibians, birds, tenrec, flora and vegetation
    in the Zotero item. On export, you'd get exactly that field back in the bibtex generated.

    How would you expect an "extra" field to be handled on import?
  • @emilianoeheyns , @dstillman is there somewhere where all the itemtypes and the corresponding bibtex @itemtype keyword are listed?
  • @emilianoeheyns The extra fields contain key:value pairs that we couldnt fit under any standard field. We thought of putting all such entries into the Extras as Key: value items.

    If we are doing this, is there a delimiter we need to use between different Extra items?

    eg:

    PA: Site 71, Site 30
    NPA: xyz, abc, kjhg
    Theme: biogeography, paleontology, habitat loss, amphibians, birds, tenrec, flora and vegetation
  • You can just have them in separate non-standard bibtex fields, so something like

    PA = {Site 71, Site 30}
    NPA = {xyz, abc, kjhg}
    Theme = {biogeography, paleontology, habitat loss, amphibians, birds, tenrec, flora and vegetation}


    I don't think either bibtex importer will treat extra in any special way (AAMOF I'm certain BBT will not as I am its author).
  • Ok, that's informative. In short it means "extra" is not some standard field type, but instead if something is not recognized by zotero as a standard field during import, it's put under "extra"?
  • extra is not a standard bibtex field, no. BBT recognizes some non-standard fields too, but the fallback behavior is to copy it to the zotero "extra" field. The standard bibtex importer might have a different handling, I'm not as familiar with that as I used to be.
  • Thank you @emilianoeheyns , that is most helpful. Just a quick clarification, would the Key and values be separted by "=" or ":" . I though one of the zotero help pages said to use ":"
  • what's in "Theme" looks like keywords to me BTW; if you have

    keywords = {biogeography, paleontology, habitat loss, amphibians, birds, tenrec, flora and vegetation}

    both BBT and the standard bibtex importer would translate that to zotero tags.
  • WRT key/value separation: just as you would have any bibtex field. Your entry might look like

    @book{Ratsirarson_et_Goodman_2000_Recherches_pour_le_développement,
    author = "Ratsirarson, J.
    and Goodman, S. M.",
    date = "2000",
    title = "Monographie de la forêt d'Ambohitantely",
    journal = "Recherche pour le Développement, Série Sciences Biologiques",
    editor = "J. Ratsirarson & S. M. Goodman",
    bookTitle = "Monographie de la forêt d’Ambohitantely",
    publisher = "Centre d’Information et de Documentation Scientifique et Technique, Antananarivo",
    volume = "16",
    pages = "pp. 1-152",
    PA = {Site 71, Site 30},
    NPA = {xyz, abc, kjhg},
    Theme = {biogeography, paleontology, habitat loss, amphibians, birds, tenrec, flora and vegetation}
    }
  • The help pages talk about the Zotero "extra" field, not an bibtex "extra" field. The colon-separated key-value stuff relates to the zotero "extra" field.
  • You might actually prefer the standard bibtex import here. I hadn't considered the option to use prefixes like "Publication Title". I'll have to change that.
  • Ah got it! So to create the bib file for zotero to import from, we use the "=". But for annotating an entry within zotero extra field we use ":". Thank you!
  • That is correct.
  • Great. An additional query. What would be the type for legal document? I belive it would be Statute in Zotero (?). BBT seems to list it under 'Misc' but how would we force it into Statute in Zotero ?
  • Upcoming release of BBT will have the same "Publication Title" behavior as the standard bibtex importer.
  • If you open an issue on github for BBT we can see whether we can come to a change that works.
  • New release of BBT now puts "cheater syntax" fields in extra for Zotero-supported fields that are not supported for the type. Non-supported fields will still be put in extra with the "tex." prefix.
  • Can you please elaborate considering our example above, how should we build bib so that the three fields above "PA, NPA, Theme" can be consumed in by zotero? Also can you please helps us in understanding what is meant by cheater syntax. Regarding prefix part, will be it tex.PA, tex.NPA and tex.Theme as new fields in our bib file?
  • They're always consumed by Zotero (at least if you're using BBT -- I think the same goes for the standard bibtex import). The question is more: where do you want that data to end up?

    The cheater syntax really isn't something you should be worrying about in this stage; when you import, stuff that doesn't go to fields in the item itself go into the "extra" field, prefixed by a label. That is called the "cheater syntax". BBTs approach here is to fit imported data into Zotero fields, widely recognized "cheater syntax" labels, and lines prefixed by tex.[bibtex field name] as a final fallback. The latter is entirely BBT-specific; if you re-export the item, those lines are output again as [bibtex field name] = { ... }, but all other uses in Zotero (other export formats, or regular citing in Zotero) will likely ignore those.

    I'd suggest turning the question around. Where do you want the field data to end up? If you tell me that, I'll tell you what is possible.
  • We would like is for it go into three 3 separate fields:
    PA ={XX, YY, ZZ...}
    NPA = {AA, BB, CC...}
    Themes = {PP, QQ. RR...}

    Is that possible?
    Is it possible to have custom fields in Bibtex and Zotero?
  • I think you're still approaching it from the wrong end. You are talking about importing data into Zotero right? So the key question is how do you want the item to look in Zotero when the import is done? If you tell me that, I can tell you how the input should look.

    If your question is instead "what should my item in Zotero look like to get this output on export to bibtex" then the answer is "yes that's possible, using the cheater syntax", but before I go into detail I think it's important to first decide which of these two is the problem you want solved.
  • edited May 18, 2020
    Oh and the answer to
    Is it possible to have custom fields in Bibtex and Zotero?
    is "yes, sort of", but that's hardly helpful :)

    To split them off:
    Is it possible to have custom fields in Bibtex?
    Yes, but that's only because bibtex allows pretty much anything; "custom" just means "ignored by pretty much every citation style". The fields you name are simply valid in bibtex, which doesn't care much about what fields you have. Styles do, but bibtex will take pretty much anything. So they're not really "custom" to bibtex.
    Is it possible to have custom fields in Zotero?
    Only sort of, and that's where the cheater syntax comes in. You can add something like

    Journal Abbreviation: XYZ

    or

    Original Date: 1345

    in the "extra" field, and some translators (such as BBT) or the Zotero internal citation engine will pick those up and will use them in their rendering. Additionally, BBT has it's own private cheater syntax, recognizable by the leading tex. in the keyword.
  • Sorry, I am not able to understand. Just to give the context:

    A team had prepared a bibliography of about 3000 records in a spreadsheet and these have some 3 custom fields. We want to import these into Zotero and add 2000 more records.

    We were told that we should convert the spreadsheet into Bibtex formation and import Bibtex file into Zotero. We want to import them into Zotero with the three custom fields showing in the Zotero Extras field.

    ===============
    I tried adding in the Extras field three lines:
    PA: India
    NPA: Bandipur
    Theme: National Park

    And when I export this record in Bibtex format, these go as notes...

    note = {PA: India
    NPA: Bandipur
    Theme: National Park}

    And when I import is back into Zotero, they go as Notes.
  • Do I understand correctly that you want these three fields to round-trip cleanly? That is, you want to import them, and have them show back up on export? If that is what you want, that is exactly what BBT will do. They'll go into extra as
    tex.pa: XX, YY, ZZ...
    tex.npa: AA, BB, CC...
    tex.themes: PP, QQ. RR...

    and will come back out as
    pa = {XX, YY, ZZ...}
    npa = {AA, BB, CC...}
    themes = {PP, QQ. RR...}


    If this still does not achieve what you want, and given this is a one-time effort, I might cautiously suggest using BBTs test import/export format, which is a simplified dump of the internal Zotero item structure in JSON. Given you're going from CSV to bibtex, you're probably comfortable with scripting, so JSON should be easy. This route would allow you to put whatever you want in whichever (supported) field you want, including the extra field. This format has no documentation, but if you export a few samples from Zotero in that format it should be pretty clear what goes where. It may change whenever my testing needs change, so it is not a good long-term storage/interchange format.
  • The question is how should my bibtex file look for getting the three fields into PA/NPA/Themes into the Extras field.

    Trying to reverse engineer, whatever I put into the extras field in the UI when exported in Bibtex go in as Notes.
    ============
    note = {tex.PA: India
    tex.NPA: Karnataka
    tex.Theme: bandipur},
Sign In or Register to comment.