CSL Json Export

Hi there,

Is there anyway to edit the CSL Json export file so as to get my Zotero tags in there?

I'm using the Json file from my Zotero database (+2000 entries) as a feed for an online dataTables render. I would like to just export the file to an FTP and have the database update.
  • @Rintze @fbennett CSL does have a keyword variable, so we could do this legally. It's not used in any style, afaik, so it wouldn't break anything. Any objections/thoughts? Frank, citeproc-js just expects a string there, I assume?
  • FYI: at the moment I'm using the XML export, which I edited (the "Endnotes xml" file), and then I'm converting to JSON. I would like to skip this middle step.
  • I think to customize this, you should be able to hook into the
    doExport function: https://github.com/zotero/translators/blob/master/CSL JSON.js#L68

    You already have the item (which contains item.tags) and you have the CSL JSON created by ZU.itemToCSLJSON(item), so you'd just need to add the tags into the JSON in the format you'd like (likely an array?) before pushing the item to the data array.

    We can see if we'd want to modify the itemToCSLJSON function more generally -- that's my question above -- but a) that's going to take longer and b) it'd almost certainly include the tags as a (comma or semicolon delimited) string, which isn't ideal.
  • I think I understood the gist of it, but the truth is I have no idea how to code js to get that done.
  • I'd have to test this myself, but my starting point would be something like
    function doExport() {
    var item, data = [];
    while(item = Z.nextItem()) {
    var itemJSON = ZU.itemToCSLJSON(item);
    itemJSON = itemJSON.push(item.tags);
    data.push(itemJSON);
    }
    Z.write(JSON.stringify(data, null, "\t"));
    }
  • This doesn't work. There's a javascript error during the CSL Json output.
  • yeah, as I said, it was untested code, more designed as an outline than working code. I'm not going to have time to debug this, sorry. We can see what people say about putting the functionality in the upstream function.
  • Yeah, I understand. But let me tell you, javascript/jquery datatables fed by Json arrays are very useful at the moment, and I think it would really benefit Zotero to work in tandem with them with a simple (but thorough) export. My implementation already works rather well but tags would be useful in order to have simple category filters (or other kinds of metadata tricks).
  • Completely agree. We're working on a full JSON export in JSON-LD format based (mostly or entirely) on schema.org. No ETA, but hopefully going to happen first half of this year.
  • Try this code instead (not tested):
    function doExport() {
    var item, data = [];
    while(item = Z.nextItem()) {
    var itemJSON = ZU.itemToCSLJSON(item);
    itemJSON.keyword = item.tags.join(", ");
    data.push(itemJSON);
    }
    Z.write(JSON.stringify(data, null, "\t"));
    }
  • It partly works! But it returns [object Object].
  • itemJSON.keyword = item.tags.map(o => o.tag).join(", ");
  • That is perfect, my friend. Thank you very much!
  • CSL does have a keyword variable, ..., citeproc-js just expects a string there, I assume?
    Yes. See https://github.com/citation-style-language/schema/blob/v1.0.1/csl-data.json#L350

  • edited January 17, 2021
    Here is a version of the CSL JSON translator that exports attachments, notes, tags, and "extra": https://github.com/dlindem/zotero_lexbib_rdf/blob/af0379536a0fec316df62ceb2ab745e650b95f90/zotero/LexBib JSON.js#L109
  • These things can also be done with BBT CSL JSON, with a postscript. Adding attachments by default was discussed here, where it was suggested that someone pitch the idea on xbiblio-devel. If it would get agreement there, I see no issue in adding it to BBT CSL JSON exports.
  • Hello everyone. Back in 2021, this forum posting was extremely useful to me as I was looking for a way to get JSON export with tags. I was able to do this with a modification of the CSL JSON.js translator, using some combination of the suggestions above. I created a new "JSON with Tags.js" translator for this purpose which served me well.

    The use for this ability is as follows: A friend of mine put together a nice little TypeScript script which loads a JSON list of exported Zotero items in a webpage and turns its tags into buttons (https://github.com/henriksen/bookie). When clicked, the buttons display a list of sources with URLs linking to the titles.

    This was created for the purpose of sharing large lists of open access historical books/documents that are often available on archive.org. See an example here: https://froginawell.net/frog/primary-sources-the-malay-world/#books

    Yesterday, I discover that at some point the modified JSON export with Tags translator, using one of the methods from above was broken. I believe this may have been due to some kind of change in the way translators are done. I was also not able to get things up and running with BBT as an alternative. Nor was david_lindemann's link above working for me anymore.

    I ended up updating the modified script to bring in changed code in the latest CSL JSON.js I could find, and it seems to be functioning again after fiddling with the doExport() function again. Sharing the modified translator below in case a) this is useful to anyone else or b) this is not a good way to go about it, in which case, happy to accept suggestions for alternatives. For now, happy it is up and working again.

    Any reason why JSON export can't include tags by default? Something to do with CSL standards?

    CSL JSON with tags:

    https://gist.github.com/kmlawson/0f3eb75f462bca340897a4d1daf2c708
  • Any reason why JSON export can't include tags by default? Something to do with CSL standards?
    Basically, yes. CSL does have a keyword category, but CSL JSON has a defined schema where that's just a single string, which isn't a great choice for tags, and Zotero has never supported it. Most problematically, adding tags as an array might break citeprocs.

    BBT does support various JSON export formats, though, and @emilianoeheyns might have suggestions of which to use?
  • Thanks! I think I'll stick with this hack for the time being. BBT export uses different names for fields and handles dates in the JSON different, so using that would mean I would have to go and tweak the bookie script as well. Fortunately, this is now working again!
  • BBT export uses different names for fields and handles dates in the JSON different
    Different in what way? Most of the BBT CSL generation is done by Zotero itself, I only augment here and there, and CSL-JSON dates are in a standard format, so I don't know what "handles dates in the JSON different" means? Perhaps that I parse them differently, but export them in the same format? If you find an instance where Zotero does it different from BBT and you think the Zotero variant is better, I'd love to hear about it. Can't fix what I don't know.
  • In general, I would say that any differences between Zotero and BBT CSL JSON export, the BBT version is better
  • edited October 27, 2023
    No critique of the wonderful BBT at all! I have a script which was created by someone else: it worked with a hack of CSL JSON in 2021, but doesn’t work with BBT export. I updated the hack to work with newer 2023 Zotero. I posted update here to help others who may see this post and see that the old solution here (not BBT, which was never explained in original, just mentioned as possible alternative) no longer works.

    I was just here to report a *solution* for benefit of others, it was not to report a problem! :-)

    CSL JSON saves dates differently than BBT, has “author” instead of “creators” etc., all of this is fine and perhaps better, but it just means it won’t work out of box for me with Bookie code written back in 2021 by someone else and it is far easier for an amateur like me to mess with one thing than several!

    Thanks again to everyone: repeat - nothing to see here unless you are interested in my very one-off special solution! I was just trying to be helpful.

    If you want JSON and you want tags with your JSON you can add a single translator like the one above, or you can install the large, powerful, and useful collection of the tools of bbt, which adds a bit of time to every launch of the Zotero app (but worth it for what it offers).

    Whatever works for you!
  • edited October 28, 2023
    I'd still very much like to see a sample of such differences.

    "CSL JSON saves dates differently than BBT", if that means ""CSL JSON saves dates differently than BBT CSL JSON", means that one of them is off-standard, as the way dates are represented in CSL JSON is fixed. They can't be structurally different and both be valid, and if BBT CSL JSON is out of spec, I'd love to fix that.
  • Wait -- you are saving in BetterBibTeX JSON, not Better CSL JSON. These two are entirely unrelated formats.
  • edited October 30, 2023
    Hi @emilianoeheyns,

    Thanks, glad you understood what was going on. I wish you luck in your debugging!

    Here is CSL JSON example:
    https://pastebin.com/UMVL9SYs

    Here is BetterBitTeX JSON example:
    https://pastebin.com/V1dG3nku

    That includes tags (the topic of this forum thread) but, as I mentioned above, does not work with the Bookie code unmodified because of differences with this format without further work.

    Here is an example with your Better CSL JSON:
    https://pastebin.com/ugXGQuQh

    Which is not helpful here since it doesn't include tags (defeating the purpose of using BBT). This is why this is not the export format I'm referring to as it offers no advantage over the original. You have a post above with something to do with postscripts, which looks like an interesting alternative solution but not one that was explained here and I couldn't make much sense of the github thread or the documentation pages for this.

    The absolute simplest solution for my specific need was to duplicate and modify the translator again and add

    if (item.tags) {
    cslItem.tags = item.tags;
    }

    That saves me messing with Bookie. That is pretty much the modification to that file I linked above.

    Thanks so much for your hard work on BetterBibTex - a wonderful resource.
  • edited October 30, 2023
    I wish you luck in your debugging! ... Here is BetterBitTeX JSON example
    There's nothing to debug. BetterBitTeX JSON is not relevant to the discussion.
    The absolute simplest solution for my specific need was to duplicate and modify the translator again and add
    You can do that with https://retorque.re/zotero-better-bibtex/exporting/scripting/, which won't be overwritten on updates.
  • edited October 30, 2023
    Ok, I think I’m a bit tired now. You failed to understand the problem, you never provided an actual solution inside this thread (which I have, hack though it is), and you forward people to massive pages that don’t actually include the concrete solution for them (instead you offer more work for people who may not be coders as they try to decipher things). You fail to see the “relevance” of a format that is the only one you have that includes tags on export, when that is, in fact, the whole point of this thread (exporting tags in JSON, CSL is not the topic). In short, you have not been helpful here, at all. I think I’m done. If you have an actual solution to the problem you are welcome to post it here. I, however, have no more time and will move on to other tasks.
  • Suit yourself. I failed initially to understand the problem because you brought in irrelevant information; you said you had a script that only understands CSL JSON, and if that's so, then BBT JSON is out. Then you failed to understand my questions and that is somehow a knock against me. I pointed at a structural solution, which you reject, which is fine. It was meant to probe whether you were open to this solution. If yes, then I usually write these postscripts for users.i don't expect users to know this level of detail m

    And BBT json is absolutely not the only format to export tags (RDF does too for example).
Sign In or Register to comment.