Zotero to CSL type and field mappings

aurimas

I needed to get a list of all fields for all item types, so instead of wasting a couple minutes looking through them on Zotero, I decided that I'll create a script that will generate this list automatically and since the current mapping is out of date, I thought it might be useful in general.

To get to the point, I ended up making a Zotero extension that outputs field mappings in an xml format, which is then transformed and styled.

It's available at https://github.com/aurimasv/z2csl where typeMap.xml is already pregenerated and typeMap.xsl and typeMap.css go along with it.

This is sort of my "hello world" extension, so I'm sure it's not the greatest piece of work out there (and if someone cares to point out some amateur mistakes, I'm excited to learn). I also lack any sort of artistic talent, so the styling is ugly, but I think it does the job.

The only thing that it lacks is mapping Zotero label to Zotero field, which for the most part is self-explanatory anyway. Other than that, it can probably replace http://gsl-nagoya-u.net/http/pub/csl-fields/

EDIT:It's also my first time using XSLT

fbennett

There is also this, if you're interested. I think it might still be slipping up on some of the item types that are virtualized inside Zotero, but it's mostly right.

https://bitbucket.org/fbennett/zoterofieldmap/overview

(I posted it in the hope that someone would take up the chalice and rebuild the field assignment pages, but there's no need to revisit it if what you have is working.)

(edit: your approach will be easier to maintain, as it uses the native Zotero functions. Mine worked off the database, which was handy for avoiding the need pull data from a running Zotero instance, but made for complexity in the code.)

Rintze

Awesome.

As an aside, I think the CSL project would really benefit from hosting something like typeMap.xml as part of the CSL specification. Currently we have no documentation on what the different CSL item types represent and how exactly they differ from each other in terms of fields. When separate pieces of software (e.g. Zotero and Mendeley) add differing sets of fields to the item types, this can lead to some unexpected differences in CSL output.

fbennett

There are variations between styles as well. Style-level test suites would be an efficient way of assuring or encouraging consistency.

aurimas

Mine worked off the database, which was handy for avoiding the need pull data from a running Zotero instance, but made for complexity in the code

That's awesome! My initial attempt was also in that direction, except I was trying to write it as a bash script. It got a bit messy and complicated (probably because my bash is not that great), so I dropped it and went with the Zotero extension. But it's cool that you actually managed to do it the way I thought it should work.

I think the CSL project would really benefit from hosting something like typeMap.xml as part of the CSL specification.

Absolutely, let me know if there are some tweaks you have in mind.

Rintze

Absolutely, let me know if there are some tweaks you have in mind.

I'm still trying to figure out exactly what information we would like to store, and in which format/structure. I had been thinking about JSON before, but XML might be better since we can easily manipulate it with XSLT and create HTML presentations.

Basically we need CSL field assignments for all CSL item types. We need clear textual descriptions for each item type that make it clear how the item types differ from each other. And it would probably be helpful to provide item type-specific labels/descriptions for each of the CSL fields (these are the Zotero labels at e.g. http://gsl-nagoya-u.net/http/pub/csl-fields/map.html) to further standardize how fields are presented to users (e.g. Zotero uses the label "Publication" for the CSL variable "container-title" for the CSL item type "article-journal).

Gracile

Thank you aurimas!

aurimas

I put the type map on github pages
http://aurimasv.github.com/z2csl/typeMap.xml

fbennett

@aurimas: Shall I redirect the old gsl-nagoya-u.net pages to this address?

aurimas

I think so. It doesn't look as good, but it should be more correct. I'll also try to do what Rintze was suggesting above.

fbennett

Sounds good. I'll just provide a reference cross-link for the present, until z2csl is ready to take over completely.

Gracile

Zotero "docketNumber" is mapped to CSL "number" but it does not appear in typeMap xml: http://aurimasv.github.com/z2csl/typeMap.xml#map-case
Why?

aurimas

I see the problem. Will try to get it fixed today.

Gracile

Thanks!

aurimas

Updated. The fields in the parentheses indicate "base" fields in Zotero. I didn't catch that before, but CSL mappings are done on base fields.

fbennett

Another little wrinkle is that each Zotero item type is associated with a "primary" creator field (marked with a non-nil value in the third column of itemTypeCreatorTypes table). The "primary" creator field always maps to "author". So for audioRecording -> song, "performer" is mapped to "author".

aurimas

@fbennett Fixed

LiborA

Hallo, http://aurimasv.github.com/z2csl/typeMap.xml is from Zotero 3.0.9 Source. Are any diferencies between 3.0.9 and 3.0.11? I need list of all fields for all item types but I don´t know how use z2csl extension. Thx

Gracile

No: http://www.zotero.org/support/3.0_changelog

aurimas

Thanks to Frank, the Zotero <=> CSL mappings now include Zotero UI labels.

I also fixed the CSL variable mappings discussed here.

http://aurimasv.github.com/z2csl/typeMap.xml

scove

Thankyou, Thankyou, Thankyou! I was tearing my hair out trying to find some sort of documentation like this that will help with writing my style.

It will also help with writing the documentation for which types to use in Zotero for the particular uni i am writing the script for. Again, Thankyou!

Lubos

Thanks for this great resource aurimas!

The only section I would love to have added for convenience is a matrix of all CSL item types by CSL variables. The cells could contain either a "Yes" in the spots where the variable is available or the name of the actual Zotero UI label.

A second matrix could be added for all Zotero item types by Zotero UI labels and the cells could contain the CSL variable names.

Come to think about it, both of such matrices could be combined into one.

aurimas

AFAIK CSL imposes no restrictions on what variables are available for which types. I think, in theory, everything is available for all types.

I'll see what I can do for the Zotero matrix. If you need this some time soon, you may want to try and add it yourself. It should not need any extra data from the Zotero plugin, just some more XSLT code.

Rintze

CSL imposes no restrictions on what variables are available for which types.

Currently, no. We might eventually want to provide some guidance, e.g. based on the long-promised revisiting of the Zotero data model.

I think putting in restrictions in the CSL spec might be a no-go, unless all implementations agree to revisit their data models as necessary (considering that we've been clamoring for changes to the Zotero data model for years, I don't think this is very likely).

Lubos

@aurimas: Sorry, what I meant for the first matrix was that the content be based on actual item field labels Zotero provides to the CSL processor.

The XSLT bit sounds tempting but I am not sure I have the required bandwidth.

tog

@aurimas

Hi there,

I am struggling figuring out how Zotero fields map to CSL variables. I searched the web and found

https://github.com/aurimasv/z2csl

which sounds exactly to be what I am looking for. I am working with Zotero standalone. Could you kindly give me advice on how to install this extension.

Thanks in advance

Thomas

fbennett

Unless you are hacking the internals of Zotero itself, the extracted data that Aurimas has published should serve:

http://aurimasv.github.io/z2csl/typeMap.xml

tog

@fbennett

Thanks a lot for that pointer. I will go for typeMap.xml

Thomas

southbam94

@aurimas
Very nice job. thanks for your great work so far!! I'm still confused why – after all those years – there is no official mapping from Zotero or am I missing something? I think it's fundamental for creating an individual citation style which I'm want to do right now. But I'm wondering if there were changes in Zotero 7 or is your mapping from 2022 still accurate?

damnation

Mainly additions.
Between the CSL specs and this you'll be fine:
https://forums.zotero.org/discussion/93617/announcement-csl-1-0-2-launch-date

adamsmith

The above site was updated for the last version of Zotero 6 (Aurimas doesn't work on Zotero anymore, but gave me -- one of the co-maintainers of CSL -- edit access; it's also open source and can be cloned, of course).
Unfortuately the add-on used to automatically create the mappings doesn't work in Zotero 7, so the page is currently missing two newly added item types: Standard and Dataset. Everything else is still correct, I believe.

@dstillman -- would one of you be willing to update the add-on? I agree we really do need an accurate mapping and with more changes coming, having the only complete mapping become outdated is going to become increasingly problematic.