Book sections have dois too

mark

Nowadays, academic publishers assign dois not just to journal articles and books but also to chapters ('book sections' in the Zotero universe).

In fact I can perfectly well add a book section to Zotero using "Add items by identifier" (example: 10.1075/slsi.26.05cou), but the resulting item doesn't have that doi or identifier anywhere in Zotero. That seems inconsistent and (going forward) problematic, as it is a form of information loss.

aurimas

DOIs will be added to most item types in the near(?) future.

bwiernik

For the moment, you can cite the DOI by saving it in the Extra field with the following format:

{:DOI: 10.1234/56789}

Note the colons before and after DOI

mark

Good to hear that this will be added, though I know that "near (?) future" could be anything.

Proposal. While we're not there yet, a good solution (if it can be done) would be to have the "Add item by identifier" function put the DOI in the "Extra" field, to get not just the ability to cite DOIs (as bwiernik suggests), but more importantly, to achieve forward compatibility with the new functionality. I imagine that when the new functionality arrives, a simple conversion function could pick up DOIs from the Extra field and put them where they belong.

It would be rather cumbersome to have to add DOIs to a library later, and especially painful if you'd have to do it for items for which you had the DOI when you imported them! (And yes, I know I can do it manually now, but as an avid Zotero user, I dislike manual labour that could have been automated!)

mark

Is my interim proposal above possible at all, or is 4.2 sufficiently close by to make it not worth the time to implement?

aurimas

Yes, we should be doing this now that the {:DOI: } notation is being more widely recommended. 4.2 (probably 5.1 at this point) is getting closer, but it's not quite there yet.

dstillman

Yes, we should be doing this now that the {:DOI: } notation is being more widely recommended.

Where is that being recommended? Is that what citeproc-js requires? It's really ugly. Hasn't our traditional recommendation just been to have "Key: Value" lines for these temporary fields?

adamsmith

yes, it's how citeproc-js can pick up any CSL variable from the Extra field.

dstillman

And it doesn't pick up "DOI: 10.1234/56789"? I understand this is a temporary workaround, but I'd prefer for us not to add something that looks like a mistake to the visible fields. Aren't we already putting "PMID: <number>" in the Extra field in some cases?

adamsmith

The PMID and PMCID was added by Simon and is passed on to citeproc as the respective variables. The curly bracket solution is parsed by citeproc-js proper when in the notes variable (and works with every CSL-JSON variable, including itemType).

I think the concern is that using something simpler can lead to false positives. How unlikely is it, e.g. that someone would start a note with "title: "?

aurimas

No, it doesn't pick up DOI label (at least to the best of my knowledge). It wouldn't be difficult to add support for the "Label: value" notation for all CSL variables, but we don't currently do it.

bwiernik

If I recall corectly, Frank implemented this functionality because that is how extra variables in MLZ are stored. He also doesn't like the appearance, but never meant for it to be widely visible.

nickbart

Documentation on the "{:DOI: 10.1234/56789}" syntax: http://gsl-nagoya-u.net/http/pub/citeproc-doc.html#supplementary-fields

adamsmith

So @Dan -- it'd be very easy to apply the same parsing we apply to PMID and PMCID and extend it to DOI. The question is if we should go further than that and include all CSL-JSON variables.
E.g. we're using the {:: syntax currently for datasets as they are imported by translators to set them to the dataset item type.

edit: as bwiernik says, Frank won't care about how we handle this in regular Zotero. citeproc-js will obviously work either way. He needs the curly brackets for his supplementary data mechanism.

zuphilip

Curly brackets notation might be better for multiple field-value-pairs?

aurimas

Can you elaborate on that?

adamsmith

I assume zuphilip means it's easier if you store multiple fields? But the way we do that with PMID/PMCID is to just use new line as the delimiter, which I don't see why it shouldn't work, so I don't have any reservations in that respect.

dstillman

it'd be very easy to apply the same parsing we apply to PMID and PMCID and extend it to DOI. The question is if we should go further than that and include all CSL-JSON variables.

I think we should apply the parsing for anything we're using ourselves, and then store it nicely. If people want to use the {:: notation on their own, that's fine, but it shouldn't be necessary for us to use it, and I don't think we'd want to automatically migrate such values later when we gain better item type support — the ones we'd migrate are the nice-looking ones we know we're using. (Dataset might be one exception here if we're using {:: notation for that already.)

dstillman

(I mean, maybe we could migrate certain {:: fields too where we had support for them, but that's a separate issue, and I don't think it's a given.)

zuphilip

Yeah, I posed it as a question. Maybe better formulated neutral: How to deal with possible multiple field-value-pairs? E.g. would it be

extra = {:itemType: dataset, :version: 4}

adamsmith

Currently, multiple pairs with Frank's markup are
extra = {:itemType: dataset} {:version: 4}

What we're using for PMID/PMCID and what we would be using for anything else in the "pretty" version would be
extra = PMID: 123456
DOI: 10.123455

etc. where our reg-ex would only check start of new lines for the field.

aurimas

I don't think we'd want to automatically migrate such values later when we gain better item type support — the ones we'd migrate are the nice-looking ones we know we're using.

I don't think there's much downside to this though. Given the awkward notation, the intention of the user is clear, so if we can support the field at the time of migration, we should do it.

It turns out that Zotero is not the only client that's "recommending" using this notation in the Extra field. Mendeley does so as well. So if we import these fields from Mendeley, we should automatically migrate them to the proper Zotero fields as they become available.

But overall, I agree that this notation is unnecessarily cumbersome and we can easily support more user-friendly "Label: value" format. I think it would be ok to support that for all CSL fields (given that a field is not already assigned).

fbennett

('Twas a stop-gap measure of course. Cumbersome, but unambiguous. :-)

nickbart

How is the the "pretty" solution supposed to work? Upon exporting to CSL JSON, are the variables in the "Extra" field moved to their proper places? I.e.,

[
    {
        ...
        "note": "",
        "PMID": "1234567",
        "DOI": "10.1234/567",
        ...
    }
]

rather than

[
    {
        ...
        "note": "PMID: 1234567\nDOI: 10.1234/567",
        ...
    }
]

adamsmith

That's already what Zotero does -- test it with "PMID: 1234567"(note that it also leaves it in the extra field, which is probably negotiable)

nickbart

Hmm, doesn’t seem to work here (Firefox 37.0.1, Zotero 4.0.26.4). When I create a new entry with just "PMID: 1234567" in the Extra field and export it to CSL JSON, this is what I get:

[
    {
        "id": 13925,
        "type": "article-journal",
        "note": "PMID: 1234567"
    }
]

adamsmith

oh sorry, I think that's something that the current beta implements (though it still has some bugs related to just that, so I wouldn't install it for production purposes right now).
Original commit: https://github.com/zotero/zotero/commit/f0bd1e77ffab6dbc7fbd600dc1acc11844aa2e02

Pull request for the relevant bug:
https://github.com/zotero/zotero/pull/659

nickbart

Ok, now in 4.0.28.3, PMID works as described, but DOI still doesn’t, and neither does {:DOI:10.1234/567}.

Is this getting any closer to being rolled out?

Let me add that I continue to feel it’d be much better to move (rather than copy) the fields in question, in order to free the note field from clutter and to allow it to be used for its original purpose, annotations, too.

bwiernik

I believe you need a space after :DOI: . Otherwise, I can guarantee you that the {:DOI: 10.1234/567} format works. I use it extensively. What citation style are you using?

nickbart

Thank you, but my question was not about the Zotero -> citeproc-js -> word processor workflow (which works well), but about Zotero’s export to a CSL JSON file (for which conversion to “proper” fields has been announced but isn’t there yet, except for PMID) – which would be very helpful for a Zotero -> zotxt -> pandoc workflow.

adamsmith

There may be more reluctance about that, since Dan doesn't really like the {:variable:} workaround, whereas the PMID thing was officially implemented as a workaround by Zotero, but we can see.