Date format mangling on import of DOI

bmaranville · December 5, 2017

It seems like sometimes when importing citations from DOI, the date format is converted to legacy/non-optimal forms: e.g. importing this DOI: 10.1007/s10948-013-2148-x
leads to a date in Zotero (string): "8/2013",

The ISO format for dates is used in many cases and seems like it would always be preferred (in this case, it would be "2013-08")

The XML data for this DOI has an unambiguous format with labeled year and month, so the conversion to legacy format string seems to be happening inside the Zotero client. (see http://api.crossref.org/works/10.1007/s10948-013-2148-x/transform/application/vnd.crossref.unixref+xml )

bmaranville · December 5, 2017

Though, to be honest I am accessing the Zotero data through the API and the main issue I'm having is that dates are being returned as "raw" string in a CSL-JSON export from Zotero, rather than the much easier-to-use date-parts array. If the export filter converted the dates back into date-parts I wouldn't be too concerned about the internal storage format for dates in the Zotero schema.

adamsmith · December 5, 2017

Yes, Zotero does this,
https://github.com/zotero/translators/blob/master/CrossRef.js#L298
but it's not really a mangling: Zotero stores dates in both ISO and string format (with string displayed and ISO visible on hover-over) and it's not clear to me that month/year is actually any worse than ISO for human readability -- if anything it's probably more widely used, which I assume this was originally coded.

bmaranville · December 5, 2017

Thanks for the quick reply - I agree that the formatting is fine for human readability.

I tried to figure out why the Zotero API is returning a raw date string for "issued" fields with format=csljson, and I am stumped. The code at https://github.com/zotero/zotero/blob/master/chrome/content/zotero/xpcom/utilities.js#L1655 suggests that a "date-parts" object will be returned, but I am getting a raw string instead. (see https://api.zotero.org/users/4484687/collections/G3J73ZTD/items?format=csljson )

adamsmith · December 5, 2017

The local export code to CSL JSON does export date parts, as the code you point to implements.

@dstillman any thought on why the server/API does this differently (wrongly, I'd say)?

bmaranville · December 5, 2017

I don't know if this is it... but there seems to be a logic error in https://github.com/zotero/dataserver/blob/master/model/Cite.inc.php#L321

... there's a continue; statement in a weird place, that seems to always block the rest of the foreach loop from running. Is it supposed to be inside the

if (Zotero_Date::isSQLDateTime($date)) {...}

block?

adamsmith · December 5, 2017

That does look off, yes. That's dstillman's code, he'll chime in.

dstillman · December 6, 2017

Looks like I never actually finished porting that logic from the client — it's been broken since 2010. Fixed now. Thanks!

dstillman · December 6, 2017

Note, though, that CSL-JSON isn't a complete export format. Unless you're passing the data to a CSL processor, you would generally want to use the default JSON export and get the parsed YYYY-MM-DD date from meta.parsedDate.