Translator error: Sage, lots of extra data in Extra

bwiernik
When I import this article from Sage: https://journals.sagepub.com/doi/10.1177/1534484311411077?icid=int.sj-full-text.similar-articles.1

I get a huge amount of extraneous data stored in Extra:

Audio File Type: other
Company: SAGE PublicationsSage CA: Los Angeles, CA
Distributor: SAGE PublicationsSage CA: Los Angeles, CA
Institution: SAGE PublicationsSage CA: Los Angeles, CA
Label: SAGE PublicationsSage CA: Los Angeles, CA
Letter Type: other
Manuscript Type: other
Map Type: other
Post Type: other
Presentation Type: other
Publisher: SAGE PublicationsSage CA: Los Angeles, CA
Report Type: other
Thesis Type: other
Website Type: other
  • adamsmith
    I believe that'd be a by-product of https://github.com/zotero/zotero/commit/3de54455f68bbcd376505de5a5d3503831e8b901 combined with the fact that our Sage import is broken (and this uses EM).

    That doesn't seem desirable behavior for EM, though...
  • dstillman
    Working on this. That commit included code to avoid adding redundant base-mapped fields when a valid field for the type was given, but I overlooked the case where none of the given fields were valid.

    (This is actually due to outdated translator behavior, though. There shouldn't be a need for a translator to specify more than the base field. Zotero should automatically assign the value to a valid base-mapped field if a base field is given.)
  • bwiernik
    Also just noticed that the date being imported from Sage is wrong. It’s giving the online date instead of the publication date.
  • dstillman
    OK, the EM translator should still be fixed not to save all these base-mapped fields, but the latest beta will remove the redundant ones, leaving this in Extra:

    Publisher: SAGE PublicationsSage CA: Los Angeles, CA
    Type: other

    (Technically "Audio File Type" is mapped to "Medium", but I'm treating it as equivalent to "Type" for this purpose.)
  • bwiernik
    edited February 17, 2020
    Type: is going to be a problem. That will change the CSL Type of the item from article-journal to other. IF that's going to be saved at all, it should be changed to Type: genre.
  • dstillman
    edited February 17, 2020
    Hmm, well, the question, then, is what to do if there are base-mapped fields for "Type" and none are valid for the item type. Should we just throw Type-mapped values away if not valid?
  • bwiernik
    The basic issue is that the Zotero base field “Type” is mapped to CSL genre or medium, not CSL “type”. So to avoid conflicting with parsing CSL fields from Extra, the base field Type should not be stored (it should either be mapped to the item type or stored as “genre”).

    In most cases, for item types that don’t have a mapped field for “Type” (e.g., articleJournal, bookSection), the data that might appear there is probably low quality or unnecessary. The “other” in the example link should be discarded—retaining it in any field will yield bad citations. I suspect this would be the case for most such items.
  • adamsmith
    Yes, agree with bwiernik on both the type vs. genre issue (i.e. _if_ we're going to keep type it should go into genre:) and throwing out type for EM entirely.
  • dstillman
    edited February 17, 2020
    it should be changed to Type: genre
    I assume this meant Genre: [type value]? As in, the CSL genre field, which is mapped to our Type-mapped fields?

    I can't tell when data is coming from the EM translator specifically, but if we don't think invalid-for-item-type Type fields from translators are generally useful, I can just add a special case to remove those.
  • dstillman
    OK, in the latest beta, this is all that will be left in Extra:

    Publisher: SAGE PublicationsSage CA: Los Angeles, CA
Sign In or Register to comment.