Retracted Items - false positives possible?

Gurdas_Sandhu
Thank you for the retracted items feature. Is it possible this service could throw false positives?

It marked one item in my database as retracted, but it seems the article had a correction but is still citable.

Original article link: https://academic.oup.com/bioscience/article/68/4/281/4644513

Retraction notice: https://retractionwatch.com/2018/04/02/caught-our-notice-climate-change-leads-to-more-neurosurgery-for-polar-bears/
  • DWL-SDCA
    edited June 19, 2019
    RetractionWatch includes both retracted articles and articles with corrections. Some of the "corrected" articles have errors such as author names in the wrong order or misspelled -- things that should be fixed in any manuscript if only for bibliometry reasons. Other corrections may completely invalidate one or more of several research results. Even if the overall results aren't affected by the correction, often the correction will be to numbers in table cells -- items that could be the reason the paper was referenced.
  • bwiernik
    This does raise the question (I haven’t checked)—@dstillman does the current system distinguish retracted vs corrected items? Can it? I know of many articles that have been corrected and noted as such in the RetractionWatch database for minor corrections that probably don’t warrant the full red banner warning (maybe a yellow banner warning?) from Zotero?
  • dstillman
    edited June 19, 2019
    does the current system distinguish retracted vs corrected items? Can it?
    Unfortunately not. RW has a "RetractionNature" field, but the only value ever used is "Retraction". See update below. In this case, the "Reason" given is "Conflict of Interest", explained as "Authors having affiliations with companies, associations, or institutions that may serve to influence their belief about their findings". The retraction DOI in the RW DB (and provided as the retraction notice link in Zotero) is also now a 404, because the initial DOI was incorrect from the publisher. That's since been fixed by OUP, but RW hasn't updated their DB with the new DOI. We'll put in a manual correction for now.

    We can ask RW if it's possible to start using the RetractionNature field, since it's true that the "This item has been retracted" message we show is a bit misleading in some cases — and even their blog post about this case, which we link to, requires some close reading to understand the situation.
  • dstillman
    Actually, it looks like the data we're currently getting from RW just doesn't include the corrections they have in the database — and this one happens to be incorrectly classified, despite the blog post. So we could look at getting the corrections data as well and showing a different message, but the message we show now is correct. We'll remove this entry on our side for the moment.
  • adamsmith
    (I'd find getting correction notifications useful)
  • Gurdas_Sandhu
    Thanks, Dan and others. For items marked as Retracted, there's nothing I can do to remove that label for the false positives, right?
  • dstillman
    I'd find getting correction notifications useful
    Talking about this a bit, our concern is that RW's correction data is much more limited, with (as far as we know) no real ambition to make it comprehensive in the same way, so there's a risk in implying that Zotero can reliably warn you about corrections.
  • Kevin.Russell
    In principle, retraction warnings can be really useful. In practice, since you're throwing false positives, users need a way to disable the warnings, preferably on an item-by-item basis.

    I've got an article that was flagged as a "mistake attributed to a Journal Editor or Publisher" and replaced in a later issue. Problem is, the PDF and all the metadata are for the later corrected version — different issue number, page numbers, DOI, everything. The paper's from 2011, so I think it's safe to assume that Retraction Watch is never going to get around to fixing whatever the problem is on their end. It's up to Zotero to help me not to have to see a giant red warning filling half my screen every time I look at this paper for the rest of my life.
  • dstillman
    Problem is, the PDF and all the metadata are for the later corrected version — different issue number, page numbers, DOI, everything.
    What's the DOI?
    The paper's from 2011, so I think it's safe to assume that Retraction Watch is never going to get around to fixing whatever the problem is on their end.
    That's not a safe assumption. Retraction Watch only launched its database last October, and this is their first collaboration to expose the data more widely. We can report problems back to them and, more immediately, we can fix problems on our end in the meantime, as I say above.

    You can turn off the feature altogether if you want to, but we're not going to provide a way to simply hide warnings — if anything, we'd create a mechanism to report false positives within Zotero. But until we have a sense of how widespread false positives are in the RW data, just reporting them here like any other bug — or emailing support@zotero.org if you don't want to post publicly — will let us fix them for all Zotero users. We've removed the mislabeled correction reported above, so that should have been unflagged (as long as you're running Zotero 5.0.68, which adds removal processing).
  • Gurdas_Sandhu
    edited June 24, 2019
    The false positive for my database went away, so thanks. The "Retracted Items" folder also disappeared since the only item in it was fixed. I wonder (not sure how I feel about this) if it is better to have the folder always there, isn't that how Duplicate Items and Unfiled Items behave?

    "we'd create a mechanism to report false positives within Zotero."
    That would be great.
  • dstillman
    Better to have the folder always there, isn't that how Duplicate Items and Unfiled Items behave?
    Yes, but unlike Unfiled Items and Duplicate Items, which are likely to be used regularly while working in Zotero, having retracted items will be rare and brief for most people, so I don't see any reason to clutter up the collections list. If you do have a retracted item you want to keep, you can hide the virtual collection, and it will reappear if another retracted item is found.

    If anything, I think it would actually make sense to handle Unfiled Files/Duplicate Items the same way, but we can't determine whether those are empty in an efficient manner.
  • dstillman
    @Kevin.Russell: For what it's worth, yours wasn't, strictly speaking, an error in Retraction Watch. The original publication was incomplete and was retracted, but then the publisher reused the original DOI/PMID when republishing it in a later issue. That would seem to defeat the whole purpose of a unique identifier, and it's not really clear what RW should do in that case. Anyhow, I've removed that entry on our end.
  • aswst16
    I'd like to report another example of a false positive. Article doi:10.1001/jama.2018.12615 was retracted to correct minor statistical errors. It has the same DOI post-correction b/c the conclusions did not change. I cite it often and am getting sick of the pop-up. Clicking the "don't remind me again" checkbox is not working on my installation.
  • dstillman
    I cite it often and am getting sick of the pop-up. Clicking the "don't remind me again" checkbox is not working on my installation.
    It looks like the checkbox currently only works when you press Refresh, not when the alert pops up while inserting another citation. We'll fix that.

    If you insert an item that's already retracted and click through the warning, we're also continuing to show the warning when you reselect that item from the Cited section of the citation dialog, which we probably shouldn't do (or we should have the same checkbox in that dialog too).
    It has the same DOI post-correction b/c the conclusions did not change.
    Do we know how common this is? As I say, this really seems like bad practice on the part of the journals.

    Retraction Watch tagged both of these articles with the "Retract and Replace" reason, which has the description "The permanent change of an item to a non-citable status, with a subsequent republication by the same journal after substantial changes to the item". That's correct in the more abstract sense of citability, but not in the more literal sense of citing a given item by its DOI.

    If this is somewhat common, other than having people complain to journals, we probably should add an option for items with the "Retract and Replace" tag to hide all further warnings, which you could use once you've retrieved an updated PDF (and possibly metadata). If you have the original PDF, the retraction warning is still important, so we shouldn't actually blacklist these — and we should probably restore the warning for the one above.
  • aswst16
    I know that this is common for this specific journal and for the JAMA family in general. They've decided that this is their editorial policy for "honest mistakes" -- see https://jamanetwork.com/journals/jama/fullarticle/2652632 and https://jamanetwork.com/journals/jamapsychiatry/fullarticle/2466828 . Given that this is a flagship journal family for clinical biomedical studies, it seems that this will be a common problem, or at least one affecting many users.

    I agree with the idea of somehow being able to check the metadata of the PDF. An obvious idea would be that if the PDF creation date is later than the retraction date, the warning should be suppressed. As far as I can tell, that did not fix the issue with the item I'm describing.
  • dstillman
    edited June 30, 2019
    Crossref agrees that reusing the DOI is bad practice on the part of journals:

    https://twitter.com/CrossrefOrg/status/1145393244588707840
  • aswst16
    I concur with that opinion. I must, however, note that journals do many dumb things, and good reference manager software would ideally help the user in the face of the inevitable and omnipresent sub-optimality of the world in general.

    Or, in less flowery language: they are way too high and mighty to change, so Zotero compensating for their bad practices is my only hope of not being alerted to death.
  • dstillman
    In the latest Zotero beta, we've fixed the repeated prompts in some cases after checking "Don't warn me about this citation again" in the word processor plugin, and we've made that a permanent choice — once you say not to warn about an item, you never see a citation warning for it again in any document.

    For items with the "Retract and Replace" reason, I've added a "Hide warning for replaced work…" link in the item pane, which pops up a dialog that explains that some journals incorrectly reuse the DOI/PMID and lets you permanently hide the retraction warning if you've confirmed that you have the latest version of the work. This isn't ideal, since it's given as an option even for replaced items that genuinely shouldn't be cited, from journals doing the right thing, but we don't really have a better option. Journals like JAMA are just breaking DOIs here.
  • aswst16
    Thank you. Will go snag the beta.
  • DWL-SDCA
    The retraction policies of publishers can border on bizarre.

    Example of retraction because publisher published the the "wrong journal". There is nothing wrong with the article itself. I learned that the article was submitted for publication in BMC Health Serv Res but was instead published in a different journal.

    "The Publisher has retracted this article [1] because it was published in this journal in error. This article is republished in BMC Health Services Research [2]."

    1. Isr J Health Policy Res. 2019;8:44. https://doi.org/10.1186/s13584-019-0330-8
    2. BMC Health Serv Res. 2019. https://doi.org/10.1186/s12913-019-4268-x
Sign In or Register to comment.