Duplicates after Mendeley import

dstillman

This discussion was created from comments split from: Available for beta testing: Online Mendeley importer.

Pitosalas

I used it and it seemed to work. But there are a whole bunch of "duplicates". And when I click on the pdfs of the many of them, they give the error:

The attached file could not be found at the following path:

/Users/pitosalas/Zotero/storage/ULCTQ8KE/Osterweil - 2007 - A Future for Software Engineering.pdf

It may have been moved or deleted outside of Zotero, or, if the file was added on another computer, it may not yet have been synced to zotero.org.

When I go back to my mendeley on the same computer, the files are there. I also have increased my Zotero storage to 3g and I don't believe its full.

dstillman

That would happen if you tried to import some other way first (e.g., RIS import) but the process was interrupted. The online importer shouldn't create duplicates, and certainly not ones with missing files, since the items it imports include identifiers from the Mendeley database and are simply updated on a repeat import attempt.

Sort by Date Added in the middle pane (right-click in the items list header to add that column) and check the dates of the items, and you'll find that your duplicates are from separate dates (and that it's earlier ones with missing files). You may be able to just delete entire earlier import runs.

Alternatively, if you want your database to just match your Mendeley database exactly, you can delete your Zotero data directory, restart Zotero, replace the online library to wipe out all your data, and then run the Mendeley importer again.

Pitosalas

Thanks... As it happens I did hit the storage limit on the free tier and the sync was interrupted. So I will follow your suggestion!

dstillman

No, it's not that:

1) Zotero is a local program that stores all data and files on your computer by default, and running out of online storage has no effect on the import.

2) Again, Zotero's Mendeley importer doesn't create duplicates. These would be from previous import attempts — e.g., from prior years — using other methods, such as RIS/BibTeX import.

Pitosalas

Got it. Let's separate the problems I saw.

(By the way I have since followed your original advice and deleted everything and allowed the import to run to completion. Actually not sure how to tell that it's completed. But the storage didn't run out.)

Maybe what I saw was not duplicates related; maybe it was just a bunch of "pdf" icons in the list which when I clicked on the gave the error I cited above. Could that have been caused by the interrupted import?

dstillman

An interrupted RIS or BibTeX import could leave some broken attachments, but it's not worth worrying about — we don't recommend using those methods to import, and whatever you did in the past doesn't matter at this point if you've since wiped your database and reimported using the official importer.

Pitosalas

Thanks. Can I ask what it means if Zotero claims to papers are dups but in fact to me they look nothing alike?

dstillman

What do you mean by "nothing alike"? Can you take a screenshot, upload it somewhere (e.g., Dropbox or Google Drive), and provide a link here?

Pitosalas

https://snipboard.io/xvSMh4.jpg

Pitosalas

And when I select the other one: https://snipboard.io/zSEPsU.jpg

dstillman

Those items both have the same invalid DOI, and DOIs override all other duplicate detection. You should delete those values.