Duplicate items

johnryle

My Zotero (version 5.0.24) is not showing duplicate items: the Duplicate items list is empty, despite many exact duplicates having been imported. There are about 2000 items in the library, mostly pdfs (a single library organised in a dozen collections). Duplicates list was working OK previously; and Zotero working fine in every other respect, syncing smoothly etc. Any advice?

bwiernik

Are the items just unattached PDFs, without parent item metadata? Zotero duplicate detection works on parent item metadata, not PDF content. You should always be working with parents items, as that enables most of Zotero’s functionality (Word processor integration, formatting citations, searching, etc.).

https://www.zotero.org/support/getting_stuff_into_your_library

johnryle

That was the problem. Solved now. Thanks

johnryle

Another duplicate problem. In the folder on my hard drive where Zotero stores attached files (Zotero > storage) many of the files (several hundred) appear to be duplicated under different folder names, some with multiple copies (up to a dozen copies of the same file). These duplicates are not visible in the Zotero database, which has been weeded of duplicates (and the Trash file emptied). The existence of the duplicates (76 gb of a total of Zotero library of 322 gb) is revealed by a search using the Mac search function or by using a duplicate-finder program. The duplicates are stored in the standard Zotero manner as single files each in its own folder with a random capitalised 8-character alphanumeric name. In an individual case—by clicking "Show File" in Zotero—it is simple to see which of several copies of the same file is the one actually linked to from Zotero. Then the other copies that Zotero seems to have generated can be safely removed. But this would take a painfully long time. Are these duplicates that have been removed from the Zotero database long ago and are still lingering, or are they somehow generated by Zotero? They are not affecting the day-to-day working of my Zotero database. What, if anything, is to be done about them?

takiefer

Bump. Same.

dstillman

@johnryle: Are you saying there are multiple folders with copies of the same files within them, or duplicated files within the same folder? It's unlikely the latter is caused by Zotero, except for webpage snapshots, where it's common and normal to see multiple .html files with similar names. If the former, you'd see multiple copies if you had duplicates of an item in a Zotero library (including in the trash) or if you had the same item in multiple libraries. It's also possible you could end up with orphaned folders from past bugs in Zotero or third-party plugins. (I think there's currently a bug where files can be left behind if you try to copy items with attachments to another library and the operation fails.)

We're planning to add functionality in a future version to automatically clean up orphaned folders in the 'storage' directory. In the meantime, there are some third-party scripts that can clean up orphaned files, but you'd need to be comfortable running Perl or Python programs.

@takiefer: This thread has multiple issues, so you'd need to say more.