Duplicates: How to delete

pmz

I have a large sized library (8-9 years on Zotero), but 95% is duplicates. Many have 15-25 duplicates. They're not from imported endnote files; all are native to Zotero. I've accessed Zotero from ten different computers over the period. It's not worth the time to delete individual references by hand and I suspect whatever is causing it would continue. Any tips of how to delete all duplicates automatically or how to prevent the ongoing accumulation?

adamsmith

There is no way to delete duplicates automatically.
Normal Zotero usage does not produce any duplicates. How are you moving data between computers?

pmz

I've just accessed Zotero through syncing, which seems to have created the duplicates. Is there a setting that would prevent duplication during syncing, or is there some other likely cause?

Another possibly related issue: My new computer shows only part of my library, but not the main part. I can access that through the web but it doesn't show up in the firefox version.

dstillman

There's no way for Zotero syncing to produce duplicates. If you had the same library on different computers from exporting and importing, though, those separate items would sync to both computers, and that would appear to you like duplicates were created. (Zotero doesn't actually care what they look like to you. It just uses internal identifiers to determine if they're the same items, and those identifiers are lost during export/import.)

It's also not possible to see only part of your library (compared to what's on zotero.org) if you've synced. Zotero syncs everything at once. See Changes Not Syncing.

pmz

Found the function to find and merge duplicates! I hadn't checked for it for a few years and I'm happy it's been added to Zotero. Thanks and sorry to repeat old issues.

http://www.zotero.org/support/duplicate_detection

adamsmith

right, but that still means you have to do each set of duplicates manually. It also won't help to prevent duplicates if you're doing something that causes them to appear systematically.

pmz

Thanks Dan and Adam! It appears that the dups were created on two days in 2010 at the same time, so I must have been importing Zotero files into a new computer. Please thank the dup detection & merge features software writer - they work very well.

Also, I have access to my full library now and my next five research seminars will be productive for a change as I click on the "Merge X Items" button, the recent dup detection/merge feature, for a few hours. It will be nice to get back into Zotero and use its new functions. Thanks again for the help.

pmz

Adam and Dan: It would be nice to program the "Merge" function to allow it to automatically go through all the duplicates in your library. My numbers of duplicates and numbers of repetitions (21 or 42 copies of each) may not be common, so this may not be a priority for the programmers. It doesn't seem like it would be a major programming commitment. So far I've spent about 3 hours merging by hand and am not done; it's slow. My guess is that others could also use automated (optional) duplicate removal. Most similar automated software has a click box and warning that the process can't be undone.

dstillman

Note that you can generally delete entire batches of imports at once just by sorting by Date Added in the middle column.

pmz

Thanks, Dan. I'm deleting by batch already but it's still a long process. I have major duplication issues.