Today, when I checked my group Zotero database (H-Buddhism bibliography), I found that all of the 8,000+ items had been duplicated, such that we now have 16,000+ items. It is not possible to export and re-import without Firefox freezing. We can't go through "merge 2 items" 8,000 times!! There needs to be a more powerful solution for this kind of problem.
acmuller: If you're experiencing a problem or would like help with something, you should start a new thread. But the main way you get duplicate items in Zotero is by using export/import, and in that case you could just sort by Date Added and delete the duplicated items.
Figure out the date when these items were duplicated (look at Date Added), then create a saved search for "Date Added" "is after" <slightly before the duplication>. Open the saved search, select all (Ctrl/Cmd + A) and delete.
Thanks for the tip. I have indeed found that the dupes were all added the same day (same minute, actually). I don't know how to create a saved search, but will begin to try to figure it out now. Any pointers on how to do this would be greatly appreciated.
I am reading this thread and couldn't find how I can merge different item types even though they are the same thing. For example, I have a journal article which appears twice from 2 different sources and one is a manuscript item. I am not able to merge all three of them even though they are the same thing. Is there a way to merge them or at least merge the 2 journal articles together?
Thanks Adam. I thought there may be another way to merge because when you change a conference paper to journal article, some fields will be missing. However, that didn't bother me much because I will add a note if I know that conference is an important piece of info. So far all works well. Thanks Adam!
Reading the above I cannot figure out how to easily delete duplicates. I thought: 1 - get them to a temporary collection 2 - sort on name 3 - delete the ones I want to keep (delete from collection that is) 4 - then 'delete collection and items'. I guess that should delete them from My Lib. as well. The previously (step 3) deleted items should still be in My Library...
But how do I get all duplicates in a collection?!?! Dragging does only drag one duplicate item at the time
You can't with a collection, but you can do exactly this with a colored tag (select all in duplicates, hit the number key for the tag you want to use, then proceed as you describe substituting "Tag" for collection)
Thank you Adam, but please ilucidate me on this. I never used colored tags. What do you mean by stating "hit the number key for the tag you want to use" My duplicates do not have a single tag in common but a lot. I can imagine though that it makes sense to add a color or even nonsensical tag like "abacadabra" to all duplicates, but I do not grasp how to do that.
See https://zotero-manual.github.io/zotero-manual/tags#colored-tags
And yes, the idea is exactly what you say--where you wanted to use a temporary collection, just assign a tag to all duplicates and the proceed just like you suggested for collections.
I've read that manual before posting. But to assign a color, there has to be a unique tag for duplicates first, right? That's my problem. Or was: Zutilo did the trick. Now I have duplicates tag: "___" But I do not need steps 1-4, I simply select ___, sort on name, delete duplicate.
I still like to know what you meant but my problem seems solved/worked-around.
Still: a delete button in the duplicate area would help, and being able to see all differences between duplicates (like attachments, tags, etc) would help even more.
(and you can just create a tag with Zotero on any item, then turn it into a colored tag. No need for Zutilo for that, though it's a fine add-on to have in any case)
I hope not ;). All and only duplicates have the tag "___". So in My Lib. I select ___. After sorting the dups stick together. I switch to and fro the dups, maybe copy some tags, other details, pick the best attachment (e.g. with annotations), etc. And delete one of the dups. The best former-dup stays in. Works fine so far.
See your last post, there must be a misunderstanding: If I have the tag "Zot" in duplicates, I can color it. Right. But all other items with the tag "Zot" will be colored, not only the duplicates. So basically I still have no clou what your procedure consists of...
I guess I'm not understanding the problem you're trying to solve is after all (what you're doing would seem to be possible just within the Duplicates view), but if you're happy with your solution, doesn't really matter.
I mean:
1. Find an item that you know is duplicate. Add a tag to it that doesn't yet exist in your library (that'd be on the right under "Tags")
2. Turn that tag into a colored tag
3. Assign to all duplicates.
More questions! Unless I miss something, I can not do all in Dups view I want. Check tags? Edit Authors? Delete useless note? Hit enter to open attachment? Dups view is a handsome Dups generator, but e.g. Procite which is dead for a while now had a better Dup-processor. But this workaround works fine! I'll manage
Your last post: how do I assign a tag to all dups without Zutilo?
Once you have the tag (steps 1 and 2 above) select all in Duplicates view and press the number key associated with that colored tag. (see the colored tag instructions).
More questions! Unless I miss something, I can not do all in Dups view I want. Check tags? Edit Authors? Delete useless note? Hit enter to open attachment?
all possible. You can select individual items in duplicate view using ctrl+click (Cmd+click on Mac I believe) and then can modify and view them like anywhere else in Zotero. (There's a post by me and Aurimas further up in this thread on that).
Finally we meet! 1: _only_ colored tags in Dup view because they are numbered shortcuts 2: Ctrl works. Got it. I prefer a Dup-tag, because theres no Dub-view between the ctrl+clicks, but I get the point now. Will work perfect for incidental dubs, but I have a lot after importing stuff from a trainee thank you for your patience
Looking for an update on this. Is there still no way to automatically detect and merge duplicates? Like luisiafalves above, I am working on systematic reviews, pulling in articles from multiple databases and then need to deduplicate thousands of articles. Any progress on this feature since 2015?
We have no trouble merging duplicate citations but are still troubled by the inability to merge/delete child attachments. Can find no documentation. Has this ever been addressed?
Sorry if I missed this in a previous comment, but when I merge two items, only the tags from the "master" are preserved. I'd have thought the tags from the two items would be combined. Am I missing something obvious? Thanks.
All tags from both items should be preserved when merging and that's the case for me. Are you sure it isn't for you? Can you consistently replicate tags disappearing.
Hm, I just did a clean test and it did work as expected. Was I just seeing (or not seeing) things? I swore tags disappeared on me. Anyway, I'll keep an eye out. Thanks for responding.
Upgrade Storage
Advanced/Saved Searches: https://www.zotero.org/support/searching#advanced_search
Just sorting is probably sufficient for this.
Thank you for your help in advance.
1 - get them to a temporary collection
2 - sort on name
3 - delete the ones I want to keep (delete from collection that is)
4 - then 'delete collection and items'. I guess that should delete them from My Lib. as well. The previously (step 3) deleted items should still be in My Library...
But how do I get all duplicates in a collection?!?! Dragging does only drag one duplicate item at the time
I can imagine though that it makes sense to add a color or even nonsensical tag like "abacadabra" to all duplicates, but I do not grasp how to do that.
And yes, the idea is exactly what you say--where you wanted to use a temporary collection, just assign a tag to all duplicates and the proceed just like you suggested for collections.
I still like to know what you meant but my problem seems solved/worked-around.
Still: a delete button in the duplicate area would help, and being able to see all differences between duplicates (like attachments, tags, etc) would help even more.
So basically I still have no clou what your procedure consists of...
1. Find an item that you know is duplicate. Add a tag to it that doesn't yet exist in your library (that'd be on the right under "Tags")
2. Turn that tag into a colored tag
3. Assign to all duplicates.
Your last post: how do I assign a tag to all dups without Zutilo?
select all in Duplicates view and press the number key associated with that colored tag. (see the colored tag instructions). all possible. You can select individual items in duplicate view using ctrl+click (Cmd+click on Mac I believe) and then can modify and view them like anywhere else in Zotero. (There's a post by me and Aurimas further up in this thread on that).
1: _only_ colored tags in Dup view because they are numbered shortcuts
2: Ctrl works. Got it. I prefer a Dup-tag, because theres no Dub-view between the ctrl+clicks, but I get the point now. Will work perfect for incidental dubs, but I have a lot after importing stuff from a trainee
thank you for your patience