unwanted duplicate entries in the bibliography - how to remove them

dinosgr
Hello dear zotero friends,

I need your help.
Here is the description of the problem:


I have finished writing my document. I am using the Harvard citation style so my documents is full of (Author, Year) citations. No problem here.

In the end of my document there is a bibliography section that contains all references that I' ve used all over my document.


HERE IS THE PROBLEM:

Some of the items in the bibliography section are duplicates. Example:


Jameson, J.M., 1998a. Air pollution and behaviour. Nature 18, 231–233.
Jameson, J.M., 1998b. Air pollution and behaviour. Nature 18, 231–233.

I don't know how these "a" and "b"'s appeared. It is the exact same citation. How do I remove these duplicates.

THANK YOU FOR YOUR HELP.
I really appreciate it
Warm Regards


edit// PS. I forgot to mention there are NO DUPLICATE entries in my zotero library!!
  • DWL-SDCA
    How did you check for duplicates? Did you look in your trash? Were you working on your manuscript alone and from a single computer or have you been working from more than one computer or with a group library?

    Worst case: Make a copy of your document with live Zotero links. Find all of the times the Jameson article was cited. Delete the reference mark fields. Save the document. Reinsert the Jameson references. Your new bibliography should not have an extra Jameson record with disambiguation.
  • dinosgr
    edited July 13, 2013
    Hello, thank you for your reply my friend.

    In the Zotero standalone program , on the left side of the screen , when I click on Duplicate Items, the list is empty.

    The trash is also empty.

    Could you please explain to me in more detail this procedure:

    Worst case: Make a copy of your document with live Zotero links. Find all of the times the Jameson article was cited. Delete the reference mark fields. Save the document. Reinsert the Jameson references. Your new bibliography should not have an extra Jameson record with disambiguation.



    THANK YOU VERY MUCH
    Warm Regards

    PS. It's not just the jameson citation... my document contains 500 citations and about 50 of them are duplicates. I need an "automatic" way to fix all duplicates at once.
  • DWL-SDCA
    It's not just the jameson citation...
    That means that your problem is more than that one reference and someone with more experience will need to help.

    Do make a copy of your document and work from the copy.

    Try the steps at:
    http://www.zotero.org/support/word_processor_plugin_troubleshooting#debugging_broken_documents
  • dinosgr
    edited July 14, 2013
    Thank you.

    I remember now that when I was inserting a citation in my document when I was shown the drop-drown menu / citation search bar (the one line red window with the "Z" on the left, I don't know how to describe it), I could choose the EXACT same citation either from the upper half of the results or the lower half of the results "My Library"). Could this have anything to do with my problem. I have been choosing the citation I wanted to insert AT RANDOM (either from the upper or lower half of the dropdown menu).

    Thanks
  • adamsmith
    So the most plausible theory of what happened is this:
    Zotero stores reference information for items in the document, so even if you delete an item from your library, it's still in the document and you can keep citing it (that's useful e.g. for collaborating).

    Now what I think happened is that you somehow deleted the version of the items that you cited originally - one way would have been if you exported and imported a library either to restore it or to move it to a different computer (instead of syncing). Another scenario would be if you deleted instead of merged duplicates.

    And then it's exactly as you said - you sometimes inserted the version stored in the document and sometimes the new version from your library. But for Zotero, these were now two separate versions -- and they show up as such in the library.

    Unfortunately, I can't think of a good way to fix this quickly that doesn't entail replacing one set of those citations individually. Which word processor are you using?
  • dinosgr
    You are right! That's exactly what happened! We have some very bright people in here like you adamsmith!

    That's exactly what happened. I moved my document + zotero library to another computer one week ago - forgot to mention that - (these duplicates in the bibliography were not there when I was working on my old computer!).

    So the problem was caused due to moving the library to a different computer???

    The first computer I used was a laptop with word 2000
    The computer I am using now is a different laptop with word 2007

    Any ideas how to fix the problem?

    THANK YOU VERY MUCH
  • adamsmith
    I don't have a good idea how to easily fix this at the moment, but FWIW, if you had moved your library using one of the methods described here this wouldn't have occured:
    http://www.zotero.org/support/kb/transferring_a_library
  • dinosgr
    Hello adam,

    I used the export library / import library feature.
    You mean the export / import thing isn't working as it should??
    THANKS
  • arggem
    Export/import _is_ working...but that's not how you move a library. See the warning at the bottom of the page adamsmith linked to.
  • dinosgr
    OK, I see. Unfortunately moving the library the correct way is not an option anymore for me.

    Would you be so kind to tell me the exact strategy I should follow in order to fix the problem (the duplicates in the bibliography) even if I have to do it manually? Only 50 citations are problematic, it should take me 3-4 hours. I don't mind doing it manually but the problem is I don't know how to do it! :-(

    I would REALLY appreciate some help now!
    THANK YOU, You are all great.
    Warm Regards
  • arggem
    I don't know if this will work or not, but if I understand the problem I think it should.

    Step 1. Make a backup copy of your current document! _now!_

    Step 2. Make a list of all the duplicate entries in your bibliography.

    Step 3. Find each occurrence of that reference in your document and delete it.

    Step 4. Re-insert it.

    Theoretically steps 3 and 4 should only be necessary for the citations made before the move. If you know which ones they are, delete them first. The citations inserted after the move should be ok, I think.

    I would try this with one reference, then regenerate the bibliography and see if it fixed that duplicate.

    Then move on to the next. Unless you have many citations of each of these references, this (really tedious process) should take very long.

    But make a copy first.

    Let us know if that works. If not, I hope someone else has a fix.
  • dinosgr
    edited July 14, 2013
    First of all THANK YOU, I really appreciate your help.

    I made a backup. OK.

    I am now working on the file.

    I went through all the entries my bibliography. I am having about 50 duplicates. When I came across the first duplicate I just used the CTRL+F function to search for all instances of this citation in my document. I just deleted the old citations and re-inserted them again choosing the respective citation STRICTLY from the "MY LIBRARY" section of the drop down menu (the RED insert citation window with the big Z to the left). I kept using the CTRL+F function to find the next instance of this citation. HERE IS A TIP: I just had to remove ONLY the citations with the [year]b (with the b letter attached next to them), NOT ALL of them which saves me time).

    After replacing all the "b" versions of my citations all over the document I went to the bibliography and THANK GOD I can see that the first duplicate had been removed. I will keep doing this for the remaining 49 duplicates and let you know how it goes.

    IT SEEMS to be working for now. Thank you arggem.

    I only have to replace (remove and reinsert) only the "b" versions of my citations - this is key I think).




    EDIT/// What I wrote above is not accurate. I had to replace the "a" versions of my citations and when I did this the "b" versions disappeared. So I had to replace the a's, not the b's!
  • dinosgr
    Dear friends,

    Thank you all for you help.

    I managed to remove all the duplicates after a 2 days of intensive work using the method you told me. I really appreciate your help.

    I have one last question:

    How do I add a blank line between references in the bibliography?

    THANK YOU VERY MUCH
  • DWL-SDCA
    Some citation styles automatically add a new line between each reference. You said that you were using one of the Harvard styles. Most of the Harvard styles (with the exception of Harvard - University of Gloucestershire and Harvard - University of Limerick) include a line between references. If you must use one of those styles but need the line space, you can follow my suggestion below. Or, you can select another of the Harvard styles in the repository.

    You should be able to adjust the spacing of "paragraphs" in your bibliography by making changes to the style settings in your word processor. After a quick look at the thread postings above I didn't see you specify the word processor you are using. Someone can probably be more specific with instructions if you let us know.
  • dinosgr
    Thank you my friend.
    I am using microst word 2007.
    The Harvard citation style that I am currently using does NOT add a blank line between references. I want my references to be separated by a blank line.

    Could you please tell me how I can add this blank line (microsoft word 2007)

    THANK YOU VERY MUCH
  • adamsmith
    You'll need to change this in the citation style or switch to a different one.
    If you do need to change the citation style, you'll need to add/change entry-spacing="1" in the line starting with <bibliography...
    general instructions here:
    http://www.zotero.org/support/dev/citation_styles/style_editing_step-by-step
  • Simon
    You could also modify the bibliography style within Word to change the spacing after each paragraph to something like 12pt.
  • dinosgr
    Thank you all very much. I owe you much.
  • dorotajaw
    Hi guys,

    I'm having exactly the same problem as dinosgr - I've just moved my library to a new computer, and some of my citations get duplicated. First, I imported the bibliography using the 'import' key, but - having read your advice - I deleted it, and tried moving it once again, and this time I think I did it the right way. The items get duplicated anyway, and Zotero is taking really long to process them.

    Can anyone help?
  • bwiernik
    @dorotajaw You shouldn't use any sort of import function to move a library between computers; Zotero treats imported items as new items, which will break Word citations. Instead, follow these instructions to transfer a library:
    https://www.zotero.org/support/kb/transferring_a_library
  • dorotajaw
    Thank you so much for your response, @bwiernik!

    I checked both ways of transfering the library. When it comes to Option A (copying the library folder), once I did this, I could not open any of he files (as if Zotero computer did't see them), so I had to try Option B (library syncing).
    This works pretty well (the files are there, and Zotero can 'see' them).

    Unfortunately, this has not solved my problem: I still get unwanted duplicates.
    I honestly have no clue as to what could be the reason for this :(
  • bwiernik
    Please don't worry, we are just in the middle of troubleshooting is all. Okay, now that you have your library synced over properly, in your Word document find a citation duplicate. Edit the citation, then click on the blue bubble for the reference in the Edit Citation window. Do you see an option to open the item in My Library (or another library)? What about for the other version of the item?
  • adamsmith
    The way I understand the problem, the most likely reason for the duplicates is actually that the imports synced.
    @dorotajaw if you look at the date added for the duplicate items, is one of them from about the same time for all duplicates?
  • dorotajaw
    Dear @bwiernik and @adamsmith, thank you for your responses!

    @bwiernik, I'd best use an example to illustrate what's going on. The problematic citation is a reference to Sperber and Wilson (1986).

    After adding a new citation for this item on my new computer, some earlier citations of Sperber and Wilson (1986) remained to be in the form: "Sperber and Wilson (1986)"; however, some more recent ones turned into "Sperber and Wilson (1986a)", and only the newest - freshly inserted one - became "Sperber and Wilson (1986b)". Strangely enough, in the References list I only have info for two, and not three - Sperber and Wilson 1986a and for Sperber and Wilson 1986b.

    In my original Zotero database (on my old computer - Windows), the pdf is tagged with two dates:
    [1986] 1995 - the 1995 version is the new edition of a book first issued in 1986.
    I've just noticed that the database on my new computer (Mac OS X) does not have the 1995 date at all - as if it somehow got lost in the syncing process.

    Now, in a substantial part of my LibreOffice document, the format of the citation is perfectly okay - it's Sperber and Wilson (1986). When I click on the blue bubble, as you recommended, I cannot open this item in my library, and what is written in the blue bubble is Sperber and Wilson (1995) (date for the new edition instead of the old one). The same happens for those citations that are marked Sperber and Wilson (1986a) - no option to open the pdf in my library, and the blue bubble goes Sperber and Wilson 1995.

    It is only in the citations that go "Spearer and Wilson (1986b)" that I can open the pdf file, and the date in the blue bubble is correct.

    @adamsmith, I'm not sure - where can I check for the dates you wrote about? Have been looking for these in the document and in zotero, but couldn't find them. Could you please let me know where to search?

    Because of these problems - thinking that they were caused by library syncing, or perhaps incomplete (?) syncing - I worked on my old computer today, but had a similar problem with another citation. Had to delete all citation of the same item manually from the whole text (as well as the references list), and re-inserted them again. This did the trick, but I don't think I'll be able to do this with all the other citations. Also, I have no clue as to what it is that I'm doing wrong - these problems have to be caused by something.

    Once again, thank you so much for your willingness to help me out with this!!!
Sign In or Register to comment.