most of my citations are invisible, but zotero thinks they're there!?

edited January 3, 2021
Hello! I just imported my Mendeley library (via .bib, not by clicking the Mendeley import, because when I clicked on that I couldn't figure out where to find the library) into Zotero today. I'm on a Mac running Mojave, and I can't upgrade to Catalina or Big Sur right now for various reasons, so that's not an option.

I had 436 references in my Mendeley, and Zotero itself is saying "436 items" in this view if I'm in my library. But only about 50 items are actually viewable in the list of citations, and if I search for a reference I know should be there (by typing in the author's surname or a word in the title, for example), it says there's nothing with that keyword. So how can it be that it knows 436 references are there but somehow at the same time DOESN'T know more than the 150 or so that are showing up. I have seriously scrolled through a million times, both by mouse and by clicking on one and then the down arrow to view each one. They're not all there. If I find the directory on my computer (Users>my full name>Zotero), it says the entire folder is 16.3mb, so it's not a problem with running out of storage. It also says there are 910 items, but maybe "items" are not the same thing as references? I didn't change any of the paths or anything when I set up zotero after I installed it.

I'm so confused. I have checked documentation and also searched the forums and I really can't find anything addressing this. The version I downloaded was the one available on the download button, so that's 5.0.94. Because I'm on Mojave, do I maybe need to downgrade to version 4? I can't imagine that would fix anything since it claims to be aware of 436 references, but....idk. Help?
  • Could you take a couple of screenshots that illustrate this, upload them somewhere (dropbox, imgur,com) and link to them?

    Might be that you're looking in the online library rather than the tool, might be that you're looking in a specific collection, maybe something else entirely. Invisible items don't exist in Zotero and I don't think anyone has reported what you're describing.

    Mojave shouldn't be a problem -- Zotero is compatible with all versions from El Capitan (10.11) onward.
  • Sure! The screenshots are here: https://app.box.com/s/heoehpra9jy7fs4rmj6bqwv33i0vtwpt

    They'll show that whether I am in the main part of the library or the "unfiled" part, it says there are 436 items, but far fewer than that appear in the middle column. I also put a picture of what happens when I search for something I know should be there (an article by an author with the last name Doyle) but isn't showing up on the main library.

    Thanks for your reply!
  • I'm not really clear on what you think this is showing.

    Just with the two screenshots you provided, you've shown something like 80 different items, and that's just the very top and bottom of the list. Based on the size of the scrollbar — and the fact that you're showing only authors from A–B and T–Z — there are pretty clearly several hundred more in the middle.

    If an item didn't import from the .bib file for some reason, we can help you debug that — If you want to post the .bib file somewhere and provide a link to it, or (if you don't want to share it publicly) email a zip of it to support@zotero.org with a link to this thread, that'd probably be easiest. But there's absolutely nothing wrong with the item count it's showing you.
  • Hmm -- this looks right. I'm counting about 40 items just on the first screenshot and that's not even all the way through first author B: items with first author A-Br make up about 10% of my (significantly larger) library, so that'd be in line with ~400 items total.

    Some things to try:
    - I assume you've quite and restarted Zotero?
    - Have you synced and looked how things look online?
    - Do you have a sense which items are missing? (as in, is there anything systematic?)
    - Relatedly, have you scrolled through the items and checked/counted?

    The oddest thing is the search that doesn't work -- I have no real explanation for that. It does work, I assume, for items you do see?
  • edited January 4, 2021
    Yeah no, in total if you scroll through them all on the screen it's showing about 240 of them. But it claims that all 436 are there. So yes, like I said, I have searched for specific items I know should be there (based on my memory and the fact that they are in my Mendeley) and they're not showing up. If I search for something I can see plainly, then yes, it does pull those up. It seems like the ones that are missing are more often items that aren't books or journal articles--e.g. Twitter threads, blog posts, magazine articles, webpages....but with 200-odd missing items, I can't say for sure if that's the rule. Do you think that's the problem? One of the reasons I wanted to quit Mendeley was that it has very few format options and my research has a lot to do with social media and other internet-specific culture, so I liked that Zotero lives in this century and understands that a blog post is different from a webpage, you know?

    Quitting and reopening Zotero didn't change anything, and the online version, when synced, also claims to have 436 items while only showing that same 240 or so.
  • I don't think we've ever seen Zotero miscount the number of items so this is quite confusing. Could you try this:
    Click on My library, make sure the 436 count is showing, then in the middle panel select all (cmd+a), the right-click/ctrl+click --> Create bibliography from selected items and then select either Nature or IEEE style (both of which number the bibliography) and then copy the bibliography to clipboard and paste into a document (Word, Pages, Google doc, doesn't matter). Do you get a bibliography counted up to 436? Are the missing items there?
  • Okay, just tried Nature, IIEE, Chicago, and APA. They all yielded only the 240ish that were visible, even though Zotero tells me "436 selected."

    Honestly I'm just glad you're also confused. It makes me feel better about all this. I'm not a dunce, and I even have a master's in library science, so I like to think that if I can't figure something out, it's the software's problem, not mine. (I realize that's unreasonable; I'm not that great. But hopefully you get my meaning.)
  • WAIT.

    It IS showing 436 in IIEE. Word must just have been slow and I was too quick in coming back here. It does mean that some of my most recent adds (according to Mendeley) didn't make it on the export, even though I synced before I created the .bib file, but I'd rather manually add, like, five references than 200, so that's not the end of the world.

    It doesn't explain why they're not showing up in Zotero, only in the copied over data, but maybe that rings a bell for you even though it doesn't for me?
  • I think you're still misunderstanding this.

    The count in Zotero is not wrong — I promise you. If you can't accept that, post the screenshots of each page and we'll count them, but there are going to be 436 items.

    The items that literally aren't in Zotero won't show up in the search, obviously.

    So again, it's just a question of items not making it from Mendeley to Zotero, either in the export or the import. As I say, you can post the .bib for us to look at, or you can open it in a text editor and look at it yourself. If the items you're missing aren't in the .bib file, this has nothing to do with Zotero and you'd have to ask Mendeley for help.
  • edited January 4, 2021
    Okay, so there are 436. You win on that point. I apparently counted to 300 twice when I did it before. But I just opened the .bib, and the missing items (the total number of citations apparently adds up to 512 in my Mendeley), are indeed there, so something got bugged in the import, not in the export.

    So, apologies. I am dumber than I thought. Also, so is Zotero, because it screwed things up as well, and that's probably where I got flustered with numbers. You and Adam are on point, and I really am grateful!
  • Again, we'd need to see the .bib to tell you more. You can upload it somewhere and provide a link or email it to support@zotero.org with a link to this thread.
  • Box doesn't seem to be offering a way to download that file. That may be a setting you have to toggle.
  • Whoops. I think I set that. Try now.
  • @web_page{Doyle2011,
    Right, so this is, as far as I know, just something Mendeley made up. @web_page isn't a valid BibTeX type, which is why Zotero doesn't import it, and no other BibTeX tool following the specification would either. The problem here is Mendeley.

    As a workaround, you can try selecting items of those types in Mendeley and exporting to RIS instead.
  • I'll try that. Thank you so much! Can I buy the two of you a coffee or something?
  • You could also do a search&replace for @web_page to @online (which is valid BibLaTeX) and which Zotero will import.
  • Nor is it valid biblatex.

    The newest BBT will accept @web_page and will treat it like @online. Not the first snafu found on the mendeley bibtex export.
  • by the by, for those playing at home or anyone who has a similar problem and comes across this post, I finally got around to this (had to pause research to take care of job apps and work), and neither @ online NOR @ web_page seem to be readable. I had to do @ electronic before anything would go. I'm on Mac Mojave, Zotero 5.0.95.3. If I wanted to manually add a citation, there is nothing for webpage, online, OR electronic, so it's only if I have the file that I can do that at all.
  • BBT will parse all of the above.
  • It didn't. I imported five times, and nothing worked until I did @ electronic
  • Do you have a sample for that?
  • edited February 17, 2021
    Sample of which? The .bib I linked to above showed the filetypes I had, and the ones with @ online or @ web_page repeatedly would not import. When I did a find+replace and changed those to @ electronic, they imported no problem.
  • I added support for those entry types but didn't test on the whole file. I've added that file to my test suite; there was an invalid command in the bib file (\ud) that BBT tripped over. 5.2.118 will ignore unknown commands by default, and now imports all 512 entries in that bib file. But there are more wonky entry types in that file:

    • book_section
    • generic
    • newspaper_article
    • film
    • magazine_article

    I'll add support for those too if you have a moment.

    This was really output by Mendeley? I shouldn't be surprised really, but the invalid tex command is weird.

  • 5.2.120 will import those 5 invalid types too.
  • (if they imported no problem it is possible they were imported using the standard Zotero translator because BBT would have tripped up over the \ud command, unless you changed the setting from ignore to something else)
Sign In or Register to comment.