Opening Zotero -it takes more than 50 minutes-

Yesterday I began using the Zotero Standalone version after using the Firefox Zotero version since 2012. In the transition to one version to another, I realised that Zotero is taking now much much longer to open and upload all my data. More than 50 minutes each time I close the program and I am opening it again. I also realised that sometimes the program is very slowly and it needs time to begin to work properly -five minutes or so- and that is able to block more easily than before. I have a big database -33.000 references with lots of images and PDFs attached- but that was not a problem with the Firefox version, that was opening and uploading all the data usually after 2 minutes. Could anyone tell me if they are experiencing the same problem? Does anyone know if this is normal or in case is not, how to improve it?
Thanks a lot
  • No, that's not at all normal, of course. Make sure you have Zotero 5.0.25, and if it's still slow with that submit debug output for Zotero startup using the "Restart with Logging Enabled" option from Tools -> Debug Output Logging and post the Debug ID here.

    And just to check, is your database on a local hard drive, or is it on some sort of network share?
  • Thanks a lot for the answer and the fast help; so much appreciated. The problem is still there with Zotero 5.0.25. The database works now faster and better, yet still taking a long time to upload and be opened.
    My database is, as it was before, on my local hard drive. It has never been in any kind of network share.
    Now, I cannot find the "Restart with Logging Enabled" option that you told me to activate. In my Tools bar only these options are available:
    Create Timeline
    RTF Scan...
    -----
    Install Browser Connector
    Add-ons
    Thanks again


  • Actually, I have found the option you told me about. It was at the Help option.
    The Debug ID is D2147460991.
    Thanks a lot for taking the trouble!
  • edited November 13, 2017
    Yep, sorry, meant Help.

    It looks like you interrupted that before it was finished loading the library data? We'd ideally want to see the whole thing.

    But there's definitely something odd going on there. It loads most item data in about a minute, but then an attempt to load notes in your library had taken at least 2.5 minutes (and possibly much longer) by the time you cut it off.

    If you do get it open, how many total items do you have, including child items?
  • edited November 13, 2017
    Can you also check your database integrity from the Advanced → Files and Folders pane of the Zotero preferences?
  • Yes, probably I cut it off; sorry about that. It took 1 hour to upload everything again. I'll be doing the same "Restart with Logging Enabled" thing tomorrow -it's late now here- and I'll send you the Debug ID as I did before. I hope this time I will not cut it off -I think I did it no noticing it.

    I have a total 53451 items, including child, notes, PDFs, images and so on.

    I checked the Database integrity and nothing happened. Apparently is all right?

    Thinking about some of the Notes attached to some items, I noticed that some of them have lost the images I have included there with some words. These Notes show only a 'broken image' icon, indicating an image was there but not any more. Could this be the problem in getting the notes uploaded?

    Thanks a lot for taking the trouble. It means a lot for me, since Zotero is, truly, my entire professional research life.
  • I checked the Database integrity and nothing happened. Apparently is all right?
    It should tell you when it finishes.
    Thinking about some of the Notes attached to some items, I noticed that some of them have lost the images I have included there with some words.
    Oh, if you have embedded images in notes, that could definitely be the problem. Those have never been supported, and it's just been an unfortunate fluke that it's been possible to add them. They've always produced sync errors, but if you're not using syncing you wouldn't get those errors.

    I'll have to look into why they're showing up as broken, though. As far as I knew they still displayed in 5.0, because even though we now have the ability to block them, we decided to still allow them until we could properly add support for embedded images.

    As for the immediate issue, I'll see if there's anything we can do to speed this up.

    How big is the zotero.sqlite database in your Zotero data directory?
  • Strangely, the Database integrity tool does not report anything. Nothings happens. Maybe is not working properly?

    I feared that... about the embedded images. I got many of them. They are very useful mixed with the text in the Notes. I think you should recommend not to do it though, to avoid further problems.

    My Zotero.sqlite is 1,35 GB.

    Thanks a lot for all the help. I love Zotero and I have been given seminars about how good is for research. It would be a tragedy for me not be able to use it as before.
  • As for the embedded images, I have to say that most of the Notes containing them are still having them. Only a few Notes have the 'broken image' icon. And I think these correspond to images that were uploaded with the .jpg format. The others, which are the majority of the images I uploaded to the Notes, are Screenshots, not .jpg images. I do not know if this helps, but I wanted to share with you this.
  • Strangely, the Database integrity tool does not report anything. Nothings happens. Maybe is not working properly?
    It's probably just still running. A 1.35 GB database is pretty massive, so that could easily take a long time to run.

    The size is likely from all the embedded images. The full-text index can also take up a lot of space, but we almost never see databases that size. (If/when we support embedded images properly, they won't be stored in the database.)

    I've created an issue for this, and we'll keep you posted.
  • Thanks a lot for the info. I started to use embedded images I think in the last two years. Should I stop doing that?
    The thing that I do not understand is why is taking so much time to do upload all the data in the Standalone version when three days ago the Firefox version was uploading everything in just less than 3 minutes.
    Anyway, I do hope we'll be able to find a solution to this problem!
    Thanks a lot
  • It's not "uploading" (which would refer to syncing) — just loading. Because of technical changes around database access in the underlying architecture, Zotero 5.0 loads all note content at startup. Normally this isn't a problem, even with a huge number of notes, because text doesn't take up much space, but if you have hundreds of megabytes of embedded images, it would be a problem. We don't test with embedded images, so we didn't see this.

    But we'll see if it's possible to defer image loading until the notes are accessed, as was the case in 4.0.
    I started to use embedded images I think in the last two years. Should I stop doing that?
    Well, the current architecture definitely isn't designed with that in mind. We do hope to have proper support for it in the future, since we know some people find it very useful. But if you're not syncing, and are taking regular backups of your database (with historical copies), and are OK with the (other) general slowdowns it causes (viewing those notes, searching), you could probably keep doing it. When we do add proper support, we'll try to migrate existing embedded images into the new format.
  • Thanks a lot for the explanation. It's fascinating. I can tell you that the combination of images and texts when creating Notes about the references -specially manuscripts and art items- is extremely important methodologically speaking, specially for people in my field.
    Anyway, I hope we can get a solution for the present problem sooner than later. Do you think I could hope a less slow loading -thanks for the clarification- in a reasonable time? As you can imagine, waiting one hour to get the database is not ideal for research!
    Anyway, I am very happy that, it seems, we have identified the problem and that I now understand what is going on. So good to know where the problem is!
  • I am running again the "Restart with Logging Enabled" . Here is the Debug ID: D62805800.
    I hope this time you can see the whole process of loading the database.
Sign In or Register to comment.