Interminable delays--even on new superfast computer-

tomfjackson

Debug Ids: 1984823225 and D1169039600
Dear Zotero Techies,
I just bought a new computer hoping to speed everything up: Core i7 + 24 GB memory, 256 GB SSD + 2 TB 7200 SATA.

But Zotero still freezes Firefox constantly, especially when I try to search "My Library" which has 8870 items. (the entire library is 7.8 GB which seems quite large to me).

I copied the entire library from my old computer onto a "Zotero" folder on D: drive, the 2 TB hard drive. Deciding to save space on the SSD.

The only thing I can think of is to put it all into the profile directory on C: the SSD Drive. But I don't even feel confident closing this program because it constantly shows that it is syncing with the server (Getting updated data from the sync server). When the sync icon stops, I get an upload error message (my error messages have been sent).

Help! I spent a year building this library, bought a new computer because of interminable freezes in programs, and here is the same problem $1400 later.
Thanks, Tom

aurimas

When the sync icon stops, I get an upload error message (my error messages have been sent).

You have to tell us what the Error Report ID was. Also helpful if you tell us what the error actually said.

I just bought a new computer hoping to speed everything up

That's a pretty extreme first step in trying to troubleshoot something (unless, of course, you wanted the new computer anyway). The size of your library is not large enough to cause tremendous slowdowns even on a less powerful computer.

But Zotero still freezes Firefox constantly

Does Firefox recover eventually? Do you end up closing Firefox? How long do you wait?

If Firefox is able to recover, could you produce a Debug ID for an action that causes the temporary freeze-up? https://www.zotero.org/support/debug_output

adamsmith

A couple of things: What's your total number of items: https://www.zotero.org/support/kb/item_count (I know you say 8870 items, but I'm assuming that's top-level items?)

If you turn off auto-sync, do you get the same problem?

What do you mean by "interminable"--are we talking literally interminable, are we talking quasi-interminable (like 10+mins) or are we talking "feels like interminable" because it's so annoying but it's maybe 10secs or so? None of that should happen, but troubleshooting would differ.

If you do have a lot of full text attachments and use quick search to search for them, you will get a brief freeze (though should be seconds, not "interminable"), so how about when you change the search settings to "All Fields and Tags"?

(Getting updated data from the sync server). When the sync icon stops, I get an upload error message (my error messages have been sent).

You need to actually start a thread for those. Error report IDs that aren't followed up with a forum post referring to them are never looked at.

tomfjackson

Report ID: 1984823225 (from latest sync error). It looked like the program was trying to upload 4.7 GB to the server, after adding maybe 10 news articles and deleting several groups.

I also turned on Debug output logging. Here is the ID:

The Debug ID is D1169039600.

aurimas

What was the error for the sync when you clicked the exclamation mark?

tomfjackson

"A file sync error occurred." then instructions on sending the report. Working on things you mentioned.

tomfjackson

8870 items unexpanded
21198 items expanded to include pdfs and notes. Thanks.

aurimas

I also turned on Debug output logging. Here is the ID:

Was that for an operation that resulted in a freeze-up? What was the operation? And since you submitted the log, the answer to our previous questions is that Firefox recovered from the freeze? How long did it take?

tomfjackson

That operation was a sync error.
Firefox freezes for a minute or so each time I type a character into the search box.
After ten minutes of syncing "data" an upload starts (I got to 12% before the (!) error message, but why would it be trying to upload 4.7 GB of data?

dstillman

Try closing the tag selector in the bottom left of Zotero, particularly if this new computer has a touchscreen — there's a known issue where that can slow down large libraries.

But that'd be if normal operations in Zotero (editing items, switching collections) are slow, which shouldn't happen. Searching a 21,000-item library may not be that speedy, but you can make it faster by entering " before the search to turn off search-as-you-type and also by using Title, Creator, Year instead of the other search modes.

adamsmith

but why would it be trying to upload 4.7 GB of data?

That seems to be a full file sync for whatever reason, could be because you never performed one before. That does occasionally fail, but will likely complete if you just let it auto-sync for a couple of days, since file syncing is incremental.

tomfjackson

Tag selector has been closed. " " does speed things up. I turned off auto-sync.

This upload issue is the larger one now. Syncing goes on for a long time, then you get the green bar, which hangs up at 12% of an enormous upload that it seems to me should not be happening.

Would it help to move the library from D: to C: within the Mozilla profile? The library has no problems in the old computer.
Thanks! Tom

tomfjackson

I am thinking of using the SSD. Please advise. TJ

adamsmith

No, the sync isn't affected by where you have your data folder, you can save yourself that trouble (that said, like pretty much anything else, database access and thus performance is going to be faster from the SSD. But I'd think marginally so).
Did you use file syncing before? Are your files synced online? Do you want them to be? Do you have 5+GB of Zotero storage?

tomfjackson

1. Getting updated data from sync server --- lasts for 30 seconds now.
2. but it is currently trying to upload 4.2 GB to the server, that is 10245 files. Why is it trying to do this? Progress is at 5%. Turned off touch screen.

adamsmith

I answered 2. 15mins ago.

tomfjackson

Okay. Perfect. Thanks! Whew. That was a more modest 800 MB upload. If I move to SSD, should I chose the profile directory first, then close, then copy? Tom
And I don't use the touch screen for firefox or zotero (I have 3 monitors, so if touch screws something up, I'll move Zotero to the other non touch monitors)
BTW got the new computer as well for photo processing and voice dictation).
Best, Tom

adamsmith

If I move to SSD, should I chose the profile directory first, then close, then copy?

yes, though really I wouldn't expect much speed-wise, so if things are OK now I'd just leave it and save space on SSD.

tomfjackson

I still get hung up for 5-30 seconds when I type something in the "All fields & tags" box. Is this library just too dang big? Should I put all searches in " "? Thanks for your help. I'll let you know if I go SSD and what the results are. Tom

adamsmith

you just need the " at the beginning, you don't need to enclose the search. All that does is to prevent the search from starting while you type. It's possible that's due to the size of your library, yes. It's a pretty massive search at that point.

DWL-SDCA

Re SSD -- no noticeable difference

We have 2 otherwise identical Mac Pro (not the new model) machines except one of them has a 512 gb SSD instead of a fast 512 Gb HDD. Without going into further details, these are production machines and only have Zotero, Zotero SA, Firefox, Chrome, BBEdit, and Transmit programs installed. Zotero tag selector is closed, Auto Synch is off. At most, a complex full search or clearing the trash is 2 to 3 seconds faster on the SSD. That said, sometimes it feels like the HDD machine wins the empty the trash race by a moment or two. The Zotero contents: 3500 to 4000 records -- no attachments no notes.

We bring in the items to Zotero, edit the Zotero records, export them in MODS format, and trash the records. The MODS file is uploaded to a bibliographic database server to be further edited and indexed. The records are then made available to the public.After changing production to include Zotero, production time was cut by more than half. Zotero is wonderful for this purpose. (I also use Zotero for my writing. My doctoral thesis has 410 references.)

tomfjackson

I copied over to 7.8 GB to SSD and have just a bit more speed too. Perhaps worth it to use this out of the 256. This is a newspaper article database downloaded from ProQuest with about 8000 pdfs for my second book on civil rights in 1963 and the media (some overkill with research assistants, but ripe for coding).

I wouldn't know how to do it otherwise. I won't be adding unpublished archival pdfs from my photos to Zotero. Using evernote for that.
I just have to avoid typing in the quick search box when in "My Library" without " first.
Otherwise this has become much much faster. Thanks.
Don't know why I was uploading the whole thing again, but thanks for figuring that out!

At some point I'll want to take this offline but will still want access. Looks like setting up groups and disabling syncing keeps the records on the server.

aurimas

So when you use the quick search, are you searching "Everything", "All Fields & Tags", or "Title, Creator, Year"? Not sure what you _want_ to search for, but typical searches can probably be limited to "Title, Creator, Year" or "All Fields & Tags", which would exclude file attachment contents and should show a significant speed-up. If you need to search file contents though, then I guess there's not much you can do.

tomfjackson

No I am searching cities, such as "Danville" in All Fields and Tags, and it still hangs up after typing only one character (now its 15 instead of 30 seconds). So I'll just have to use " before all searches now.