Problems with sync (Zotero Standalone 4.0.1)

uGLJJINKYvnyAGYRcjBj

Report ID: 1119678734

Every time when I click the sync icon in Zotero Standalone 4.0.1, I receive after a few minutes the message "Invalid response from server.Please try again in a few minutes." I did wait for about 10 minutes for several times, I also tried restarting my computer, but every time I get the same error message. The Zotero Error Report mentions eight times the exact same error:

[JavaScript Error: "(null)" {file: "moz-nullprincipal:{12d82b16-6a00-4eb4-a8b3-f6b97e937ddb}" line: 33 column: 37 source: " title="Zotero Reference Test pane">"}]

If I check my library on www.zotero.org, it says: "No items found". (I started syncing only yesterday, but there has not been one single successful sync yet.)


I would appreciate very much if you could help me; thanks in advance.

dstillman

Did you generate that Report ID after triggering the sync error and before restarting Zotero? There's no relevant error in there.

If so, can you restart Zotero and provide a Debug ID for a sync attempt that produces the error?

uGLJJINKYvnyAGYRcjBj

Thanks for your response. Yes, I did generate the Report ID after triggering the sync error and before restarting Zotero.

I now followed your advise, which also referred to the steps as outlined on the Debug Output Logging page to which you linked:

I enabled Debug Output Logging in Zotero Preferences, restarted Zotero, tried a sync, uploaded the outcome after the sync stopped (by clicking on "Submit to Zotero Server") and received this Debug ID:
D965103045

dstillman

How many items are in your library, including child items? Is your trash empty?

uGLJJINKYvnyAGYRcjBj

4946 excluding child items, 20453 including child items (some of them including hundreds of words). Trash is empty. Database integrity is OK. It would be the first time to sync this library, but until now without success.

uGLJJINKYvnyAGYRcjBj

In the meanwhile I tried a workaround, and as it seems with success: I did a RDF-export of my library, closed ZSA, moved "zotero.sqlite" and "zotero-sqlite.bak" to a backup folder, restarted ZSA, imported the RDF-file, and now the library syncs without any avail. I do not seem to have lost any information, although zotero.sqlite is now only 19,1 MB (it was 25,8 MB).
Can I trust that I did indeed not lose any information or fields through the RDF-export-import method (is it 100% reliable)? It would be really cumbersome, of course, to compare more than 20000 items against the original database ...

adamsmith

It's not 100%, no, though pretty close.
I'm not exactly sure what the limitations are. The two issues I'm aware of are 1. the "related" info isn't exported/imported via RDF and 2. old Word/LibreOffice documents you have created using the database will no longer be recognized.

uGLJJINKYvnyAGYRcjBj

Well, I did not use the "Related"-function yet, and I think that I can live with the second issue (I only recently switched from EndNote to Zotero).
Any ideas about what might have triggered the sync-problems before the export-import intervention?

dstillman

This was quite likely just a result of 1) high server load earlier, due to a bug in 4.0, and 2) lower server load now, due to our fixing it. Exporting and importing is almost never something we recommend for troubleshooting, and it's unlikely that it fixed this. (It's not impossible, if, say, you had previously deleted a huge number of items, but it's not likely.)

uGLJJINKYvnyAGYRcjBj

Thanks, Dan. I will try to sync the original library (which, I realize, involves several steps in order to get the library in the cloud empty first), if exporting and importing probably did not fix the problem and is not recommended. No, I did not delete any items before, but exported, imported and synced the same amount of items.

dstillman

Well, wait, if you've already synced, you don't want to sync again with the original library—you'll just end up with duplicates.

dstillman

You can do Restore to Zotero Server first to clear the server library, though.

dstillman

OK, now you appear to have manually deleted all items. You don't want to sync directly into that library. You'll want to do a Restore to Zotero Server from the original library, if you want to try to sync with that. (I wasn't really saying to do this, for what it's worth. If your previously exported/imported library seemed fine, there was no particular reason to erase it. That's more just general advice, particularly for existing libraries.)

Really, I'd suggest that you just pause for a moment here and let us confirm that what you're doing is the best course of action.

uGLJJINKYvnyAGYRcjBj

Thanks for your concerns. I am afraid that I saw your warnings too late, as I already started syncing the original library with Zotero Server. But I have kept backups of the original library. If anything goes wrong I will do the the Restore to Zotero Server, as advised, but I first await your confirmation, OK?

dstillman

You have nothing queued right now, and no items on the server. I've also cleared the delete log in your library, so you can now just do a normal sync of the library and it should be fine—no need for the Restore.

uGLJJINKYvnyAGYRcjBj

Well, syncing the original library ended with an error. All items are still in my library on the computer, and the library on Zotero Server is still empty.
I also made a copy of the library that resulted from the export-import operation and did sync well, before I went back to the original library.
What would be your advise to do next? Retry first (I am not sure whether ZSA was still syncing when you consulted my library on the server)?

dstillman

Well, syncing the original library ended with an error.

Report ID?

uGLJJINKYvnyAGYRcjBj

Report ID = 980564419

dstillman

First, upgrade to 4.0.3, though it probably won't help with this.

Next, I'm not sure why the errors aren't making it into the Report IDs, but how about another Debug ID for the sync attempt?

uGLJJINKYvnyAGYRcjBj

I upgraded to 4.0.3, enabled Debug Output Logging, triggered a sync (which ended with an error) and submitted the debug output to Zotero server. ID = D1905522012.

dstillman

Can you try again now? It might still fail, but I'll have more debug information. (No need for more from you.)

The servers are still under high load, which is likely the underlying cause, but the fact this is still failing suggests that there's a lot more data being uploaded from this library than from the exported/imported one, which is curious. I should have some more info after your next attempt.

uGLJJINKYvnyAGYRcjBj

I tried again, and it failed indeed. Debug ID is now D2065488339.
When I browse through the library, I cannot see any differences compared to the exported/imported one. Plus the number of items (including child items) is exactly the same.

dstillman

Sorry, my mistake—the changes I made weren't properly in place. I'll need one more attempt. No need for another Debug ID.

uGLJJINKYvnyAGYRcjBj

No problem. My next attempt ended again with an error ("invalid response from server"). What to do next?

dstillman

Your upload is going right now, though it's possible it won't finish.

The reason you're having trouble with this library—and why I asked about this above—is that the upload contains nearly 100,000 (yes, really) deleted items, and 12,000 deleted tags.

uGLJJINKYvnyAGYRcjBj

Oh, I see. When transferring from EndNote to Zotero last week, I deleted several times the imported collection until I succeeded into modify the Refman(RIS) Export filter in such ways that I got results (there are many errors in this filter; it almost seems as if EndNote deliberately discourages switching that way). I also deleted tags. BUT: I emptied the trash. Thus emptying the trash does not actually delete items, but only makes them invisible?
If this data bank is now screwed up this way, wouldn't it be safer if I create a fresh zotero.sqlite first, re-import the original items from EndNote (I kept a copy of the exported file) and then sync? Or is there another way to get rid of the 100,000 deleted items and 12,000 deleted tags?

dstillman

The data is gone, but Zotero keeps a log of the item keys in order to propagate the deletions to other machines. The reason Zotero syncs these, even if the server library is empty, is that it doesn't know that you didn't, say, copy your database to another computer before deleting the items, such that the items still exist there. However, I think we actually just ignore the deletions on the server if the objects don't already exist there, so the deletions wouldn't be synced to the other computer anyway, and we could probably skip uploading these (though only if the server library was empty).

In any case, we can probably optimize this better with the sync overhaul coming in Zotero 4.1.

We'll see if your current upload goes through as is. If not, Restore to Zotero Server has the side effect of clearing the local delete log, so that should be an easy fix.

I'll keep an eye on the upload and let you know if it fails.

uGLJJINKYvnyAGYRcjBj

Thanks for your explanation. Does this log of item keys also account for the size difference between what I call the original library (which is now again in Zotero) and the exported/imported library (RDF-export)? If so, will this increased size have no disadvantages in the long run if the collection further grows, and wouldn't that then be a good reason to start over again with a fresh zotero.sqlite? Or will a Restore to Zotero Server lead to the same results?

uGLJJINKYvnyAGYRcjBj

I am now starting the upload, as Restore to Zotero Server remains an option that will not be gone after that, but I would appreciate if you also could reply to my last post.

dstillman

The upload went through, so you should be all set now.

Zotero clears the delete log after a successful sync. The file size might not go down automatically (and/or immediately), but it's just empty space that will be reused as necessary and that won't have an effect on performance. (If you're particularly concerned you can run your database through the DB Repair Tool, which will rebuild it.)

dstillman

Uh, starting what upload?