Syncing after database problems [Debug ID: D938691295]

mc

I few weeks ago I had some database problems with 5.0 beta, which were fixed. A couple of days ago I tried a data sync but about 250 conflicts came up. After resolving 2-3 conflicts Zotero would crash and on re-start the exact same conflicts would re-appear.

I decided to delete the online database entries (which I have since found out is not the thing to do) and sync by resetting the data sync history because this should(?) merge the local database and blank online database or overwrite the online database. Instead, items were deleted from my local database (was the online version seen as the most up-to-date version?).

A second problem became apparent when I looked to replace zotero.sqlite with a previous backed-up version -- Zotero is not creating daily backups of zotero.sqlite (I checked the backup settings and they are the default, 1440 minutes and 2 copies). The last backup was from at least a month previous. I have sent in a debug ID for this: D938691295. Luckily, I had an external backup, so nothing was lost.

Can my online database be "reset" so I can sync again? How do I get Zotero to backup zotero.sqlite?

dstillman

A second problem became apparent when I looked to replace zotero.sqlite with a previous backed-up version -- Zotero is not creating daily backups of zotero.sqlite

This seems to be working for me — I have an automatic backup from earlier today — but I'll take a look at this.

Can my online database be "reset" so I can sync again?

I don't really understand the issue you're trying to resolve here. Can you explain what you're trying to do?

dstillman

You're saying that when you sync all items are deleted (because you deleted them online), and you want to undo that? If so, try adding all items into a new temporary collection before syncing — that should mark the items as changed, which should generate conflicts, letting you choose all local versions.

The "Restore to Zotero Server" option that existed in 4.0 isn't currently available in 5.0.

mc

Sorry for the delay in getting back to you.

Syncing: When I sync approx. 4,000 out of 4,200 items are deleted including attachments. When I added all items to a temporary collection, I have the items but the attachments are still deleted (I do not save my attachments online). I made sure to select local for conflicts.

Backup: Still not getting a backup of zotero.sqlite

dstillman

OK, different idea: instead of adding all items to a collection, create a new temporary tag, assign a color to it (right-click -> Assign Color) and note the associated number key, press + to expand all items in the middle pane, Select All, and press the correct number key on the keyboard to assign the tag to all the items. They should then all be marked as having changed, not just the parent items.

dstillman

Re: backups, try setting extensions.zotero.backup.interval to 1, enable debug output logging, and then leave your computer idle for 5 minutes. It should create a backup, and the debug output should show this happening. (It might be easier to start Zotero with real-time debug output enabled so you can see when the five minutes are up.)

mc

Sync:
Adding the temporary tag worked. I added a step because Zotero became non-responsive: close and re-start Zotero so that zotero.sqlite-journal was written to zotero.sqlite. As a result of all this, my database has grown from 17,287 KB to 28,081 KB without any new data. The subsequent sync, when I put my database back to the way I wanted it, took a long time (about 10 minutes).

Backup: I changed extensions.zotero.backup.interval to 1 and waited (Debug ID D1371298406). No backups created.

dstillman

The size increase is expected — 5.0 stores a copy of every item it uploads, which helps it resolve conflicts automatically during future syncs. We might be able to store those compressed in the future to reduce the file size.

Re: backup, that was waiting 5 minutes? If so, I'd guess that something on your system is preventing Zotero from properly detecting that the system is idle. Zotero waits for 5 minutes of system inactivity and then checks whether the last backup is older than the interval. I just tested this on Windows and it does work.

mc

Sync: I spoke too soon. Although all my items and attachments were present after syncing, approx 1,000/4,000 items were categorized as"unfiled items". So it worked and didn't. I'll try again later.

Backup: are there any hidden preferences that could cause backups not to work (besides extensions.zotero.backup.interval and extensions.zotero.backup.numBackups)? Otherwise I will use a new profile and see what happens.

Thanks for your patience and help.

mc

Sync: To get sync to work I eventually had to edit the zotero.sqlite database to change the various "date modified" fields to a more recent date/time than was in the online version. I know, not recommended, but it worked.

Backup: A new profile solved this problem, there may have been junk hanging around that was causing the problem.