Zotero Database Corruption Solutions

innovation1
edited June 20, 2024
My Zotero database got corrupted due to an unintended computer shutdown while Zotero was open.

Even the *compressed* database sql file (640MB) is too large for the Zotero Database Repair Tool: https://www.zotero.org/utils/dbfix/

So I am now considering the Manual alternative solution: https://www.zotero.org/utils/dbfix/manual_

1. Can you confirm that these manual instructions for Windows users are up to date: https://forums.zotero.org/discussion/74334/recuperation-of-zotero-data#:~:text=Windows

Download the,the new database."

2. Is it possible to email the corrupted sql file to a Zotero specialist who can attempt to resolve the issue?

-------------
Further context:
-I'm using Windows 10 OS
-Only part of my library (entries up until Feb, 2024) appears to have synced online (For some reason, my library entries since Feb, 2024 are missing from my online library, but my collections folders, including those made yesterday, are all in the online library).
-I have a Zotero automatic backup of the database as it was in April.
  • adamsmith
    The instructions are generally up to date, but this is more finnicky in Windows than on Mac/Linux, given the poor terminal support -- so you might get an error and this won't run, but you definitely won't break anything in the process.
  • innovation1
    edited June 20, 2024
    Using GPT-4o, I was able to resolve the issue. But it has raised some concerns:

    1. My local Zotero library has stopped syncing with my online Zotero library, even though, in Preferences, it is configured to sync every time I change any data in the library.

    2. Zotero has not automatically backed up my database since April, although it ought to backup the user's database every 24 hours by default (https://www.zotero.org/support/preferences/hidden_preferences#:~:text=backup.interval,hours )

    Can you help resolve these issues?
  • dstillman
    1. See Changes Not Syncing.

    2. How big is your uncompressed database? Is your Zotero data directory in the default location on your local disk? Does your computer have an HDD or SSD? Your database is likely too big — perhaps combined with the speed of your computer/disk — for the automatic backups. I'd guess that it's timing out.
  • innovation1
    1. Issue resolved, but still raises questions.

    The issue with syncing was due to a simple "sync error" caused by the excessively large metadata of certain items: e.g., an extremely long book abstract, and an entry of an author's name (as displayed on ProQuest) which included dozens of alternative spellings of the author's name:

    "Bacon, Francis, 1561-1626 (Bacchon, Francesco; Baccon, Francesco; Baccon, Franciscus; Baco; Baco Verulamius, Franciscus; Baco de Verulamio, Franciscus; Baco van Verulam, ...; Baco von Verulam, ...; Baco von Verulam, Franz; Baco von Verùlam, Fr.; Baco, Fr.; Baco, Franciscus; Baco, Franz; Baco, von Verulam; Bacon de Verulam, Francois; Bacon de Verulamio, ...; Bacon de Verulamio, Franciscus; Bacon de Verulamius, Francis; Bacon of Verulam, Francis; Bacon of Verulem, Francis; Bacon of Verulem, Franciscus; Bacon von Verulam; Bacon von Verulam, ...; Bacon von Verulam, Franz; Bacon, ...; Bacon, F.; Bacon, Fr.; Bacon, Francis; Bacon, Francis, Viscount St. Albans; Bacon, Francis, of Verulam; Bacon, Franciscus; Bacon, Franciszek; Bacon, Frantz; Bacon, Franz; Bacon, Franz von Verulam; Bacon, François; Bacone; Bacone, ...; Bacone, Francesco; Bacone, Francisco; Baconius de Verulamio, Franciscus; Baconius, Franciscus; Bacono, Francesco; Baconus de Verulamio, Franciscus; Baconus de Verulamo, Franciscus; Baconus, Fr; Baconus, Franc.; Baconus, Franciscus; Bagone, Francesco; Bako von Verulam, ...; Bako von Verulam, Franz; Bako, Franz; Bakon; Bakon, Francisk; Bejkŭn, ...; Bekon, Frensis; Bėkon Verulamskij, Francisk; Bèkon, Frensis; Bėkon, Frėnsis; Francis Lord Verulam; Francis Lord Verulam, Viscount St. Alban; Franciscus; Franciscus, Baron de Verulamio; Franciscus, of Verulamo; Lord Bacon; Saint Albans, Francis Bacon, Viscount; Saint Albans, Francis Bacon, vicomte de; St. Albans, Francis Bacon, Viscount; Verulam, Francis B., of; Verulam, Francis Bacon; Verulam, Francis Bacon, baron; Verulam, Francis Bacon, of; Verulam, Francis of; Verulam, Francois de Bacon; Verulam, François, de; Verulamio, Franciscus B.; Verulamio, Franciscus Baconus de; Verulamio, Franciscus de; Verulamio, Franciscus, baro de; Verulamio, Franciscus, de; Verulamius, Francis Bacon, of; Verulamius, Franciscus, de; Verulamo, Franciscus, de; Verulamus, Franciscus, de)"

    However, Zotero had previously synced all other entries, except these excessively large ones. Has the syncing system changed in such a way that now, if one item is too large to sync, none of the items will sync?

    2.
    Uncompressed database size = 2GB
    Computer uses a HDD, not SSD, and yes, the database is in default location on that HDD.
  • dstillman
    Has the syncing system changed in such a way that now, if one item is too large to sync, none of the items will sync?
    No, it should continue syncing other items, unless there are a bunch of similar items. If syncs in batches, and if a whole batch fails, it will give up. If you're getting a sync error, you should always resolve it ASAP.
    Uncompressed database size = 2GB
    Computer uses a HDD
    OK, yeah, that's a huge database, on a slow disk. That could easily take a long time to back up and might fail for various reasons (including disk space, but presumably you've checked that). You might even want to disable automatic backup attempts by setting extensions.zotero.backup.numBackups to false from the Config Editor in the Advanced pane of the Zotero settings. It should probably skip automatic backups for databases above a certain size anyway.
Sign In or Register to comment.