Upgrade StorageBacking up Zotero and deduplication: can we ignore the *bak files with daily backups?
I do daily or almost daily backups of my laptop (to external disks and remote storage), including the Zotero directory. For backups I use Restic (https://restic.net/) which employs deduplication.
I've reread carefully https://www.zotero.org/support/zotero_data#backing_up_your_zotero_data and https://www.zotero.org/support/zotero_data#restoring_your_zotero_data_from_a_backup1 but I cannot find answers to these two questions:
1. Is it safe to run the backup while Zotero is opened? Or should I close Zotero before running the backup? In case it could matter, I am using Linux.
2. With daily backups, do I need to keep backups of zotero.sqlite.bak and zotero.sqlite.1.bak?
Why? I've been noticing that my daily backups can sometimes grow by as many as 600 MB just from the sum of the stored (after dedup and compression) zotero.sqlite + zotero.sqlite.bak + zotero.sqlite.1.bak. My zotero.sqlite file is of about 960 MB. I am not adding hundreds of megabytes worth of new data, but it is my understanding that even adding just a few entries, especially with full text indexing, can lead to the SQLite files changing a lot at the byte level, which would render the deduplication of Restic almost powerless.
I think that, if I close Zotero before launching Restic's backup (or if a backup of zotero.sqlite made while Zotero is running is always recoverable), I might just backup zotero.sqlite (and forego keeping backups of zotero.sqlite.bak and zotero.sqlite.1.bak). Otherwise, I think I should keep both zotero.sqlite and zotero.sqlite.bak so I never lose much more than 24 hours worth of Zotero state. In both cases, with daily backups zotero.sqlite.1.bak would not be a file I need to backup. But I am concerned I might be missing something.
(P.S. Heavily edited after I realized a previous idea that depended on calling sqlite3 to make a backup would not work, as I think Zotero keeps a lock on the zotero.sqlite file).
I've reread carefully https://www.zotero.org/support/zotero_data#backing_up_your_zotero_data and https://www.zotero.org/support/zotero_data#restoring_your_zotero_data_from_a_backup1 but I cannot find answers to these two questions:
1. Is it safe to run the backup while Zotero is opened? Or should I close Zotero before running the backup? In case it could matter, I am using Linux.
2. With daily backups, do I need to keep backups of zotero.sqlite.bak and zotero.sqlite.1.bak?
Why? I've been noticing that my daily backups can sometimes grow by as many as 600 MB just from the sum of the stored (after dedup and compression) zotero.sqlite + zotero.sqlite.bak + zotero.sqlite.1.bak. My zotero.sqlite file is of about 960 MB. I am not adding hundreds of megabytes worth of new data, but it is my understanding that even adding just a few entries, especially with full text indexing, can lead to the SQLite files changing a lot at the byte level, which would render the deduplication of Restic almost powerless.
I think that, if I close Zotero before launching Restic's backup (or if a backup of zotero.sqlite made while Zotero is running is always recoverable), I might just backup zotero.sqlite (and forego keeping backups of zotero.sqlite.bak and zotero.sqlite.1.bak). Otherwise, I think I should keep both zotero.sqlite and zotero.sqlite.bak so I never lose much more than 24 hours worth of Zotero state. In both cases, with daily backups zotero.sqlite.1.bak would not be a file I need to backup. But I am concerned I might be missing something.
(P.S. Heavily edited after I realized a previous idea that depended on calling sqlite3 to make a backup would not work, as I think Zotero keeps a lock on the zotero.sqlite file).