Upgrade Storage(Sub-)Collections lost in one Group Library. How to proceed?
Dear developers of Zotero,
TLDR: it seems that I have lost the structure (sub-)collections of one group library, where I am the only member of. Hopefully, we can restore this structure of (sub-)collections. I have a relevant backup for this given Group Library.
Recently, I switched from my Fedora desktop to my Fedora laptop. For this this switch, I followed the usual guideline for Zotero. As in, after starting up Zotero (Flathub) on my laptop, I replaced the new data directory with my backup (which is now my current data directory). After checking quickly everything went fine by the total amount of items per library that I own my self, I deleted this most recent backup itself. However during this process, I seem to have lost the structure of (sub-)collections of one group library that I own, and I am the only member. And perhaps also some articles too. Thus I still have most articles, but these are all "unfilled items", as there is no longer a structure of (sub-)collections. I have double checked this with my Web Library, these are lost there too.
For my research I have several group libraries, for each step in my research; this is semi-according a PRISMA-protocol. During this phase of my research, I have excluded and include articles, and then sorted by reason. Aka, I have sorted my articles into the collection Excluded or Included, and then into a sub-collection, which is the reason. As in "Article 29.876 is Excluded for reason X, thus I put this article in (sub-)collection Excluded/X". Yeah, I have over 30K articles for this research...
However, I have now lost this structure of (sub-)collections of this given Group Library. So that I still have most articles left, but these are all "unfilled items", as there is no longer a structure of (sub-)collections For consistency and ideally, I would like to replace mere this given Group Library. Where I have already deleted the most recent backup, there I still have a relevant backup for this research. The main difference between these two backups, is my personal library; which have changed significantly between these two versions of backup. As in, I have not worked on my research between these backups (thus no changes in the related Group Libraries), but I have worked on my personal Library significantly.
Before I do anything weird (again), I am asking here for guidance. Although everything seemed to go just fine, afterwards I just lost two things. I lost the structure of (sub-)collections of a given Group Library, and I have lost some articles. As I still have a relevant backup for this research, I hope I can mere replace that given whole project. In case we can restore mere the related Group Libraries, then that will be nice. However, I can also 1) create another/new account, 2) use this new account to export my whole research (again, over 30K of articles) by using that relevant backup, 3) while keeping that backup, deleting this new data directory of my new account, 4) switch back to my current/original/preferred account, and 5) import my research back into my preferred account. Or I can do something similar, but than not switching my research of 30K items, but instead replacing my personal library of 6K items (with 30 GiB of attachments).
In case there is another protocol to follow, or any other related advice, I am very interested to follow that. With 30K of processed articles, I would like to use either protocol to continue my research, without redoing the PRISMA process with these 30K...
Thanks in advance,
86ul
TLDR: it seems that I have lost the structure (sub-)collections of one group library, where I am the only member of. Hopefully, we can restore this structure of (sub-)collections. I have a relevant backup for this given Group Library.
Recently, I switched from my Fedora desktop to my Fedora laptop. For this this switch, I followed the usual guideline for Zotero. As in, after starting up Zotero (Flathub) on my laptop, I replaced the new data directory with my backup (which is now my current data directory). After checking quickly everything went fine by the total amount of items per library that I own my self, I deleted this most recent backup itself. However during this process, I seem to have lost the structure of (sub-)collections of one group library that I own, and I am the only member. And perhaps also some articles too. Thus I still have most articles, but these are all "unfilled items", as there is no longer a structure of (sub-)collections. I have double checked this with my Web Library, these are lost there too.
For my research I have several group libraries, for each step in my research; this is semi-according a PRISMA-protocol. During this phase of my research, I have excluded and include articles, and then sorted by reason. Aka, I have sorted my articles into the collection Excluded or Included, and then into a sub-collection, which is the reason. As in "Article 29.876 is Excluded for reason X, thus I put this article in (sub-)collection Excluded/X". Yeah, I have over 30K articles for this research...
However, I have now lost this structure of (sub-)collections of this given Group Library. So that I still have most articles left, but these are all "unfilled items", as there is no longer a structure of (sub-)collections For consistency and ideally, I would like to replace mere this given Group Library. Where I have already deleted the most recent backup, there I still have a relevant backup for this research. The main difference between these two backups, is my personal library; which have changed significantly between these two versions of backup. As in, I have not worked on my research between these backups (thus no changes in the related Group Libraries), but I have worked on my personal Library significantly.
Before I do anything weird (again), I am asking here for guidance. Although everything seemed to go just fine, afterwards I just lost two things. I lost the structure of (sub-)collections of a given Group Library, and I have lost some articles. As I still have a relevant backup for this research, I hope I can mere replace that given whole project. In case we can restore mere the related Group Libraries, then that will be nice. However, I can also 1) create another/new account, 2) use this new account to export my whole research (again, over 30K of articles) by using that relevant backup, 3) while keeping that backup, deleting this new data directory of my new account, 4) switch back to my current/original/preferred account, and 5) import my research back into my preferred account. Or I can do something similar, but than not switching my research of 30K items, but instead replacing my personal library of 6K items (with 30 GiB of attachments).
In case there is another protocol to follow, or any other related advice, I am very interested to follow that. With 30K of processed articles, I would like to use either protocol to continue my research, without redoing the PRISMA process with these 30K...
Thanks in advance,
86ul
1) Disable automatic sync
2) Make another back-up of your data directory just to be safe
3) Restore from your back-up
4) Make sure your _group_ library looks alright
5) In the Zotero preferences und Sync, find and click "Reset Options"
6) Important: under "Library" in the dropdown, select the group library for which you've lost the collection structure
7) Check the "Replace Online Library" radio button and then click "Reset"
8) Wait for the process to finish, then re-enable automatic sync
This will restore the group library to what you're seeing after back-up, while your changes to My Library (and any other libraries) since the back-up will sync back down.
It seems that I don't have to do the crazy stuff that I had in mind. Glad that I asked for guidance first, before ruining my Libraries once more.
I hope to do it all tonight.
0) Syncing my library on my laptop for the last time.
1) I disabled the automatic sync (both options, thus including the attachments), and I disconnected all internet (neither LAN, nor WiFi).
2) After closing Zotero, I renamed the current data directory from Zotero into Zotero-20260330.
3) Copying several backups/data directories from their backups, and restarted Zotero to check for the group library with still the original structure of (sub-)collections of the problematic group library.
4a) When I used a wrong backup (one that has the group library without the structure), I repeated step 2 for that backup too, but then with the date of that backup version.
4b) When I finally found a good backup (one that has the group original structure both intact, and each [sub-]collection has the amount of items according to my spreadsheet), I double checked the other Group Libraries for this research project too. Only after verifying every used Group Library to my spread sheet, only then I continued to step 5.
[5-7]a) After connecting back to the internet, I went to Setting > Sync > Reset, and did reset every Group Library of my research. When I received a warning message that there is a difference of 3k items between the versions of the given Group Library (the version on my laptop, and the version online), then I checked the online version too. However, I still continued with resetting the online version of the given Group Library from the version on my laptop.
[5-7]b) I repeated step of 5-7a with every Group Library of my research, 3 in total
8a) Because I have a question, I did not proceed with step 8, but wrote this below:
Regarding step 8, I have one question left, this time regarding my own Personal Library. As I have stated in the starter above, I have worked recently only on my Personal Library. Besides changing and adding items, I also have attached at lot of files to these items in my own Personal Library. I have done this by Zotero itself, thus without any plugins. Is there a way to keep somehow those new connections too?
Before I wreck this up again, I guess that I can do the following, which is based on the used protocol above. In short:
8b) Renaming the reset data directory of tonight to Zotero-20260331 (which is the reset data directory)
9) As my Web Library is fine, I can rename Zotero-20260330 back into Zotero (aka re-using the data directory of this morning, that one without the structure of [sub-]collection)
10) Verifying the state of the now current data directory, but only my Personal Library
11) Re-enable automatic syncing of all my Libraries on my laptop, with the Web Library. As the Group Libraries on the web have all articles and the the preferred structures, re-enabling the automatic syncing should 'update' the Group Libraries on my laptop, with those in the Web Library, as these are as rest during steps 0-7
11) Hoping everything went just fine, and continue with my research...
Again, I only have done steps 1 to 7b so far, and did thus not continue from step 8 onwards.
Oh, and in the mean time, I found out that this all is not due to Zotero, but all backing up incorrectly, thus by my own fault. I used a discontinued program to back up the intended backup. As in, both the intended backup and the used backup are from around the same time, but done by two different programs. That one who made the intended backup is discontinued, and thus no longer works properly. Either way, always double check important files, regardless the used program/protocol to back up. Jeb, yet another lesson learned.
There's nothing more you need to do then my step 8 above.