All references dissapeared from a group library, folders still there

Hello, I have managed a group library with over a dozen people for nearly 3 years. I had opened and synced all the references to my zotero today and confirmed we had all the references. When I added another user later, that user said the library was empty. I have confirmed this is true. Our folder structure is in the library, but every single reference has disappeared. None of the group members has done anything to the library today.

My sql lite backup was from a long time ago, so I do not have any of the recent work.

What in the world happened to the references?

I have gone through these procedures:
https://www.zotero.org/support/zotero_data#restoring_your_zotero_data_from_a_backup1

Unfortunately, none of those work. All group member use Sync and the Zotero app is an updated version (most people are recent within the last year).

Current file: zotero.sqlite (362 MB)
Backup file: zotero.sqlite.bak (423MB). Note: I cannot simply restore from these backup because I have about 20 group libraries shared amongst dozens of researchers and colleagues.

There is a storage folder, but it does not contain folders with dates. They are all hexadecimal. There are about 5,500 items in the storage folder.

My current directory is my home directory. I have full admin privileges.

I just used Zotero today on my computer. I moved one file from a folder and then closed. I then have my colleague
  • edited August 22, 2023
    Did you check the trash? Most likely somebody moved all the items to the trash and you can restore them from there. Beyond that, we'd need the first few letters of the group library to comment, but this definitely would've been something someone did.
    Backup file: zotero.sqlite.bak (423MB). Note: I cannot simply restore from these backup because I have about 20 group libraries shared amongst dozens of researchers and colleagues.
    You can — the Replace Online Library function, mentioned in the restore instructions, is library-specific. No other libraries would be affected, and syncing after using Replace Online Library would just pull down the more recent data from all your other libraries (or you could switch back to your current database before syncing, but that's not necessary). You should make an immediate backup of all zotero.sqlite.bak files in the data directory, in case you need them.

    Again, though, you probably just need to restore the items from the trash.
  • edited August 22, 2023
    There is a storage folder, but it does not contain folders with dates.
    This is a misunderstanding. The folder names are 8-character random item keys. The thing about dates is just telling you to look at the dates of the folders to see when those attachments were added to Zotero, so you can tell if this is the same data directory you were using previously. But that section isn't relevant to your situation anyway.
  • Yes, I have 32 folders with yesterdays date (around the time this happened). I have looked at the files inside each folder. There is a cache file (very small, e.g. a few KB) and the other files are attachments or document. All the files are for a different group library than the one in question.
  • You didn’t respond to anything in my first message.
  • Yes, totally missed that. There are no items in the trash.

    I'm looking back at the instructions on restoring here: https://www.zotero.org/support/zotero_data

    The instructions do link to the Replace Online Library as a sub items within restoring Zotero data. I did not look through that thoroughly because when I discovered my back up was over a month ago, that was not a good option because many more references have been added since then.

    It would be helpful to indicate that restoring could be from an individuals automatic backup or and group members most recent (good) sqlite file.

    In the meantime, I am seeing if anyone in my group has a more recent sqlite file that we can try the Replace Online Library function with.
  • zotero.sqlite.bak is from a month ago? If you've been using Zotero on this computer, you should have zotero.sqlite.bak and zotero.sqlite.1.bak from the last couple days in the data directory.
  • edited August 23, 2023
    If you email the group name to support@zotero.org with a link to this thread, we can likely also tell you who in the group did this.
  • I haven't been using Zotero myself since mid July. This usually goes in waves when I am writing papers or preparing grants. Some of my students will have more recent versions.

    It would be lovely to have a mechanism to restore a group/library online, just as there is a backups locally. Some kind of version control would allow us to see the counts of the number of references. I could then easily see when a specific group went from hundreds (or thousands) to zero.
  • So I have heard back from my students who have most recently used it. One of the students has a file that is from a couple weeks ago, but that file is only 1MB (it should be much larger). The other student has had Zotero open the past few days but apparently wasn't putting in new references, was just opening them. Her group also shows nothing.

    That's very odd, because yesterday her and I both saw all the references.

    So, the only option now is for me to restore from a month ago or if there are other options you can identify.

    The only think I think that may have triggered this is when I added another person who then performed a sync and everything somehow disappeared. Again, nothing is in the trash.
  • Again, we can tell you who did this, but you need to tell us the group name via email.

    Just adding a member and having them sync would never do this. Someone has to have deleted all items and emptied the trash.

    As for backups, anyone using Zotero regularly should have zotero.sqlite.bak and zotero.sqlite.1.bak files from the last couple days. If someone hasn’t been using Zotero, those files would be older. Beyond that, it would just be any backups someone made of their own system (e.g., automated hourly Time Machine backups on a Mac).
  • edited August 24, 2023
    So of the libraries you're a member of, it looks like there's one (beginning with "F") that 1) has dozens of collections, 2) has a single current item, and 3) has thousands of recently deleted items. I'm assuming that's the group in question.

    Those items were deleted — along with three collections — and purged from the trash on August 5. Again, if you email us, we can tell you the group member that did this.

    These deletions would've been synced down to anyone who synced since August 5th. If you saw the items more recently, you just hadn't synced that library — perhaps you have auto-sync disabled and have been syncing manually, or you started up Zotero after not using it for a while and the initial auto-sync was still going when you first looked at the group.

    So that means that if someone had been using Zotero and had been syncing, this change would've occurred weeks ago, so the items wouldn't be present in any recent automatic backups. But it also means that your backup from a month ago may not be that out of date — there couldn't have been any changes in this library since the 5th.
  • I have emailed to find out who may have deleted these files. All group members are still a part of the project so if anything happened, it should probably be a user error.

    Thank you for your help.

    In the future, it would be great to have version control online so we can backup a single group that way.
  • Zotero support was able to identify that a group member deleted the group or files from the group.

    First question, if a group member deleted the references (which I know would be an accident), why are they not in my trash?

    Second, question, is a group member able to delete a group? My group still shows up, but it is empty.
  • edited August 29, 2023
    First question, if a group member deleted the references (which I know would be an accident), why are they not in my trash?
    Because the group member also emptied the trash. I can't tell you why they did this, but it's what they did.

    Any group member that had been syncing would've been able to restore the group data for at least a couple days with the automatic local backups Zotero makes, but beyond that you would need your own backup (e.g., the automated backups from Time Machine on a Mac).
    Second, question, is a group member able to delete a group? My group still shows up, but it is empty.
    They can't delete the group itself, but they can make changes to it, and that by necessity involves deletions. We could consider making it impossible for non-admins to purge items (and soon collections) from the trash for 30 days, but there are downsides to that — e.g., it's fairly easy to save an item to the wrong library, so if someone saved something personal to a group by mistake, making it impossible to delete an item isn't ideal. There could perhaps be some sort of time limit — e.g., you could purge an item from the trash if it was created within the last 15 minutes.
  • These are good ideas. Honestly, editing and deleting a small number seems reasonable. It would make a lot of sense to flag a situation where a user was deleting all the references and then purging all the trash!

    At some level, it would be advantageous to notify the group owner if a significant number of references were deleted (or purged) so there was some kind of warning and a way of addressing this.

    In our case, I'm certain this was an accident. I don't have all the details yet, but it sounds like a student was turning in a borrowed university laptop and they were trying to delete everything from their computer. I think this person didn't realize it was deleting everything from all users in the group.

    So, providing an owner with a notification would be great, as would a notification to the user if they are deleting an entire folder and purging the contents (e.g. a warning that this will remove from all other shared users in the group).
  • Currently, the two steps to permanently delete items in a shared library are exactly the same as in the personal library:
    I. Move to Bin
    "Are you sure you want to move the selected items to the Bin?"
    -> You can directly press Enter to validate Ok.
    II. Delete
    "Are you sure you want to delete the selected items?"
    -> You can directly press Enter to validate Ok.

    I think that the easiest things to improve would be:
    1) Add the warning suggested above at step II in a shared library: "⚠️ The items will be permanently deleted for all members of this shared library." A similar warning could also be given in step I, but that is probably obvious and and less critical anyway. So better keep the warning at the most important step only to make it more efficient.
    2) Change Delete to Delete Permanently at step II and the text below to "Are you sure you want to delete permanently the selected items?" (for both personal and shared libraries). This would be more consistent with the option given in the right-click menu on items in the Bin. And it does not cost much to be extra clear about this.
    3) Make the default option to be Cancel instead of Ok when pressing Enter at step II (for both personal and shared libraries). Purging the trash of a library is not something that needs to be optimized for efficiency. So the small extra effort needed to press Ok is probably well worth it considering the extra protection it provides.

    You can argue that no one reads dialogs anyway, but I still believe that giving more precise information in the delete popup is important.

    Beyond these simple steps, in line with your ideas, an "Admin Bin" could probably be a good solution:
    - Only admins can purge the bin permanently, with the additional warnings mentionned above.
    - Other members can purge the bin for all users, but it goes to an Admin Bin sub-collection of the Bin collection only visible by admins, where they need to make a final purge to make it permanent. The member should be notified about this when purging the bin.
    - Members can still make a permanent purge of the items that they have added to the shared collection. But all the items selected to be purged must be added by that same user. And eventually add an extra layer of validation (a tick Also delete permanently for the Admins before validating the purge) to make sure this is the intended behaviour. This is intended only for confidential items that were added by mistake, so it makes sense to make the process more painful.
    - At step II, admins will always see the option to also purge the Admin Bin by clicking inside a tick box if the Admin Bin is not empty (which means that another member has purged the bin). This keeps the normal maintenance of the shared library very simple, with just a simple additional step triggered by purging event by other members.
  • These are great ideas! I agree that many people do not read the dialogs. I do think a message about deleting the items for all users in the group permanently would help.

    It seems though, having an Admin bin that only allows permanent purging would be far more reasonable and a good fail-safe.
Sign In or Register to comment.