Library gone, totally deleted – what happened?

Hello,

Emergency has brought me here. My whole library is completely gone, and it seems Zotero has deleted it by itself. Here is what happened:

After not having used Zotero during Christmas and New Year, I saw an article on Google Scholar that I wanted to save, so I opened up the Zotero application without looking at it and then hit the save button in the Zotero Firefox add-on, as usual. Action failed for some reason, but when I followed the article to its ScienceDirect page, it worked. I went back to the Zotero application and found that this article was the only article in my whole library.

All the bells and whistles went off in my head, but my last hope slowly drained as I followed this guide through: https://www.zotero.org/support/zotero_data#locating_missing_zotero_data and realised that none of the suggested actions would work, because none of the files associated with my library seems to exist anymore. In my Zotero folder at /home/username/Zotero all the files and folders were created only a few minutes ago, in the same instance that I fired up the application or saved the article (except for most of the translators, which were created a few months back). It seems all my old library files just got overwritten. Or somehow completely wiped out on an earlier occasion. Question is, how is this even theoretically possible? Is there some severe bug?

My Zotero is version 6.0.15 and I use Linux Mint 20.2 (Ubuntu 20.04) and I have not performed any disk cleanups or similar, and definitely not fiddled with the Zotero files myself. A couple of weeks ago everything was there, and now it isn't. What could possibly have happened? For various reasons (that mostly have to do with a shortage of storage space) my last backup is over a year old. I never thought the library files could just get deleted by themselves.

Any help much appreciated.
  • The page asks for specific information, which you didn't provide, and we can't really help without that. But there's no chance that Zotero did this by itself. You either unlinked your account and told Zotero to delete your data or you somehow deleted the files outside of Zotero. Or, as the page explains, this wasn't the data directory you were using previously and your data is elsewhere on the computer. We'd need the requested info to say more.

    There's no code in Zotero to delete the database and 'storage' folder other than by unlinking or switching accounts from the Sync pane of the preferences.
  • dstillman: Sorry, I'll try to be more formal.

    --------------------------------------------------------------------------
    If you've gone through these steps and aren't sure what to do, post to the Zotero Forums with the following info:

    • The names, sizes, and dates of all files beginning with “zotero.sqlite” in your current data directory

    "zotero.sqlite", 1.4 MB, metadata says it was created in the same moment I launched the Zotero app yesterday night.
    "zotero.sqlite-journal", 156 kB, created in the same moment.
    "zotero.sqlite.bak" 1.4 MB, created half an hour after the first two files.

    • Whether there's a 'storage' folder containing subfolders with dates corresponding to your previous usage of Zotero

    There are three subfolders under 'storage', containing the files associated with the one article I saved yesterday. No other subfolders or files.

    • Whether your current data directory is in the default location (“Zotero” in your home folder)

    Yes, it's the default, /home/username/Zotero. I have also performed an extended file search on my computer, but the old library files are nowhere else to be found either.

    • When you last used Zotero on this computer, and what happened on your computer since then

    I can't remember exactly when I used it the last time, but it was some time before Christmas. Although it was probably even longer since I last saved an article into the library. Not much has happened on the computer since then. No new apps installed, no old apps uninstalled. A few regular system updates have been applied. If you'd like to look into the details, here's a part of the dpkg log with all the upgrades made on my computer since the beginning of December: https://pastebin.com/ywk72TYm

    I would also like to add that I haven't had a Zotero account until today, so all of my usage has been offline and I have never configured any account or sync in the app. And I most definitely haven't touched the files myself. I haven't performed any type of file operations these last weeks, other than moving a few stuff around on my desktop and Downloads folder.

    • What you've tried so far

    I've confirmed the above. What now?
  • You say that you do have a back-up from a year ago. Can you check if you have a Zotero folder there in /home/username/Zotero ?
    Whether it existed previously would determine whether to look for a way it has gotten deleted or whether it exists elsewhere.

    Also, how did you install Zotero? I believe there are unofficial Flatpack and Snap packages out there, and it'd be good to know if you used either of those.
  • Bloody hell! The backup's gone too!! It was in a folder on my desktop, but now the 'storage' folder and zotero.sqlite.bak files are gone, and the zotero.sqlite file is just 1 MB! What the hell is actually happening here??

    The details:
    I installed Zotero on this computer when I got it about one and a half year ago, using the standard suggested way for Ubuntu Linux distros described here: https://www.zotero.org/support/installation, adding the https://github.com/retorquere/zotero-deb repository in my PPA. I then copied my Zotero folder from my old Windows machine (which should have been the default C:\Users\username\Zotero) to my new Linux Mint machine, pasted it in my home directory replacing the autogenerated Zotero folder there, and also kept a backup copy of it on my desktop. Not sure if I also had to do something else to make it work, but it has worked flawlessly since then anyway. Until what I described happened yesterday night. And now that I check the backup folder that has been lying on my desktop for almost a year and a half, the 'storage' folder in it is gone?!

    It's as if some kind of file operation has deleted everything named */Zotero/storage on my computer, plus the corresponding sqlite files. How on earth could this be even theoretically possible? The Zotero app never knew about my backup folder on the desktop, and it all worked a couple of weeks ago. Since then I've had Christmas and New Year, and now suddenly my entire library has ceased to exist, including from the detached backup. Is there some kind of log to find out what has happened?
  • I assume you've looked in your trash? I have no idea what happened but it very much does sound like a system operation. Obviously it's not anything Zotero would (or could) do.
  • I always look in my trash before I empty it. But I have not been near any of these Zotero folders. What could possibly have performed such a system operation? I could kill for a log file telling me what this sorcery is all about.
  • I mean, technically something like rm -r */Zotero/* or so, but that doesn't just run miraculously nor would you have run that on purpose. I'd susepct somethign else going on -- you may be misremembering something, or you changed your username and left files in an old home directory or something along those lines.
  • No, that's not the case. From what I've found so far, my best bet is a sudo apt autoremove a little while back, but that's almost as illogical as anything else.

    I really didn't think this was possible. I will have to start over from scratch. Are you sure there isn't some kind of log? Somewhere to see deleted files history? Or even better, somewhere where Zotero or the Firefox add-on logs what gets added to the library (URLs, DOIs, something)?
  • edited January 6, 2023
    apt doesn't have anything to do with your Zotero data directory and wouldn't remove that.

    Zotero create a log during some version upgrades (e.g., Zotero 5 to 6), but it's stored in a 'logs' folder in the data directory. It doesn't sound like you have that either.

    When people report this, it's almost invariably due to their running disk-cleaning software to free up disk space and accidentally deleting a large Zotero SQLite database file and the database backups in the same folder. Usually the 'storage' folder is left behind. But if a tool just identified large folders directly within the home directory and/or desktop, you might have accidentally deleted the entire Zotero folder.

    If you had previously used Zotero syncing and tried to set up syncing with a different account, Zotero would warn you — quite clearly — that it was going to delete your existing local data and make you confirm before doing so. But that wouldn't be the case if you hadn't previously set up syncing with a different account, and it obviously wouldn't affect some other folder on your computer.

    Other than that, the most likely explanation is that you're just misremembering where you had your Zotero data directory and the preference somehow got reset, though in that case you should of course be able to find your data directory somewhere else. If you actually had a full-disk backup, you could check the prefs.js file within the backup of the Zotero profile directory folder and look for a dataDir line to see if Zotero was perhaps pointed at another directory, but if you don't have a full-disk backup that wouldn't be possible.

    Anyway, this clearly isn't something that Zotero did, so I'm afraid we're not going to be able to help further. We always recommend regular, automated backups to an external device (i.e., not stored on your desktop) as well as using Zotero syncing, in case something happens to your local Zotero data.
  • edited January 6, 2023
    It was in a folder on my desktop, but now the 'storage' folder and zotero.sqlite.bak files are gone, and the zotero.sqlite file is just 1 MB!
    Oh, but if you're saying this folder still exists and there's just a 1 MB file (with what creation date?), you either copied an empty, unused data directory as the backup or you at some point pointed Zotero at that folder on your desktop. Unless you pointed Zotero at that folder today, I'd guess the former, which lends credence to the theory that you were using a custom data directory elsewhere on your computer and the pref just got reset somehow. Or you had it on an external disk that got unmounted?
  • The debs don't touch the data or profile dirs in any way BTW. It is safe to install/uninstall/reinstall the deb, or even alternately run the tarball/deb version, without affecting the Zotero data.
  • dstillman: Look, you are most welcome to lecture me about off-system backups, which I guess I deserve – but I can assure you that I haven't used any disk cleanup software on any computer at all for the past few years, nor have I deliberately deleted files in bulk. I've gone through my terminal command history for the last two months, but nothing there seems even remotely close to account for what has happened, sudo apt autoremove was the only thing that had to do with deletion/cleanup whatsoever. The only thing I can imagine is that this is the result of a side effect of some software that I am not aware about. I also remember clearly that the backup folder on my desktop took up a few hundred megabytes when I put it there, not 10 MB as it does now, so something has obviously happened along the way – I did not backup the wrong folder from the start. Anyway, it doesn't matter right now. What I need now is anything that can help me rebuild the library. I noticed that my profile folder (/home/username/.zotero) is still intact and retains the old profile files. I don't know what the different files there do, but I noticed an sqlite database called 'places.sqlite' that took up 5.2 MB. Does this contain any clues to what articles I have saved before? Or is there some other file(s) in the profile directory that I can use to try to find my articles again?
  • Nope sorry. Anything data related is in the data directory.
    The only other place that has *some* metadata would be documents in which you cited with the LibreOffice add-on
Sign In or Register to comment.