Library has disappeared

bev2105 · January 19, 2017

I use Zotero standalone and have been working on this library for my thesis for years now, so I am totally reliant on it. It is stored in my Dropbox folder and hasn't had any problems until now. I noticed today that when I open my library, only a few items that I added over the past few days are showing. Nothing else is visible - the library literally contains just 8 items. In the 'storage' section of my Zotero folder in Dropbox, I can still see folders containing thousands of references and PDFs (3.3 GB), so I think the data is still there. My data directory location is definitely correct and I haven't made any updates or changes to Zotero or my computer recently. I'm guessing that I might have lost the library a few days ago, but I don't know exactly when.

In case it helps, here are the other files I can see in the zotero folder:
zotero.sqlite (dated to 2012), 16.3MB.
zotero.sqlite-journal (dated to today), 156KB.
zotero.sqlite.1.bak (dated 15 January 2017), 16.3MB
zotero.sqlite.bak (dated yesterday), 16.3MB

I would love to have any advice on how I can restore the entire library, and what might have happened to it -- I can't figure out where I went wrong or how to fix it. Thanks so much!

dstillman · January 19, 2017

Storing your data directory in Dropbox is strongly discouraged, because it's trivial to corrupt your database. (The upcoming Zotero 5.0 will warn you if you try to start with your data directory there.)

First make a backup of the .sqlite files outside of Dropbox, and then rename zotero.sqlite.bak to zotero.sqlite and see if that restores your data. If not, try zotero.sqlite.1.bak.

Once you find your data, you should 1) check your database integrity from the Advanced → Files and Folders pane of the Zotero preferences and 2) move your data directory out of Dropbox.

bev2105 · March 1, 2017

Thanks so much for your response, @dstillman . I tried what you suggested but unfortunately neither zotero.sqlite.bak nor zotero.sqlite.1.bak were able to restore my library. Both are exactly the same size (16.3MB) which is far short of the size of my storage folder (3.3GB). I believe my library is intact in the storage folder.

I have a back up of everything on an external hard drive dating back to November 2016, when everything was working fine. Should I copy over the entire storage folder and zotero.sqlite file from the hard drive to my computer? Is there any other way of accessing the storage folder other than through zotero.sqlite?

One final question - I realise now that Dropbox isn't a good place for my Data Directory so obviously I want to move it ASAP. Would it be better to change the location of the data directory *before* or *after* attempting to restore the library from my back up drive?

Thanks again for your help.

dstillman · March 1, 2017

Both are exactly the same size (16.3MB) which is far short of the size of my storage folder (3.3GB).

No, unless you see a backup database with a substantially larger zotero.sqlite, this doesn't indicate a problem. 16 MB is a database with a good amount of data. The 'storage' folder contains attached files — that's going to be far larger than the database itself.

If you run your database through the DB Repair Tool and post the Upload ID here, I can confirm that there's data in it. You can also try the repaired file, though if you checked your DB integrity already that's unlikely to help.

Would it be better to change the location of the data directory *before* or *after* attempting to restore the library from my back up drive?

Yes, you should move it out of Dropbox before doing anything. It's also a good idea to first temporarily disable auto-sync in the Sync pane of the Zotero preferences, to avoid an accidental full sync in case you get the steps wrong.