Data missing after 5.0 upgrade

etancoigne

Hi,

I used to use Zotero 4.0 for Firefox, along with a dedicated data repository (Documents>Bibliography>Zotero_data; zotero.sqlite created on November 2015, last modified today; size: 5.1Mo). I have a Mac Sierra 10.12.6.

Zotero disappeared from Firefox while (accidentaly) updating Firefox to Firefox 57.0. I discovered Zotero 5.0 and installed it. Then I pointed to my data directory. However, the library remained blank (even after restarting Zotero, or my computer).

I copy-pasted my data in the Home>user>Zotero folder (with appropriate path in the Zotero preferences). It did not work either.

I then tried to downgrade to Zotero 4.0. I successfully installed it on an older version of Firefox. Then I pointed to my personal directory and got notified that my data had to be opened with Zotero 5.0. So I used a back-up I have (never opened with Zot5.0), and pointed to it. The library remained blank.

I could sync with my online data but I will then definitely loose some of the data I have locally (e.g. paths to locally stored PDFs. I did not synchronize the PDFs, due to the size of my library...).

Any idea that could help me solve this?

Thank you very much.

Elise

adamsmith

FWIW, the synced version would still contain any links even if you did not sync files.

But for the main problem, check where Zotero is looking for your data, i.e. the Zotero data folder as set in the preferences https://www.zotero.org/support/zotero_data and then check on the various zotero.sqlite files in there -- what are their respective sizes?

etancoigne

Thanks for your reply. I will then do the sync if I fail restoring my data.

There are 3 sqlite files in the actual folder:

zotero.sqlite (5.1Mo),
zotero.sqlite.76.bak (717 Ko),
zotero.sqlite-journal (430 Ko)

dstillman

What exactly do you see in the left and middle panes, and can you provide a Report ID? If there's actually a 5.1 MB zotero.sqlite file in the active data directory, your database shouldn't be blank. (And Zotero 5.0 also should automatically detect your existing data directory, including a custom data directory location.) Is there anything in the trash in Zotero?

(zotero.sqlite.76.bak is a little strange, though — if you had an up-to-date Zotero 4.0 database, the 5.0 upgrade would name that zotero.sqlite.77.bak. You could get that if you weren't on the latest Zotero 4.0 version, though. If you look in the 'storage' subdirectory and sort by date added, do the dates correspond to times (including recent ones) you've used Zotero?)

Definitely no need to sync until we debug this further.

etancoigne

Hi, thanks for your reply. Here is my report ID: 658941188. It says that there is no pb.

In the left pane I see a fresh new Zotero: "My library", then subfolders "my publications", "duplicate items", etc.; all empty.
In the middle pane I see the Zotero quick start guide.

However I do not have a "storage" subdirectory in my active data directory. I have "locate", "pipes", "styles", "translators", and then the 3 aforementioned sqlite files.

I remember I had a 77.bak at some point... I'm currently trying to restore the data repository and see if it had the "storage" repository and the 77.bak file.

dstillman

OK, that's almost certainly not the data directory you were using before, then — there's no way for the 'storage' directory to be removed (unless it was somehow removed outside of Zotero). It's also very strange that an empty database would be 5.1 MB, so I'm not sure what's going on there. You should look for the data directory you were previously using.

dstillman

(Really the only way I can think of to have a 5 MB empty database would be if you deleted all items in the database and emptied the trash, but that still wouldn't remove the 'storage' directory.)

etancoigne

I checked previously in my Firefox preferences and it confirmed that the data directory was this one.
I checked for other zotero.sqlite files to detect another data folder on my computer. The files I came up with are 2y.o. (never been opened since 2015). This 5.1Mo sqlite file in my current active directory is actually the one that has been accessed by Zotero for every trial I did these last days.

My back-up doesn't have a "storage" directory either: only "locate", "translators", and... a 717Ko sqlite... (?!)

(the trash is empty. Sorry I did not see your question on this)

I think that I will simply make the sync, then.

dstillman

The only way you could have no 'storage' directory is if you never added any attachments to Zotero. And if this was your previous database, you'd also still have the groups you belonged to.

My back-up doesn't have a "storage" directory either: only "locate", "translators", and... a 717Ko sqlite... (?!)

Then you either never had attachments or weren't backing up the right directory.

etancoigne

Thinking of it, I may have manually removed the "storage" directory at some point, given that my PDFs are stored in a dedicated folder outside the Zotero folder. (I may have done it to deduplicate my PDFs).

Maybe this could be the reason why Zotero 5.0 does not recognize my current data directory, if it misses this subfolder?

dstillman

The 'storage' directory doesn't affect what you see in the database, only whether you can open stored attachment files.

You could try running your database through the DB Repair Tool to see if there's a problem with the database. A new database is ~1 MB, so there's something strange going on here.

dstillman

(For what it's worth, though, simply deleting the 'storage' directory wouldn't ever really be the right thing to do, unless you were using download-on-demand file syncing (and even then it would have unwanted side effects). Files would only exist there if they were actually referenced in the Zotero database.)

dstillman

In the middle pane I see the Zotero quick start guide.

Oh, I misread this. If you see the Quick Start Guide, this is an empty Zotero 4.0 database. You can see the date it was created by looking at the Date Added value for that item.

For an empty database in Zotero 5.0, you see a welcome message, but not the Quick Start Guide.

So this definitely isn't the database you were using before.

etancoigne

Thanks for your help. I especially created this folder for managing my bibliographical data a few years ago. So it should at least include an outdated database (if not the good one)... I'm very surprised. I do not have other recent sqlite files on my computer, so if this one is not the good one, I really have no idea where the good one can be.

I'm going to sync my data with my online account, then. I'm sorry we did not fix this - kind of frustrating... I hope this was not too much a waste of time for you.

Thanks again for your valuable help. I was getting crazy behind my computer :)

Elise

dstillman

Just to be sure, you'd also want to check for a 'zotero' data directory in your Zotero profile directory and Firefox profile directory (both of which could easily be excluded from a system-wide search).

etancoigne

Thanks. I checked the Firefox profile directory (I used Zotero with Portable Firefox). But I don't have a hidden sqlite there. It's mysterious!

I also did the repair (on both the restored and actual databases) but it did not change anything. I had check their integrity in Zotero previously and everything was fine.

So, now I stop obstinating and do the sync :)

etancoigne

Finally, after the sync my zotero.sqlite database has a 23.2Mo size... so the previous one was definitely not the good one. I can't figure out what happened during the update, but at least I have all my data, including the links to my local PDFs.
Thanks
Elise