Upgrade StorageSome files of older items being lost and not syncing with Zotero Storage
I have a large Zotero database that I'm syncing with Zotero Storage. For a while I have had the issue that Zotero appears to lose track of my files, that once were part of my collection.
An example is shown in the screenshot from my Zotero Web library, where all items used to have PDFs attached (at some point in the past), but now for three items the link symbol indicates that no PDFs are being stored.
https://s3.amazonaws.com/zotero.org/images/forums/u2465227/yzlvgyy39e0w1agi4cia.png
The main issue that I'm confused about an need help with is how to resolve this situation (without losing my highlights and annotations on the PDFs).
When I add the missing file in the Zotero desktop application, I can read and see it locally, however after triggering and completing a sync cycle, it does not get uploaded to Zotero storage and is not available from the web library (I'm using less than 50% of my storage quota so that is not the problem).
I found that resetting the sync history and triggering a new sync does result in the newly added file to be uploaded, but it feels wrong to do this every time I add a file locally.
Is there another way to get Zotero to sync my files, or is the reset the only way?
An example is shown in the screenshot from my Zotero Web library, where all items used to have PDFs attached (at some point in the past), but now for three items the link symbol indicates that no PDFs are being stored.
https://s3.amazonaws.com/zotero.org/images/forums/u2465227/yzlvgyy39e0w1agi4cia.png
The main issue that I'm confused about an need help with is how to resolve this situation (without losing my highlights and annotations on the PDFs).
When I add the missing file in the Zotero desktop application, I can read and see it locally, however after triggering and completing a sync cycle, it does not get uploaded to Zotero storage and is not available from the web library (I'm using less than 50% of my storage quota so that is not the problem).
I found that resetting the sync history and triggering a new sync does result in the newly added file to be uploaded, but it feels wrong to do this every time I add a file locally.
Is there another way to get Zotero to sync my files, or is the reset the only way?
All three of those items have duplicates in your library, and those other items have the files that you just added. So the files are getting uploaded — you're just looking at the wrong items.
As for the files being "lost", if the files aren't in the online library, they were never in the online library (unless you used "Purge Storage in My Library" from your storage settings page at some point). And if they're not in the desktop app on the computer where you're looking now, they were never on that computer (because they were never in the online library for them to be downloaded to that computer).
This just means the files weren't uploaded from the computer where they were added, most likely because you were at your storage quota at the time and never synced (or never synced with file syncing enabled) on that computer after adding a subscription. You're actually just missing nearly all files from before the day you first added a subscription, except for a handful from one day in 2015, which I'd guess are the ones that had filled up your free storage quota.
So this is just what's covered in Files Not Syncing: if you still have access to the computer where the files were added to Zotero, you just need to sync that computer with file syncing enabled, and then they'll be available in the online library and on all your other devices.
I wished that it was only confusion, but this is not true. The newly added pdf files that you saw in the web library only appeared there after executing "Reset File Sync History" from the Sync Preferences.
It is true that there are quite a few duplicate items, but that is not the issue. Historically, the database originates from importing a multi-GB large Mendeley library, which was only partly successful. Metadata was correctly imported but many files were not. I'm not completely sure at which point the duplicates appeared - I'm now trying to eliminate them manually one-by-one.
Here is an example where the file was and still is missing in the Web library but was added in the Desktop app:
https://s3.amazonaws.com/zotero.org/images/forums/u2465227/dbdhojmhmzhkk896b1fj.png
I've hit the sync button several times (and it completed without complaining) after adding the pdf. Nevertheless, the web library looks like this:
https://s3.amazonaws.com/zotero.org/images/forums/u2465227/pi2y2jxfv48acdsjdzbi.png
The page came out of search for an author's name, so there are no duplicates for this item. You can even see the correct time stamp from when I added the pdf in the desktop app. Just the pdf file doesn't show up.
The current usage of storage is a 49.8%, by the way, so that is not where the problem lies.
If it is relevant, I'm using Zotero 6.0.37 on macos 13.6.6.
I'm guessing you're adding files via the Locate button in the file-not-found dialog? There's actually a known issue where that doesn't cause files to be automatically uploaded, so I'm guessing that's what you're seeing.
Everything I said above would still apply — any missing files have always been missing from the online library and any computer where they're currently missing, but if they do exist on another computer you can sync that one now that you have storage space. But beyond that, for now you just need to run Reset File Sync History after adding any files via the Locate button.
I never used linked files and, indeed, the icon presumably just means that there is url available.
I'm also aware that what I sloppily called "loss of files" may not have been an actual loss from the online library but rather the synchronisation not working properly such that files that I had in a local library on one computer were not transferred when I switched computer and relied on synchronisation with Zotero storage (without realising that not all files had been synchronised to the online storage).
Thanks again for your help in getting to the bottom of this.