Some files of older items being lost and not syncing with Zotero Storage

I have a large Zotero database that I'm syncing with Zotero Storage. For a while I have had the issue that Zotero appears to lose track of my files, that once were part of my collection.

An example is shown in the screenshot from my Zotero Web library, where all items used to have PDFs attached (at some point in the past), but now for three items the link symbol indicates that no PDFs are being stored.
https://s3.amazonaws.com/zotero.org/images/forums/u2465227/yzlvgyy39e0w1agi4cia.png

The main issue that I'm confused about an need help with is how to resolve this situation (without losing my highlights and annotations on the PDFs).

When I add the missing file in the Zotero desktop application, I can read and see it locally, however after triggering and completing a sync cycle, it does not get uploaded to Zotero storage and is not available from the web library (I'm using less than 50% of my storage quota so that is not the problem).

I found that resetting the sync history and triggering a new sync does result in the newly added file to be uploaded, but it feels wrong to do this every time I add a file locally.

Is there another way to get Zotero to sync my files, or is the reset the only way?
  • edited 27 days ago
    I think this is just confusion.

    All three of those items have duplicates in your library, and those other items have the files that you just added. So the files are getting uploaded — you're just looking at the wrong items.

    As for the files being "lost", if the files aren't in the online library, they were never in the online library (unless you used "Purge Storage in My Library" from your storage settings page at some point). And if they're not in the desktop app on the computer where you're looking now, they were never on that computer (because they were never in the online library for them to be downloaded to that computer).

    This just means the files weren't uploaded from the computer where they were added, most likely because you were at your storage quota at the time and never synced (or never synced with file syncing enabled) on that computer after adding a subscription. You're actually just missing nearly all files from before the day you first added a subscription, except for a handful from one day in 2015, which I'd guess are the ones that had filled up your free storage quota.

    So this is just what's covered in Files Not Syncing: if you still have access to the computer where the files were added to Zotero, you just need to sync that computer with file syncing enabled, and then they'll be available in the online library and on all your other devices.
  • Thanks for the quick response!

    I wished that it was only confusion, but this is not true. The newly added pdf files that you saw in the web library only appeared there after executing "Reset File Sync History" from the Sync Preferences.

    It is true that there are quite a few duplicate items, but that is not the issue. Historically, the database originates from importing a multi-GB large Mendeley library, which was only partly successful. Metadata was correctly imported but many files were not. I'm not completely sure at which point the duplicates appeared - I'm now trying to eliminate them manually one-by-one.

    Here is an example where the file was and still is missing in the Web library but was added in the Desktop app:
    https://s3.amazonaws.com/zotero.org/images/forums/u2465227/dbdhojmhmzhkk896b1fj.png
    I've hit the sync button several times (and it completed without complaining) after adding the pdf. Nevertheless, the web library looks like this:
    https://s3.amazonaws.com/zotero.org/images/forums/u2465227/pi2y2jxfv48acdsjdzbi.png
    The page came out of search for an author's name, so there are no duplicates for this item. You can even see the correct time stamp from when I added the pdf in the desktop app. Just the pdf file doesn't show up.

    The current usage of storage is a 49.8%, by the way, so that is not where the problem lies.
    If it is relevant, I'm using Zotero 6.0.37 on macos 13.6.6.
  • That item shows a linked file -- linked files are never accessible in the web library and would only sync via a 3rd party service. If you didn't link this on purpose, it may have been Zotfile, which (if that's in case installed) you'd want to make sure to set to use Zotero's own storage.
  • But files not syncing is different from files getting "lost". There's no way for files to disappear from the web library unless you ran Purge Storage in My Library. If a file you can open locally isn't appearing in the web library, you should follow the steps on Files Not Syncing and provide the requested info.
  • edited 24 days ago
    @adamsmith: It's actually not a linked file — that's just the icon that appears when there's a URL or DOI to launch but no other available file. There definitely is a stored file here that's not available online, but it also never was (again, unless @Joachim.Brand purged storage). We'd need more info to say why.
  • edited 24 days ago
    You can even see the correct time stamp from when I added the pdf in the desktop app.
    @Joachim.Brand: The 2024 date, you mean? That's not about the PDF — that's just the Date Modified of the parent item. The child PDF attachment that's online hasn't been touched since 2018.

    I'm guessing you're adding files via the Locate button in the file-not-found dialog? There's actually a known issue where that doesn't cause files to be automatically uploaded, so I'm guessing that's what you're seeing.

    Everything I said above would still apply — any missing files have always been missing from the online library and any computer where they're currently missing, but if they do exist on another computer you can sync that one now that you have storage space. But beyond that, for now you just need to run Reset File Sync History after adding any files via the Locate button.
  • Yes, thank you for the additional comments. The known issue https://forums.zotero.org/discussion/comment/457735/#Comment_457735 reported by @NitzanSchwartz looks exactly like the issue I'm having. I have indeed used the Locate button in the file-not-found dialog and the file is available locally but only gets synced after resetting the File Sync History. I can use that as a workaround in the meantime while the issue is being resolved.

    I never used linked files and, indeed, the icon presumably just means that there is url available.

    I'm also aware that what I sloppily called "loss of files" may not have been an actual loss from the online library but rather the synchronisation not working properly such that files that I had in a local library on one computer were not transferred when I switched computer and relied on synchronisation with Zotero storage (without realising that not all files had been synchronised to the online storage).

    Thanks again for your help in getting to the bottom of this.
Sign In or Register to comment.