Random storage files converted to links, annotations lost

plafourc
Hi,

To sync my library across multiple computers, I use the zotero sync function for everything except the files, which I manually sync via git.

Today when I booted zotero on a machine I hadn’t used in some time, it decided to remove some seemingly random pdf files from the storage and convert them into links instead. This is a real pain because I do need the files to stay in the storage for git to share them, and most important of all, I have lost all annotations in the process.

Is it possible to correct the damage? That is:
-Get the pdfs back in the storage in a batch. I do have a backup of the storage folder through git but zotero seems to have lost the pointers to the files.
-Restore the annotations. If this is impossible I want a way to store the annotations in separate files that I can backup via git so I don’t lose any in the future.

Finally, if anyone has an idea of where the issue comes from and how to prevent it that would be great.
  • dstillman
    Zotero did not do that — it has no functionality to do so. You did this with some plugin (either ZotFile or one of the Z7 equivalents).

    You can convert linked files to stored files via Tools → Manage Attachments → Convert Linked Files to Stored Files, but beyond that I'm afraid you're on your own. Your best bet is probably to restore from a backup from before you did this.
  • tim820
    If the annotations were added in the Zotero reader then they were in the Zotero sqlite database (at least on the machine where they were added), not in the PDF. And neither Zotfile nor any Zotfile-like Z7 plugin does anything with annotations as far as I know (not since Zotero v6, when extraction of externally-added PDF annotations was taken over by Zotero from Zotfile).

    So lost Zotero-added (not externally-added) annotations would seem to suggest a Zotero data-related issue not a PDF file-related issue would it not ?

    How did you "remove some ... pdf files from the [Zotero local ?] storage and convert them into links" ? Where is your Zotero data directory located ? How did the lost annotations issue actually show itself ?
  • dstillman
    So lost Zotero-added (not externally-added) annotations would seem to suggest a Zotero data-related issue not a PDF file-related issue would it not ?
    It was clearly caused by whatever did this. Attachments can't actually be converted to different types — the old attachment has to be deleted, the new one has to be created, and the annotations have to be transferred over via a helper function that Zotero provides. When ZotFile didn't do that for Zotero 6 annotations, it resulted in data loss, and something similar presumably happened here. I couldn't say what @plafourc did — I'm just saying this wasn't done by Zotero, which doesn't create linked files in any sort of automated way and doesn't delete its own annotations.

    @plafourc should restore from a backup of their entire data directory from before this happened, which will restore both the database and the stored files, and then use Replace Online Library to overwrite the online library.

    https://www.zotero.org/support/zotero_data#restoring_your_zotero_data_from_a_backup_and_overwriting_synced_changes
  • plafourc
    Thank you for your help I will try to restore from backup as suggested.

    I do have zotfile in my system, which might be the issue.

    As for the annotations it looks like the database items corresponding to the stored pdf files have been removed, and fresh items with links to pdf files were created instead. Hence the vanishing of notes I guess.
  • tim820
    @dstillman what happens to PDF annotations in the Zotero database when it cannot find the PDF file that they originally belonged to (stored or linked) ? Are they kept ? At what point does Zotero need/try to verify that connection ?

    For example, if a user deletes a PDF attachment (stored or linked; and from within Zotero or outside Zotero) that had annotations, and then re-adds the PDF again (stored or linked), can the annotations persist, or are they lost ?
  • adamsmith
    I'd be very surprised if they weren't tied in the database to the file (id) to which they belong, so deleting a file in Zotero (not on the harddisk) deletes the annotations and once you've emptied the trash, they're gone.
  • dstillman
    Yup.
Sign In or Register to comment.