Id 551212249 - Zotero 1.5 DELETES all folders in its storage directory after hangup

mhomilius
The Problem occured while opening a large pdf in firefox using acrobat with zotero running at the bottom of firefox.
Acrobat or Firefox opened a dialog, that it is taking too long to load open the file.
The serious issue is, that zotero then deleted all stored items, which completely breaks the whole library. The items still appear in zotero, but can't be located anymore.
Is there any way to get the lost files back, e.g. are they stored as binary objects in the sqlite database?
  • noksagt
    Attachments aren't stored as binary objects in the sqlite database, but nor should they be deleted. Also note that Zotero-1.5 is not the latest version of the beta.

    Are you using the syncing functionality in 1.5 on multiple machines? If so, are you using the WebDAV syncing & are you sure that your attachments were all synced in the first place?
  • mhomilius
    Your are right with the version of Zotero, it happened on 2.0b3. But something similar happened last week, before I updated to 2.0b3. Should have been 1.5 then.
    I am not using any syncing functionality.
  • noksagt
    Did you report the first event & how "similar" was it? Did it happen with Acrobat thtat time as well? Are you able to do anything to reliably reproduce this?

    Can you disclose more about your platform (OS, firefox version, acrobat version, other plugins installed, location of zotero directory, etc.).

    If you use your OS's file manager, what do you see in your storage directory?
    http://www.zotero.org/support/zotero_data

    There should be no way for Zotero to actually remove these files accidentally, but there had been an issue reported in 1.5 that the capitalization of the storage sub-directories didn't match what was in the database.
  • ebollrich
    It is actually my computer and I asked for a friend's assistance with the problem.

    My Firefox version: 3.0.10, Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 3.5.30729)
    Add-ons in Firefox other than zotero: microsoft.net framework assistance 1.0

    Windows version: windows xp home edition, german.

    Zotero: 2.0b3, german.

    Adobe Reader: version 8.1.5
    But I am quite sure now, that the problem is independent from Acrobat Reader.

    I didn't use zotero the last days because of the aforementioned incident, but started yesterday evening again. I downloaded some articles from jstor directly into zotero. Before I have shut down the computer, I exported the library this time (I didn't do that last times) - I used the zotero function "export library - zotero RDF, all (notes and data)". Today, I started zotero and downloaded some articles again. I tried to open a pdf in zotero which I downloaded yesterday, but zotero said it cannot locate the file. After looking into the zotero storage folder then, I realised that there are just the folders from the articles I downloaded today. All the folders from yesterday and before were deleted again. The zotero storage directory is in the firefox profile folder.

    Either it starts deleting all folders older than one day always I start downloading pdfs into zotero again, or there might be a problem with the automatic backup. I read that normally "Zotero makes a backup copy within your data directory of your library if the last backup is more than 12 hours old" which should be named zotero.sqlite.new. I don't have that file in the mozilla firefox zotero storage.

    I din't make any changes in the storage, nor did firefox hang up this time (it happened last time because I tried to open a large pdf while having zotero open at the bottom of firefox). And I also din't have a problem with the Acrobat Reader. Also, my firefox profile is still the same.

    The following items are shown in the zotero storage now: storage, styles, translators, pdfinfo-Win32.exe, pdfinfo-Win32.exe.version, pdftotext-Win32.exe, pdftotext-Win32.exe.version, zotero.sqlite, zotero.sqlite.1.bak, zotero.sqlite.49.bak, zotero.sqlite.bak, zotero.sqlite-journal. In the storage there are just folders added today with just the pdf files + snapshots I downloaded today.

    Would be great to find a solution. Thanks.
  • noksagt
    there might be a problem with the automatic backup. I read that normally "Zotero makes a backup copy within your data directory of your library if the last backup is more than 12 hours old" which should be named zotero.sqlite.new. I don't have that file in the mozilla firefox zotero storage.
    Zotero.sqlite and the backups of this file do not store your file attachments.
    Would be great to find a solution.
    Nobody else has reported this issue & I don't see how Zotero can be deleting your attachments. Are you sure this isn't because of something outside of Zotero (quota, malware, some other program that may prevent permanent changes to your home directory, etc.)?
  • dstillman
    This certainly shouldn't happen, and I can't think of anything that would trigger it, but, just to be sure, I've reviewed the attachment-handling code and put in a couple additional safeguards to check for problems.

    Do you see the now-missing folders in the 'files' directory of the RDF export? (Note that the folder names would be different.) If so, did you do anything else in Zotero between when you exported and when you noticed the missing files? Have you tried re-saving the items you added since the export?

    Obviously, we would very much want to know if you could reproduce this. If you're concerned that this is an ongoing problem, you might want to keep the storage folder open while you work to make sure that nothing is happening to the attachments.
  • ebollrich
    The files which zotero cannot find anymore are in the RDF export on my external hard disk drive. I opened all folders (with different names) and crosschecked it. But they are not in the firefox storage anymore where zotero normally saves the items.

    After I exported yesterday evening, I started working with zotero again today, saved some pdfs and later on I realised that all the files from yesterday (the ones from the weeks before were already deleted) cannot be located anymore and that the files are not on my computer anymore. I did an RDF-export again now. So far, all the files from today are still in zotero and in the RDF export storage. There are much more files because every zotero entry has seemingly 2 files, I have seen now, 1 for the snapshot and 1 for the pdf - that is correct? I also realised when I open the folder with the snapshots and links and close it afterwards without changing anything in the folder, the folder properties say that the folder was edited at the time when I opened it. This does not happen when I open the folder with the pdf-file inside.

    I didn't have any problems with zotero when I started using it. The first time the files were deleted after firefox hang up with zotero open at the bottom of firefox while loading a huge pdf (4,62 MB) while adding tags, i.e. working with, in zotero. I upgraded my zotero version 4 days or something before that incident, but it worked before. I didn't change settings in firefox.
  • dstillman
    There are much more files because every zotero entry has seemingly 2 files, I have seen now, 1 for the snapshot and 1 for the pdf - that is correct?
    Yes. Every Zotero attachment with files would have a separate folder.
    I also realised when I open the folder with the snapshots and links and close it afterwards without changing anything in the folder, the folder properties say that the folder was edited at the time when I opened it.
    Probably due to the thumbs.db preview file created by Windows.
    The first time the files were deleted after firefox hang up with zotero open at the bottom of firefox while loading a huge pdf (4,62 MB) while adding tags, i.e. working with, in zotero.
    Assuming you're referring to the "Unresponsive script" warning, did you get this last night or this morning (after last night's export)? When you got it the first time this happened, did you click Continue or Stop? Also, could you say exactly what you were doing in Zotero while loading the PDF the first time?
  • ebollrich
    I didn't get the unresponsive script warning this time. Last time I added some tags and information about the document while the pdf was loading, I did so until firefox hung up and I couldn't add anything anymore. I couldn't click continue or stop, I had to close firefox with the windows task manager. There was no dialog at all.

    But since the problem occured now indepently from problems with firefox or huge pdfs I don't know if it is something else. Maybe it is something with the sync. There is a similar entry now in the forum: "2.0 Beta: The attached file could not be found.": http://forums.zotero.org/discussion/4682/the-attached-file-could-not-be-found/#Item_3

    At least it sounds like the problem I have.
  • dstillman
    Wait, so you are using sync (metadata and WebDAV), or are you not?
  • ebollrich
    In settings the field "sync automatically" is ticked (yes). But I don't use it manually! I never realised that something was synchronised - so I don't know if it does, but the tick is there.
  • dstillman
    No, if you haven't entered a username and password then syncing isn't enabled, regardless of the checkbox.

    So, we'll keep looking into this, but my advice would be to back up your data directory regularly, perhaps keep the 'storage' folder open in the background for the next few days to alert you right away in case this happens again, and let us know if you can reproduce it.
Sign In or Register to comment.