Move from dropbox to webDAV?

rjclement
Hi, I am using Zotero Firefox on Windows Mac and, hopefully, iPAD. Currently I am storing my pdfs as attachments on dropbox using Zotfile.

I was considering moving the library to a WebDAV storage server (probably Box)and would like some advice. Particularly:

1) is this a good idea, and

2) how to I go about moving my data from dropbox to WebDav (i.e. is there a set of instructions somewhere to guide me)?

Thanks
  • adamsmith
    1) generally yes. Biggest advantage is probably that you'll be able to work with Paperships, but there may be other pluses (e.g. in working with groups, same sync for Snapshots and PDFs).

    2) There's two steps two this:
    a) Reverse the location of files. For that, Set File Location in the General Tab of ZotFile from "Custom" back to "Store in Zotero". Then select all of your items in Zotero --> Manage Attachments --> Rename Attachments (which will also move them back into Zotero).
    You now have all your attachments stored in Zotero
    b) set up WebDAV sync. This should be straightforward. Here's a box.com example from Cornell
    http://mannlib.cornell.edu/news/link-your-zotero-files-cornell-box
    (You want option 2)
  • rjclement
    Thanks Adam, that worked great.

    However, I seem to have one problem that has arisen because of the move.

    Although the pdf's moved from dropbox to my WebDAV server (Box), it appears that the snapshots that are associated with many of the PDF's have become lost. This is OK, I dont really need them, but when I try and open the PDF in Papership, it says that the file can not be found - though I can open the PDF in Zotero. Is there a way to bulk remove these lost attachments and sync the result so it applies to papership?
  • adamsmith
    The Snapshots aren't really associated with any PDFs -- they're just attached to the same item. They'd likely have stayed within Zotero the whole time anyway. Are you saying these are suddenly completely gone in Zotero?

    I'm also not quite sure why you think that's related to the failure of the PDFs to open in PaperShip? That would seem more like a problem communicating between PaperShip and your WebDAV?
  • rjclement
    Yes, after moving the PDFs from Dropbox into WebDAV the snapshots in the items that contained both PDFs and snapshots no longer exist (or at least zotero can't find them). There are some items with just snapshots that still exist and can be opened.

    The reason I think it is an issue in Papership is because items with just a pdf attachment allow me to open the pdf in Papership, but items that contain a pdf and a missing snapshot would not let me view the pdf in Papership. I then deleted the missing snapshot in zotero and synced. This change did not show up in Papership, as it still showed two attachments for the item from which I had deleted the snapshot, and I still could not view the pdf in Papership. It did not appear to be a synch problem because new items that I added to zotero after moving the data to WebDAV showed up in Papership after synching and I could view these.
  • adamsmith
    OK, so you still see them in Zotero, but when you try to open them, you get a message saying "may have been moved or deleted" -- correct?
    Could you provide a debug ID for trying to open one of them?
    http://www.zotero.org/support/debug_output
    You could also just look at the debug. I'm looking specifically for one of the lines that says something like
    Attachment file 'xyz' not found
    (where xyz is the full filepath). Post that here to speed things up, otherwise we'll have to wait for Dan to look at your debug output.

    Another thing I'd that'd be ideal for testing would be to sync this library to another computer to see if sync works for those PDFs in general.
  • rjclement
    Alright, I open an item that has an PDF and a snapshot attached. I click on the PDF and it opens in Firefox. When I click on the snapshot I get the error:

    "File Not Found
    The attached file could not be found.
    It may have been moved or deleted outside of Zotero."

    The error dialogue has a cancel button and a Locate... button. If I click on the locate button it takes me a folder along the following directory path (which is empty):

    Computer_Name
    Macintosh HD
    Users
    RobertClement
    Library
    Application support
    Firefox
    profiles
    aekxe9dd.default-1426846782999
    zotero
    storage
    IGHAZIW

    No additional error information is available from Papership on the IPAD (only that the file doesn't exist). I'll try moving my windows machine over to the WebDAV server tonight to see if that has the same issues.

    I also checked the folder on my dropbox that was previously used to hold the library. The folder structure is still there but there are no files (PDF or snapshot) in the folders - as I would expect.

    Thanks ever so much for looking into this.
  • adamsmith
    Are you saying the folder IGHAZIW exists, though? Or is there no such folder? If there is no such folder, this may still all be in order (could you have originally added the item on your other machine?), but if the folder exists but is empty, that would be a sign of something having gone wrong.
  • rjclement
    Yes, the folder exists. There appear to be several thousand folders with similarly coded names. Some of the folders have a PDF file in them and some of them are empty.
  • adamsmith
    Odd. I'd say make a back-up of your Dropbox and your zotero data folder before you do the same on Windows, but then do run the whole thing there and sync and see if that works.

    The folders in principle are right -- there should be one per attachment -- they just shouldn't be empty.
  • rjclement
    OK, I started my WinPC without internet connection and backed up the Dropbox zotero attachment library, and also the Apps/Roaming zotero files and sub-folders. I then reconnected to the internet and tried to start zotero but got the following error:

    "A potential problem was found with your WebDAV server.

    An uploaded file was not immediately available for download.
    There may be a short delay between when you upload files and
    when they become available, particularly if you are using a
    cloud storage service.

    If Zotero file syncing appears to work normally, you can ignore
    this message. If you have trouble, please post to the Zotero Forums"

    I suspect this is because dropbox had deleted all the pdf files in my zotero dropbox library, because they had been removed on my Mac when zotfile moved the files from dropbox to zotero as a precursor to putting them on WebDAV

    I couldn't get around this so I removed zotero/zotfile from firefox, restarted, and then added zotero back to firefox.

    This took a bit of tinkering but finally I got WebDav set up on my WinPC as well and synching ran and completed.

    Now, the problem is similar if not a bit worse

    1) Most of the PDF's are attached to the library items.

    2) There still exist snapshots that are not linked to file. When I go to the location of these missing snapshots the folder exists and I will get a path something like:
    C:\Users\rclement\AppData\Roaming\Mozilla\Firefox\Profiles \rxqkup03.default\zotero\storage\9XKNHTT9
    In which there is a file named: .zotero-ft-cache

    3) There are now also missing PDF's or PDF/snapshot combinations which, when I look at the location do not take me to a specific folder but instead to:
    C:\Users\rclement\AppData\Roaming\Mozilla\Firefox\Profiles \rxqkup03.default\zotero\storage
    I should add that I know that at least some of these missing PDF's had existed earlier today on my Mac, so they must have been lost (or not yet synched) to WebDAV or from WebDAV to the WinPC, or somehow lost in the process.

    I havent had the chance to go back to the Mac or iPAD to check the status of the files on those devices.

    Hope this helps.
  • dstillman
    I didn't read this whole thread, but I'd be surprised if Files Not Syncing didn't more or less cover the issue, with the addition that one of the computers here may mean PaperShip (and I don't know how PaperShip file syncing works). But basically, you have to find the computer where the files were originally created and make sure they sync from there.

    There's no way for files to disappear without the associated items being deleted in Zotero, so they either 1) were moved or deleted outside of Zotero or 2) were never on the computer you're checking to begin with. The steps in the linked page can help you figure out which. Note that folders can exist with just .zotero-ft-cache due to full-text content syncing, so that's expected and doesn't mean that the files were somehow deleted.

    The "An uploaded file was not immediately available for download." doesn't mean what you you say — that's just a message that can appear for cloud-based storage services, where, as the message says, an uploaded file isn't immediately available, as it should be with a real WebDAV server. It's harmless as long as the files appear shortly after they're uploaded.
  • rjclement
    Yes, it appears that some of the PDF's that were not loaded had just taken a bit more time, they were there this morning after I started zotero. There are a few pdfs which were not attachments that are missing but this is only an insignificant handful which I hadn't had the change to get the meta-data for.

    However, the snapshots are still causing a problem. The current status being:

    On my Zotero account the offending items contain both attachments (pdf and snapshot) and appear to have the correct meta-data

    On both Mac and WinPC both the pdf and snapshot appear as attachments. I can open the pdf but the snapshot gives a file not found error, which takes me to folder containing the .zotero-ft-cache file.

    On iPad it shows two attached items but I can not open the PDF (you dont get a choice on which to open) and it simply says the file can not be found.
  • rjclement
    The response I got back from Papership:

    "Moving files to WebDAV is indeed a bit fiddly. Only one other PaperShip user has reported the same type of error with snapshots, likely related to the fact that Zotero has changed their snapshot format a couple of months back.

    But this is also a problem with PaperShip itself, because it expects to have all attached files downloaded before presenting them, which is problematic here.

    We’ll try to improve that with a future update, but in the meantime, you should be able to delete the snapshot file with Zotero, then sync to PaperShip to only have the PDF. Please note that because PaperShip doesn’t currently support the trash, you also need to empty your trash with Zotero to sync.

    You can also trigger a “full sync” at any time by opening this link directly on your device (it will open the PaperShip app): szn-papership://fullsync"

    I performed a full sync of Papership but didn't solve the problem. If deleting the offending snapshots would help then I could do that - but that would be a problem because I dont want to delete all snapshots, just the ones that are not available locally.
  • dstillman
    likely related to the fact that Zotero has changed their snapshot format a couple of months back
    I'm not sure what this is referring to, for what it's worth. (We changed the encoding of filenames within the ZIP files used for snapshots, but that was years ago.)
    If deleting the offending snapshots would help then I could do that - but that would be a problem because I dont want to delete all snapshots, just the ones that are not available locally.
    You can't search for missing snapshots, but you can expand all items by clicking in the middle pane and pressing +, searching for "snapshot" in the search bar, and then looking for empty blue dots instead of filled ones.
  • rjclement
    OK, I went through and deleted the snapshots that were 'missing' as indicated by the empty blue dot, I then synched the database and also synched Papership. This seems to have done the trick as I can now open the PDF's in Papership. There are still a handfull of 'missing' PDF's (on all platforms) but nothing I can't reacquire.

    Thanks for your help.
Sign In or Register to comment.