Is it possible to copy the whole zotero folder to another computer first, then sync through webdav?

Jackliu008
Dear Zotero users,

I have a large zotero library (30G) in my laptop and now I would like to sync the library with a new desktop computer. I setup the webdav sync using ADrive, but, as you can imagine, it will take very long time to upload the storage data to ADrive server (after 5 hours, I only get 2G done). I imagine it will take at lest the same amount of time to download the data to the other computer.

So my question is: to save the download time (from ADrive to my desktop), is it possible that after I sync my laptop with ADrive, I just copy the whole Zotero folder from my LAPTOP to the desktop so that there is no need to download the data from the server (to my desktop)? Then I can start the synch between the desktop and ADrive for new modifications.

I have a feeling that this may not work, but I'd really need your suggestions.

Thank you very much for your time and help.

JackLiu
  • adamsmith
    yes, this will absolutely work. Described as "Option A" here:
    https://www.zotero.org/support/kb/transferring_a_library
  • Jackliu008
    Thanks a lot!
    But I wonder, after I copy the Zoetro to the new computer, if I setup the sync between ADrive and the copied Zotero in the new computer, will the files uploaded in ADrive (from my laptop) still need to sync back to my desktop Zotero, even though these files are already copied there (in the desktop)? In other words, does the Zotero copied to the new computer "know" the files in the new desktop are the same as in the ADrive webdav and therefore no sync is needed for these files? This is the part I want to achieve in order to save the download (sync-back) time.

    To illustrate my question again:
    Usually we may sync computer A to webdav, then from wedav to computer B, but now what I want to try is to first sync computer A to webdav, then, copy zotero from computer A to B, next sync computer B to webdav only for sync of future modulations between Computer A and B through webdav.

    Thanks again.
  • adamsmith
    yes, Zotero knows the files are the same (the attachments have the same database ID and the files are in subfolders with the same, unique key).
  • Jackliu008
    This is great! Thanks a lot for your help!
  • Jackliu008
    Dear Adam,

    I encountered another question and I wonder whether I could have your help.

    I uploaded about 16G data to the ADrive through webDAV sync, which took about 4 days (due to the slow upload speed at my home). There are still 9 G left. After I realized that the upload speed is much faster at my school (on my new desktop), I copied the whole zotero folder to the new laptop, and set up the sync again.

    I thought Zotero in my new desktop should continue the sync from where I stop. But after a few hours, 14 G data were uploaded and it still uploading. The data in the ADrive now are already much larger than I have in my computer Zotero storage folder.

    So I wonder whether my desktop started a new upload, i.e., did not continue from where my laptop left, instead, it will upload all the data again. If so, should I redo the upload to get rid of the (potential) replications in the ADrive? Or is it normal that the data in the webDAV take more space than that in the computer?

    Thanks a lot.
  • adamsmith
    yes, if it's mainly PDFs, they could be a bit larger than what you have locally. What you did should work; Zotero wouldn't duplicate files on the WebDAV.
  • aurimas
    However, if during your file transfer the Date Modified timestamps were updated, Zotero may be re-uploading some of the files that are already on the WebDAV server.
    The data in the ADrive now are already much larger than I have in my computer Zotero storage folder.
    That doesn't sound right actually. If anything, the files should be smaller, since Zotero ZIPs them up before uploading (some PDFs are already "optimally" compressed, so the file size _may_ increase marginally). Are you sure you selected the same exact path on the WebDAV server to upload your files to? You can check your WebDAV storage and see how many files are in the Zotero folder. It should be twice the number of attachment _folders_ you have locally in your storage directory (there's a small .prop file included with each attachment). Note that each attachment _folder_ may contain multiple files that end up getting ZIP'ed up into a single package for upload.
  • Jackliu008
    Thank you very much, adam and aurimas.

    I suspect something may have gone wrong:

    My zotero storage is slightly more than 24G. I uploaded 16.1G from my laptop to ADrive. Then I copied the Zotero folder to my desktop at school and set up the sync using exactly the same path as used in my laptop Zotero. Within a day, my desktop uploaded 20.1 G, making the total data in ADrive 36.1G, when it stop, i.e., finished, the upload.

    When I got home and opened Zotero in my laptop, it first showed to check data in server for above 10 minutes, then started to upload again. It shows there are still about 3900 files to upload. After a night of uploading, it uploaded 60% of the files and increased 2.7G data in ADrive. Calculating this way, these 3900 files will give another ~5G data in ADrive. Put together, it will make 41G data in ADrive, this roughly equal to the original 16.1G I uploaded from my laptop, plus my whole zotero storage. But I am not sure whether this number relation is just coincident, or indeed some part of the data was duplicated. (I did notice that in windows, not zotero, the files in storage folder that were copied to my desktop indeed showed new modified time, i.e., they all have the time when they were copied in to the new computer.)

    This morning, my desktop at school first showed a syn id error. When I clicked again, no error information any more and it started to check server. No files are uploaded or downloaded, and it seems satisfied :)

    In ADrive, I cannot check the files. When I click Zotero folder, it shows to display the files but the files within zotero folder never shows up. When it takes too long, it says I need to login again. I suspect in zotero folder there are too many files and take too long to display.

    In this situation, I am thinking of two options:
    1. wait until the 40% of the 3900 files in my laptop to be uploaded, then if everything works fine, just let it be.
    2. if this definitely won't work, can I just deleted all the data in the ADrive, then start the sync again only in one computer (likely my desktop because it has faster upload speed). After it is done, than turn on the other computer to let it check to see what happens.

    Sorry for the long story, and I really appreciate your time and help.
  • Jackliu008
    Here is the ending of the story: finally both of my computers showed it finished the sync. But when I click the attached file in Zotero, it says the file cannot be located.

    Magically, ADrive showed the total storage takes 20.1G. I also reset sycn history in my desktop Zotero and let it reupload several Giga files, but still attached PDFs cannot transfer between the two computers via ADrive.

    I cannot see the files in ADrive. Tech support from ADrive said this is because there are too many files (>36000) and display them will take too long.

    Since I don't know what is the problem, I am not sure whether it will work if I delete the data in ADrive and redo the sync.

    Any suggestions will be very much appreciated.
  • aurimas
    Since I don't know what is the problem, I am not sure whether it will work if I delete the data in ADrive and redo the sync.
    Should be safe to delete everything on ADrive, reset file sync history in Zotero, and then redo the sync .But if ADrive is showing 20GB, isn't that correct? (i.e. the data is not duplicated)
  • Jackliu008
    Yes, it seems the data is not duplicated. The data on my computer is slightly more than 24G, but, as you mentioned, if they are compressed, they could take less space.

    I have already reset file sync history (although without deleting all the data in ADrive) and now both computer seems to finish uploading. I never saw they downloaded anything.I checked the storage folder on both computers and indeed there was no new downloaded files/folders.

    So is it possible that something is wrong when the computers try to download data from ADrive? It seems less likely the uploading failed.

    Thank you very much for your help.
  • aurimas
    when I click the attached file in Zotero, it says the file cannot be located.
    is this on one computer or on both? Is that the case for all attachments?
    So is it possible that something is wrong when the computers try to download data from ADrive? It seems less likely the uploading failed.
    not sure I follow. Can you elaborate?
  • Jackliu008
    "is this on one computer or on both? Is that the case for all attachments?"

    As mentioned earlier, I copied the whole zotero folder from one computer to another. Then I set up the sync in both computers and spent a few days to get all files synced with ADrive. However, all attachments for for newly added items in one computer cannot be found in the other computer.

    When I checked the Zotero storage folder in both computers, I found that no folders/files were downloaded from ADrive, because the latest folders are only the ones that were added in the local computer. Because I cannot see the content of the folder in ADrive, I don't know whether the newly added attachments were indeed uploaded. But I did not see any error information after I added new items. So probably these items were uploaded. If so, the problem may be the download part, i.e., for some reason, one computer cannot download the attachments uploaded from another computer.

    Again I cannot be certain whether the newly added attachment were indeed uploaded into Adrive. But is there a record in Zotero that can show whether all files were uploaded?
  • aurimas
    Are you completely certain that the sync settings on both computers are the same? If that is the case, and both computers say that the sync has completed, then we'll need a debug ID from both computers for the first sync after you reset file sync history (disable automatic syncing and click the sync arrow to perform the sync)
  • Jackliu008
    aurimas, thank you very much for your help. I will try the debugging procedure later.

    Before that, I notice that very often there is a sync error when I just open Zotero or when I activate my computer from sleep mode. I submitted the error report, # 1846625033. I wonder whether it has some clue for the problem i have.

    thanks again.
  • Jackliu008
    I forgot to mention if I click the sync button on the right upper corner, zotero start to sync and no error information any more.
  • dstillman
    [JavaScript Error: "xmlhttp.responseXML is null" {file: "chrome://zotero/content/xpcom/storage/webdav.js" line: 1654}]
    Your WebDAV server (or something in between you and it) is returning invalid responses, and you're getting sync errors. (I'd be surprised if the server passes "Verify Server" in the Sync pane of the preferences, but regardless, you're getting various WebDAV errors during the sync process.) You'll need to resolve those with your WebDAV provider.
  • Jackliu008
    Thanks Dan Stillman.

    For both computers, "verifying serve" always shows it is successful. But I also clicked the debug button (zotero in my laptop) and found a report with many items that seem to be related to the sync of attachment files. I don't know whether the information is helpful (I sent the debug report, debug id: D1750130694). But it also indicates something seems wrong with webdav.

    I tried a couple of times. Although Zotero did not explicitly show error info most of the time, the debug file always contains lines indicating webdav errors. Does this mean indeed the communication between my computer and ADrive have problems? Or is it possible that my web browser has problem?
  • dstillman
    That debug output is incomplete — it only includes the beginning of a sync attempt before any errors — but yes, this is a problem with your WebDAV provider. If you try providing a Debug ID again I might be able to tell you more, but this isn't going to be something we can fix on our end.
  • Jackliu008
    Thanks a lot, Dan.

    To make sure this is the right way to tackle the problem, I provided again a debug file (D295208265) after I open Zotero on my laptop. This time Zotero did not give error information.

    I will get a debug file from my desktop tomorrow when i get to school. I hope this can help to confirm whether the same problem occurred in both computers.

    Thanks again, and I really appreciate your time and help.
  • dstillman
    Here's the error:
    (3)(+0000000): HTTP PROPFIND <propfind xmlns='DAV:'><prop><getlastmodified/></prop></propfind> to https://[...]@www.adrive.com/webDAV.php/zotero/

    (3)(+0005219):
    Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 84 bytes) in /usr/share/php5/symfony-1.4.8/lib/plugins/sfDoctrinePlugin/lib/vendor/doctrine/Doctrine/Connection/Statement.php on line 314
  • Jackliu008
    Hi Dan, thanks very much again.

    I checked the debug file from my desktop Zotero and found the same errors that you pointed out above.

    I am not familiar with these computer terms, so, just to confirm, this has nothing to do with my own computers, right? If this is purely a webdav issue, I will send the two lines of error info to ADrive tech support to see whether they can help to solve the problem.

    Thank you very much.
  • dstillman
    It's an issue with the WebDAV software on the server, yes.
  • redcloud111
    Jack

    Did you ever get Zotero working with ADrive? I am looking for an affordable solution for a large database. Let me know. Thanks!
Sign In or Register to comment.