downloading ALL attachments at EVERY sync cycle

ali_send
This has to do with file syncing on multiple computers, using a Webdav sync server.

Hi, I'm a long-time user! (about 1850 items with 2.7 Gb attachments).
I have two computers (home/work) and am using Webdav (4shared.com) option for file syncing.

What happened: I have restored my cloud copy from my local library by using "reset file sync history" on my home computer. Then, I restored the copy on my work computer using "restore from zotero server". It all went down just fine.

The problem: since then, every time I sync zotero (at home after updating at work, or vice versa), it downloads the entire 2.7 Gb!

It seems there is something wrong about the sync history, but I've been unable to figure it out.

Any idea?!

Thanks,
Ali
  • adamsmith
    how can you tell it's downloading the entire 2.7GB? It's likely just checking the attachments.
  • ali_send
    Thank you adamsmith for your prompt reply.

    Well, this is what happens: the sync icon (circular arrow) goes around for a few minutes (perhaps to build a list of what to and not to download/upload). Then, the download/upload icon and the progress bar comes up and starts downloading about 1300 items (those with attachments in my library). This process also takes a few minutes.

    I looked at it more carefully this time; it seems that it does not actually download a file that has not been changed (it skips over it). But, why try to download it in the first place? Zotero was not like this before. Is this what you consider normal?

    Zotero used to recognize the changes and proceed to download/upload accordingly. The process was much faster too.

    Imagine that I made some notes to a pdf at home and then sync it. Next day at work, when I hit sync again to get that one file updates, I have to sit back and look at it (for about 5 minutes or so) to go over 1300 items. sometimes it chokes in between and I have to press sync again.
  • adamsmith
    no, it's not normal. But it's important to be clear on what's actually going on.

    Dan will have to look at this, but please note that the Reset options in the sync tab specifically say that they should be used only in rare situations and not for general troubleshooting. Please be mindful of these warnings; messing with the reset functions without a clear idea of why and what you're doing can easily cause problems.
    You triggered this by resetting the file sync history, which doesn't sound like it was necessary. This still should only cause this type of check once, though, so we'll look at what happened.
  • dstillman
    Two possibilities:

    1) When you click the sync icon manually, Zotero will check all local files for changes each time. If you use auto-sync, it will only check local files periodically. So if you have a lot of files and/or slower disk access (is this Windows?), you should use auto-syncing. We're hoping to improve this in the future, but doing it efficiently requires some platform-specific features that are more complicated to implement.

    2) If you have a lot of files with missing attachments in your library, Zotero might be checking for those, though it should only do that after changes are made the WebDAV server. We'd have to look at a Debug ID for a sync attempt to determine whether this is the issue.
  • ali_send
    The Debug ID is D1694356496.

    I will look forward to your response.
  • dstillman
    edited March 17, 2015
    I'm not totally sure what, but something very strange is happening. Disable your other Zotero-related extensions and see if that helps. Also check the time, date, and time zone on your system to make sure they're correct. If you still have trouble, provide another Debug ID for a sync.
  • ali_send
    I've disabled all zotero-related extensions. The issue persists.
    New Debug ID is D2054683243.

    Some more details that might help:

    The sync fiasco only happens when the action follows an earlier sync on a different computer. That is, e.g. I make a change + sync on computer one and then sync on computer 2 (to update my local copy). However, after the local copy is synced, following local change/syncs are normal.

    During the abnormal syncs, there is a subset of files (~992) that always undergoes the futile download process.
  • ali_send
    Dan, is there any update on your feedback?
  • dstillman
    Sorry for the delay. I may have fixed this in the latest 4.0 Beta. Could you install that over your current version and see if you can reproduce this? (You'll want to install it on both sides. If you use Standalone on either, you'll need to close it and perform the syncs with Firefox.)

    If you can still reproduce it, another Debug ID with the beta would be helpful.
  • claude70
    I've been having this problem too. I'm syncing between 3 computers - 2 running Windows 7, one running Windows 8.1, using my uni's Webdav storage. For months every time I did a sync on any computer with Standalone it would take around 30-40 minutes to run through all the attachments, although it didn't actually transfer any files. Eventually I found a few missing attachments, deleted them, and all was fine for a few weeks. Then it started again when I returned from an overseas trip, and this time I can't find any missing attachments.

    I've installed 4.0 beta in Firefox and been using that, but I still get the same problem. Setting sync to "automatic" just freezes up Firefox.

    I've submitted a couple of debug reports from different computers to see if there's anything obvious going on!

    ID is D2098981725
    ID is D1679816063
    ID is D1444686683
  • ali_send
    edited June 28, 2015
    I am still struggling with the issue. And, now I am sure there are others.

    It seems to me there sometime is a glitch over a specific file. I also often manage to find the file and temporary remove it, which solves the issue for a short term.

    @Dan (or Adam): The sync error is not user-friendly at all. I am unable to figure out from the error message or error report what went wrong. Is it possible to include in the error, the filename of which the sync stuck/stopped on? It would be even better to show the filenames as the sync is in progress (not the encrypted filename, but some information that allows us to actually find the corresponding entry in the dataset).

    Thanks.
  • dstillman
    ali_send: You never responded to my last post above. And I'm not sure what your latest post has to do with the rest of the thread, which isn't about sync errors.
  • ali_send
    Dan, what is unclear to you about our problem?

    1) We have multiple computers and we use Webdav sync server.

    2) Make a change on one file + sync on computer-1, then sync on computer-2 will cause Zotero (attempting) to download all attachment files (or a substantially large subset of them). Not all of these get actually downloaded but this slows down the sync process.

    3) Sometimes the sync is unsuccessful. It stops in the middle of the process and gives you an error about Webdav server. You cannot get around this by pushing the sync botton again or by setting it in automatic sync mode. There is a glitch (like a missing attachment or something).

    4) If the glitch happens when there are e.g. 9 files left for the sync to be completed, I can find the file which makes the problem. However, if it happens when there are 900 files left, it is not possible to pinpoint the problem.

    I didn't reply to your last post because I didn't see it. You send it after 20 days from my last post and by then I already given up on it.

    FYI, I downloaded the new Zotero on two new computers. Both of them have the problem #2 above. One of them has the problem #4 as well. More precisely, on this computer, every time Zotero tries to download and sync 1054 items and when it gets down to 986, it gets stuck and says "A WebDAV file sync error occurred. Please try syncing again."

    Thanks for your help.
  • fbennett
    edited July 24, 2015
    ali_send: Reading through the discussion, I see that one possible cause mentioned is a time mismatch among the machines involved in the sync. It would be good to check that, if you haven't done already.

    After that, a fresh debug ID would help Dan to look more closely - not much can be done without examining the data in the log file.
  • dstillman
    edited June 29, 2015
    Dan, what is unclear to you about our problem?
    I understand the initial issue, but you just hadn't mentioned a sync error before your previous post. That's what I was asking about.

    And yes, I'll need a new Debug ID from the latest beta.
  • ali_send
    I am submitting two debug IDs generated on two different computers. Both computers are in the same time-zone, both are using the same operating system, and running the latest Zotero (basically one is my laptop and the other my desktop).

    Here are the IDs:

    1) The Debug ID is D78262007
    2) The Debug ID is D1170937268

    Both computers have the problem #2 stated in my previous post (i.e. Zotero downloads many attachments) but the first ID (D78262007) is from the computer which also has problem #4 (Zotero stucks in the middle of the sync).

    I hope the contrast between these would shed some light on the issue.

    Thanks
  • dstillman
    Neither of those are from the 4.0 beta, so there's nothing new to look at.

    But a new version of Zotero is now out with the fix from the beta version, so you should upgrade to 4.0.27 on all your computers and see if the problem still occurs. If so, we'll need new Debug IDs from that version.
  • ali_send
    edited July 1, 2015
    The first debug ID is from Zotero 4.0.27
    The second debug id was from Zotero 4.0.26.4 (the last 4 is for 4.0 beta, right?)

    Do you want me to send screenshots of 'About Zotero' window?
  • ali_send
    edited July 1, 2015
    Alright, I have updated Zotero on all machines to 4.0.27.1

    The good news is that sync is now successful on all machines (i.e. no glitch; problem #4 is gone).

    The not so good news is that Zotero still attempts to download a lot of files.

    Here is a Debug ID: D906302962

    All Zotero related plugins and extensions were disabled on all machines.

    I sync on computer-1 (without actually changing or updating any file) and then sync on computer-2 (which the above debug report is generated on). Zotero tried to download 989 items (my library has a total of 2011 items). None of those items actually got downloaded (the remaining download was 0 mb all the time). The whole process took about 3-4 minutes.

    This situation does not bother me as long as the sync is successful and is not too slow. However, I would like to know how I might be able to fixed it.

    I look forward to your reply.
  • ali_send
    Dan, Is there any update?

    By the way, I just realized that there are some items with missing attachments.
  • dstillman
    edited July 18, 2015
    OK, sorry for the long delay. My fix above may have been the right one, but it was definitely in the wrong place in the code. Try the latest beta and let me know if this still happens.
  • ali_send
    Hi Dan,

    Thank you for attending to this issue.

    I'm sorry to say that the fix didn't do the job. I tried on two machines and the situation is still the same; Zotero attempts to download attachments. In fact, it used to download 989 items, now it does it for 1562 items (out of 2070 total items) which is effectively every item that has an attachment.

    Here are two debug IDs from two different machines.

    The Debug ID is D801630705.
    The Debug ID is D200858604.

    Thanks.
  • dstillman
    OK, I think you'll be happier with the latest beta. Give it a try and let me know.
  • ali_send
    Hi Dan,

    It seems to be working fine. I say "it seems" because I had a little issue on one of the computers (3 attachments were missing and I just manually copied them into their folder).

    Thank you very very much. I really appreciate this. It should have taken a lot of time and I hope the fix would be something that others also benefit from.

    Cheers,
    Ali
  • dstillman
    Great, thanks for the debugging help. This fix will be included in the next release (and will indeed help others).
Sign In or Register to comment.