high data usage with webdav sync

rebelprincenyc

Hi,

Let me preface this by saying that I know webdav sync is not supported but I wonder nonetheless if anyone can help.

I sync my library to a box.com account via webdav. Roughly since the latest Zotero update, Zotero has used a very large amount of data to sync. My zotero library is 26GB and each sync uses around 7-10GB (according to Little Snitch Network Monitor). I don't know where to start trying to address this problem. I've blasted through a ton of data in the past few weeks. If anyone has experienced a similar issue or has ideas about where I might start looking to diagnose, I'd be very grateful. Thanks.

I am running Zot 5 on one iMac and one MacBook, each with latest High Sierra.

dstillman

WebDAV syncing isn't unsupported — just problems with specific WebDAV servers.

Can you provide a Debug ID for 30 seconds or so of file syncing while it's using lots of data?

rebelprincenyc

Thanks, & sorry for the exasperated tone!

Here is the debug ID: D605921354

I used opensnoop command in terminal and it appears that the data ramps up when Zotero is reading from folder ~/Library/Caches/Zotero/Profile/[randomstring].default/cache2/entries/

dstillman

onProgress with 651464296/4246498603

OK, that's an easy one — you appear to have a multi-gigabyte file that Zotero is trying to upload (and I'd guess that it's failing at the end).

Paste 'MUBTGZI5' into the Zotero search bar in All Fields & Tags mode to see the attachment in question.

dstillman

Actually, that looks like a download, not an upload. Not sure what happens at the end.

rebelprincenyc

Interesting. MUBTGZI5 returns nothing in zotero all fields & tags, but I do have a folder in my 'storage' folder with that name, which is empty on my local machine but contains a 4GB file on the server (no idea how that file became 4gb). I've deleted the file in the Box web interface & I'll try sync again.

dstillman

Make sure you're in the library root or the trash when searching. The attachment item has to exist, or else Zotero wouldn't be trying to download it.

rebelprincenyc

sure enough, it does show up; when i choose 'show file' it says 'file cannot be found. i deleted it & emptied trash

rebelprincenyc

Wow, thanks, deleting that file and attachment seems to mostly have solved the problem. I am still getting a lot of data usage on upload at a certain point and I'm not sure why. I haven't made any changes to the files I sync via webdav so nothing should need to be uploaded or downloaded. I did a debug again when it was using a lot of upload data:

D899730233

If you don't mind taking a look, is there any clue what's using up data? again thanks very much for your help.

dstillman

What counts as "a lot" of data usage?

That's Zotero checking all files against the server — downloading the tiny .prop file for each one — but not actually uploading anything, so it shouldn't be a lot of data. It also shouldn't do that under normal usage, only if you used Reset File Sync History lately.

If you let it complete, does it happen again on the next sync?

If so, can I see a Debug ID for the very beginning of a sync after a previous one completed?

rebelprincenyc

OK, thanks, it looked like it was uploading at ~1-2MB/s for a few minutes. I will resync w/ a full log and see what happens. Thanks again for the help.

rebelprincenyc

Yep, now it's syncing in under 20 seconds. Earlier it took 45min start to finish. thanks!

debug ID for last sync is D1183619966 but it should look normal now I think (as I guess the last one did).