It's been a while since I've worked on ZotDroid. A lot has happened in the intervening months, including an awful lot of travel.
I've just released version 1.2.0. Here is the important top level bit for these using personal webdav:
*** IMPORTANT *** * Please read this if you use personal webdav for storing attachments * --- This version changes the default download path. Previously "Documents/ZotDroid/temp" would hold the PDFs and other attachments if you used personal webdav.
This path has now changed to "Documents/ZotDroid/attachments". This means any downloaded attachments will need to be moved to this directory in order to be recognised by ZotDroid.
This brings the WebDav option into line with the Zotero storage. It also allows for syncing using other programs or methods (such as copying the files directly over).
You can delete all the items inside the 'temp' directory such as PDFs and similar, once you've moved them to the correct 'attachments' directory. This saves you downloading everything again.
This change should allow easier syncing using other methods such as dropbox, simply by specifying a different folder location. The temp directory is still required to hold intermediate files.
I apologise for any inconvenience this may cause. I believe it will provide better functionality going forward.
---
* Added initial top-level note support. * Changed sync to be an icon on the top level toolbar * Moved the reset and sync options to the general settings * Minor UX tweaks.
I've changed a bit of the UX. I'd like to spend more time on that but I've focused on top level notes and where the various attachments are stored.
This upgrade, at worst, will mean people will have to download all their existing attachments again. I'm really sorry for that but it brings things a bit more into line with Zotero storage and makes the checks for existing attachments a little easier. I'm hoping this leads to better syncing with things like dropbox.
Hope this reassures folks I'm still around! Cheers Ben
Really good to hear from you and to see this update.
I've installed the update, moved the few downloaded attachments I had, and also did a 'reset & sync' as ZotDroid search wasn't working at the top level.
Search works fine after the reset.
A couple of points:
- It's worth letting users know that with this update, personal webdav is turned off and the settings are reset, so they need to be added-in again.
- I downloaded an attachment with v1.2.0 but the file would not open. It looks like ZotDroid has downloaded the .zip file to "Documents/ZotDroid/temp" and a .pdf file with the correct title is put into "Documents/ZotDroid/attachments", but it is empty - 0 bytes. I did this with a couple of attachments with the same result.
Do you need any other information to resolve this bug?
Sorry to here you've been having trouble. I've not had issues on my simulated or real devices (finally! I got myself an Android phone I can test with!). I did notice a couple of times that, for some reason, adb wanted to remove the old version and reinstall the new one. Not sure why that was if I'm honest. Only occurred occasionally.
Which version of Android are you running at the moment and on what device? I can test a bit closer to your setup to see.
Thanks for getting back to me. I'm on Android 9 on a Note 9.
Would it be cleaner if I just uninstall / reinstall ZotDroid? I've got very few attachments stored on the phone so I can easily download those that I need again.
Also, there was a query about adding annotations to PDFs on the phone and whether that copy gets sync'd back to the main WebDav storage - I think we've talked about that before but I can't remember the answer!
No, we don't have annotations being fed back but I think this needs to be the next feature as lots of people have asked for it.
Perhaps the Zotero developers can help me here? Does Zotero automatically sync back PDF attachments if they've been changed externally? If so, how does that work inside Zotero? If not, what do I need to do in order to upload a new zip file to webdav or a new basic file to zotero storage?
This is perhaps the first major 'writing back' to a zotero account and therefore, is *high* risk for me. I don't want to mess up folks' PDFs you see and that could happen if I get it wrong.
I do want to add this though because it will be immensely useful!
I can't speak as a developer but I understood that the annotations (highlighting, comments, notes) I add to a PDF in whatever PDF reader / editor I am using, become an editable layer in the PDF file (unless the PDF is flattened) and as long as that PDF is saved back to storage, then these are all available to my other Zotero clients.
The challenge would be how to handle simultaneous annotating of a PDF by two clients which are then saved back to WebDav storage - but it's not something I'd ever do...
I absolutely understand your desire to make sure that is done right!
You've hit the nail on the head - sync is hard! :S One of the hardest really because of timing and risk.
However, I suspect the Zotero devs have solved this and I can jump on their method to save myself some headaches. I'll delve a little more into the API but if a nice dev is reading this, please send me a clue! :D
Hey there! I just got Zotdroid and I was super excited to start using it. However, it simply won't sync. No matter what I do I cannot start the syncing process. It immediately terminate after the window "Sync 0.0% complete" or something similar. (The window opens and closes too quick to read.)
I can log into my Zotero account without an issue and I've already restarted my tablet and re-installed the app itself.
@dstillman thanks for that. I think this might be the next feature to add. I think folks really seem to want it.
@j.j.l. Not clear what it might be. Never seen a sync quit like that before. Likely there is a crash of somekind. Which Android version are you using and on what device?
@richard.masters I've run a few tests on a couple of real devices and simulated ones and I've not been able to reproduce your problem. A reinstall might be the easiest thing at this point while I work on cleaning up the code a bit more. By any chance are you using any special settings such as saving the pdfs in a custom directory, or having the database live somewhere else than the default? Cheers Ben
Just looking back through the thread - you are using the personal webdav right? Which provider are you using? It sounds like the zip files are being downloaded but not unzipped correctly. I've managed to get Android 9 running and it seems ok so far. I'll double check the unzip routine.
Actually, a thought occurs - when you reinstalled ZotDroid did you delete the entire /sdcard/Documents/ZotDroid directory? I know I didn't mention it but I think that directory is kept over reinstalls. That means any attachments that already existed wouldn't have been redownloaded.
@richard.masters hmmmm. I can't see why Zotdroid would quit in the middle of a download like that. Maybe if you'd run out of disk space perhaps but I doubt that. I wonder if pCloud has some strangeness to it? Again unlikely.
I suspect I'll need to get a good bugtracking site setup. Currently I'm using gogs which is fine but I'll need to get some sort of better bug tracker so folks can submit things in an easier way than via this forum (which I feel is better for requests and discussion).
hi @richard.masters and @OniDaito no problems as described above with pcloud. Though after the recent update (thanks!) i initially had some issues, sorted by wiping user data and moving attachment directories as suggested etc.
on a seperate note: does anyone have any issues with basic syncing in terms of updating collections? Essentially, i have to 'reset and sync' each time in order to update. new icon does not seem to make any difference to this. Not a major issue by any means, still a great app that i'm using pretty much daily now! Thanks again @OniDaito
I paid for this app (without realizing it -- since the app doesn't allow you to try out before you pay !!) and I can't see an obvious way of moving the library to the external card on my cell phone (LG G7 Thinq). I have very limited space on this phone and don't want to use it all up by putting my zotero library in the internal storage. Is there anyway to keep the library on the external memory card?
@hari1 I think you can get a refund if it's not right for you - at least that was my belief - through the google store. If not I need to set that if I can. I've had that option when I've bought things.
To move to the external card, you can set the path in the settings section. Usually it's something like /sdcard/Documents/ZotDroid
@richard.masters - still looking for the error. Can't see it at the moment. Might have to find a different approach
@henge - by updating collections do you mean the items in the collections or the collections themselves?
@OniDaito sorry, not being clear again! I mean just a regular update, syncing to the zotero server, so that new items I've added on another device are synced across to my tablet/phone.
@henge oh dear. Again, I'm having a lot of trouble replicating these errors at the moment. Hopefully, during the usual cleanup between versions, I'll catch it.
@OniDaito Re. moving the zotero to external SD card on the mobile device, I see 2 "Locations" in the Settings under General: 1. Downloads Location 2. Database Location
Can I set both of these location to the external SD card (in different folders but under the same main folder). Can you recommend a path?
Main features are more fields for each record. Also the first steps have been taken regarding uploading altered PDFs - storage of the MD5 sums and dates. I'm hoping to get some sort of comparison working next, then uploading.
Can folks let me know what Android apps they use to 'alter' their PDFs please? That way, I can test things out a little better.
@richard.masters - I can't be sure but I *think* I found the bug affecting you. This version may have fixed it.
It is good to see further development of ZotDroid Ben - the future looks very promising!
But now for the embarrassing bit... After installing your latest update the zip problem was still there so I investigated further by running ZotDroid on an older Android phone. It worked on that so I went through the settings on my main phone and discovered I didn't have the correct WebDav settings - I had used www.pcloud.com rather than webdav.pcloud.com:443!
I had checked with "Test WebDav settings" and that came back successful so didn't give it another thought.
So, completely my fault - many apologies for wasting your time.
Thanks for letting me know! I think I'm partially responsible there because I should alter the test webdav fucntion. It merely checks a webdav directory can be browsed - it doesn't go all the way and download an attachment and unzip it. The problem with such an approach is that some attachments can be quite large. But perhaps I can scan a library and download a small one? It would be a bit more user friendly.
Yes, progress is slow - I've much on my plate but I do want to keep improving things.
Upgrade Storage
It's been a while since I've worked on ZotDroid. A lot has happened in the intervening months, including an awful lot of travel.
I've just released version 1.2.0. Here is the important top level bit for these using personal webdav:
*** IMPORTANT ***
* Please read this if you use personal webdav for storing attachments *
---
This version changes the default download path. Previously "Documents/ZotDroid/temp" would hold the PDFs and other attachments if you used personal webdav.
This path has now changed to "Documents/ZotDroid/attachments". This means any downloaded attachments will need to be moved to this directory in order to be recognised by ZotDroid.
This brings the WebDav option into line with the Zotero storage. It also allows for syncing using other programs or methods (such as copying the files directly over).
You can delete all the items inside the 'temp' directory such as PDFs and similar, once you've moved them to the correct 'attachments' directory. This saves you downloading everything again.
This change should allow easier syncing using other methods such as dropbox, simply by specifying a different folder location. The temp directory is still required to hold intermediate files.
I apologise for any inconvenience this may cause. I believe it will provide better functionality going forward.
---
* Added initial top-level note support.
* Changed sync to be an icon on the top level toolbar
* Moved the reset and sync options to the general settings
* Minor UX tweaks.
I've changed a bit of the UX. I'd like to spend more time on that but I've focused on top level notes and where the various attachments are stored.
This upgrade, at worst, will mean people will have to download all their existing attachments again. I'm really sorry for that but it brings things a bit more into line with Zotero storage and makes the checks for existing attachments a little easier. I'm hoping this leads to better syncing with things like dropbox.
Hope this reassures folks I'm still around!
Cheers
Ben
Really good to hear from you and to see this update.
I've installed the update, moved the few downloaded attachments I had, and also did a 'reset & sync' as ZotDroid search wasn't working at the top level.
Search works fine after the reset.
A couple of points:
- It's worth letting users know that with this update, personal webdav is turned off and the settings are reset, so they need to be added-in again.
- I downloaded an attachment with v1.2.0 but the file would not open. It looks like ZotDroid has downloaded the .zip file to "Documents/ZotDroid/temp" and a .pdf file with the correct title is put into "Documents/ZotDroid/attachments", but it is empty - 0 bytes. I did this with a couple of attachments with the same result.
Do you need any other information to resolve this bug?
Thanks,
Richard
Sorry to here you've been having trouble. I've not had issues on my simulated or real devices (finally! I got myself an Android phone I can test with!). I did notice a couple of times that, for some reason, adb wanted to remove the old version and reinstall the new one. Not sure why that was if I'm honest. Only occurred occasionally.
Which version of Android are you running at the moment and on what device? I can test a bit closer to your setup to see.
Cheers
Ben
Thanks for getting back to me. I'm on Android 9 on a Note 9.
Would it be cleaner if I just uninstall / reinstall ZotDroid? I've got very few attachments stored on the phone so I can easily download those that I need again.
Also, there was a query about adding annotations to PDFs on the phone and whether that copy gets sync'd back to the main WebDav storage - I think we've talked about that before but I can't remember the answer!
Many thanks,
Richard
No, we don't have annotations being fed back but I think this needs to be the next feature as lots of people have asked for it.
Perhaps the Zotero developers can help me here? Does Zotero automatically sync back PDF attachments if they've been changed externally? If so, how does that work inside Zotero? If not, what do I need to do in order to upload a new zip file to webdav or a new basic file to zotero storage?
This is perhaps the first major 'writing back' to a zotero account and therefore, is *high* risk for me. I don't want to mess up folks' PDFs you see and that could happen if I get it wrong.
I do want to add this though because it will be immensely useful!
B
I can't speak as a developer but I understood that the annotations (highlighting, comments, notes) I add to a PDF in whatever PDF reader / editor I am using, become an editable layer in the PDF file (unless the PDF is flattened) and as long as that PDF is saved back to storage, then these are all available to my other Zotero clients.
The challenge would be how to handle simultaneous annotating of a PDF by two clients which are then saved back to WebDav storage - but it's not something I'd ever do...
I absolutely understand your desire to make sure that is done right!
Best regards,
Richard
You've hit the nail on the head - sync is hard! :S One of the hardest really because of timing and risk.
However, I suspect the Zotero devs have solved this and I can jump on their method to save myself some headaches. I'll delve a little more into the API but if a nice dev is reading this, please send me a clue! :D
B
Single-file uploading is documented here:
https://www.zotero.org/support/dev/web_api/v3/file_upload
The actual upload step for WebDAV would be different, but the rest applies. The WebDAV code is here, if that helps.
As noted there, for Zotero file storage, if you store the pre-change PDF you can upload a binary diff, but that may not be worth the trouble.
(HTML snapshot uploading is similar but undocumented, but also likely irrelevant for your purposes.)
I just got Zotdroid and I was super excited to start using it. However, it simply won't sync. No matter what I do I cannot start the syncing process. It immediately terminate after the window "Sync 0.0% complete" or something similar. (The window opens and closes too quick to read.)
I can log into my Zotero account without an issue and I've already restarted my tablet and re-installed the app itself.
What could be the issue?
Cheers
@j.j.l. Not clear what it might be. Never seen a sync quit like that before. Likely there is a crash of somekind. Which Android version are you using and on what device?
Have you had any luck diagnosing my "0 byte" attachment bug?
Do you need any more information?
I haven't uninstalled / reinstalled Zotdroid but could do if you think that might help?
Thanks,
Richard
By any chance are you using any special settings such as saving the pdfs in a custom directory, or having the database live somewhere else than the default?
Cheers
Ben
I think they are in the standard locations:
/storage/emulated/0/Documents/ZotDroid
/data/user/0/computer.benjamin.zotdr0id/databases
Thanks
Richard
I've uninstalled ZotDroid & reinstalled it, downloaded an attachment, but I'm still getting 0 bytes.
The zip file has been downloaded to
/sdcard/Documents/ZotDroid/temp/
and is 80KB.
There is a PDF file in
/sdcard/Documents/ZotDroid/attachments/
with the correct name, but it is 0 bytes.
Cheers
Richard
Just looking back through the thread - you are using the personal webdav right? Which provider are you using? It sounds like the zip files are being downloaded but not unzipped correctly. I've managed to get Android 9 running and it seems ok so far. I'll double check the unzip routine.
Actually, a thought occurs - when you reinstalled ZotDroid did you delete the entire /sdcard/Documents/ZotDroid directory? I know I didn't mention it but I think that directory is kept over reinstalls. That means any attachments that already existed wouldn't have been redownloaded.
B
Yes, I'm using pCloud.
There seems to be a problem with the downloading function.
I checked the zip file downloaded via ZotDroid - it is 80KB and can not be unzipped on Android or when copied over to a Windows device.
The zip file on pCloud is 2.7MB and when I download that via Zotero on Windows it is downloaded, unzips, and is can be read okay.
Let me know if there is anything else I can check.
Thanks,
Richard
I suspect I'll need to get a good bugtracking site setup. Currently I'm using gogs which is fine but I'll need to get some sort of better bug tracker so folks can submit things in an easier way than via this forum (which I feel is better for requests and discussion).
I'll look into this bug as soon as I can.
B
It will be interesting to see if anyone else is having this problem.
It could be pCloud but it performs very well in every other respect so I doubt it. (I hope not as I bought a lifetime subscription...!)
Best regards,
Richard
no problems as described above with pcloud. Though after the recent update (thanks!) i initially had some issues, sorted by wiping user data and moving attachment directories as suggested etc.
on a seperate note: does anyone have any issues with basic syncing in terms of updating collections? Essentially, i have to 'reset and sync' each time in order to update. new icon does not seem to make any difference to this.
Not a major issue by any means, still a great app that i'm using pretty much daily now! Thanks again @OniDaito
@OniDaito - I have now tried to download a few attachments and the zip file downloaded is always 79.52 KB, whatever the size of the original!
Thank you for any help you can provide.
To move to the external card, you can set the path in the settings section. Usually it's something like /sdcard/Documents/ZotDroid
@richard.masters - still looking for the error. Can't see it at the moment. Might have to find a different approach
@henge - by updating collections do you mean the items in the collections or the collections themselves?
I mean just a regular update, syncing to the zotero server, so that new items I've added on another device are synced across to my tablet/phone.
1. Downloads Location
2. Database Location
Can I set both of these location to the external SD card (in different folders but under the same main folder). Can you recommend a path?
I've just pushed v1.2.1
Main features are more fields for each record. Also the first steps have been taken regarding uploading altered PDFs - storage of the MD5 sums and dates. I'm hoping to get some sort of comparison working next, then uploading.
Can folks let me know what Android apps they use to 'alter' their PDFs please? That way, I can test things out a little better.
@richard.masters - I can't be sure but I *think* I found the bug affecting you. This version may have fixed it.
Progress continues! :)
B
It is good to see further development of ZotDroid Ben - the future looks very promising!
But now for the embarrassing bit... After installing your latest update the zip problem was still there so I investigated further by running ZotDroid on an older Android phone. It worked on that so I went through the settings on my main phone and discovered I didn't have the correct WebDav settings - I had used www.pcloud.com rather than webdav.pcloud.com:443!
I had checked with "Test WebDav settings" and that came back successful so didn't give it another thought.
So, completely my fault - many apologies for wasting your time.
Richard
Thanks for letting me know! I think I'm partially responsible there because I should alter the test webdav fucntion. It merely checks a webdav directory can be browsed - it doesn't go all the way and download an attachment and unzip it. The problem with such an approach is that some attachments can be quite large. But perhaps I can scan a library and download a small one? It would be a bit more user friendly.
Yes, progress is slow - I've much on my plate but I do want to keep improving things.
Cheers
Ben