Upgrade StorageZotable: a modern Zotero client for Android
Hey everyone! I've just published Zotable, my new Zotero client for Android, to the Google Play Store. It is based on Zandy but with a few new features and a seriously overhauled interface that implements Google's Material Design principles, including a floating action button (like in Evernote, Gmail, Google Keep, etc) to quickly add items via barcode scan, ISBN lookup, or PDF upload.
It is available for $2 on the Play Store (https://play.google.com/store/apps/details?id=com.mattrobertson.zotable.app) but is free software under GNU Affero license. The full source code is available for download and can be used to build the app for free in Android Studio (https://goo.gl/yCIB7H).
Check it out and let me know what you think!
It is available for $2 on the Play Store (https://play.google.com/store/apps/details?id=com.mattrobertson.zotable.app) but is free software under GNU Affero license. The full source code is available for download and can be used to build the app for free in Android Studio (https://goo.gl/yCIB7H).
Check it out and let me know what you think!
A couple of questions (these aren't just for me--I consult on Zotero occasionally and these are questions that e.g. academic librarians recommending products to their patrons will be interested in):
1. How are you planning on providing support for this?
2. How committed are you to this in terms of the amount of time/how long you'll maintain this? Obviously you can't promise, but anything you could say about that would be helpful.
3. Any new features still planned?
4. In particular -- what about groups? And is there an easy way to annotate PDFs and re-upload them (e.g. the way that works in ZotPad)? Or is there one planned?
@aurimas I'll look into it. I've developed a few apps before but have never hosted any on a versioning platform, so this would be new territory for me.
As for annotations -- to be clear, I'm not asking you to add PDF annotations as a feature (that'd be a rather major undertaking) but rather to think of a convenient way to send an attached PDF to an installed PDF annotator and then re-upload the existing file. Here's a description of how ZotPad does this:
http://chronicle.com/blogs/profhacker/working-with-zotero-on-the-ipad-with-zotpad/42094
From what I understand, reading&annotating is one of the biggest uses for tablets, so this would be very, very helpful.
(the following is more of a general observation than anything specifically about your work, and some of it is not applicable to you at all)
I'm not sure how many changes went into this new app, but I rather wish that more people contributed patches to existing projects than forking them into new ones (I don't know whether you tried or not and maybe this is a good move. I'm just saying it in general). More projects means fewer contributors per project making things move slower. Of course, sometimes this is not possible when views on project development diverge between developers (e.g. OOo and LO)
The situation is a bit more sticky given that the app is put up for sale on the Play store. I'm not saying you shouldn't do it (there are plenty of expenses involved in app development), I just think it discourages volunteer contributions if the contributors feel that you're benefiting financially from their work. Maybe being open about financial gains/losses would create a better environment for contributions (again, not saying you need to do this, I'm just observing).
In any case, I think version tracking (even if not hosted online) is a must these days, even if you're not going to collaborate with others. Hosting the project online with a decent issue tracker, patch pipeline, maybe even automated testing integration goes a long long long way too. I'd highly encourage you to invest the time into adopting one of these platforms (not necessarily GitHub, though I think it's quite good).
I'd second the suggestion about GitHub/repositories.
Likewise adamsmith's comments on annotations. I've started using Zotfile to allow me to annotate PDFs on my Kindle, then bring them back to Zotero and extract the annotations.
I've been very frustrated with Zandy.
I echo concerns above about having a sustainable ecosystem for a Android Zotero app.
Maybe we should tell this idea to the Cyanogenmod guys?
But in the discussion above why is no-one mentioning Zed by Daniel Favand (@dpfavand http://www.favand.net/zed)? It has been the best Android client until Zotable, and even now I would say they each have their advantages.
Daniel acknowledged back in July that development of Zed had stalled but he was planning to pick it up again soon.
Either way, Android is badly served for Zotero support and I really welcome all developments in this area. I'm quite prepared to pay if it guarantees continued development and support. (I've just purchased Zotable.)
I also strongly endorse the call for good integration with annotation services. This is probably not the correct place to say this but if anyone can integrate Zotero and Diigo my research life would be so much richer!
After I added a bunch of items to my Zotero library, though, Zed hung up on the sync. No matter how long I waited, it never completed. I restarted Zed several times, and restarted my Kindle, but I could never get the sync to complete. I could access items in the library (expect that the sync notification covered up the attachment field so I can't open any attachments.
It's clearly doing something, because when I tried it again today, I could see entries I had just created on Standalone. Unfortunately, it's not terribly usable in the current state.
Then, I too noticed that the developer (Daniel) no longer seemed to be working on it, so it didn't seem worth pursuing further. But if he is planning to continue, that's good news.
As for Zotable, I'd love to try it if (as above) I could get an APK so that I can load it onto my Kindle. That's actually how (and why) I tried Zed.
It is good that there is some development interest in an Android client, this is really needed. But like others before I would suggest that you try to get in touch with the other developers and see if you can work on one client together. At the moment there are at least 3 apps (Zandy, Zed and Zotable) but none of them really works properly and reliably.
Zandy isn't actively developed anymore, so forking it was imo a pretty reasonable step for someone who wanted to build on it. Zed takes a completely different development approach (the html5/js/css based ionic framework as opposed to Zandy/Zotable which are written in Java). Also Daniel hasn't open sourced Zed, so it'd currently be impossible for people to jump on that to help (that's the reason I didn't mention it above).
It's because genuine collaboration is so hard that I'm pushing for code to be made open and easily available via github the way Avram did with the original Zandy app, which made its significant code base--in particular a java API library -- usable even after he stopped putting much work into it.
I found Erathosthenes a very good reference manager on Android, and already use it for reading. As Erathosthenes is based on bibtex, it's easy enough to setup an automatic one-way sync from Zotero to Erathosthenes:
- Export a bibtex file from Zotero with Better BibTeX (use "keep updated" to update the bibtex automatically
- (Optional if you want to read attachments) Run the bibtex file through a simple sed script that changes the path to the attachment storage directory in the "file" fields in the bibtex file, to the appropriate directory on Android.
- Synchronise the bibtex file (and the Zotero storage directory, if you want attachments) to Android via dropbox or owncloud or other syn methods.
- Import the bibtex file into Erathosthenes (this has to be done manually...).
This is a bit fiddly, easy enough for technically-minded users but not feasible for non-technical users.It's only one-way so modifications on Androids don't get into Zotero, but as I'm mainly interested in reading on the Tablet, it's ok.
The one thing that Android does not have is a decent Zotero client.
In short:
Zotable gathered all the info from my Zotero library (slowly but surely)BUT it does not sync.
If fact; I scanned a book with Zotable, it never turned up in Zotero, and magically vanished from Zotable after a little bit.
I have added some things to my (online and standalone) Zotero, and they do not turn up in Zotable
I have created some notes in Zotable, and they do not turn up in Zotero.
I have checked all the settings and compared with Zotero settings and it all seems in order (user ID, API key).
So.. it _gathered_ the Zotero library and structure, but now it seems static: it is not getting the new stuff from Zotero, nor is it uploading to sync.
I appreciate any help; I am on a mission to gather and study library and book relations.. My primary goal for this is not the mobile access per se, but rather the scan function directly to Zotero: this is gold for my enquiries.
It is very frustrating that we don’t have a good android app yet. I would surely pay for such software as I did for Zandy. However, Zandy also has problems opening some of my attachments, but at least it works in 70% of cases :)
I use Erathostenes basically as he/she suggests (see details in https://github.com/rdiaz02/Adios_Mendeley#using-a-tablet, where I also have some scripts for automating all the steps).
I do like that with Erathostenes it is me who decides how and what to sync; in particular, using Syncthing (https://syncthing.net), I keep in sync the PDFs so PDF annotations are always synced between tablets and computers (and this is two-way). (I also use Syncthing to send the BibTeX file to the tablets whenever there is a change in the computers).
This way of dealing with the PDFs is something that I find much, much nicer than any WebDAV scheme. For instance, I never have to worry about realizing, while off-line that, ooooops, I forgot to download a particular PDF when I still was online. I carry my complete, updated, synced, PDF collection on my tablets.
Moreover, Erathostenes is able to (partially) use JabRef's groups and, thus, Zotero's collections.
So up to here almost perfect. (I cannot modify the Zotero entries themselves, but I can live with that since I use the tablets mainly for reading PDFs).
The catch? Erathostenes is painfully slow loading/reloading my > 3000 references bibtex file. On an Asus TF201 it can take over 10 minutes to load the db and over 30 to deal with the JabRef groups; a Nexus 7 can do it in between 10 and 20' total. Changes in Zotero, thus, are painful to update in the tablets. Moreover, Erathostenes will often just hang, and you need to kill it and restart it.
I have tried Zotable (and Zed, and Zed Lite, and Zandy), but the way PDFs are dealt with is a complete showstopper for me. So I continue using Erathostenes.
An alternative approach that I want to try is using Referey (https://play.google.com/store/apps/details?id=com.kmk.Referey). Referey is an old Android app for Mendeley that works just great (you sync the db and the PDFs as you see fit, and it works from there, including searching, sorting by several criteria, and using the Mendeley equivalent of collections, with multiple levels of nesting). What I'd need to do is prepare a script that would take the key pieces from the Zotero db and create a minimal fake Mendeley db that Referey can then work with. But I haven't had a chance to devote any time to this. No, I do not want to use Mendeley to do this Zotero-to-Mendeley sqlite structure conversion, even if Mendeley is supposed to import Zotero dbs, for at least two reasons: I no longer keep (nor want to keep) Mendeley in my machines, and after six years it seems that Mendeley is still unable to deal properly with Zotero's collections (http://feedback.mendeley.com/forums/4941-general/suggestions/389900-allow-importing-nested-hierarchical-zotero-collect). I suspect the direct Zotero-to-Mendeley mapping of the key pieces of the sqlite dbs so that Referey works should be doable.
(Modifying Referey to have it deal with Zotero's sqlite db might be an option, but I have not been able to locate a copy of the source code ---I've asked the author, without success so far---, nor do I know anything about its license or, for that matter, android programming so I am not sure I'd be able to do anything useful with the Referey code).
Sure enough, using Referey on a Zotero-to-Referey db would only allow using/modifying the PDFs, not modifying the Zotero db itself, so it ain't a full solution, but it is probably more than enough for those of us who use tablets mainly for PDF reading/annotation.
I prepared some code that takes a Zotero db and produces a db that Referey will understand. This will give you access to the PDFs in your tablet and preserves tags and collections, and it is much faster than the BiBTeX route. Available from https://github.com/rdiaz02/Zotero-to-Referey
Even if I set aggressive syncing, collections and subcollections only have partial data. It seems that to have the full library on the tablet, I would first need to click on every single lower-most subcollection so that an update is requested from each of them.
- Is anyone else experiencing this issue?
- Is this the intended behavior of "agressive syncing"?
- If it is, could there be something like "_really_ aggressive syncing" that syncs the whole library without the need to go through every single subcollection?
I've never used Mendeley, (I'm a third year masters student now, so I've just started to research), but I've been going crazy with trying to figure out the best way to use zotero (stand alone or on firefox, download add-ons or not, another app, or what) so that I can read, annotate and save my pdfs on the go in such a way that they easily sync with zotero on my home computer.
Then I realized I wasn't the only one.
I've managed to install syncthing on both my imac and android tablet successfully, I think - it does sync my zotero storage folder on both devices, at least part way.
I've definitely gotten stuck on the Referey part. I've installed it on my android tablet, but it says database not found and I don't know what to do. I've tried several different paths for the sql and the folder storing the pdfs, but I'm not doing it right.
As you can tell, I need a lot of help. I don't know any code, and even simple stuff is pretty hard for me.
I've opened my file manager, ESfileexplorer and copied fullpath of the locations of both the sql and the pdfs folder on my tablet, and then pasted it into the appropriate space in referey, but nothing - error accessing database.
Luckily I don't have much in zotero so I could easily just start from scratch, but even if I did that, what is the best way to go to use it with an android tablet and imac?
Before we get going, though, using the code in the Zotero-to-Referey github page is probably not easy. Even if I use it daily, it is a hack that I've only tested in Linux; it also requires a working installation of R (and debugging it requires knowning a little bit of R). If you only want to read and annotate PDFs maybe a standard usage of Zotfile would be simpler.
Referey and locating the db: you have to tell Referey the complete path of the sqlite db. What sqlite db? The sqlite db that the R script created in your computer. So the workflow is something like:
1. In the computer, using the R script create a Referey db from you Zotero db. So we do Zotero db-> Referey db. Let's call this newly created db "rf.db".
2. Sync (e.g., using syncthing, or whatever) the "rf.db" from the computer to the tablet.
It is the "rf.db" that Referey needs to see.
When you use Syncthing to sync stuff, you have to specify the directories you sync. Syncthing uses the term "folders" to refer to a set of files under its control. Now, the same Syncthing folder can, of course, refer to different directories in different machines (computers, tablets, etc). For example, suppose in Syncthing you call something "My_Zotero_Referey_db". This is the name of the Syncthing folder in all of your devices: this is the name of the entity that will be shared between your computer and your tablet. This is what you specify in the "Folder ID" box in Syncthing. In your computer, when you configure Syncthing, this Syncthing folder might be the directory "/home/my_username/Ref" and in your android tablet it could be "/sdcard/referey_stuff" (this is what you write in the "Folder Path" box in Syncthing).
Now, in your tablet, you would tell Referey that the "rf.db" lives under "/sdcard/referey_stuff/rf.db". Note I am giving the full path, which includes the directory (the "Folder Path" sensu Syncthing plus the file name).
If you are getting errors accessing the db it could be because the path is not correctly specified, or because the db has an incorrect format (e.g., the db was not transformed appropriately for Referey).
So a few things you might want to check:
1. Is R really converting the file successfully? For instance, does the db file produced from R have a size different from 0? And is R giving errors or warnings? (R errors or warnings should show in the .Rout file).
2. The db path in Referey, is it really to a file that exists (with that path) in your tablet?
3. Is this file that you are telling Referey about the file that was produced as the db from the run of the R code?
Once Referey sees the db, you can then worry about getting the PDF directory correct. In terms of syncing (folders, directories, etc) similar considerations apply to the PDF folder. I use separate folders for the PDFs and the db but that is largely irrelevant, although convenient.
I haven't yet discovered the bookmarklet referred to that will allow saving of references. Hints on this appreciated.
There's a lot of ongoing work on the website, so I have high hopes for some improvements on the website, too.