Upgrade StorageTransaction Costs of Coordinating Zotero Library and the iPad (through iAnnotate)
I had been waiting to buy an iPad until I was pretty sure I could read and annotate my Zotero library on it, and then sync my annotations back into Zotero with minimal effort. After reading about the experiences of others, I decided iAnnotate was far enough along to do at least that much, so I bought an iPad two months ago.
Without much trouble, I have been able to use the "Aji Reader Service" software that comes with iAnnotate to sync my Zotero storage folder with my library in iAnnotate on the iPad. Essentially, the way I have it set up, the software uses a wireless network (not the cloud) to allow the iPad to store a copy of the .pdf attachments in my Zotero storage folder on the iPad hard drive. the .pdf files can then be read and annotated (comments, highlights, underlining, etc.) in iAnnotate, and then saved on the iPad with the annotations. The software then allows the iPad to push the annotated .pdf back into the Zotero storage folder where it originally came from. Then, if I use Zotero to open one of the .pdf attachments I annotated on my iPad and pushed back into the Zotero storage folder, it comes up with the annotations I made on the iPad.
It is certainly worth the MONETARY cost of the iPad to have all your journal articles on the iPad. It is simply amazing to be able to instantly pull-up any article I've saved in Zotero, especially at conferences and during class. There are other unbelievably useful academic tools available on the iPad (such as reviewing RSS feeds of abstracts for OnlineFirst or recently published journal issues*) that make it worth the MONEY, but I'd like to get some feedback on ways to reduce the EFFORT it costs to maintain a Zotero library on the iPad.
At the moment, these are the transaction costs I have had to pay:
1) iAnnotate only displays .pdf attachments, so screen shots, text documents, and other non-.pdf attachments need to be converted into .pdfs in order to access them on the iPad. I am fortunate enough to have access to Adobe's .pdf maker, but PrimoPDF is a free .pdf-maker that is available on the web (along with many, others I'm sure).
2) Every .pdf attachment needs to be carefully named, as the iAnnotate library only displays the .pdf file name. The "rename file from parent metadata" feature does a good job of this, but you have to make sure to use it.
3) You really get much more from your library if your .pdf attachments are readable and indexed. For many articles, this means running an OCR program on the .pdf. Fortunately, the Aji Reader software does the indexing when it copies files from the storage folder to the iPad, it just takes a really long time (at the moment, I have a little over 1100 .pdfs on the iPad from Zotero).
4) You need to be very conscious of where the most recent version of a file is when syncing; if you tell the iPad to pull an old file from the storage folder when it should have pushed the newly annotated file into the storage folder, it will overwrite your annotations. I do most of my reading and annotating on the iPad, so I am usually pretty confident in first pushing annotated files into the Zotero storage folder on my computer, and then telling the iPad to pull copies of any new .pdf attachments I have saved through Zotero on my computer. But, it still takes a bit of conscious effort.
5) The syncing process can take some time (especially when you have a larger collection), and you can't (or shouldn't) use the iPad for other things while it is syncing. You also need to do this on a wireless network, which I don't have access to while at the office.
6) There also seems to be more than a few dead attachments in the Zotero storage folder that get onto the iPad. As the other points have suggested, your Zotero library needs to be pretty meticulously maintained to make this work, and in the process of cleaning-up my Zotero library I have copied and deleted references and attachments. This has led to files, attachments, and even empty folders in my Zotero storage folder that are not part of my cleaned-up library that appears in Firefox. These dead attachments get pulled over to the iPad with everything else in the storage folder, and they take up sync time and iPad memory. I haven't figured out a good way to take care of this yet, but I'm sure it will take a good bit of time as well. I'm hoping to come up with something better than clearing all the .pdfs off the iPad, cleaning-out the storage folder "by hand," and then having the iPad re-pull all the attachments again.
7) Another effort intensive feature of coordinating the Zotero and iAnnotate libraries involves the conversion of iPad annotations into Zotero notes. iAnnotate can generate summaries from the highlighting and comments made on a .pdf, but that summary needs to be generated, emailed, copied, and then pasted into a note that needs to be created in the Zotero library. Including finding the citation under which to put the note, this involves at least 6 steps for every article annotated on the iPad. I haven't done too much of this because it is so labor intensive.
8) I have also noticed that .pdf attachments saved in group folders appear in separate places in zotero storage, so this means they appear twice in my iAnnotate library on the iPad.
9*) While I am not as eager as others to use the iPad to actually create entries in Zotero (I prefer to somewhat limit my iPad capacity to simply reviewing articles), I have used a combination of other applications to ultimately get materials read on the iPad into Zotero. As mentioned earlier, another great academic use for the iPad is reviewing newly published journal articles. By searching the journal publishers' websites and the features within the research databases my university subscribes, I was able to find RSS feeds for most of the journals relevant to my discipline. I have entered these RSS feeds into GoogleReader, and by using the free MobileRSS app on my iPad (as well as the GoogleReader interface on my laptop) I receive the title, author, abstract, and link to the full-text for recently published and OnlineFirst content from all the journals I follow (as well as relevant blogs, news, websites, government agencies, research centers, etc.). The MobileRSS app allows me to "star" especially interesting content, as well as writing a note about the RSS content. Back on my laptop, I can go to GoogleReader through Firefox, go to the content I have "starred" on the iPad, and then use Zotero to capture the content in the almost-old-fashioned-way.
I hope this is helpful for those who aren't yet sure about the pecuniary costs of the iPad as a journal reading tool (as well as those have not yet tried coordinating Zotero and iAnnotate libraries), but I am hoping this can be the start of a productive discussion about reducing the transaction costs of coordinating the Zotero library with the iAnnotate library on the iPad.
D.McC.
Without much trouble, I have been able to use the "Aji Reader Service" software that comes with iAnnotate to sync my Zotero storage folder with my library in iAnnotate on the iPad. Essentially, the way I have it set up, the software uses a wireless network (not the cloud) to allow the iPad to store a copy of the .pdf attachments in my Zotero storage folder on the iPad hard drive. the .pdf files can then be read and annotated (comments, highlights, underlining, etc.) in iAnnotate, and then saved on the iPad with the annotations. The software then allows the iPad to push the annotated .pdf back into the Zotero storage folder where it originally came from. Then, if I use Zotero to open one of the .pdf attachments I annotated on my iPad and pushed back into the Zotero storage folder, it comes up with the annotations I made on the iPad.
It is certainly worth the MONETARY cost of the iPad to have all your journal articles on the iPad. It is simply amazing to be able to instantly pull-up any article I've saved in Zotero, especially at conferences and during class. There are other unbelievably useful academic tools available on the iPad (such as reviewing RSS feeds of abstracts for OnlineFirst or recently published journal issues*) that make it worth the MONEY, but I'd like to get some feedback on ways to reduce the EFFORT it costs to maintain a Zotero library on the iPad.
At the moment, these are the transaction costs I have had to pay:
1) iAnnotate only displays .pdf attachments, so screen shots, text documents, and other non-.pdf attachments need to be converted into .pdfs in order to access them on the iPad. I am fortunate enough to have access to Adobe's .pdf maker, but PrimoPDF is a free .pdf-maker that is available on the web (along with many, others I'm sure).
2) Every .pdf attachment needs to be carefully named, as the iAnnotate library only displays the .pdf file name. The "rename file from parent metadata" feature does a good job of this, but you have to make sure to use it.
3) You really get much more from your library if your .pdf attachments are readable and indexed. For many articles, this means running an OCR program on the .pdf. Fortunately, the Aji Reader software does the indexing when it copies files from the storage folder to the iPad, it just takes a really long time (at the moment, I have a little over 1100 .pdfs on the iPad from Zotero).
4) You need to be very conscious of where the most recent version of a file is when syncing; if you tell the iPad to pull an old file from the storage folder when it should have pushed the newly annotated file into the storage folder, it will overwrite your annotations. I do most of my reading and annotating on the iPad, so I am usually pretty confident in first pushing annotated files into the Zotero storage folder on my computer, and then telling the iPad to pull copies of any new .pdf attachments I have saved through Zotero on my computer. But, it still takes a bit of conscious effort.
5) The syncing process can take some time (especially when you have a larger collection), and you can't (or shouldn't) use the iPad for other things while it is syncing. You also need to do this on a wireless network, which I don't have access to while at the office.
6) There also seems to be more than a few dead attachments in the Zotero storage folder that get onto the iPad. As the other points have suggested, your Zotero library needs to be pretty meticulously maintained to make this work, and in the process of cleaning-up my Zotero library I have copied and deleted references and attachments. This has led to files, attachments, and even empty folders in my Zotero storage folder that are not part of my cleaned-up library that appears in Firefox. These dead attachments get pulled over to the iPad with everything else in the storage folder, and they take up sync time and iPad memory. I haven't figured out a good way to take care of this yet, but I'm sure it will take a good bit of time as well. I'm hoping to come up with something better than clearing all the .pdfs off the iPad, cleaning-out the storage folder "by hand," and then having the iPad re-pull all the attachments again.
7) Another effort intensive feature of coordinating the Zotero and iAnnotate libraries involves the conversion of iPad annotations into Zotero notes. iAnnotate can generate summaries from the highlighting and comments made on a .pdf, but that summary needs to be generated, emailed, copied, and then pasted into a note that needs to be created in the Zotero library. Including finding the citation under which to put the note, this involves at least 6 steps for every article annotated on the iPad. I haven't done too much of this because it is so labor intensive.
8) I have also noticed that .pdf attachments saved in group folders appear in separate places in zotero storage, so this means they appear twice in my iAnnotate library on the iPad.
9*) While I am not as eager as others to use the iPad to actually create entries in Zotero (I prefer to somewhat limit my iPad capacity to simply reviewing articles), I have used a combination of other applications to ultimately get materials read on the iPad into Zotero. As mentioned earlier, another great academic use for the iPad is reviewing newly published journal articles. By searching the journal publishers' websites and the features within the research databases my university subscribes, I was able to find RSS feeds for most of the journals relevant to my discipline. I have entered these RSS feeds into GoogleReader, and by using the free MobileRSS app on my iPad (as well as the GoogleReader interface on my laptop) I receive the title, author, abstract, and link to the full-text for recently published and OnlineFirst content from all the journals I follow (as well as relevant blogs, news, websites, government agencies, research centers, etc.). The MobileRSS app allows me to "star" especially interesting content, as well as writing a note about the RSS content. Back on my laptop, I can go to GoogleReader through Firefox, go to the content I have "starred" on the iPad, and then use Zotero to capture the content in the almost-old-fashioned-way.
I hope this is helpful for those who aren't yet sure about the pecuniary costs of the iPad as a journal reading tool (as well as those have not yet tried coordinating Zotero and iAnnotate libraries), but I am hoping this can be the start of a productive discussion about reducing the transaction costs of coordinating the Zotero library with the iAnnotate library on the iPad.
D.McC.
I too have just acquired an iPad for the very same reasons.
I haven't investigated the Aji Reader Service yet however, though that looks like a good workflow.
I am currently using the Dropbox functionality to export individual PDF's as and when I need them. Basically, I select the item I am interested in and click the 'Show File' button in Zotero to bring up the folder that the file is stored under. Because I have mapped the Dropbox folder on the PC as a network drive I can right click the file and send it to Dropbox using the context menu that appears (i.e. using the 'Send to' option).
Using iAnnotate the file can be downloaded and re-synced back to Dropbox after annotation. This does mean that you have to manually drag and drop the newly-annotated file back into the appropriate Zotero item afterwards, but it is reasonably quick to do. If you just want to move a few files at a time it is a workable method in my experience.
I also agree that use of an RSS reader to view recent articles is a great idea. I look forward to the Zotero Everywhere API being implemented by the iPad app writers. I can see huge opportunities to integrate Zotero into a number of information processing tools. Being able to send a citation to Zotero direct from an RSS entry would be great!
As a previous Apple-sceptic I have to admit that my experiences of the iPad have altered my perceptions considerably. If you are a Zotero user considering an iPad I agree it is worth every penny, even as it stands. As integration becomes better implemented I imagine it will become a killer combination.
Has anyone found a way to work around this problem?
I spent many hours this week trying to switch to Mendeley because of their flat directory structure, but the bugginess of the program became too much for me. I much prefer Zotero and I'd really like to stick with it.
Thanks!
Thanks for sharing all the information. Here are a couple of comments:
1) Not that I know anything about that, but maybe Firefox mobile will allow to run zotero on honeycomb tablets at some point. There also is a pdf annotation software for android: https://market.android.com/details?id=com.cerience.reader.app
But who knows whether that will ever happen (and maybe it's something the ipad users don't want to hear )
2) Has anyone tried PDF Expert as an alternative to iAnnotate? Seems very nice
3) You can use the zotfile plugin to more conveniently name your pdfs. You don't even have to store in the zotero storage folder but you can also link them from any place.
1) Install DropBox on your computer and iPad
2) Write a script that will scan your Zotero items and move these to a human readable folder structure in DropBox and then setting up a symbolic link from the original location to the new location.
When you edit your item in iAnnotate, the annotated PDF would be synced with DropBox that zotero items are linked to. If you add or remove items in Zotero, you would need to redo the linking, so this should be set up as a scheduled task.
See this tread for a script for making a human readable folder structure of your Zotero items. This could be modified to do the step 2 relatively easily.
http://forums.zotero.org/discussion/16148/create-a-humanreadable-folderstructure-from-zoterostorage/#Item_21
I have gotten into the bad habit of pulling citation information (and the article .PDFs) for materials that I would like to read, rather than just the materials I have already read and summarized. With this in mind, I have decided to break-down the syncing process into one method for "already read and reviewed items" and another process for "as yet unread items." Here is the gist of what I have in mind:
For "already read and reviewed items"...
Pretty much the same process I described in my earlier post. These types of materials seem fairly appropriate to exchange through the Aji Exchange software that comes with iAnnotate. Even though it takes a while, these types of materials shouldn't be modified all that often, so it should be ok if the longer exchange process is only done periodically (I'm thinking about once every few weeks).
For "as yet unread items"...
First, I turned off the automatically attach associated .pdf option in zotero. (I hate to lose this great function, but I think I'm over-using it and it is getting me into trouble.)
Second, I've created a Dropbox folder for unread articles
Third, I've added the zotfile plugin (which I actually wasn't aware of...thanks Greg).
Fourth, I've configured the zotfile options to put files into that Dropbox folder and leave a link in zotero. (This configuration will really only help with newly added materials that I haven't read yet, but it will allow me to access .pdf articles through zotero, even if I haven't read them yet, without including them in my zotero storage folder.)
Fifth, I'm planning to use the saved search options in both "My Library in zotero" and "Windows Explorer in my zotero storage folder" to identify those .pdfs that are "read" (=stay in zotero) or "unread" (=moved to the designated dropbox folder, and then hyperlinked to citation in zotero). I'm not looking forward to taking the time to do all this, but it seems like a necessary step.
After that, the rest of the steps are procedural. Essentially, 1) pull the readings from the dropbox folder into iAnnotate as ready to read; 2) read and annotate the download article in iAnnotate; 3) when done, email the annotation summary to myself; and, 4) using the list of emailed notes as a guide, migrate the actual .pdfs for the articles I've read from the dropbox folder to the zotero storage folder.
I'm going to work on this transition over the next few weeks and I'll post again about how it works. If anyone else is interested in using dropbox, following this link will get you an extra 250MB of space over the 2G that come with every registration (and I get an extra 250MB as well): http://db.tt/JpBvdxX
I hope this helps, and I'm interested in any suggestions you all may have.
D.McC.
There is a beta version of the zotfile plugin at the end of this discussion, which also allows you to change the name and location of already added attachments (also in batch processing):
http://forums.zotero.org/discussion/5301/3/zotero-plugin-to-rename-move-and-attach-your-pdfs-to-zotero-items/
There are some other useful news things but it's kind of a mess to find them because you have to go through the whole discussion...
Greg
I've started moving things over to this two-sync-source approach, and I'm becoming more convinced this is the way to go.
Unless you want to pay for Dropbox space, you're really limited in your ability to store ALL you .pdf articles there. However, the 2-10GB of space you can get for free make is great option for what could be thought of as a reader's RAM. The larger and less frequently accessed articles in zotero's storage folder (and synced through the Aji Exchange program) are more like the reader's harddrive.
Since Dropbox is a much better option for quickly moving readings and annotations between the iPad, it is a great place to store your queue of readings that will need to be read and modified more frequently than can be accommodated by the Aji Exchange program.
After they're read, annotated, and don't require the faster and easier syncing through Dropbox, moving the .pdf into zotero and allowing the Aji Exchange server to keep the article in sync with the iPad is perfectly appropriate. (It may take longer, so you'll probably do it less frequently, but it will still keep your annotations synced and there isn't a limit on the amount of data that can be transferred.)
I guess now all we need is the right combination of plugins/add-ons to make the logistics of this process more efficient and user-friendly.
"zotfile" seems like the right tool for pulling attachments in through zotero, moving the .pdf article to an "unread articles" folder in dropbox, and then leaving a link to the re-located article under the proper citation in zotero.
Once the articles are in the "unread articles" folder in dropbox, iAnnotate on the iPad can be used to pull the desired article(s) onto the iPad, either one at a time or all at once, they can be read and annotated, and then sent back to dropbox.
This gets us to a point where there is a mix of read/annotated and unread articles in the "unread articles" folder in dropbox. My plan had been to email annotations to myself, and then use those emails as a guide for individually relocating the articles I've actually read from the "unread articles" folder in dropbox back into zotero (as well as converting the emailed iAnnotate summary into a note for that article). Obviously, I'd like to find a better solution than doing this for every article individually.
I'm not yet aware of something that could do this automatically. Does anyone know of something that works like a "reverse-zotfile" that would use the linked file under a citation in zotero to relocate the externally saved .pdf article back into zotero?
My ideal solution would be a feature built into iAnnotate on the iPad (which might need to work in combination with an iAnnotate add-on for zotero) that would allow a reader to initiate all of the following steps by pushing a button after reading and annotating an article:
1) send the annotated article and summary back into dropbox
2) execute a "reverse-zotfile" process that moves the .pdf article from the dropbox folder to zotero (which places it in zotero's storage folder)
3) along with #2, turn the annotation summary into a note in zotero
4*) here's what would be a really nice step...the next time the Aji Exchange program is used to sync the zotero storage folder with iAnnotate on the iPad, the version of the article that was already saved locally on the iPad (when it was downloaded from the "unread articles" folder in dropbox) is automatically deleted from the iPad when that same file arrives through iAnnotate.
I can think of a few other nice tweaks and features that would be nice, but these seem like the next most important in the process of moving towards a more efficient zotero+iPad+iAnnotate integration.
So what do people think? Does anyone know of a reverse zotfile?
Thanks,
D.McC.
Question: Is 1 already done automatically by iAnnotate? I mean at least for the pdf? It would be great if iAnnotate also automatically saves the annotation summary as a txt file on dropbox (or wherever the pdf is coming from). You should ask them whether they can implement that - it might also be relevant for non-zotero stuff.
Also, have you ever tried 'PDF Expert' to annotate? I am still wondering how that compares to iAnnotate...
http://www.bobdeschutter.be/?p=498
The only feature that might be missing is syncing annotated PDFs back to Zotero, but if you are just tiniest bit capable of scripting then you can easily just adjust the included Python script to have a two-way sync. I might write that one day, but right now I don't need it and I don't have time for it. And I'm just hoping Zotero brings their own app to the iPad in the near future. :-)