ZotFile Reader
Streamlines process to work with Zotero and your pdf reader applications on tablet computers (iPad, Android tablets etc).

https://addons.mozilla.org/en-US/firefox/addon/zotfile-reader/

DONATION
If you are using this add-on, please consider a donation
http://www.columbia.edu/~jpl2136/zotfile.html


DESCRIPTION
ZotFile Reader is a Zotero plugin that streamlines the process to work with Zotero and your pdf reader applications on tablet computers (iPad, Android tablets etc). Building on my zotfile plugin, this add-on pushes attachments to a folder that syncs with your pdf app (such as a dropbox folder),renames them according to the zotfile rules, automatically checks whether these pdfs have been annotated, and prompts the user whether s/he wants to pull them back into zotero. It also adds a tag to the attachments on the 'reader' so that these attachments can easily be listed through a saved search. In particular, it offers the following functions:

1) Move Attachment File to Reader
This function moves the attachment to the reader destination folder specified by the user, renames it according to the zotfile rules, changes the attachment so that it links to the new location, adds the tag '_READ', and adds a time stamp. The folder should be set in the preference dialog and should be a folder that syncs with your tablet (dropbox...). The tag allows the user to easily find the attachments on the reader (you can also add a saved search for this tag). The time stamp is added to the attachment note field so that zotfile reader can recognize when the attachment was modified (e.g. annotated).

2) Remove Attachment File from Reader
This function pulls the attachment from the reader and moves it back to it's original place using zotfile's options. The function is similar to zotfile's 'Rename Existing Attachments' function but it also removes the tag.

3) Scan Reader (accessible through Zotero Actions -> Scan Reader...)
This functions checks whether any of the attachments on the tablet were updated using the time tag and asks the user whether s/he wants to a) pull the attachment, b) update the time tag, or c) do nothing.

Background and Foreground Mode
In background mode, zotfile reader leaves all zotero attachment files where they are and moves copies of the file to the destination folder when you push attachments and replaces the zotero attachment file with this copy when you pull files. Everything happens in the background. The mode is useful when you want to sync the attachment files in your zotero library across multiple computers or when you index your attachments. The disadvantage is that there are two copies of the file.
The foreground mode moves the attachment file to the destination folder and links to this file from within zotero so that there always is only one copy of your files. You can not, however, sync linked attachments to the zotero server.

GETTING STARTED
Here are the steps to get started:
1) install zotfile 1.6.2 from
https://addons.mozilla.org/en-US/firefox/addon/zotfile
2) (optional) change the zotfile renaming options so that you like them.
3) Set up some folder that syncs with you tablet. One possibility is to get a dropbox account, which syncs with PDF Expert, iAnnotate, etc.
4) install zotfile reader from
https://addons.mozilla.org/en-US/firefox/addon/zotfile-reader/
5) change the destination folder in the zotfile reader options to the folder set up under 3.
6) (optional) add a saved search with the tag '_READ'
7) please donate: http://www.columbia.edu/~jpl2136/zotfile.html

REQUIREMENT
This add-on requires zotfile 1.6.2 or higher: https://addons.mozilla.org/en-US/firefox/addon/zotfile


THIS ADD-ON IS UNDER DEVELOPMENT SO..
This add-on is still actively developed. The next big thing I plan is to integrate the extraction of annotations from pdf files. Because this add-on is still in an early stage, things might substantial change. Although I will put some efforts into the transition from one to the next version, you might have to enter some preferences again or (which is very unlikely) move your attachments again.

Most apps save them in the standard PDF format so that they are compatible with Preview/Adobe Reader but the annotation are not added as notes. So the best solution would be to extract the annotations using javascript...

yes, the plugin uses zotfile's renaming rules to rename and move the file to the dropbox or whatever folder. There is a separate sub-folder option though.
You can already check it out, by the way. It's just an early version but should be fully functional.

One other comment:

It would be nice to have a way of syncing the changes without pulling the file back. (I.e. pulling removes the file from my readings folder, but often I just want to update and read on.)

But that is something for later. Right now it is already a nice tool for organising required readings; the more important things to fix would be the workflow-related issues in my previous post.

Yes, user scenario 1 looks perfect for my kind of workflow, which also involves syncing across multiple clients.

As far as I can tell, the pdf indexing happens when the attachment is changed from "imported" to "linked" (i.e. when I do a "push") and also when it is changed from "linked" to "imported" (when I do a "pull"), though I'm less sure about the latter. In both these cases there is a lot of harddisk activity and a timeout in exactly the same way as when I import a new PDF by drag and drop, that's how I connected the dots.

@Roxy075: Make sure you don't confuse the Zotfile core with the Zotfile Reader addon. "Attach New File(s)" is a Zotfile option that has nothing to do with the reader integration. All the reader integration is supposed to do is pushing files to a designated folder (which can be inside your Dropbox) and pulling them back into Zotero if they've changed.

@ mark: I am going to add this.

@ roxy: Batch renaming should work both for zotfile and zotfile reader and you do not need to 'pull' them when you never pushed them. You can just use the zotfile's rename existing attachment function.

@ mark: Here is a new version with an implementation of two modes (I call them back- and foreground mode). You will have to start again with pushing files to your reader (the best would be if you pull all of them back before you start with this version). Can you test whether it works and report here (maybe also the foreground mode even when you don't plan to use it)?

http://www.columbia.edu/~jpl2136/zotreader_02.xpi

@roxy: Batch pulling should work but I still have the impression that there are some misunderstandings. There is nothing to pull if all of your attachments are stored in zotero. Push means to move from the current attachment location to the zotfile READER destination folder (which might be dropbox). Pull means to move them back from the zotfile READER destination folder to the location defined by the zotfile destination folder (which is different from the zotfile READER destination folder). So pulling does not make any sense when they are all stored in Zotero.

Background pushing and pulling works great; no time-lag! Clever solution, too (I never realised you could add notes to attachments directly like that). Foreground mode works still seems to work the same, including the long delays associated with detaching and re-attaching the item.

A common use case that is not addressed is the following: (1) I'm pushing a book to my reader (2) I make annotations (3) I want to update my Zotero copy with the annotations, but (4) I also want to keep reading the book on my reader. This would be a case of updating the item without pulling it back. I think it's pretty common for people to (want to) do this.

Conceivably, this would be a third (middle) option in the Zotfile Reader menu: something like "Sync changes". Presumably, this should also work the other way around: if I update my Zotero copy while the file is checked out, pulling it back should warn me that the file will be overwritten. Right now it doesn't. (But I recognize this is a minority case: if I push a book to my reader, I should be reasonably expected to update it only there until I pull it back in.)

Updating the copy of the file in Zotero is currently possible with the 'Scan Reader...' function (Zotero->Actions->Scan Reader). It's not pushing changes from the copy in zotero to the reader. Maybe I will add some warning if the att file in zotero was also updated.
I will consider adding another right click menu item although the menu becomes quit populated with zotfile & zotreader stuff.

The _READ tag has two functions: 1) 'Scan Reader' finds the items on the reader based on that tag, and 2) you can add a saved search to zotero, which allows you to quickly and easily list all the attachments on the reader. Currently, you have to do that manually but I might add some code so that the saved search is added automatically when you install the plugin.
There is a hidden option to change the tag. It's hidden because changing it will not update the tag for the attachments you have already pushed. So the option should be used with caution.

6. Use relative Dropbox URLs, with the system-specific part coded in the Zotero Reader settings. This would probably involve sth like a %Dropbox% variable in the PDF text field in background mode, rather than a hardcoded URL as it is now.

I.e. on PC1 my Dropbox is F:\Dropbox, on PC2 my Dropbox is D:\Work\Dropbox . Allow me to set the Dropbox URL in the settings and take it from there, so that pushing, pulling, and Scan Reader work wherever my Dropbox happens to be (once I have configured that right).

7. Grey out "Pull from reader" if an item is not on the reader. Grey out "Push to reader" if an item is on the reader (partly addresses point 1 above).

Hi, here is the next version. It includes three new things:
- Pulling in background mode is now more intelligent. There are now four scenarios: a) Only the file on the reader has changed -> File in Zotero is replaced, and tag is removed; b) Only the file in Zotero has changed -> File on reader is deleted, and tag is removed; c) Both file have changed -> User is prompted; d) Neither the zotero file nor the reader file has changed -> File on reader is deleted, and tag is removed.
- zotreader now checks whether an item has already been pushed based on the _READ tag. It only allows pushing if the item currently is NOT on the reader so that multiple pushs are avoided. Currently, the user is not prompted whether s/he wants to replace the file on the reader, which I might add later.
- non intrusive alert if 'Scan Reader' did not find any modified files

Can someone, presumably mark, test this?

http://www.columbia.edu/~jpl2136/zotreader_03.xpi

yes, I agree. Just didn't know how to do it right away but will look into it at some point.

Anyone here who knows something about java programming? I came across this
http://ecomware.com/notes/itext-export-annotations
and was wondering whether I can use it to extract annotations. I have no clue about java though and don't know how to get this running. I just need some external application that I can call from the plugin. So far I only found solutions that extract annotations, which are in a note and not highlighted text.

No, that actually does not work. It is only extracting highlights that are copied to a hidden note, which is only possible in Adobe Acrobat with an certain option and not with any other program or PDF App (as far as I know). Most programs I found (such as the Automator function in Mac OS) only do that and it's not helpful for my purposes. Maybe the java solution does that too but I don't know.

Here is a quote from the discussion you linked to making that point:
"However, for the Highlights to be extracted, one still has to use Acrobat's option to automatically copy highlighted content into a hidden note (and this is probably not going to change, since everything else would require me to basically rewrite pdf2txt in javascript)."

Here is the first test version with 'project folders' - i.e. user specified subfolders. Project folders can be turned on and off through the zotfile preference window.
This is a test version only and it would be good to get some feedback. Not only whether everything works (background and foreground mode?) but also about the general implementation. I am very open to suggestions. Does it make sense to have an extra window for the project folders? Would it make more sense to integrate the preferences in the main zotfile pref window? Should project folders be turned on by default? etc.

http://www.columbia.edu/~jpl2136/zotreader_04b1.xpi

Here is an screenshot that illustrates the new feature:
http://www.columbia.edu/~jpl2136/zotreader-project-folders.png

ps: project folders do not work in zotero tab might right now... i hope to fix that soon.

I really like this idea: http://www.columbia.edu/~jpl2136/zotreader-project-folders.png

I have just installed the latest beta 0.4b2 and have three comments:
1.The folder structure offered based on the collections is not in the expected order. Check your PNG above. I would expect that in the path the top collection should come first, then the subcollection, then subsubcollection etc.
2.Would it be possible to push multiple attachments at once with an option to simply use the first offered folder or all offered folders?
3.Would be great if this new collection->folder feature could also make it to the Zotfile plugin.

1) For my particular case, I would only offer the collections that the item is actually in.
2) Sorry, I meant attachments of multiple items. For example, if one goes to My Library and selects a few items that belong to different collections.

By the way, I am on Windows 7 64bit and it is working well, except for my point 1.

Thanks Joscha, the nested subfolders work now as expected.

For the multiple item selection, I would like to see two new menu items in the "Manage Reader Files" submenu: "Use the first available collection path for each item" and "Use all available collection paths for each item"

One more thing that may need some thinking is what happens if a user selects an item with an internally stored attachment such as a snapshot of a webpage. In this case it would be sensible to allow the user an option to have an additional subfolder level (configured based on item's metadata) created automatically to avoid file-name conflicts etc.

I suppose the value would be to have multiple items from different collections copied to their respective subfolders automatically.

I understand that having this done to all available collection paths for each item will create duplicates but that would be for the user to deal with. Even now, the user can simply repeat the action for the same attachment and choose a different folder.

The "use additional subfolders" option is indeed a good solution. Although, having this activated only for internal attachments would be a great bonus. I do not follow your comment on file-name conflicts, since internal attachments in multiple items may have the same name. I thought that is why Zotero creates a randomly named subfolder when attaching them.

The new version is now under review with some small changes compared to the last beta a updated. The review will take about a week but you can already download 0.4 from here:

https://addons.mozilla.org/en-US/firefox/addon/zotfile-reader/versions/

New features
- sub-folders based on zotero collections (screenshot)
- user defined sub-folders (screenshot1,screenshot2)
- redesigned UI (expandable menu to
- functions are renamed ('move' and 'remove' instead of 'push' and 'pull')
- restore selection after moving and removing attachments
- new pref.: confirmation for moving attachments that are already on the reader
- menu items are disabled if not applicable
- additional smaller changes and bug-fixes

I can try to compile and package this for 32-bit Linux systems, though I can't say C++ is my strong suit.

Hi,

here is the first version that extracts highlighted text from PDFs. The extraction is currently only working on the mac. There is a good chance that it's not going to work at a different computer but I am hopeful. Can anyone confirm whether it works or not? The extraction can be started manually from any attached pdf or happens automatically when you remove attachments from the tablet.

Best!

ps: In general, the quality of the extraction should be pretty good. There are some situation that produce problems though. Note annotations with line breaks is one of them.


New Features
- Extraction of annotations and highlights from PDFs (currently only on Mac)
- empty folders are deleted after changing user-defined subfolders
- BUG: pop-up menu does not work in second window


http://www.columbia.edu/~jpl2136/zotreader_05b2.xpi

My FF does not complain... The URL was wrong but your error report sounds like something different, right? Maybe FF doesn't allow binaries in extension packages? Can you confirm that it's not the URL?

Hey, it's a c++ program that extracts all note annotations as well as the highlighted text from your pdf attachments in zotero. You can use it in two ways: a) when you remove an attachment from your tablet, the annotations are automatically extracted and a note is added to your zotero item. b) you can do it manually with any pdf attachment by clicking Manage Reader Files->Extract Annotations in the right click menu.
At least that is supposed to be the way it works. There are still some cases, which make it fail but it should already work pretty well. At least I hope it does on other macs. It would be good to hear whether it does....

Pdfs from iAnnotate and Preview should both work...

There are multiple steps involved and it's important to pinpoint the where exactly the problem is. First, the plugin calls the binary using the location of the pdf and the output text file as an argument. Second, the program does the extraction work and saves a text file. Third, the plugin opens the text file and creates a note for the zotero item. Fourth, the plugin deletes the text file.

My guess is that the binary still does not work at other macs... :(
Can you download the file below, extract it and run the file 'ExtractPDFAnnotations-MacIntel'. The terminal should open and tell you something like 'Error: Couldn't open file '': No such file or directory.' You can also pass the path of an annotated pdf as an argument, in which case you should get the annotations printed to the terminal window. If it throws some other error at you, the binary does not work and it would be good to know what the error message is.

http://www.columbia.edu/~jpl2136/ExtractPDFAnnotations.zip

ps: and can you post your FF and OS version?