Zotfile Reader - Zotero plugin to work with tablet (iPad, Android tablets etc)

245
  • Joscha
    1) I will change that. I am wondering whether it would also make sense to offer the lower level folders as options even when the item is not in these collections. In the example, that would add '/05_Projects' and '/05_Projects/Event Project'. Does that make sense or is it inconsistent?
    2) I don't really understand what you mean. Right now, it's possible to move multiple attachments at once tot he same folder. It's not possible to move attachments to multiple folders so.
    3) I thought about that. The problem I have is that this would add two expandable menus to the right click menu (one for 'attach new file' and one for 'rename existing att'). I was wondering whether that is getting to much. I am also not sure how to handle the case one someone just wants to import attachments (in which case the folders don't make sense). But generally, I am inclined to add this to zotfile at some point.

    By the way, had anyone tried this on Windows (with the collection based folders). Does it work?
  • Lubos
    1) For my particular case, I would only offer the collections that the item is actually in.
    2) Sorry, I meant attachments of multiple items. For example, if one goes to My Library and selects a few items that belong to different collections.

    By the way, I am on Windows 7 64bit and it is working well, except for my point 1.
  • Joscha
    edited August 1, 2011
    Here is another version with the following changes:
    - functions are renamed ('move' and 'remove' instead of 'push' and 'pull')
    - folders created from collections are correctly sorted
    - new preference: How to handle attachments that are already on the reader (ignore, prompt, replace)
    - tooltip for menu items includes the folder an item is moved to
    - Bug: folders created from collections should now be correct for more then one subcollections (in the screenshot below, the 'ASR revision' folder was wrong)

    Here is the new beta:
    http://www.columbia.edu/~jpl2136/zotreader_04b3.xpi

    Here are the two options for project folders:
    a) Project folders based on Zotero collections (automatically created from selected item)
    http://www.columbia.edu/~jpl2136/zotreader-project-folders.png
    b) User defined project folders
    http://www.columbia.edu/~jpl2136/zotreader-project-folders2.png


    @Ludos: Currently, you can move the attachments of multiple items but the folders are only created for the collections of the first selected item. I will change that at some point.

    By the way, the max. number of folders is 10 at the moment. Maybe I will change that to 15 or even 20 (but that might get a little long).
  • Lubos
    Thanks Joscha, the nested subfolders work now as expected.

    For the multiple item selection, I would like to see two new menu items in the "Manage Reader Files" submenu: "Use the first available collection path for each item" and "Use all available collection paths for each item"

    One more thing that may need some thinking is what happens if a user selects an item with an internally stored attachment such as a snapshot of a webpage. In this case it would be sensible to allow the user an option to have an additional subfolder level (configured based on item's metadata) created automatically to avoid file-name conflicts etc.
  • Joscha
    I don't really see the value of "Use the first available collection path for each item" because you can simply select the first collection and all items are moved to this folder.
    I also don't think that "Use all available collection paths for each item" makes sense because it would create multiple copies of the same file. These multiple copies just create conflicts when you change the file at multiple locations (e.g. annotate the file).

    There shouldn't be a file-name conflict because these items have different file endings (pdf, jpg etc.). In addition, you can use the option 'use additional subfolders...' and the rule '\%T' to create further sub-folders based on the item type (or other metadata). Currently the logic is [Base folder]/[project folders]/[metadata folders].
  • Lubos
    I suppose the value would be to have multiple items from different collections copied to their respective subfolders automatically.

    I understand that having this done to all available collection paths for each item will create duplicates but that would be for the user to deal with. Even now, the user can simply repeat the action for the same attachment and choose a different folder.

    The "use additional subfolders" option is indeed a good solution. Although, having this activated only for internal attachments would be a great bonus. I do not follow your comment on file-name conflicts, since internal attachments in multiple items may have the same name. I thought that is why Zotero creates a randomly named subfolder when attaching them.
  • Joscha
    edited August 6, 2011
    Hi, here is a new beta version with lots of changes for user-defined subfolders.

    http://www.columbia.edu/~jpl2136/zotreader_04b6.xpi

    Changes:
    - vastly improved user defined subfolders:
    - improved interface
    - easy to change the order of subfolders
    - when the user changes the location of a subfolder, zotreader provides the option to move the attachment files to the new location

    - 15 instead of 10 max folders
    - lots of other smaller changes and bug fixes

    Any problems with this version? Otherwise I will submit it for review...
  • Joscha
    edited August 9, 2011
    The new version is now under review with some small changes compared to the last beta a updated. The review will take about a week but you can already download 0.4 from here:

    https://addons.mozilla.org/en-US/firefox/addon/zotfile-reader/versions/

    New features
    - sub-folders based on zotero collections (screenshot)
    - user defined sub-folders (screenshot1,screenshot2)
    - redesigned UI (expandable menu to
    - functions are renamed ('move' and 'remove' instead of 'push' and 'pull')
    - restore selection after moving and removing attachments
    - new pref.: confirmation for moving attachments that are already on the reader
    - menu items are disabled if not applicable
    - additional smaller changes and bug-fixes
  • Joscha
    The extraction of annotations and highlighted text is pretty much working now. I had to write a c++ command line tool based on the poppler library to accomplish the extraction of highlighted text (note annotations are pretty easy but highlighted text is very complicated). Now I just need to integrate this tool in the plugin AND the c++ program needs to be compiled for different systems. I only have a mac so can anyone do this for other systems?
  • ajlyon
    I can try to compile and package this for 32-bit Linux systems, though I can't say C++ is my strong suit.
  • Joscha
    Great! I will send you an email.
  • Joscha
    edited August 19, 2011
    Hi,

    here is the first version that extracts highlighted text from PDFs. The extraction is currently only working on the mac. There is a good chance that it's not going to work at a different computer but I am hopeful. Can anyone confirm whether it works or not? The extraction can be started manually from any attached pdf or happens automatically when you remove attachments from the tablet.

    Best!

    ps: In general, the quality of the extraction should be pretty good. There are some situation that produce problems though. Note annotations with line breaks is one of them.


    New Features
    - Extraction of annotations and highlights from PDFs (currently only on Mac)
    - empty folders are deleted after changing user-defined subfolders
    - BUG: pop-up menu does not work in second window


    http://www.columbia.edu/~jpl2136/zotreader_05b2.xpi
  • samuelas
    Hi--Firefox tells me that the latest version appears to be corrupted and will not install.
  • Joscha
    My FF does not complain... The URL was wrong but your error report sounds like something different, right? Maybe FF doesn't allow binaries in extension packages? Can you confirm that it's not the URL?
  • samuelas
    No, it was just the url, thanks.

    I'm curious about the extraction function, how to access it and what it does.
  • Joscha
    Hey, it's a c++ program that extracts all note annotations as well as the highlighted text from your pdf attachments in zotero. You can use it in two ways: a) when you remove an attachment from your tablet, the annotations are automatically extracted and a note is added to your zotero item. b) you can do it manually with any pdf attachment by clicking Manage Reader Files->Extract Annotations in the right click menu.
    At least that is supposed to be the way it works. There are still some cases, which make it fail but it should already work pretty well. At least I hope it does on other macs. It would be good to hear whether it does....
  • samuelas
    It sounds like it will be a fantastic feature. At the moment I'm not getting anything to show up: it does "extract" the annotations, that is to say, it is processing them, but not taking out the data, so I end up with a note that says something like: "Extracted Annotations (Fri, 19 Aug 2011 14:10:46 GMT)" but nothing else.

    Now perhaps it can't read what I'm doing--but it DOES recognize which files have been annotated of course and which haven't. I tested with annotations made with iAnnotate and with Preview--it recognized both but extracted neither. Did I explain this well?
  • Joscha
    edited August 19, 2011
    Pdfs from iAnnotate and Preview should both work...

    There are multiple steps involved and it's important to pinpoint the where exactly the problem is. First, the plugin calls the binary using the location of the pdf and the output text file as an argument. Second, the program does the extraction work and saves a text file. Third, the plugin opens the text file and creates a note for the zotero item. Fourth, the plugin deletes the text file.

    My guess is that the binary still does not work at other macs... :(
    Can you download the file below, extract it and run the file 'ExtractPDFAnnotations-MacIntel'. The terminal should open and tell you something like 'Error: Couldn't open file '': No such file or directory.' You can also pass the path of an annotated pdf as an argument, in which case you should get the annotations printed to the terminal window. If it throws some other error at you, the binary does not work and it would be good to know what the error message is.

    http://www.columbia.edu/~jpl2136/ExtractPDFAnnotations.zip

    ps: and can you post your FF and OS version?
  • synoptic
    Hi there, great plug-in , really helping to streamline my workflow.

    I was wondering if it is currently possible to pull an annotated PDF from the reader but store it as a copy in the Zotero item, rather than have it overwrite the original? Ideally I would like to append a prefix such as 'Annotated_' to the filename in order to differentiate the annotated and non-annotated versions.

    Also, it appears as though the plug-in has problems pulling annotated copies of the files if the 'Use Zotero to Rename' option is set but the files have been pushed without being renamed in the first place. Does that make sense?

    Great work though, exactly the sort of thing that is needed.
  • Joscha
    no, that is not possible at the moment but it's already on my list. The filename will most likely get a 'postfix', which is either set to something like _annotated or can be changed by the user. I am pretty sure that it will still take some time though...

    I will look into the potential bug but I am wondering how you can push without renaming?
  • synoptic
    Hey Joscha, great to know you are working on an implementation of this feature, I look forward to trying it out.

    Regarding the 'bug'...

    If I import an item into Zotero from Science Direct, for example, and also have the associated PDF imported automatically, I get an attachment with the name 'ScienceDirect Full Text PDF' even though the actual file name has been converted according to the parent metadata (e.g. 'Chen et al. - 2007 - Dual-energy X-ray radiography for automatic high-Z.pdf').

    I then push this file onto my reader's dropbox folder without first converting the item according to parent metadata, make some annotations and upload back into the dropbox folder before finally attempting to pull it back into Zotero. Although the file appears to be in the Zotero storage folder I cannot open the annotated copy from within Zotero itself.

    I will play around to see if this is always the case...
  • prjohnston
    I think zotero/zotfiles/zotfile reader are great bits of software and I am trying to use them to make my paper reading, annotating and storing much easier.

    Yesterday I downloaded zotfile reader 0.4 from Mozilla and it is really great to have the zotero file structure preserved in the reader (and subsequently on Dropbox). However, I do have one question. In trying to copy files to the reader and into the appropriate folders simply by highlighting the entries using "Manager Reader Files", the program asks if I wish to move all the files into the folders, not just the PDFs. This is despite the fact that my Zotfile preferences specifies only the PDF filetype (I have not ticked the "Attach all files ..." box) and I have ticked the "Only move attachments ...." box in the zotreader settings. Have I missed something? I would have thought that with this setup I should only be moving PDF files?

    Otherwise, great stuff ....

    Thank you.
  • Joscha
    @synoptic: okay, I see. Now it's more clear. I assume you are running zotreader in background mode, right? I will look into it when I find time. Let me know when you find out more.

    @prjohnston: Do you just want to move the pdf attachments to the reader and not the other attachments? If that is the case, the "Attach all files ..." option has nothing to do with it. You have to check "Only move attachments ....", which I think you did, AND specify the file types you want to move in under 'General Settings' but this change will also affect zotfile. Does that solve your problem?
  • prjohnston
    Thanks, Joscha, for your reply. I had already tried the set up you suggested, including restricting the filetype just to PDF. To give you a more concrete example: I have a folder in zotero with three entries. To these entries I have attached 3 PDFs and 2 HTML files. When I highlight all entries (with command A on my mac) and select "manage reader files -> move to subfolder on reader" the java script asks me if I want to move five files. Given my setup, I would only be expecting to move the three PDFs?

    I do have another problem, this time of my own making. I set the zotreader directory to be in my dropbox, which is fine and copied some files to it. For various reasons I then deleted the directory under the dropbox and now the reader won't copy files it thinks are already there. Is there some way I can reset zotreader and start again from the beginning?

    Thanks again.
  • Greg
    edited August 24, 2011
    I also realized that the counting is not always accurate. The question is whether the html files are also moved despite the settings. Are they?
    I get prompted when I want to move files that are already on the reader so that it should work. You can also manually remove the '_READ' tag from the attachment and delete the attachment note.
    Or maybe zotero's link points to nothing because the files does not exist. I am not sure how zotreader behaves in that case...
  • Joscha
    @prjohnston: Yes, the counting is only based on the number of attachment files. I am not sure whether I will change that because it would require to check every attachment before prompting the user. Probably not a big computational burden but still...

    @prjohnston: Linked attachments that point nowhere are ignored so that might be the reason. Are you using the foreground mode? Otherwise there should be a prompt and you should be able to 'remove' to the reader.
  • Joscha
    edited August 26, 2011
    Here is a new beta for 0.5. The extraction of annotations should now work for macs. You have to install the program separately though (ZotFile Pref->ZotReader->Download & Install...). Windows and Linux users should get a greyed out 'Download & Install...' bottom and a warning message saying 'This feature is currently only available...'.

    Can a mac AND a windows/linux user confirm this behavior?


    New Features in 0.5
    - Extraction of annotations and highlights from PDFs (currently only on Mac)
    - user-defined sub-folder configuration is now in a separate window
    - empty folders are deleted after changing user-defined subfolders
    - BUG: pop-up menu now works in second FF window

    Changes from 0.5b2 to 0.5b3
    - user-defined sub-folder configuration is now in a separate window
    - extraction tool is now separate download from the preference pane
    - improved extraction (does not fail with line breaks)
    - first note line is bold
    - new option for 'true page' and full in-text citation



    http://www.columbia.edu/~jpl2136/zotreader_05b3.xpi
  • mark
    Here's a Windows user. Yes, the button is (sadly) greyed out.
  • Joscha
    I know, kind of sad. Maybe I still have a windows laptop somewhere in ny but I am not even sure and won't be able to check before mid September. I am actually more hoping for a compiling expert...
  • Joscha
    edited August 26, 2011
    Here is a new beta that is compatible with Zotero Standalone.

    http://www.columbia.edu/~jpl2136/zotreader_05b4.xpi


    New Features in 0.5
    - Extraction of annotations and highlights from PDFs (currently only on Mac)
    - Compatible with Zotero Standalone
    (for instructions see here and remember to first install zotfile 1.6.4)
    - user-defined sub-folder configuration is now in a separate window
    - empty folders are deleted after changing user-defined subfolders
    - BUG: pop-up menu now works in second FF window

    Changes from 0.5b3 to 0.5b4
    - Compatible with Zotero Standalone

This is an old discussion that has not been active in a long time. Before commenting here, you should strongly consider starting a new discussion instead. If you think the content of this discussion is still relevant, you can link to it from your new discussion.

This discussion has been closed.