ZotFile - Advanced PDF management for Zotero

  • Why don't you just use 'custom location' under general settings to put all pdfs into your tablet folder so that you don't even have to deal with sending files to the tablet etc.?

    Update is pretty much what 'send' does when the attachment already is on the tablet (except the extraction but that can be changed easily)...

    A modified tag is probably what I am going to do to easily access modified attachments. I just would like to somehow automatically update this tag.

    When you modify the code to keep an original copy of the file in foreground mode, I suggest that you connect it to the tablet.storeCopyOfFile option, which currently doesn't do anything in foreground mode.
  • "Why don't you just use 'custom location' under general settings to put all pdfs into your tablet folder so that you don't even have to deal with sending files to the tablet etc.?"

    That's easy to answer - because I haven't used Zotfile to attach the files, I drag them to the Zotero window and use the renaming function in Zotero. I just checked the General Settings and it seems like I can't set the subfolders according to my Zotero collections, which is possible if I send them to tablet (love this feature!).

    "Update is pretty much what 'send' does when the attachment already is on the tablet (except the extraction but that can be changed easily)..."

    Yeah, I think the main thing I miss is the extraction of annotations. :)

    Thanks for the tip about tablet.storeCopyOfFile!
  • One bug report and one suggestion:

    Bug report: When I close all FF windows and open one again (without quitting FF), Zotfile context menu from the list pane (e.g. Rename Attachment) has 'warning' above it and doesn't work. It recovers only when I restart FF. I'm on FF 10.0, OSX 10.7.2, but it happened with FF 8, too.

    Suggestion: I hope ZotFile could automatically rename the PDFs at the time they are attached: for example, when we save items with 'Preferences -- General -- Automatically attach associated PDFs..' option on.

    The suggestion is what you saw on the mail. I'm posting just to share it with everyone. Thank you again for the great plugin!
  • ohthere, both the bug and the feature request are added to the list... thanks for the report.
  • edited February 9, 2012
    For Standalone users, Zotero 3.0.1 currently does not update zotfile automatically. 3.0.2 will solve this problem but for the time being, I recommend that Standalone users update to zotfile 2.0 manually.
  • Great Work!

    I'd like to use the "move and rename" feature to store only a link of the PDF files on a shared folder in our intranet with other users with who I share a group library. It seems to work in the personnal library but not in the group library with zotfile "Location of files" setting.

    Any suggestion?

    Thanks again!
  • afabl, Zotero does not allow any linked attachments for group library items simply because the link is only valid on your computer (you also can not create the manually). So zotfile also can not add a linked attachments to a group library item - it's just not possible in zotero...
  • Hello, just a quick question, I installed zotfile and a new category came up under the menu, something about "files sent to tablet". I deleted that category accidentally and now I can't get it back.

    Any tips?

    Btw, thank you so much for your great work! :D
  • Star7, just go to 'zotfile preference'->'adv. settings' and press the button to add the search again...
  • I installed ZotFile on the stand-alone Zotero App (Mac) and I am now looking for user instructions without much luck. I tried the 9 menu buttons on the top to no avail. Right-clicking on individual entries drops a menu that, at the bottom, lists: "Attach New File" and "Manage Attachments". I suspect those are the ZotFile items but I hesitate to try because the pdf is no longer the last downloaded item in the downloads folder.

    I appreciate the new installation instruction web page, but it might be necessary to provide a few more details for those who need the first handhold to start using it.

  • Very excited about the annotation extraction feature. That being said, I can't get it to work.

    I've tried a number of different files highlighted with ezPdf Reader on Android, or Foxit under Windows/Linux(wine) and when I click on extract annotations in Zotero I get the same result - the progress bar reaches a certain point and then just sits there. I have to escape out of the process to end it and no result are produced. The only relevant error I can seem to find is:

    [JavaScript Error: "zitem.getType is not a function" {file: "chrome://zotfile/content/zotfile.js" line: 535}]
  • SamR, yes, you found the correct menu entries. Zotfile works through 'Attach new File' and "Manage Attachments". Note that you first have to set the source folder when you are using Zotero Standalone.

    In general, I won't invest much more time in the documentation. I am happy about any contributions others are willing to make!

    looplog, did you try different pdfs? Does the problem also occurs after a fresh Firefox and Zotero start (restart FF, open zotero, right-click an item and go no 'extract annotations')? Can you disable the hidden preference 'pdfExtraction.NoteFullCite' and tell me what happens then (same error message, different one etc)? If it seems to be one specific pdf, can you send me the file?
  • First, thanks for this awesome plugin! I have a question and a couple of suggestions for new features.

    I am enjoying the annotation extractions feature, but I wondered if there was a size limit to pdfs from which annotations can be extracted? It works fine for articles, but when I try to extract from pdf books, the progress bar flashes on the screen, and then disappears without creating a note.

    1. I would love to be able to optionally extract each comment to a separate note. Then I could tag and access the notes individually later.
    2. It would be great to be able to use a markup code in my annotations to designate a zotero tag. That way I could code my annotations, and when they are extracted into notes, they could be automatically tagged. It would streamline my workflow so that I would only have to read and interact with each source one time. If my first suggestion was implemented, it would be great to have the option to apply the tag either to the specific annotation, or to the all annotations in the document.

    Thanks again for your work.
  • Guppy, no there is no page limit or anything like that. Maybe it's some unsupported pdf standard but usually you should get an error message in that case. First, you should check the error console. Then you can send me the pdf if you want or try to pinpoint the problem to specific pages by splitting the pdf (assuming the annotations are throughout the book).

    I added 1 to the list. I am not sure about 2. It would be okay to add a tag to the notes with annotated text but some specific markup that triggers a tag doesn't sound like a general solution.
  • edited February 14, 2012
    Joscha: I tried a few different pdfs. Some of them are journal articles, some created by me, some ebooks, and all produce the same result. For good measure I downloaded acrobat X yesterday to see if highlighting using that program would work but it had the same result. This is with fresh Firefox restarts. I also tried the hidden preference you suggested and got the same results. I'm beginning to suspect user error...
  • Wow! Zotfile was already awesome one year ago. Now it's almost incredible!
    Unfortunately, I don't have a tablet yet, but I'm looking forward to use the new features as soon as I manage to choose one. :)
    Thanks for the great job Joscha and any other contributors!

    I have two questions regarding the preferences. Could you please precise what the following means ?
    1) Rename files when they are send to tablet ;
    2) Only with the following filetypes (pdf,doc,txt,rtf) ;
    For 1), do you mean "rename files according to a different scheme than for local files"?
    As for 2), what actions are only for those filetypes? Is there a particular utility for this option I didn't notice?

    Lastly, I would like to know if you still consider to implement a feature which I think was already suggested, namely to avoid missing fields and their following separator.
    This for sure is a detail and not a bug. It would however be a nice perfecting, especially when one is using Zotero for managing information for which lots of metadata are missing.

    Thanks again!
    Really, without ZotFile, Zotero wouldn't be so great. I'm even not sure if I would continue to use it without your great add-on!

  • edited February 14, 2012
    thanks for the great update to version 2!
  • looplog, maybe you should try a different computer but the fact that an error message keeps popping up indicates that something is going wrong. You keep getting the "zitem.getType" error right (even after changing the hidden option)? Anything reproducible would be very helpful.

    Thierry.C, here are some more information about the options:
    When 'Rename files when they are sent to tablet' is activated, attachment files are renamed using the zotfile rules when you send them to the tablet. Otherwise, the current name of the file is preserved. There are no different renaming rules so that the option is only meaningful when you save some files in zotero with non-zotfile names.
    'Only with the following filetypes (pdf,doc,txt,rtf)' mainly refers to 'Attach new file' (i.e. the file types zotfile looks for in the download folder). Actually, it might also apply to the renaming of files. I kind of lost track...

    Yes, your suggestion is part of an issue but all this is most likely going to take some time.
  • edited February 14, 2012
    Can you tell me any reazon why ZotFile freezes up Zotero for about 2 minutes after moving a file to the tablet folder? (I'm using the "create a link when moving" option). The file would sync to the folder, then to Dropbox, then I would sync and open it on the ipad, but Zotero would still be frozen.
  • Thanks Joscha for the information!

    “There are no different renaming rules so that the option is only meaningful when you save some files in zotero with non-zotfile names.”
    Oh OK, I thought it was something like that but I was not sure.

    “Only with the following filetypes (pdf,doc,txt,rtf)' mainly refers to 'Attach new file'”

    OK, then the restriction to those extensions is not related to any incompatibility with the features or the-like I guess.

    Tonight I manage to rename and move about 400 PDFs but one (great score by the way!)
    Here is a report for the disobedient file:

    My config is Windows XP 32, FF 10.0.1, Zotero 3.02, Zotfile 2.0.
    The item and associated file were obtained through DOI:10.1016/j.trd.2010.02.002
    Nothing happens when I click "Rename Attachments". I tested several times, with restart of Firefox, deleting the item and reattaching the PDF.

    I got the following error in the console:
    Erreur : uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIFile.moveTo]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://zotfile/content/zotfile.js :: :: line 830" data: no]

    Hope this helps!
    Please tell me if you need any other piece of information to diagnose the case.

  • erazlogo, I assume that happens because zotero indexes your pdf files, is that correct? When I was indexing my files, zotero always froze and the linking of tablet attachments trickers the indexing. You can either disable indexing or switch to background mode. Does that solve your problem?

    Thierry.C, looks like the file got lost at some point. Can you open the file by clicking on the attachment in Zotero? Maybe the file was missing in the first place (which zotfile should handle). I had no problems with the DOI. What are the steps to reproduce this?
  • Joscha, yes it's enigmatic to me. I can open the file and locate it in the storage directory.

    One way to reproduce this problem:
    I create the item with the DOI. I drag-&-drop the PDF file from the desktop onto the item. Now, the attachment is visible and I can open it and locate it. When I right-click on the item and select Manage Attachments => Rename Attachments, nothing happens (except the error in the console).
    For debugging purposes, I also tried to attach the PDF to a "dummy" item with just "test" as both author and title. Also by previously renaming the file as test.pdf. Same problem. I conclude that it is the file which is responsible of the problem. But I don't understand why.

    If it may help, I could send you print-screens of Zotfile's preferences. Just tell me.
  • edited February 15, 2012
    Thanks for your response yesterday, Joscha.

    Ok, previously (2 days ago) I couldn't extract annotations from large pdfs, but if I saved the annotated pages into a smaller file, then the extraction worked. (BTW - no errors appeared in the console, and annotations were done with Adobe Acrobat 9 Pro.) But now, after updating to Zotero 3.0.2 yesterday, they all work. Maybe they are unrelated, but I am happy.

    Follow-up Question:
    I didn't understand your response to #2 -- "I added 1 to the list. I am not sure about 2. It would be okay to add a tag to the notes with annotated text but some specific markup that triggers a tag doesn't sound like a general solution."

    You say it would be okay to add a tag to notes, but not with a markup -- what are you thinking would be an alternative?

    Maybe I am using the term 'markup' incorrectly. I was envisioning an option in the settings that would toggle zotfile looking for something in an annotation that would indicate: (1)A tag (or tags) should be added to the note which will be created, and (2)what the tag(s) should be. Of course, I was hoping that this would work with suggestion 1, that each annotation could optionally be saved as a separately tagged note.

    My guess is that many who are doing research are organizing ideas in individual tagged notes that can be organized and sorted separately. Being able to label/tag them while you are reading (and annotating) would prevent the need to go back and re-read and process each note a second time.

    Thanks for your thoughts on this.
  • edited February 15, 2012
    Thanks for the Zotfile app. I love it, but I'm having the same problem as looplog, though not with every file. Sometimes the extraction works but other times the program seems to hang, doesn't extract any annotations at all, or sometimes extracts only some of the annotations (in this case, the first annotation I made, which was at the end of the text). Several Jave Script errors appear on the "Report Error" page, but this is perhaps most relelvant:

    JavaScript Error: "this._editor is undefined" {file: "chrome://zotero/content/bindings/styled-textbox.xml" line: 193}]

    I'm using Adobe Acrobat X to highlight text in the pdf's.

    Any help will be appreciated.
  • edited February 16, 2012
    Thierry.C, it's seems to be the file but I don't know what the problem is either. Maybe the file is locked? I created a zotero group 'zotfile'. When you apply for membership, I will add you so that you can upload the file. I wasn't able to make the group entirely public because public groups do not allow attachments.

    Guppy, I just don't want to implement a solution that is very user specific and not of general interest. But maybe you are right and other people are interested in something like this. I will think about adding something like this when I work on the handling of the extracted annotation stuff the next time.

    egerhardt, the pdf library used by zotfile (pdf.js) simply doesn't support all pdf standards yet. It should capture errors though, skip the page in question and put the error message on the console. For mac users, the poppler based extraction tool is an alternative but currently has problems with special characters. I suggest that you do the same as I just told Thierry and upload affected files to the zotfile zotero group. It would also be good to get more infos about the error messages, especially if they are related to a zotfile file (chrome://zotfile/...).

    Note that I only have about 15 MB left on my 100MB free account. Should be enough for a couple of pdfs but no random or very large pdfs. Usually the error can be pinned down to specific pages.
  • Thank you very much Joscha for involving so much effort into the problem!
    The file in question is only 260 ko so it shouldn't be a problem. Anyway, you can delete it when you wish.
    Besides, the file seems not to be protected. Once, he was "locked" in terms of Windows standard (because downloaded) but I unlocked it and nothing changed. In addition, I'm pretty sure I have other files that are "locked" or protected without causing this problem.
  • First I would like to add my thanks for a great program; then, of course, to raise a problem. Recently -- I'm not sure when it started -- everything I do with zotfile moves the attachment to the tablet, whether I want it to or not. That is, "attach a new file" creates a link in Zotero and puts the file in my tablet (dropbox) directory; and renaming a file also puts it in the tablet directory. Furthermore, the "Get from tablet" entry is greyed out, so the only way I can get it back is manually. This happens both in standalone (3.0.1 and, I think, the version before that) and in Firefox.
    Is there some setting I'm missing?
  • Hi, thanks for the great extractor program. It seems to work well on my local library.

    But I have a problem. I am using Zotfile to extract annotations from PDF's inside my group library, and I keep getting the error message:

    Error: uncaught exception: [Exception... "Component returned failure code: 0x80630003 (NS_ERROR_STORAGE_CONSTRAINT) [mozIStorageStatement.execute]" nsresult: "0x80630003 (NS_ERROR_STORAGE_CONSTRAINT)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: <TOP_LEVEL> :: line 145" data: no] [QUERY: INSERT INTO itemNotes (itemID, sourceItemID, note, title) VALUES (?,?,?,?)] [ERROR: insert on table "itemNotes" violates foreign key constraint "fki_itemNotes_libraryID"]

    I double checked the following already:
    1) I can manually create notes as a child of this zotero library item
    2) I have admin permissions
    3) I tried to extract annotations to the exact same pdf copied in my local library, and it works every time.
    4) Every pdf in my group library (I have 2 diff groups, 1 with admin, 1 with ownership) I get this exception error.

    I don't know what else to do?

  • edited February 16, 2012
    Joscha, I finally deciphered the origin of the bug:
    In fact, the file is not in question.
    The problem comes from an unauthorized character in one of the field that is used for generating a path, namely the colon (:) in the name of the publication.

    So, I think all fields should be processed in the same way the title field is processed for what concerns unauthorized characters, i.e., replace \/:*?"<>| on Windows platforms by a separator, or by nothing.

    Hopes this helps to improve your already great add-on!

    EDIT: Another problem happens when using subfolders (e.g., \%T\%w\%y) and a field is missing. In that case, one cannot use "locate the file" since the path is known as, e.g, "\%T\\%y" instead of "\%T\%y". It is however strange to me that directly opening the file works smoothly (as if the path was defined or interpreted differently).
    Anyway, I think the best solution to both avoid this error and provide a systematic classifying would be to replace missing field(s) by a common name specific to the field, as it should be done in a bibliography.
    root\Journal Article\Journal of Power Sources\2005
    root\Journal Article\Unknown journals\2005
    root\Journal Article\Journal of Power Sources\Unknow dates

    "Unknown journals/dates" replacement names are just examples. One could imagine "Undefined journals", "Other journals", "NA dates" or the French "[s.d.]" (sans date).
  • First, you have done a massive service to the Zotero community with this plug-in. I've been using it for a long time and it has transformed my use of Zotero.

    I have just started using 2.0, and am most excited about the extraction of comments/highlights from PDFs. On this I have two questions:

    1. Is there a way to choose what it tries to extract. When I bring in PDFs, I usually use Acrobat to recognize the text and save them in its Clearscan format (which renders the fonts as vectors, and so scales up and down in size more easily/smoothly). The extraction process seems to not like highlighted Clearscan text, however, so I would rather it only pulled out comments.

    2. Is there a way to debug why files might not cooperate with the extraction process. So far, I have only had success with one of my PDFs. All of the others either hang (with the Extract PDF Annotations sitting there until I hit Esc), or return very quickly but with no file of annotations created.

This discussion has been closed.