Hi there...i have a question according this nice tool. Using it since several months now and works perfect. but would it be possible to add an option that it also clears an attachment from my harddisk if i delete an entry (with attachment) in zotero?
Hi schuessi, sorry but don't think that I will add that. It's a pretty different from a programming perspective and I rather leave the way files are handled after deleting a zotero item to zotero....
There is a new zotfile beta for version 2, which streamlines the process of annotating pdf files on your tablet, extracts pdf annotations and saves them as a zotero note:
I have the last version of firefox for Mac and Zotero 2.1.10, I installed zotfile (different versions, I have tried a lot) but renaming option doesn't work! I have uninstalled and reinstalled everything several times, but nothing...! any help? Thanks
@jws02459: Yes, it is compatible with Firefox 3.0 - 10.*. Can you be more specific abou 'stopped working'?
@igg: What do you mean with the 'renaming option doesn't work'? What exactly are you doing and what happens? Are you trying to rename a pdf or a different file?
I try to rename a pdf, but nothing happens. The other options (as send pdf to tablet) work ok, but it doesn't rename pdfs and no error message appear (it just do nothing!).
@igg: sorry, but I still can't do anything with that. I need something reproducible. What are your zotfile options? e.g. for 'location of files'? Does the 'Preview of Current Renaming Rules' feature in the zotfile preference pane work? Can you also check 'Tools->Web Devel.->Error Console' in Firefox for any zotfile related stuff after you encounter the problem the next time? For zotfile bugs, the 'Source File' should be something like 'chrome://zotfile/content/...' (most likely zotfile.js). If there are a lot of things, it might be helpful to clear the error console, then execute the actoins that caused the problem and then check again.
I have checked the error console after trying to rename a pdf, it says: error.zotero.Utilities.removeDiacritics is not a function
The 'Preview of Current Renaming Rules' feature always has "update" in the button, I click and it tells me to select an item, then I can rename it manually, and then everything stays as before.
Regarding location of files, I have tried all the options but nothing changes, renaming option doesn't work!
New Features in beta 5+ - extraction of pdf annotations for all platforms (thanks to Joe Devietti) - UI change: 'Attach new File' moved back to main right-click menu - accents and other special characters are removed from filenames (only zotero 3+)
Joscha, could you please update the ZotFile website with links to the newest beta's? I like to be able to check what the latest version is without trailing this thread for updates.
And what happened to the Addons listing? Would be great if it could be auto-updated from withing FF!
Extraction of PDF annotation and highlight into Zotero is amazing. Both work well from Acrobat and Preview (I'm on Mac); the in-text citation for the highlighted snippets is particularly neat. This is a huge help.
Two questions: I have many PDFs with annotations and highlighing done in Skim; when I run "Extract Annotations" the progess bar briefly shows up but nothing seems to be extracted – is there anything that can be done about this? (Presumably connected with Skim's way of handling annotations.) I tried exporting the file from Skim with the "embedded note"-option and ran "Extract Annotations" on it – with no results. I fished out this error from Zotero's logs: [JavaScript Error: "dir.exists is not a function" {file: "chrome://zotero/content/xpcom/file.js" line: 207}]
In the last Beta you've added the removal of accents and special characters in filenames. (I have many of those.) Is it possible to leave accents and other diacritica intact in filenames, maybe as a hiddenpref? Thanks for your great job on Zotfile. kithairon
There is any way to have syncronised the pdfs in zotero and ipad (dropbox) so if you annotate something in the pdf in the ipad, it automatically changes the zotero pdf adding the annotations? (Without doing "send to tablet", "get from tablet" or "extract annotations" functions?)
@mark yes, you are right. Here is now a permanent link to the current beta, which I will also put on the (outdated) zotfile page: http://www.columbia.edu/~jpl2136/zotfile_2b.xpi I will put 2.0 on the mozilla addon listing when it's out of beta. The review process easily takes 2 weeks so that it's not an option for these quick bug fixes. I don't think that there will be any more features for 2.0. Just some remaining bugs and improvements to the pdf.js extraction.
@kithairon Skim: The problem is that Skim does not use the PDF standard to add annotations but instead their own annotation standard. As a consequence, these annotations can not be extracted with zotfile. It also should be impossible to look at them in preview. The export with "embedded note"-option should work though. I just tried it and had no problems. Can you reproduce your error? If you can, tell me what exactly you did and provide a Debug ID (http://www.zotero.org/support/debug_output). Maybe it's a specific pdf file? special characters: I have added an option to the advanced setting tab for removing special characters in the filenames. It's disabled by default (not sure whether I should enable or disable it by default). It's going to be in the next beta or you can already get it from github https://github.com/jlegewie/zotfile
@igg You can do this by changing this hidden 'zotfile.tablet.mode' option to 2. With mode=2, the pdf is moved to your tablet folder and added to the zotero item as a linked attachment. Maybe that has side effects you don't want but it should do what you asked for. I haven't tested this option recently so that I am not completely sure whether everything works with mode=2.
I have found zotfile.tablet.mode option and changed it to mode 2, but it doesn't work; nothing happens, I have to press the "get from tablet" option to get the annotations made in the tablet, it doesn't do it automatically...
Re: Skim. I checked again and tested with a few files. Glad to report that it works as you say when the pdf is exported as "with embedded notes". (Must have been the specific pdf I tried on the first account.)
The output with the pdf.js extraction method is generally a more reliable:
sometimes: - Anchored / Fixed notes (mostly; see Debug ID is D369159605) - picks up diacritica quite well (Indic, European)
doesn’t extract: text notes
There are a few pdfs where the anchored text is not picked up like in the sample below. (The Zotfile report box showed up in all of them).
The debug ID D369159605 is for a recent Lancet article, where your plug-in handles the highlighted text but doesn’t pick up the anchored note. (The Lancet, 377 (2011) 74-84. doi:10.1016/S0140-6736(10)61422-5 ). A „Zotfile Report“ comes up telling me to look at the JS error console. There the verdict is:
Error: error while reading annotations of page 7 of file:///Users/amw/Documents/Zettelbank/Zotero/storage/2DJMMP3J/Leichsenring et al._2011_Borderline personality disorder-embedded.pdf TypeError: annotation.get("Name") is undefined Quelldatei: chrome://zotfile/content/pdfextract/extract.js Zeile: 14
For some of the successfully extracted notes the annotated text in Skim occurs doubled in the Zotero note: occasionally as clean (italic) text, on other occasions garbled:
Let me know if more debug output is needed, or any other test would be useful. I'm grateful that already this much of the Skim annotations can be handled – and would love to see the handling of the anchored text notes reliably.
@igg: You first have to send the files again before anything changes. mode=2 is the 'foreground' mode. You can look at the description in the first post on http://forums.zotero.org/discussion/18737/1/zotfile-reader-zotero-plugin-to-work-with-tablet-ipad-android-tablets-etc/ So it doesn't do any syncing but there is only one version of the pdf so that the pdf on dropbox and in zotero is the same. You can access the hidden options through about:config: In Zotero Firefox: type 'about:config' in the url bar In Zotero Standalone: 'Actions->Preferences->Advanced->Open about:config' and change zotfile.tablet.mode to 2 for foreground mode Sorry, I can't check the foreground mode. I think everything should work but there are just to many scenarios so that I can't fully test everything. I am happy to fix bugs when you find them though.
@kithairon: Thanks for the extensive report. Can you mail me example pdfs with the different problems and your annotations? I count three: anchored/fixed notes problem, doubled text, and garbled text. The page that causes the problem is enough. By the way, you can also test whether extraction is possible at all by trying to select and copy/paste text. Pdfs without a clean text layer just won't work…
New Features in zotfile 2 - easily send PDF attachments to your iPad or Android tablet - extraction of pdf annotations for all platforms (thanks to Joe Devietti) - redesigned preference dialog - and many more...
Here is the (hopefully) last beta 9: http://www.columbia.edu/~jpl2136/zotfile_2b.xpi It includes some general fixes and minor improvements for the extraction of pdf annotations. I will start the mozilla review process soon if nothing comes up.
New in version 2 - Sync Zotero Attachments with your iPad or Android tablet - Extract Annotations from PDF Files - redesigned preference pane - many more features and bug fixes
@hcsiller: I could offer the different tags as options similar to the collection based subfolders. Maybe I will look into that but right now there are still some other things on my list.
@mark: Thanks. By the way, FF 11 might solve the subfolder syncing problem because one of the new features is addon syncing. I haven't tested it though and it might even complicate things with standalone. So I will probably go through a .json file on dropbox.
@theed: I fixed a number of bugs, which might have solved your problem. Let me know if you run into it again ('Send to tablet' now also catches errors and shows a 'ZotFile Error' message instead of just crashing). Not sure whether I will do something about the 'ok' vs 'yes'... (I am using the javascript confirm, which can easily be manipulated).
Excellent work with Zotfile 2, it's nice that Zotfile Reader was merged in and the extract annotations feature is awesome!
I use Zotfile with Dropbox and wonder if anyone has any suggestions how I can streamline my workflow. All syncing is handled by Dropbox, so I don't want to move files back and forth. The foreground mode is great for this purpose but it could still be better:
1. When I use "Scan Tablet Files...", I only have the option to either update the Zotero file, or to get it from the tablet. Is it possible to extract annotations from all changed files without moving them?
2. Can I tell Zotfile to automatically delete old notes when I extract new annotations?
3. I want to keep two copies of each file, the original file and an annotated copy. What's the best way to achieve this? Since I'm using the foreground mode, "Save copy of annotated file with suffix" isn't useful to me.
It would be very nice if Zotfile could copy files instead of moving them, with an optional prefix/suffix added to the filename, and add a link to the copied file in Zotero. Something like a hybrid between the current foreground and background mode; I don't mind if it's a hidden setting. This way I can have my working (annotated) copy on all my devices and keep the original in Zotero, e.g. for sharing with colleagues.
Hi! @hcsiller: yes, just select all the items and click 'extract annotations'. Maybe first test with ~10 pdfs. Zotfile extracts the annotations for all pdf attachments of selected items. There might still be problems with some pdfs but zotfile should capture any error, continue and put the error on the console. Let me know if it crashes because of a particular pdf.
@Mogger: 1) I have some improvements for the "Scan Tablet Files..." functions in mind but they are still going to take some time. The idea is to show the user a list of all modified items with different options and not prompt for each attachment separately. For now, I might add an option to 'select all modified attachments', which would make it possible to do what you want. 2) No, that's not possible right now. Maybe I will add an option to prompt for that but I am not sure because deleting the old note assumes that the user did not make any modifications. 3) I created an issue for that.
@Joscha: Awesome! I was thinking a bit about how to make Zotfile easier to use for my use case, here's what I've come up with so far:
1. In addition to "Send" and "Get", there should be an "Update" action. This is especially useful if you want your attachment in one place, e.g. on Dropbox. More about this below.
2. When using "Scan Tablet Files...", it shows how many files were modified and asks you to either update or get all attachments, or treat them individually. For the last option it could maybe show the result of a search in Zotero and let you handle them there.
3. Make it clearer in the preferences dialog what exactly "Send", "Get" and "Update" does. I was thinking something like this:
---- Send to tablet Send action: [dropdown list: Copy files, Move files] [checkbox] Add link to sent files
[checkbox] Rename files¹
Get from tablet [checkbox] Save copy of annotated file with suffix [textbox]² [checkbox] Automatically extract annotations
1. Would be nice with a "current filename" wildcard to allow you to simply add a prefix/suffix to the current name, e.g. prefix_%f_suffix. 2. This option should probably be disabled if you choose "Move files".
----
etc. The wording can (and probably should) of course be changed, but I hope you get the idea.
Regarding the notes, what's the use case in keeping them? Personally I just want the highlights from the current version of the document. Maybe there could be an option in Advanced Settings to either update the current note or create a new note when extracting annotations.
Mogger: I will think about how these ideas fit into zotfile. The scan reader function will get an update and I am thinking about easy ways to access modified tablet attachments. That might include some kind of update function but I am not really sure what you mean by that. I thought it should basically update the file on the tablet if the zotero file has changed and the other way around (or prompt if both have changed).
The use case with the notes is that I edit them after extracting annotations. These changes would get lost. But I added a ticket for an additional option...
Note that anything but bug fixes is probably going to take some time...
About "Update":
Currently Zotfile seems to work best if you want to send attachments to your tablet, read/add annotations, and then send them back to the computer. "Update" would be for those who want all files in a central place (e.g. Dropbox folder). I guess you could also see it as "Sync"; its function should be similar to getting the files from the tablet and sending them back again.
"I am thinking about easy ways to access modified tablet attachments":
An idea I had was to tag such files with a tag, e.g. _MODIFIED, and let Zotero show all attachments with that tag. The get/update action would remove the tag.
Notes:
Ah, I didn't even consider editing the notes, good that you pointed it out. And thanks for adding the Issue about the option!
No worries about time, it's completely understandable and the current version of Zotfile already works great. I might poke around in the code to try to figure out how to make Zotfile keep an original copy of the file (or I can just add it manually).
Upgrade Storage
http://forums.zotero.org/discussion/18737/3/zotfile-reader-zotero-plugin-to-work-with-tablet-ipad-android-tablets-etc/#Comment_111985
(Please comment on this beta in the zotfile reader thread)
@jws02459: Yes, it is compatible with Firefox 3.0 - 10.*. Can you be more specific abou 'stopped working'?
@igg: What do you mean with the 'renaming option doesn't work'? What exactly are you doing and what happens? Are you trying to rename a pdf or a different file?
The 'Preview of Current Renaming Rules' feature always has "update" in the button, I click and it tells me to select an item, then I can rename it manually, and then everything stays as before.
Regarding location of files, I have tried all the options but nothing changes, renaming option doesn't work!
Any ideas?
Here is the new version:
http://www.columbia.edu/~jpl2136/zotfile_200b7.xpi
New Features in beta 5+
- extraction of pdf annotations for all platforms (thanks to Joe Devietti)
- UI change: 'Attach new File' moved back to main right-click menu
- accents and other special characters are removed from filenames (only zotero 3+)
And what happened to the Addons listing? Would be great if it could be auto-updated from withing FF!
Thank you so much!
Two questions: I have many PDFs with annotations and highlighing done in Skim; when I run "Extract Annotations" the progess bar briefly shows up but nothing seems to be extracted – is there anything that can be done about this? (Presumably connected with Skim's way of handling annotations.) I tried exporting the file from Skim with the "embedded note"-option and ran "Extract Annotations" on it – with no results. I fished out this error from Zotero's logs:
[JavaScript Error: "dir.exists is not a function" {file: "chrome://zotero/content/xpcom/file.js" line: 207}]
In the last Beta you've added the removal of accents and special characters in filenames. (I have many of those.) Is it possible to leave accents and other diacritica intact in filenames, maybe as a hiddenpref?
Thanks for your great job on Zotfile.
kithairon
@mark yes, you are right. Here is now a permanent link to the current beta, which I will also put on the (outdated) zotfile page:
http://www.columbia.edu/~jpl2136/zotfile_2b.xpi
I will put 2.0 on the mozilla addon listing when it's out of beta. The review process easily takes 2 weeks so that it's not an option for these quick bug fixes. I don't think that there will be any more features for 2.0. Just some remaining bugs and improvements to the pdf.js extraction.
@kithairon Skim: The problem is that Skim does not use the PDF standard to add annotations but instead their own annotation standard. As a consequence, these annotations can not be extracted with zotfile. It also should be impossible to look at them in preview. The export with "embedded note"-option should work though. I just tried it and had no problems. Can you reproduce your error? If you can, tell me what exactly you did and provide a Debug ID (http://www.zotero.org/support/debug_output). Maybe it's a specific pdf file?
special characters: I have added an option to the advanced setting tab for removing special characters in the filenames. It's disabled by default (not sure whether I should enable or disable it by default). It's going to be in the next beta or you can already get it from github https://github.com/jlegewie/zotfile
@igg You can do this by changing this hidden 'zotfile.tablet.mode' option to 2. With mode=2, the pdf is moved to your tablet folder and added to the zotero item as a linked attachment. Maybe that has side effects you don't want but it should do what you asked for. I haven't tested this option recently so that I am not completely sure whether everything works with mode=2.
Where are the hidden options?
Could you check mode 2?
Thanks!!!!
The output with the pdf.js extraction method is generally a more reliable:
pdf.js
exracts successfully:
- underline
- highlight
sometimes:
- Anchored / Fixed notes (mostly; see Debug ID is D369159605)
- picks up diacritica quite well (Indic, European)
doesn’t extract:
text notes
There are a few pdfs where the anchored text is not picked up like in the sample below. (The Zotfile report box showed up in all of them).
The debug ID D369159605 is for a recent Lancet article, where your plug-in handles the highlighted text but doesn’t pick up the anchored note. (The Lancet, 377 (2011) 74-84. doi:10.1016/S0140-6736(10)61422-5 ). A „Zotfile Report“ comes up telling me to look at the JS error console. There the verdict is:
Error: error while reading annotations of page 7 of file:///Users/amw/Documents/Zettelbank/Zotero/storage/2DJMMP3J/Leichsenring et al._2011_Borderline personality disorder-embedded.pdf TypeError: annotation.get("Name") is undefined
Quelldatei: chrome://zotfile/content/pdfextract/extract.js
Zeile: 14
For some of the successfully extracted notes the annotated text in Skim occurs doubled in the Zotero note: occasionally as clean (italic) text, on other occasions garbled:
"$EKLGKDUPDVDPXFFD\D:KDWLVPLQGIXOQHVV"7KHQRQIRUJHWIXOQHVV \D3UDGKDQHGS@" (Wallace 0000:3)
Let me know if more debug output is needed, or any other test would be useful. I'm grateful that already this much of the Skim annotations can be handled – and would love to see the handling of the anchored text notes reliably.
So it doesn't do any syncing but there is only one version of the pdf so that the pdf on dropbox and in zotero is the same.
You can access the hidden options through about:config:
In Zotero Firefox: type 'about:config' in the url bar
In Zotero Standalone: 'Actions->Preferences->Advanced->Open about:config' and change zotfile.tablet.mode to 2 for foreground mode
Sorry, I can't check the foreground mode. I think everything should work but there are just to many scenarios so that I can't fully test everything. I am happy to fix bugs when you find them though.
@kithairon: Thanks for the extensive report. Can you mail me example pdfs with the different problems and your annotations? I count three: anchored/fixed notes problem, doubled text, and garbled text. The page that causes the problem is enough. By the way, you can also test whether extraction is possible at all by trying to select and copy/paste text. Pdfs without a clean text layer just won't work…
http://www.columbia.edu/~jpl2136/zotfile_2b.xpi
New Features in zotfile 2
- easily send PDF attachments to your iPad or Android tablet
- extraction of pdf annotations for all platforms (thanks to Joe Devietti)
- redesigned preference dialog
- and many more...
http://www.columbia.edu/~jpl2136/zotfile_2b.xpi
It includes some general fixes and minor improvements for the extraction of pdf annotations. I will start the mozilla review process soon if nothing comes up.
https://addons.mozilla.org/en-US/firefox/addon/zotfile/versions/
and the website provides some new information but 2.0 (e.g. hidden options):
http://www.columbia.edu/~jpl2136/zotfile.html
New in version 2
- Sync Zotero Attachments with your iPad or Android tablet
- Extract Annotations from PDF Files
- redesigned preference pane
- many more features and bug fixes
@hcsiller: I could offer the different tags as options similar to the collection based subfolders. Maybe I will look into that but right now there are still some other things on my list.
@mark: Thanks. By the way, FF 11 might solve the subfolder syncing problem because one of the new features is addon syncing. I haven't tested it though and it might even complicate things with standalone. So I will probably go through a .json file on dropbox.
@theed: I fixed a number of bugs, which might have solved your problem. Let me know if you run into it again ('Send to tablet' now also catches errors and shows a 'ZotFile Error' message instead of just crashing).
Not sure whether I will do something about the 'ok' vs 'yes'... (I am using the javascript confirm, which can easily be manipulated).
Is there any way to do a batch run on all existing items with attached pdf's?
I use Zotfile with Dropbox and wonder if anyone has any suggestions how I can streamline my workflow. All syncing is handled by Dropbox, so I don't want to move files back and forth. The foreground mode is great for this purpose but it could still be better:
1. When I use "Scan Tablet Files...", I only have the option to either update the Zotero file, or to get it from the tablet. Is it possible to extract annotations from all changed files without moving them?
2. Can I tell Zotfile to automatically delete old notes when I extract new annotations?
3. I want to keep two copies of each file, the original file and an annotated copy. What's the best way to achieve this? Since I'm using the foreground mode, "Save copy of annotated file with suffix" isn't useful to me.
It would be very nice if Zotfile could copy files instead of moving them, with an optional prefix/suffix added to the filename, and add a link to the copied file in Zotero. Something like a hybrid between the current foreground and background mode; I don't mind if it's a hidden setting. This way I can have my working (annotated) copy on all my devices and keep the original in Zotero, e.g. for sharing with colleagues.
@hcsiller: yes, just select all the items and click 'extract annotations'. Maybe first test with ~10 pdfs. Zotfile extracts the annotations for all pdf attachments of selected items. There might still be problems with some pdfs but zotfile should capture any error, continue and put the error on the console. Let me know if it crashes because of a particular pdf.
@Mogger:
1) I have some improvements for the "Scan Tablet Files..." functions in mind but they are still going to take some time. The idea is to show the user a list of all modified items with different options and not prompt for each attachment separately. For now, I might add an option to 'select all modified attachments', which would make it possible to do what you want.
2) No, that's not possible right now. Maybe I will add an option to prompt for that but I am not sure because deleting the old note assumes that the user did not make any modifications.
3) I created an issue for that.
Awesome! I was thinking a bit about how to make Zotfile easier to use for my use case, here's what I've come up with so far:
1. In addition to "Send" and "Get", there should be an "Update" action. This is especially useful if you want your attachment in one place, e.g. on Dropbox. More about this below.
2. When using "Scan Tablet Files...", it shows how many files were modified and asks you to either update or get all attachments, or treat them individually. For the last option it could maybe show the result of a search in Zotero and let you handle them there.
3. Make it clearer in the preferences dialog what exactly "Send", "Get" and "Update" does. I was thinking something like this:
----
Send to tablet
Send action: [dropdown list: Copy files, Move files]
[checkbox] Add link to sent files
[checkbox] Rename files¹
Get from tablet
[checkbox] Save copy of annotated file with suffix [textbox]²
[checkbox] Automatically extract annotations
Update files
[checkbox] Automatically extract annotations
----
1. Would be nice with a "current filename" wildcard to allow you to simply add a prefix/suffix to the current name, e.g. prefix_%f_suffix.
2. This option should probably be disabled if you choose "Move files".
----
etc. The wording can (and probably should) of course be changed, but I hope you get the idea.
Regarding the notes, what's the use case in keeping them? Personally I just want the highlights from the current version of the document. Maybe there could be an option in Advanced Settings to either update the current note or create a new note when extracting annotations.
The use case with the notes is that I edit them after extracting annotations. These changes would get lost. But I added a ticket for an additional option...
Note that anything but bug fixes is probably going to take some time...
Currently Zotfile seems to work best if you want to send attachments to your tablet, read/add annotations, and then send them back to the computer. "Update" would be for those who want all files in a central place (e.g. Dropbox folder). I guess you could also see it as "Sync"; its function should be similar to getting the files from the tablet and sending them back again.
"I am thinking about easy ways to access modified tablet attachments":
An idea I had was to tag such files with a tag, e.g. _MODIFIED, and let Zotero show all attachments with that tag. The get/update action would remove the tag.
Notes:
Ah, I didn't even consider editing the notes, good that you pointed it out. And thanks for adding the Issue about the option!
No worries about time, it's completely understandable and the current version of Zotfile already works great. I might poke around in the code to try to figure out how to make Zotfile keep an original copy of the file (or I can just add it manually).