Hi Joscha,
yes, setting the option in about:config works as expected.
It seems that zotfile gets confused about the files exported before the option change, so it seems one has to retrieve all files first and then make the change.
With the background option and attaching changed files as new pdf with a suffix, I have 2 comments:
1, it would be nice to have the choice of a prefix, because the suffix is often not visible in zotero due to long filenames. Even cooler would be the choice to insert the "suffix" at a desired location within the filestring (like, after author-year), as then files would still sort right. This might be doable if the _annotated were handled with a % option as in the third tab of the options dialog.
2, if I export the item again, then both files are put onto my reader. This is of course expected behavior, but it would probably make more sense if only the annotated version got moved (this would be a check whether two files are attached, one only differing by the _annotated part). Furthermore, when I import the two files back without making further changes, then both are renamed to the same name, so that the annotated is not recognizeable anymore (both files open, and one is annotated, and one not, but they are both just called the normal name without _annotated).
Short comment again about checking reader files, clicking "move back" does not really do nothing - it actually closes the dialog, but then doesn't import the file.
Thanks for your efforts. Already at this point the extension is a great help.
and yet another idea:
I realized that your extension might also solve another problem I have been reading about in the forums. Many people would like their attached files in one folder rather than in the 8-letter subfolders. You could add a function to export all marked items similar to the "background" mode, i.e. by collecting them all and copying them to 1 folder, but without giving them the "_READ" tag. This way, one could export all files for someone else to look at directly, but one wouldn't mess with one's own reader status.
Sorry that my answer took half a century but I was in vacation a couple of weeks... :-)
I tested the mentioned beta and it now renames the files in the destination folder correctly. But I found a few other issues:
1. ZotFile _does_ rename the original file which is not always desirable in my opinion.
2. The "Files on Tablet" folder does not reliably show the files on my tablet. In fact in my case the folder is always empty?!
3. Sometimes I can only select the menu item "Move to tablet", even if it is already there. ZotFile keeps adding a new file each time with a suffix of 1, 2, 3 etc. I cannot select "Remove from tablet" since it is grayed out in the menu?!?
@theed: I will think about your first and second point and might implement some changes in a future version. Probably not for 2.0 though. Meanwhile, you can move individual attachments by clicking on the attachment and not the zotero item. That should solve your problem discussed in point 2. About your second post: That is the reason why I provide a zotfile option to 'link' attachments and not to import them. Not the same but with this option you can place all your attachments in one folder and the unreadable zotero storage folder.
@Tom: 1) I will add an option to disable the renaming (I assume you are taking about the tablet features, right?). 2) Can you right-click that 'Files on Tablet' folder, click on 'Edit Saved Search' and report the options here. 3) Can you reproduce this exactly? What are the steps you take when this occurs? Maybe something with the zotfile reader tag is wrong. The attachments files that are on the reader should have the tag '_READ' and this tag should not be used or changed manually. Can you confirm that the attachments have this tag?
hi Tom, I guess that the zotfile function crashes at some point after the pdf got moved but before the tag gets added (when a zotfile function crashes FF and Zotero are unaffected). That would explain why the tag is sometimes missing. Can you check 'Tools->Web Devel.->Error Console' for any zotfile related stuff after you encounter the problem the next time? For zotfile bugs, the 'Source File' should be something like 'chrome://zotfile/content/...' (most likely zotfile.js).
yes, it does. Thanks! Nothing about the file and the line I guess? Usually FF also lists that. I will look into this for sure and follow up here on the forum. Just might take a while before I find the time.
Nope, sorry, nothing else... :-( I can do what ever you want to test the issue. But you have to tell me what to do since I'm not a computer science guy...
Just installed the Zotfile 2.0 beta. Everything seems to work smoothly and I'm very happy with the relative paths; now I can use the Tablet functions on multiple computers.
Minor suggestion: all options have moved into a submenu, and I can see the sense in that; however, "Attach new file(s)" is in my workflow one of the most common actions I choose, and so it is not so convenient that this action is now buried in a submenu which requires further waiting and/or clicking.
"Move" sounds dangerous. Most of us don't want to move but want to copy. "Send to tablet" is maybe a better option.
Similarly, "Remove from tablet" sounds dangerous. I'm not just removing it, I'm getting it back. So maybe "Get from tablet"?
Also, for attachments that are already on the reader, is it really necessary that "Move/Send to tablet" is still available as an option, besides "Remove/Get from tablet"? I found it a bit confusing. What is the use case for doing a "Move to tablet" for an item that is already there? Updating it in the case of changes on the computer side? In that case it is conceptually closer to the "Scan Tablet" function and could conceivably just happen there (so that Scan Tablet would not just scan for updates on the Tablet side but also on the computer side).
@mark: I was waiting for the hassel with the one menu to come up. I am not sure about that either and was hoping to add an option about this. But I am not able to figure out how I can control menu items in the main right-click menu. Sure, I can add them but I don't know how I can control whether they appear or not. There is a zotero function, which builds the menu but I don't know how I can extend that. With the submenu, however, I have full control. I like your name suggestions and I will make the change. "Move/Send to tablet" is also available for attachments on the tablet because you can move files again in order to change the subfolder they are in.
Maybe others should report their experience, but just going on my own use of Zotfile, in over 50% of the cases I invoke a Zotfile function it is the "Add new attachment" function. That might just warrant its default inclusion in the main right-click menu, with the additional and lesser-used actions in the submenu.
/edit Ah, I see the use of the Move option for items that already exist on the Tablet then. That is a very minor use case though, I doubt it is worth the confusion. Still, that may be just me.
I've been using your plugin on a mac for a few months with success and it's a great tool for organizing a pdf library. Thank you!
Maybe you can help me with a problem I've been experiencing. Starting about a week ago, Zotfile unexpectedly stopped working. As far as I know, I haven't changed anything, but when I try to attach a new file via Zotfile, nothing happens. I also seem unable to to access the preferences pane through the usual way. It's seems the plugin has become unresponsive. How might i fix this?
I've noticed that moving to the tablet fails sometimes.
1, I think when the pdf is a scan (sometimes the case with older literature), it is not moved. Thought it is because zotfile is trying to rename by looking for info in the file, but turning off the "rename when moving to tablet" doesn't change this.
2, when trying to move a larger number of files, in my library, not all files are actually moved. I think it is because whenever an "unmovable" item is in the list and zotfile gets to it, it breaks off. However, this is just a hunch.
Therefore, could you try and fix no. 1, and would it be possible to continue moving even after encountering an error with some file, and then pop up a window at the end that tells you which files were not moved?
and, another question: I installed the zotero standalone, and installed zotfile. zotfile seems to be running in background mode (copies files, instead of making them links in zotero lib). I cannot change this, as there is nowhere to go to config in the standalone. I thought that the default for zotfile is foreground mode, but maybe not? Any way to change the mode in the standalone?
thx
and another quick thing: in the window asking whether one wants to overwrite files that are already on the reader (probably only exists in background mode?) allows "OK" and "cancel", and cancel actually moves only those that are not yet there. Might be nice to call it something like "move only new files".
@jamesnicholson: I would start with the standard things: (a) restart FF, (b) restart you computer, (c) reinstall zotfile. Can you do these things and follow up here again?
@theed: Someone else reported that the moving sometimes fails. Maybe you are reporting the same problem, maybe not. Something reproducible would be very helpful! I am pretty sure that your first theory is wrong though. First you should make sure that you can open the files from zotfile by double clicking on them. Also, can you check 'Tools->Web Devel.->Error Console' for any zotfile related stuff after you encounter the problem the next time? For zotfile bugs, the 'Source File' should be something like 'chrome://zotfile/content/...' (most likely zotfile.js). If there are a lot of things, it might be helpful to clear the error console, then execute the actoins that caused the problem and then check again.
@theed2: The default is background mode. You can access the about:config stuff through 'Actions->Preferences->Advanced->Open about:config' in Standalone and change zotfile.tablet.mode to 2 for foreground mode.
First of all great work, you're helping a lot of people.
I have one slightly bizarre question. For some time I have been looking for solutions to get my many highlights and comments on pdfs into the relevant Zotero notes pane. It seems that Zotfile Reader's new feature does this, but only for mac and I am, of course, one of those annoying people on windows, so...
Would it be possible for me to procure a mac for a weekend, install Zotero, Zotfile and Zotfile reader, sync my library, run the extractor, sync it back to the zotero server, then sync back to my Windows PC and have all my pdf highlights and comments in the notes? With the required bandwidth (!), I see no logical reason why this wouldn't work, but can't be sure without seeing the extractor in action. I don't really want to go to the effort (and incur the shame) of asking for someone's mac for a weekend if this won't work, so your, or some other knowledgable mac user's go-ahead would be helpful.
Of course an extractor for windows would be a much more elegant and longer-term solution, but I don't have the programming skills myself. If you've found someone who wants to be paid to compile it for windows, I'd be prepared to tip in with at least 3 times the normal contribution!
Hi Julian, should be possible... I would try out the extractor on this mac though before you get started with one of the pdfs in your library. You can also go for zotfile 2.0 beta instead of zotfile reader. In theory, it should be a one click thing to extract ALL your annotations but maybe it's going to stop at some point because it runs into some problems. So I would certainly start by selecting a smaller number of zotero items and see how it goes. In zotfile 2.0, you can also adjust the html code put around each highlight/note so that you can customize how the extracted text look in your notes. I haven't heard from anyone about compiling a windows version (which I think is harder then a linux version because more libraries need to be compiled). But maybe someone feels encouraged when you put your offer out there.
It turns out that the problem was quite simple and had nothing to do with your plugin. I store my files in a dropbox folder and it was full, so the plugin was unable to move files into this folder. I cleared some space and it's back to working perfectly.
sorry I haven't replied earlier. I still get some pdfs which are not moved by the plugin. I seem to be unable to find anything common to all of the failing ones, but I can offer to send you some of them. Let me know if you want that, and if so, how I can get the pdfs to you.
It's actually a rather large percentage, maybe 10% of all articles I download.
I'm really starting to like the way folders work; and yet I feel also limited by the right-click > move to tablet interface, which makes it difficult and counterintuitive to move around items within folders on the tablet.
Now I spotted the "Create subfolders from Zotero collections" option and I realised that with some modifications this could be made better and at the same time solve the issue with managing Zotfile folders on multiple systems.
Here is the killer idea: under the option "Create subfolders from Z collections", let me choose a Zotero collection that is to be the Zotfile root folder. All subcollections within that collection would then automatically be subfolders in Zotfile — and the bonus is that this is synced information so Zotfile doesn't have to save the folder lists elsewhere.
Then let Zotfile monitor this collection (and all subcollections) — anything I drop into there should be pushed to the tablet in the way it is already done, and anything I remove from there should be gotten back from the tablet in the way it's already done. Perhaps if realtime monitoring is difficult there could be a button somewhere, though obviously some kind of automatic syncing would be optimal.
I don't know what is involved in coding this, but I bet that once you have done it, (1) you'll be happy, because you don't have to worry about adding, saving and syncing folders anymore; and (2) your users will be happy, because instead of having another set of folders to manipulate and keep track of they can just keep working with the collection UI Zotero already has anyway.
@theed: Anything reproducible would be very helpful to solve that problem. Have you checked the FF error console after a file didn't work? That might be a good starting point and maybe it's all I need. Can you also check whether the problem is connected to specific files or specific zotero items? Just attach the same file to a different zotero item and see what happens and also attach a different file to the same zotero item. What do you find? You can also go ahead an send me a file but I don't expect that particular files are the problem.
@mark: Yes, I agree that would be a very elegant solution. It does require quit some coding though and I am pretty tied up right now with other stuff. Not sure when that is going to change but I will look into this when I have more time for zotfile again... thanks for all your suggestions.
here is a small update that fixes the problems with the 'scan tablet' function. That is the only change for this version but it should fix the bug some reported here. Let me know if the problem persists.
Deleting subfolders does not work on Win7/Z3/FF8. Zotfile nicely asks me what to do with the attachments in the folders, and when I choose one of the options (e.g. remove from reader) it does something with the attachments in the folder, but the actual folder is not deleted from the list nor from the Zotfile root folder in Dropbox.
/edit Ah, it does work, but only when the folder is empty, which is reasonable I suppose but I would've expected a warning about it. (There happened to be files in there which weren't put there by Zotfile.) Proposal: Provide an explanation when a folder could not be deleted due to it not being empty.
/edit2
BTW, I'm still occassionally taken aback by terms like "remove" in Zotfile, hope you'll get a chance to change that to less alarming terms as suggested above.
@mark: yes, that is exactly how the deleting works. I can add a message that reports about the moving about files (maybe there already is one?) and also mentions what happened with the folder. Yes, the terms will change. I just did the quick fix for b4 because I realized that a wrong was called by the scan tablet function.
There is a message about moving files, but no notice that if the folder contains files that were not put there by Zotfile, it won't be deleted. Something like "Can't delete, the folder not empty."
I have already implemented some small requests (accents are removed in file names, functions are renamed ...) and more will follow. Most importantly, though, Joe Devietti is making some great contributions. Looks like zotfile will be able to extract annotations and highlighted text on all platform very soon! His solution is pure javascript and based on the pdf.js library... I will probably post a beta on this forum within a week.
Upgrade Storage
yes, setting the option in about:config works as expected.
It seems that zotfile gets confused about the files exported before the option change, so it seems one has to retrieve all files first and then make the change.
With the background option and attaching changed files as new pdf with a suffix, I have 2 comments:
1, it would be nice to have the choice of a prefix, because the suffix is often not visible in zotero due to long filenames. Even cooler would be the choice to insert the "suffix" at a desired location within the filestring (like, after author-year), as then files would still sort right. This might be doable if the _annotated were handled with a % option as in the third tab of the options dialog.
2, if I export the item again, then both files are put onto my reader. This is of course expected behavior, but it would probably make more sense if only the annotated version got moved (this would be a check whether two files are attached, one only differing by the _annotated part). Furthermore, when I import the two files back without making further changes, then both are renamed to the same name, so that the annotated is not recognizeable anymore (both files open, and one is annotated, and one not, but they are both just called the normal name without _annotated).
Short comment again about checking reader files, clicking "move back" does not really do nothing - it actually closes the dialog, but then doesn't import the file.
Thanks for your efforts. Already at this point the extension is a great help.
I realized that your extension might also solve another problem I have been reading about in the forums. Many people would like their attached files in one folder rather than in the 8-letter subfolders. You could add a function to export all marked items similar to the "background" mode, i.e. by collecting them all and copying them to 1 folder, but without giving them the "_READ" tag. This way, one could export all files for someone else to look at directly, but one wouldn't mess with one's own reader status.
Sorry that my answer took half a century but I was in vacation a couple of weeks... :-)
I tested the mentioned beta and it now renames the files in the destination folder correctly. But I found a few other issues:
1. ZotFile _does_ rename the original file which is not always desirable in my opinion.
2. The "Files on Tablet" folder does not reliably show the files on my tablet. In fact in my case the folder is always empty?!
3. Sometimes I can only select the menu item "Move to tablet", even if it is already there. ZotFile keeps adding a new file each time with a suffix of 1, 2, 3 etc. I cannot select "Remove from tablet" since it is grayed out in the menu?!?
THX!
Tom
About your second post: That is the reason why I provide a zotfile option to 'link' attachments and not to import them. Not the same but with this option you can place all your attachments in one folder and the unreadable zotero storage folder.
@Tom: 1) I will add an option to disable the renaming (I assume you are taking about the tablet features, right?). 2) Can you right-click that 'Files on Tablet' folder, click on 'Edit Saved Search' and report the options here. 3) Can you reproduce this exactly? What are the steps you take when this occurs? Maybe something with the zotfile reader tag is wrong. The attachments files that are on the reader should have the tag '_READ' and this tag should not be used or changed manually. Can you confirm that the attachments have this tag?
1.) Yes, that's right. I'm just not sure if it is a good idea to rename my entire Zotero library. But I want the files on the tablet to be renamed.
2.) That is correct: "Tag is _READ". I think the problem is, that ZotFile sometimes does not set the _READ tag correctly:
3.) Unfortunately I can not reproduce this all the time. It seems that ZotFile sometimes does not set the _READ tag (nor any other tags or text).
Regards,
Tom
Fehler: uncaught exception: Cannot add invalid tag 773 in Zotero.Item.addTagByID()
Does this help?
THX!
I will look into this for sure and follow up here on the forum. Just might take a while before I find the time.
Minor suggestion: all options have moved into a submenu, and I can see the sense in that; however, "Attach new file(s)" is in my workflow one of the most common actions I choose, and so it is not so convenient that this action is now buried in a submenu which requires further waiting and/or clicking.
"Move" sounds dangerous. Most of us don't want to move but want to copy. "Send to tablet" is maybe a better option.
Similarly, "Remove from tablet" sounds dangerous. I'm not just removing it, I'm getting it back. So maybe "Get from tablet"?
Also, for attachments that are already on the reader, is it really necessary that "Move/Send to tablet" is still available as an option, besides "Remove/Get from tablet"? I found it a bit confusing. What is the use case for doing a "Move to tablet" for an item that is already there? Updating it in the case of changes on the computer side? In that case it is conceptually closer to the "Scan Tablet" function and could conceivably just happen there (so that Scan Tablet would not just scan for updates on the Tablet side but also on the computer side).
I like your name suggestions and I will make the change. "Move/Send to tablet" is also available for attachments on the tablet because you can move files again in order to change the subfolder they are in.
/edit
Ah, I see the use of the Move option for items that already exist on the Tablet then. That is a very minor use case though, I doubt it is worth the confusion. Still, that may be just me.
I've been using your plugin on a mac for a few months with success and it's a great tool for organizing a pdf library. Thank you!
Maybe you can help me with a problem I've been experiencing. Starting about a week ago, Zotfile unexpectedly stopped working. As far as I know, I haven't changed anything, but when I try to attach a new file via Zotfile, nothing happens. I also seem unable to to access the preferences pane through the usual way. It's seems the plugin has become unresponsive. How might i fix this?
I've noticed that moving to the tablet fails sometimes.
1, I think when the pdf is a scan (sometimes the case with older literature), it is not moved. Thought it is because zotfile is trying to rename by looking for info in the file, but turning off the "rename when moving to tablet" doesn't change this.
2, when trying to move a larger number of files, in my library, not all files are actually moved. I think it is because whenever an "unmovable" item is in the list and zotfile gets to it, it breaks off. However, this is just a hunch.
Therefore, could you try and fix no. 1, and would it be possible to continue moving even after encountering an error with some file, and then pop up a window at the end that tells you which files were not moved?
thx
@theed: Someone else reported that the moving sometimes fails. Maybe you are reporting the same problem, maybe not. Something reproducible would be very helpful! I am pretty sure that your first theory is wrong though. First you should make sure that you can open the files from zotfile by double clicking on them. Also, can you check 'Tools->Web Devel.->Error Console' for any zotfile related stuff after you encounter the problem the next time? For zotfile bugs, the 'Source File' should be something like 'chrome://zotfile/content/...' (most likely zotfile.js). If there are a lot of things, it might be helpful to clear the error console, then execute the actoins that caused the problem and then check again.
@theed2: The default is background mode. You can access the about:config stuff through 'Actions->Preferences->Advanced->Open about:config' in Standalone and change zotfile.tablet.mode to 2 for foreground mode.
First of all great work, you're helping a lot of people.
I have one slightly bizarre question. For some time I have been looking for solutions to get my many highlights and comments on pdfs into the relevant Zotero notes pane. It seems that Zotfile Reader's new feature does this, but only for mac and I am, of course, one of those annoying people on windows, so...
Would it be possible for me to procure a mac for a weekend, install Zotero, Zotfile and Zotfile reader, sync my library, run the extractor, sync it back to the zotero server, then sync back to my Windows PC and have all my pdf highlights and comments in the notes? With the required bandwidth (!), I see no logical reason why this wouldn't work, but can't be sure without seeing the extractor in action. I don't really want to go to the effort (and incur the shame) of asking for someone's mac for a weekend if this won't work, so your, or some other knowledgable mac user's go-ahead would be helpful.
Of course an extractor for windows would be a much more elegant and longer-term solution, but I don't have the programming skills myself. If you've found someone who wants to be paid to compile it for windows, I'd be prepared to tip in with at least 3 times the normal contribution!
Thanks again.
In theory, it should be a one click thing to extract ALL your annotations but maybe it's going to stop at some point because it runs into some problems. So I would certainly start by selecting a smaller number of zotero items and see how it goes. In zotfile 2.0, you can also adjust the html code put around each highlight/note so that you can customize how the extracted text look in your notes.
I haven't heard from anyone about compiling a windows version (which I think is harder then a linux version because more libraries need to be compiled). But maybe someone feels encouraged when you put your offer out there.
It turns out that the problem was quite simple and had nothing to do with your plugin. I store my files in a dropbox folder and it was full, so the plugin was unable to move files into this folder. I cleared some space and it's back to working perfectly.
sorry I haven't replied earlier. I still get some pdfs which are not moved by the plugin. I seem to be unable to find anything common to all of the failing ones, but I can offer to send you some of them. Let me know if you want that, and if so, how I can get the pdfs to you.
It's actually a rather large percentage, maybe 10% of all articles I download.
Now I spotted the "Create subfolders from Zotero collections" option and I realised that with some modifications this could be made better and at the same time solve the issue with managing Zotfile folders on multiple systems.
Here is the killer idea: under the option "Create subfolders from Z collections", let me choose a Zotero collection that is to be the Zotfile root folder. All subcollections within that collection would then automatically be subfolders in Zotfile — and the bonus is that this is synced information so Zotfile doesn't have to save the folder lists elsewhere.
Then let Zotfile monitor this collection (and all subcollections) — anything I drop into there should be pushed to the tablet in the way it is already done, and anything I remove from there should be gotten back from the tablet in the way it's already done. Perhaps if realtime monitoring is difficult there could be a button somewhere, though obviously some kind of automatic syncing would be optimal.
I don't know what is involved in coding this, but I bet that once you have done it, (1) you'll be happy, because you don't have to worry about adding, saving and syncing folders anymore; and (2) your users will be happy, because instead of having another set of folders to manipulate and keep track of they can just keep working with the collection UI Zotero already has anyway.
You can also go ahead an send me a file but I don't expect that particular files are the problem.
@mark: Yes, I agree that would be a very elegant solution. It does require quit some coding though and I am pretty tied up right now with other stuff. Not sure when that is going to change but I will look into this when I have more time for zotfile again... thanks for all your suggestions.
here is a small update that fixes the problems with the 'scan tablet' function. That is the only change for this version but it should fix the bug some reported here. Let me know if the problem persists.
http://www.columbia.edu/~jpl2136/zotfile_200b4.xpi
/edit
Ah, it does work, but only when the folder is empty, which is reasonable I suppose but I would've expected a warning about it. (There happened to be files in there which weren't put there by Zotfile.)
Proposal: Provide an explanation when a folder could not be deleted due to it not being empty.
/edit2
BTW, I'm still occassionally taken aback by terms like "remove" in Zotfile, hope you'll get a chance to change that to less alarming terms as suggested above.
Yes, the terms will change. I just did the quick fix for b4 because I realized that a wrong was called by the scan tablet function.
https://github.com/jlegewie/zotfile
Code contributions and pull requests are welcome!
I have already implemented some small requests (accents are removed in file names, functions are renamed ...) and more will follow. Most importantly, though, Joe Devietti is making some great contributions. Looks like zotfile will be able to extract annotations and highlighted text on all platform very soon! His solution is pure javascript and based on the pdf.js library... I will probably post a beta on this forum within a week.