Many friends have pointed me to Zotero and I'm keen to get away from Mendeley - the only feature keeping me from switching completely is the watch folder feature. I find a lot of utility from a reference manager with a workflow where I can primarily import from browser, but also allows the flexibility to manage pdfs in folders locally and auto-import.
It takes the friction away from having to remember to always add new pdfs to their correct collection as well.
I saw a few threads about the watch folder feature on the forums, but I think there would be a lot more interest and use cases if one could "vote" on features (or if there was a corresponding open issue for it). I appreciate there are limited resources for open source dev in general though, so completely understand that other features are prioritised. This is basically just my lengthy "+1" on the watch folder feature request :)
In Zotero, you generally add items from the web, which downloads metadata and saves a PDF if available, storing the file in its own storage. You can also click the save button when viewing a PDF to save it and try to retrieve metadata, though we recommend saving from the article page when possible.
If you only have a PDF on disk — say, because you received it via email — you can drag it to Zotero, which will try to retrieve metadata for it and copy the PDF into its own storage. You can then delete the original copy on disk.
An upcoming version of Zotero will also make it possible to open PDFs on disk via Zotero in order to add them.
It's also possible to use linked files, generally with the help of the ZotFile extension. But ZotFile will convert the default stored files into linked files in a location of your choosing — it doesn't just link them where they exist now. (You can do that manually in Zotero, but it'd be tedious, and not a recommended workflow.) If you have a file on disk that you were adding to Zotero, it would generally be in a temporary folder such as a download folder, and you would drag it to Zotero — resulting in either a stored file or a linked file in a configured directory — and then delete it.
Ah ok I understand - I definitely wouldn't like to always delete the original file in my case, but I get that having an extra copy in Zotero and the two pdf copies not being linked makes it difficult to do e.g. markup. Nonetheless, it would be great if there if a one-way watch and upload feature did exist (regardless of where the files are then located afterwards), AFAICT this was removed from Zotfile as you mentioned in earlier threads.
I'll see how I can adjust my workflow to have everything hosted as much as possible via Zotero though, the Mendeley importer at least removes some of this friction. Thanks again.
So sad this feature is not likely to be ever implemented. I understand why you would prefer "add items from the web", but that's just not efficient to add to my library hundreds of articles downloaded through a couple of decades.
You can already drag hundreds of PDFs to Zotero and have it look up metadata automatically. I think a feature to automatically import all PDFs in a folder hierarchy and recreate that hierarchy is under development. The only thing that won't likely happen is a feature that automatically imports any new file in a given folder, i.e. a watch folder.
So do we have any update on watch folder, I recently moved from Mendley. Mendley mistakenly imported every file from watch folder (even excel files, temp files, SEM files). And now I realize Zotero has no watch folder feature.
TLDR: Users: we need the option! Devs: you do not need the option!
Is my summary of multiple threads? Could you please clarify whether refusing to implement the watch folder is a technical problem or an exclusive wish of the developers? The former case makes it easier to accept the necessity of learning a new workflow and paying such a price as a complete reorganization of the user's file library. Because of the many requests from the users, the latter makes me confused regarding who Zotero is developing for: users or developers?
I guess the implementation of the watch folder is indeed related to considerable efforts and technical difficulties, but I would appreciate any clues from the developers.
@pivik: We explain exactly why on the page linked immediately above your comment. I'm not sure why you think there's some mystery here. People ask for all sorts of things we don't implement because we think they're worse ways of working. If you disagree, you can try to convince us (unlikely here), modify the code yourself, use a plugin that does what you want, or use something else.
@dstillman, On the link above, I see the explanation like we believe this way is better for you. It is challenging to accept that developers may really know better about users needs than users themselves. That is why I ask whether the reason is technical efforts or developer beliefs. If the developers will never implement it, how can users get convenient access to their files without Zotero? I've seen somewhere suggestions like using system search by titles, but we cannot remember titles for hundreds of documents. I am just afraid to lose dozens of gigabits of valuable files if I transfer them to Zotero and delete the originals to avoid duplication. So, as with many others, I guess, I just want to keep the integrity of my own file library for some reasons.
As example, I may need access to files when preparing a presentation. I may want to put all used articles in one folder with my slides and make links to files. This is very useful for lectures because students cannot access articles from their homes. If I have a well-organized file library, I can easily find the required documents, which are often stored within one designated folder with a specified name (e.g., /Inflammation or /Immune System). If I use Zotero workflow, I have to open it first, then find the document, right-click, open a folder, and finally copy it somewhere. Is that correct? Are there other options?
@pivik: You're misunderstanding the argument. As it says, even if you want to use your own organizational system, that's something that you would do with a plugin in Zotero, so that Zotero can automatically rename and organize files for you as you save files to it (ideally from the web) and keep them updated based on the data in your library without file links constantly getting broken.
And you can always search outside of Zotero based on anything in the filename — title, creator, date, or whatever else you put in the file-rename string. That's one of the main benefits of automatic renaming. (But Zotero also provides many more flexible ways to organize files than the filesystem, so people who use Zotero are generally pretty content to search within Zotero.)
Ultimately, though, if Zotero isn't for you, it isn't for you. If your primary goal is manually crafting a folder structure of PDFs in your filesystem, it might not be for you.
@pivik people who need easier access to their PDFs outside Zotero often look at linked attachments. Often with management aided via Zotfile. Linked files also have limitations though (see link below). And complex named sub-folder arrangements in your OS are harder to manage in conjunction with Zotero/Zotfile. Alternatively you could use the saved system search for PDFs in local Zotero storage, with file shortcuts to those PDFs from your topic folders. It's a matter of weighing up your priorities. You can make almost any system work if you're willing to do the work. https://www.zotero.org/support/attaching_files#linked_files
You might want to try a small trial first, setting up two different approaches with the contents of different folders. You may find that setting up your folder structure as Zotero collections and embracing Zotero's way of doing things supplants your old system. Or you may come up with a hybrid that works best for you.
@tim820, @dstillman, I should have spent more time investigating documentation. Thank you for the explanations! Now I realize that I can move files from Zotero folders to wherever I want with Zotfile. I think I should try the Zotero workflow.
I want to sincerely thank you both for your time! I feel like I stubbornly asked questions that have already been addressed.
@dstillman A watch folder is a great feature simply because many people think in that way. In addition, Elsevier publishes about 18% of journal articles. Those «nice people» just changed something on their end, and zotero doesn't work. Guess what? As of 30/Jul/2024, if you want to add a paper from elsevier to your zotero library, you have to download it and manually add it to your library.
@m.a.lugo00: We're working on changing how Zotero saves attachments to better handle sites like ScienceDirect that use strict anti-bot protections. That's the actual solution for that.
I'd like to share my use case as an argument for a watch folder feature, though I'm not actually trying to convince anybody.
I read a lot on my phone. Frequently, there are pdfs received through emails, or MS Teams, and many reports such as IMF's "Fiscal Monitor" which, while available online, don't have the metadata properly set for Zotero Connect to work, even if I were on a PC.
So with a watch folder feature, if set to a Google Drive or One Drive folder (on my PC Zotero), I could read on my phone and annotate, then "send" it do that synced folder. My desktop would then sync the Drive and Zotero would pick it up, automatically importing it. Maybe to a "Temp" library which I would later carefully update on my PC.
By the way, I know that Zotfile has a send and retrieve feature for tablets, but that's only after you get the PDF in Zotero (say on a Desktop). I'm talking about adding to the library from a mobile device so that I can focus on immediately reading and annotating a document wherever I am, and only later actually tending to the library and metadata.
@sfetter12: But we have mobile apps. You just share it the Zotero iOS/Android app, annotate in the app with actual Zotero annotations, and it syncs automatically to your PC.
I'm on android and unfortunatelly can't download the Zotero app (currently full - but I'd get it after beta). I am using the Web Library through the chrome app.
Using the Web Library, adding a PDF is suboptimal. If I upload it directly, a proper item with metadata is not created (zotero desktop would at least try to fetch the metadata). If I create an item and attach the PDF, I have to go through each PDF one by one, set some metadata manually and later have a lot more cleanup to do on desktop.
I just see a lot more potential being able to select a dozen PDFs on my phone, send them all to sync on a specific folder on OneDrive/GoogleDrive, and let Zotero Desktop import them automatically with possibly many entries retrieving the proper metadata. But maybe it works better on the actual android app, I haven't been able to try yet.
Regarding annotations, since I'm using Zotfile to sync across devices (library larger than free version), unfortunately I'd rather have them saved to the PDF, which I understand isn't the ideal Zotero use case (as far as I understand linked files would not sync). Thus even if the Android app handles uploading entries as smoothly as possible, since I'd later have to convert to linked files (my entire library is in a OneDrive folder), annotating on a third-party app that saves to Onedrive would still be my preferred workflow rather than annotating on Zotero.
Well, this is only my scenario. I don't actually recommend people stepping out of the ideal Zotero ecosystem. Eventually paying for more online storage if needed would be a lot less of a hassle, but not my reality.
Upgrade Storage
Many friends have pointed me to Zotero and I'm keen to get away from Mendeley - the only feature keeping me from switching completely is the watch folder feature. I find a lot of utility from a reference manager with a workflow where I can primarily import from browser, but also allows the flexibility to manage pdfs in folders locally and auto-import.
It takes the friction away from having to remember to always add new pdfs to their correct collection as well.
I saw a few threads about the watch folder feature on the forums, but I think there would be a lot more interest and use cases if one could "vote" on features (or if there was a corresponding open issue for it). I appreciate there are limited resources for open source dev in general though, so completely understand that other features are prioritised. This is basically just my lengthy "+1" on the watch folder feature request :)
Thanks again,
Lood
In Zotero, you generally add items from the web, which downloads metadata and saves a PDF if available, storing the file in its own storage. You can also click the save button when viewing a PDF to save it and try to retrieve metadata, though we recommend saving from the article page when possible.
If you only have a PDF on disk — say, because you received it via email — you can drag it to Zotero, which will try to retrieve metadata for it and copy the PDF into its own storage. You can then delete the original copy on disk.
An upcoming version of Zotero will also make it possible to open PDFs on disk via Zotero in order to add them.
It's also possible to use linked files, generally with the help of the ZotFile extension. But ZotFile will convert the default stored files into linked files in a location of your choosing — it doesn't just link them where they exist now. (You can do that manually in Zotero, but it'd be tedious, and not a recommended workflow.) If you have a file on disk that you were adding to Zotero, it would generally be in a temporary folder such as a download folder, and you would drag it to Zotero — resulting in either a stored file or a linked file in a configured directory — and then delete it.
Nonetheless, it would be great if there if a one-way watch and upload feature did exist (regardless of where the files are then located afterwards), AFAICT this was removed from Zotfile as you mentioned in earlier threads.
I'll see how I can adjust my workflow to have everything hosted as much as possible via Zotero though, the Mendeley importer at least removes some of this friction. Thanks again.
The only thing that won't likely happen is a feature that automatically imports any new file in a given folder, i.e. a watch folder.
Users: we need the option!
Devs: you do not need the option!
Is my summary of multiple threads?
Could you please clarify whether refusing to implement the watch folder is a technical problem or an exclusive wish of the developers? The former case makes it easier to accept the necessity of learning a new workflow and paying such a price as a complete reorganization of the user's file library. Because of the many requests from the users, the latter makes me confused regarding who Zotero is developing for: users or developers?
I guess the implementation of the watch folder is indeed related to considerable efforts and technical difficulties, but I would appreciate any clues from the developers.
As example, I may need access to files when preparing a presentation. I may want to put all used articles in one folder with my slides and make links to files. This is very useful for lectures because students cannot access articles from their homes. If I have a well-organized file library, I can easily find the required documents, which are often stored within one designated folder with a specified name (e.g., /Inflammation or /Immune System). If I use Zotero workflow, I have to open it first, then find the document, right-click, open a folder, and finally copy it somewhere. Is that correct? Are there other options?
And you can always search outside of Zotero based on anything in the filename — title, creator, date, or whatever else you put in the file-rename string. That's one of the main benefits of automatic renaming. (But Zotero also provides many more flexible ways to organize files than the filesystem, so people who use Zotero are generally pretty content to search within Zotero.)
Ultimately, though, if Zotero isn't for you, it isn't for you. If your primary goal is manually crafting a folder structure of PDFs in your filesystem, it might not be for you.
https://www.zotero.org/support/attaching_files#linked_files
You can drag and drop existing PDFs from their folders into Zotero as linked files by using Ctrl+Shift (Windows/Linux) or Cmd+Option (Mac) while dropping.
https://www.zotero.org/support/attaching_files#adding_files
You might want to try a small trial first, setting up two different approaches with the contents of different folders. You may find that setting up your folder structure as Zotero collections and embracing Zotero's way of doing things supplants your old system. Or you may come up with a hybrid that works best for you.
I want to sincerely thank you both for your time! I feel like I stubbornly asked questions that have already been addressed.
A watch folder is a great feature simply because many people think in that way. In addition, Elsevier publishes about 18% of journal articles. Those «nice people» just changed something on their end, and zotero doesn't work. Guess what? As of 30/Jul/2024, if you want to add a paper from elsevier to your zotero library, you have to download it and manually add it to your library.
Thanks again!
I read a lot on my phone. Frequently, there are pdfs received through emails, or MS Teams, and many reports such as IMF's "Fiscal Monitor" which, while available online, don't have the metadata properly set for Zotero Connect to work, even if I were on a PC.
So with a watch folder feature, if set to a Google Drive or One Drive folder (on my PC Zotero), I could read on my phone and annotate, then "send" it do that synced folder. My desktop would then sync the Drive and Zotero would pick it up, automatically importing it. Maybe to a "Temp" library which I would later carefully update on my PC.
By the way, I know that Zotfile has a send and retrieve feature for tablets, but that's only after you get the PDF in Zotero (say on a Desktop). I'm talking about adding to the library from a mobile device so that I can focus on immediately reading and annotating a document wherever I am, and only later actually tending to the library and metadata.
I'm on android and unfortunatelly can't download the Zotero app (currently full - but I'd get it after beta). I am using the Web Library through the chrome app.
Using the Web Library, adding a PDF is suboptimal. If I upload it directly, a proper item with metadata is not created (zotero desktop would at least try to fetch the metadata). If I create an item and attach the PDF, I have to go through each PDF one by one, set some metadata manually and later have a lot more cleanup to do on desktop.
I just see a lot more potential being able to select a dozen PDFs on my phone, send them all to sync on a specific folder on OneDrive/GoogleDrive, and let Zotero Desktop import them automatically with possibly many entries retrieving the proper metadata. But maybe it works better on the actual android app, I haven't been able to try yet.
Regarding annotations, since I'm using Zotfile to sync across devices (library larger than free version), unfortunately I'd rather have them saved to the PDF, which I understand isn't the ideal Zotero use case (as far as I understand linked files would not sync). Thus even if the Android app handles uploading entries as smoothly as possible, since I'd later have to convert to linked files (my entire library is in a OneDrive folder), annotating on a third-party app that saves to Onedrive would still be my preferred workflow rather than annotating on Zotero.
Well, this is only my scenario. I don't actually recommend people stepping out of the ideal Zotero ecosystem. Eventually paying for more online storage if needed would be a lot less of a hassle, but not my reality.