Upgrade StorageOn Zotfile development and Zotero 7
Hi all!
After reading this [1] recent comment I decided to open this issue. Zotfile is a plugin for Zotero whose usage seems to be very extended [2,5] through Zotero users, in particular for renaming (and renaming rules) and moving files to custom locations where they are linked to Zotero as suggested in [1].
Unfortunately, "Zotfile is currently not actively developed and maintained" [3, 4] as claimed by its developer @Joscha [6,7].
On the other hand, the upgrade of Zotero to version 7 "will require many plugin changes" [8].
On my side, I have neither time nor Java/Javascript knowledge to upgrade Zotfile to support Zotero 7. Are there someone there to upgrade such great plugin?
Thank you!
[1] https://forums.zotero.org/discussion/comment/434298/#Comment_434298
[2] https://forums.zotero.org/search/?Search=zotfile
[3] http://zotfile.com
[4] https://github.com/jlegewie/zotfile/issues/522
[5] https://forums.zotero.org/discussion/5301/zotfile-advanced-pdf-management-for-zotero#latest
[6] https://forums.zotero.org/profile/10268/Joscha
[7] https://github.com/jlegewie
[8] https://www.zotero.org/support/dev/zotero_7_for_developers
After reading this [1] recent comment I decided to open this issue. Zotfile is a plugin for Zotero whose usage seems to be very extended [2,5] through Zotero users, in particular for renaming (and renaming rules) and moving files to custom locations where they are linked to Zotero as suggested in [1].
Unfortunately, "Zotfile is currently not actively developed and maintained" [3, 4] as claimed by its developer @Joscha [6,7].
On the other hand, the upgrade of Zotero to version 7 "will require many plugin changes" [8].
On my side, I have neither time nor Java/Javascript knowledge to upgrade Zotfile to support Zotero 7. Are there someone there to upgrade such great plugin?
Thank you!
[1] https://forums.zotero.org/discussion/comment/434298/#Comment_434298
[2] https://forums.zotero.org/search/?Search=zotfile
[3] http://zotfile.com
[4] https://github.com/jlegewie/zotfile/issues/522
[5] https://forums.zotero.org/discussion/5301/zotfile-advanced-pdf-management-for-zotero#latest
[6] https://forums.zotero.org/profile/10268/Joscha
[7] https://github.com/jlegewie
[8] https://www.zotero.org/support/dev/zotero_7_for_developers
I suspect that Zotfile just 'works' for most people who use it. While people have asked for additional features and some bug fixes, it sounds like those are unlikely to happen any time soon (if at all), unless maybe someone else takes over the code (although the last Zotfile update was as recently as late last year). And with Zotero already having basically taken over annotation extraction and tablet integration, there is less to maintain.
There are Zotfile users here who regularly advise on any issues people have with Zotfile. It does take a little work to set it up, and so sometimes people need a little assistance with that. But they tend to be people who like to tweak things (although maybe not actual coders), and have the confidence to do so. It's just that as with any Zotero addon, specific assistance is not really the Zotero team's responsibility.
Long term, the best solution is for someone to fork the Zotfile repo and make updates/changes on that fork. People can then use the old but working (at least on Zotero 6 as of now) Zotfile or the fork if they want new features.
However, I feel that, hopefully, perhaps in time the Zofile functions will eventually be incorporated into the main development. At least, those for renaming and moving files to custom locations. In fact, there are already some basic rules for renaming files.
There are 2 major and 2 minor functionalities that ZotFile is providing now that annotations aren't part of that anymore:
Major:
1. More complex renaming -- Zotero devs have repeatedly said they're interested in supporting some version of that.
2. Moving files to links -- that's tricky because I'm not sure to what extent Zotero wants to actively promote usage that has notable downsides (like: doesn't work with mobile apps.)
Minor
1. Attaching recently added files -- I don't think this is getting a ton of use, but I've definitely had projects where that's been quite useful. Seems maybe a bit tricky in terms of UX, but non-contentious feature-wise
2. Send to/get from tablet. That's currently still arguably the best way to get stuff onto an Android tablet to annotate, but once there's an Android app, I'd argue this is probably OK to let go off (although there's always someone who loves a feature..)
-send to/get from tablet will not disappear with a zotero app for android, at least not for those who use an android e-reader (as I do). running zotero on an e-reader is clumsy, and what is needed is exactly the send to/get from tablet functionality.
-Attach new file is something I constantly use when downloading files for example that I get sent through email
> 2. Moving files to links -- that's tricky because I'm not sure to what extent Zotero wants to actively promote usage that has notable downsides (like: doesn't work with mobile apps.)
I primarily use Zotfile for its ability to organize my papers and PDFs in a separate directory, which significantly streamlines various tasks such as searching (ripgrep, fd), fuzzy finding and launching, annotating, and linking within markdown. While I also find the advanced renaming features useful, my workflow heavily relies on the seamless integration of Zotfile and BBT plugins. It's way faster and smoother workflow for my case. Overall, having a dedicated directory for my papers is crucial for my productivity.
In my opinion, the best solution would be to integrate its main features into Zotero, although I understand there may be some resistance to this idea.
One imperfect (possible) workaround that I right now can think if ZotFile get outdated, is to rsync the Zotero data directory coupled with a pdf finder (`find`, `fd`) to regularly update the separate directory and keep the storage filled with sort of symlinks!
- did a quick search and found [this](https://github.com/retorquere/zotero-file-hierarchy) it may help in future.
- The problem with this, is it is one way! So for example, if I annotate a PDF with my iPad or favorite PDF annotator, it automatically doesn't update the corresponding PDF of the Zotero.
If I can't do the send to/get from tablet, I regretfully cannot move to zotero 7 (which I would love to do because with the size of my library the 32 bit version is becoming very slow)
Whether Zotfile's functionality remains with Zotfile itself (made compatible with Zotero 7), or it is integrated into generic Zotero, doesn't really matter to users. If it's the latter, there does not seem to be any objection from Zotfile developer @Joscha to the pioneering work on Zotfile becoming basically redundant. That's a normal process with addons that over time are shown to have 'mission-critical' functionality that deserves to be in the main app. But if it's the former, then it sounds like Zotero developers would need to collaborate with @Joscha, as they did with the transition to Zotero 6, when some Zotfile functionality was subsumed into Zotero (PDF annotation extraction). Zotero developers are presumably quite familiar with Zotfile.
In any case, it would be good to hear Zotero developers' plans for this.
About the general future of zotfile: Pretty crazy that I published the first version of zotfile almost 15 years ago as a grad student. The irony is that more people have probably used zotfile over the years than read my academic work. Unfortunately, I just don't have the time to continue the maintenance and development of zotfile. Generally, I agree with the comment from @adamsmith above. An additional option would be for someone to create a new plugin that implements some of the functionality (e.g. more complex renaming). There is a lot of old code and unnecessary complexity in zotfile so a clean slate has many benefits.
So Zotfile users who rely on Android functionality could *possibly* delay a move to Zotero 7 until the Android app is available - depending on how long that will be - and *assuming* it covers their workflow well enough (and that other critical features of Zotfile are also made available in some way under Zotero 7). But from what some Zotfile/Android users have said above, the Android app may not cover their needs. If not, another way of doing Android send/get would be required.
Of the other features still requiring a solution ...
"Major"
'2. Moving files to links'
I'm guessing that Zotfile's auto-moving/linking of attachments as they are downloaded is probably the key 'mission critical' feature for most users. It allows a workflow and file organization system that is vital to the way that people need to work, and is not achievable in any other way. An addon that did nothing but this would be good (eg a 'cutdown' v7-compatible Zotfile ?); better would be to have that functionality within Zotero itself (consistent with its wide use).
"Minor"
'1. Attaching recently added files'
This is also very useful IMHO. It shows the most recently-dated file in a designated folder (that the user has deposited there - for example when Zotero's web connector could not download/add it automatically from an obscure website), and asks the user if they want to attach it to the item. If so, the file is renamed/moved to a pre-designated Custom Location folder and added to the item as a linked file. So it's a variation on Zotero's right click Add Attachment\Attach Link To File (which requires a few more clicks; but does it rename too ?). Which would presumably make it relatively easy to add to Zotero (importantly it's not a 'watch folder', to which I believe Zotero developers are averse). But the fact that it can be done in another way by Zotero 'as-is' possibly makes it non-critical.
So we're perhaps talking mostly about attachment auto-moving/renaming that would have to be made available in Zotero v7. And Android send/get if there is no other way of doing that.
as expressed by many others here, I think the availability of zotfile (or zotfile-like functionality) would be make or break for my transitioning to zotero 7. and as someone above has already said, it's made for people like me who don't have the skills to code, but love to tweak our files.
that said, the version 6 PDF reader/annotation functions are a step up from what's available on Zotfile, and given that history I am hopeful that all the Zotfile functions that are lost (and I'm sure each of us have favourites) will get substituted either by functionality within zotero itself, or with new add-ons.
given that this is going to be a bump that most of us will come up against, I wonder if it would be worthwhile compiling a how-to for zotfile users separately, to soften the blow? not up to the developers obviously, but if we can put this together I think it would be very useful.
I quickly slapped together a Zotero 7 plugin called ZotMoov. It can be found at https://github.com/wileyyugioh/zotmoov. All it does is move added files to a directory and change it to a linked file. It would be nice to get some feedback to polish it a bit more.
It should be noted that I'm not planning on adding any features--I want to keep it simple.
Thanks!
-Will the android app be released at the moment when zotero 7 will be officially released?
-Or at least: Will the android app be released at a time before zotero 6 will become entirely defunct (or will zotero continue to offer version 6 for download after the official introduction of zotero 7)?
-Which parts of zotfile will be included into zotero at some point and what is the order of their inclusion?
-Which parts of zotfile will never be included into zotero?
This would at least give users and plugin developers some kind of planning safety. Otherwise there may be plugin developers wasting their time on developing a plugin whose functionality will be included into zotero/the android app soon.
Internal Pdf reader with amazing annotations capabilities;
with hundreds of translator (scrappers) from the web;
Betterbibtex almost ready to be released;
.
.
.
Zotero is about to be the most potent reference manager in the world.
Zotfile is the only weakness we have.
Please, somebody, a generous soul, with programming skills: it would really be great if someone can help us to update Zotfile.
The 'Attaching recently added files' is something I use all the time, since I have lots of references that are documents, reports, and others that Zotero does not pickup automatically.
The 'send/retrieve file from tablet' is something that it is also useful to me even though I use the iPad app since the beta was released, because sometimes I need to review a paper on another app and that really simplifies the workflow.
Also, I don't currently use the linked files feature since I pay for Zotero storage, but it was a life saver when I was a broke college student.
Sadly, I will have to hold off on updating to Zotero 7 until an updated or forked version of ZotFile (or an alternative) becomes available, and I know I'm not alone in this. And I know Zotero developers have a vested interest on having people use their sync solution instead, but it does feel weird that a FOSS application doesn't have that basic functionality built-in. If Zotero 7 eventually supports linked files by default, boom, problem solved.
1) Unfortunately, I am not going to update zotfile for Zotero 7 myself. I looked into it again but it would take a significant amount of time. Happy to support anyone who takes on that work. From my experience, that is unlikely and you should not expect zotfile to work in Zotero 7.
2) In the long term, it is better to find alternative solutions for your workflow anyways. A combination of integrating features in Zotero and smaller plugins for specific tasks is much more future proof.