On Zotfile development and Zotero 7

Hi all!

After reading this [1] recent comment I decided to open this issue. Zotfile is a plugin for Zotero whose usage seems to be very extended [2,5] through Zotero users, in particular for renaming (and renaming rules) and moving files to custom locations where they are linked to Zotero as suggested in [1].

Unfortunately, "Zotfile is currently not actively developed and maintained" [3, 4] as claimed by its developer @Joscha [6,7].

On the other hand, the upgrade of Zotero to version 7 "will require many plugin changes" [8].

On my side, I have neither time nor Java/Javascript knowledge to upgrade Zotfile to support Zotero 7. Are there someone there to upgrade such great plugin?

Thank you!

[1] https://forums.zotero.org/discussion/comment/434298/#Comment_434298
[2] https://forums.zotero.org/search/?Search=zotfile
[3] http://zotfile.com
[4] https://github.com/jlegewie/zotfile/issues/522
[5] https://forums.zotero.org/discussion/5301/zotfile-advanced-pdf-management-for-zotero#latest
[6] https://forums.zotero.org/profile/10268/Joscha
[7] https://github.com/jlegewie
[8] https://www.zotero.org/support/dev/zotero_7_for_developers
  • edited May 8, 2023
    Given the substantial number of Zotfile users, I would assume that any changes required in Zotfile for Zotero 7 will be managed between the Zotero team and the Zotfile developer, as I believe happened with the transition from Zotero 5 to Zotero 6.

    I suspect that Zotfile just 'works' for most people who use it. While people have asked for additional features and some bug fixes, it sounds like those are unlikely to happen any time soon (if at all), unless maybe someone else takes over the code (although the last Zotfile update was as recently as late last year). And with Zotero already having basically taken over annotation extraction and tablet integration, there is less to maintain.

    There are Zotfile users here who regularly advise on any issues people have with Zotfile. It does take a little work to set it up, and so sometimes people need a little assistance with that. But they tend to be people who like to tweak things (although maybe not actual coders), and have the confidence to do so. It's just that as with any Zotero addon, specific assistance is not really the Zotero team's responsibility.
  • I was thinking of making a post like this, thanks @iagogv. Hopefully its not to hard for Zotero team to help with the transition because it’d be really great to have Zotfile working on Zotero 7.

    Long term, the best solution is for someone to fork the Zotfile repo and make updates/changes on that fork. People can then use the old but working (at least on Zotero 6 as of now) Zotfile or the fork if they want new features.
  • edited May 8, 2023
    I support what was said above, about the need to be able to continue using Zotfile in Zotero 7.

    However, I feel that, hopefully, perhaps in time the Zofile functions will eventually be incorporated into the main development. At least, those for renaming and moving files to custom locations. In fact, there are already some basic rules for renaming files.
  • I think the right approach would really be to wrap ZotFile functionality into Zotero instead of relying on an add-on that's great but not maintained and thus also poorly supported in case of bugs.
    There are 2 major and 2 minor functionalities that ZotFile is providing now that annotations aren't part of that anymore:
    1. More complex renaming -- Zotero devs have repeatedly said they're interested in supporting some version of that.
    2. Moving files to links -- that's tricky because I'm not sure to what extent Zotero wants to actively promote usage that has notable downsides (like: doesn't work with mobile apps.)

    1. Attaching recently added files -- I don't think this is getting a ton of use, but I've definitely had projects where that's been quite useful. Seems maybe a bit tricky in terms of UX, but non-contentious feature-wise
    2. Send to/get from tablet. That's currently still arguably the best way to get stuff onto an Android tablet to annotate, but once there's an Android app, I'd argue this is probably OK to let go off (although there's always someone who loves a feature..)
  • Both minor functionalities in my view are actually MAJOR:
    -send to/get from tablet will not disappear with a zotero app for android, at least not for those who use an android e-reader (as I do). running zotero on an e-reader is clumsy, and what is needed is exactly the send to/get from tablet functionality.
    -Attach new file is something I constantly use when downloading files for example that I get sent through email
  • As I said
    (although there's always someone who loves a feature..)
  • Totally agree with everything. If anyone has a lead on someone that forked and is trying to do the porting of ZotFile to Zotero 7, I would love to help with what I can. It is quite important to have it running.
  • edited May 31, 2023
    > Major:
    > 2. Moving files to links -- that's tricky because I'm not sure to what extent Zotero wants to actively promote usage that has notable downsides (like: doesn't work with mobile apps.)

    I primarily use Zotfile for its ability to organize my papers and PDFs in a separate directory, which significantly streamlines various tasks such as searching (ripgrep, fd), fuzzy finding and launching, annotating, and linking within markdown. While I also find the advanced renaming features useful, my workflow heavily relies on the seamless integration of Zotfile and BBT plugins. It's way faster and smoother workflow for my case. Overall, having a dedicated directory for my papers is crucial for my productivity.

    In my opinion, the best solution would be to integrate its main features into Zotero, although I understand there may be some resistance to this idea.

    One imperfect (possible) workaround that I right now can think if ZotFile get outdated, is to rsync the Zotero data directory coupled with a pdf finder (`find`, `fd`) to regularly update the separate directory and keep the storage filled with sort of symlinks!
    - did a quick search and found [this](https://github.com/retorquere/zotero-file-hierarchy) it may help in future.
    - The problem with this, is it is one way! So for example, if I annotate a PDF with my iPad or favorite PDF annotator, it automatically doesn't update the corresponding PDF of the Zotero.
  • I strongly agree with migugg's comment above. My zotero workflow depends about 85% on the ability to seamlessly manage attachments using zotfile to send to Android tablet where I do my markup; then I retrieve them into zotero which automatically turns markup into notes.

    If I can't do the send to/get from tablet, I regretfully cannot move to zotero 7 (which I would love to do because with the size of my library the 32 bit version is becoming very slow)
  • bad news.
  • The bottom line is that the absence of Zotfile's functionality being available in Zotero 7 would prevent thousands of Zotero users from migrating to Zotero 7. They rely on that functionality for multiple important reasons, like needing to easily access their PDF collection external to Zotero for various academic purposes; in ways that are incompatible with Zotero taking those files into local Zotero PDF storage.

    Whether Zotfile's functionality remains with Zotfile itself (made compatible with Zotero 7), or it is integrated into generic Zotero, doesn't really matter to users. If it's the latter, there does not seem to be any objection from Zotfile developer @Joscha to the pioneering work on Zotfile becoming basically redundant. That's a normal process with addons that over time are shown to have 'mission-critical' functionality that deserves to be in the main app. But if it's the former, then it sounds like Zotero developers would need to collaborate with @Joscha, as they did with the transition to Zotero 6, when some Zotfile functionality was subsumed into Zotero (PDF annotation extraction). Zotero developers are presumably quite familiar with Zotfile.

    In any case, it would be good to hear Zotero developers' plans for this.
  • @tim820. Although I don't necessarily agree with your sentiment particularly towards the zotero developers, you raise an important point about file compatibility. Some files might be lost for people who are using zotfile to send and get files from tablet (optional setting under "Tablet Settings") and have set the ".tablet.mode" setting to 1, which is the default called "background mode" in the documentation. I do want to address that. At a minimum with a warning to current users before transitioning but hopefully with a solution. It can get complicated because different versions of the same file might exist.

    About the general future of zotfile: Pretty crazy that I published the first version of zotfile almost 15 years ago as a grad student. The irony is that more people have probably used zotfile over the years than read my academic work. Unfortunately, I just don't have the time to continue the maintenance and development of zotfile. Generally, I agree with the comment from @adamsmith above. An additional option would be for someone to create a new plugin that implements some of the functionality (e.g. more complex renaming). There is a lot of old code and unnecessary complexity in zotfile so a clean slate has many benefits.
  • Actually I am quite surprised to know that zotfile was developed by an associate professor of sociology from Harvard. Thank you for your contribution.
  • to create a new plugin that implements some of the functionality (e.g. more complex renaming)
    Just to note, we'll be integrating more advanced renaming in an upcoming Zotero version. We implemented a version of it last year, but didn't release it, and now we're working on a different, more flexible approach.
  • edited June 22, 2023
    Could you tell us if it's coming to the official release of Zotero 7? As adamsmith listed in his above comment https://forums.zotero.org/discussion/comment/434438/#Comment_434438, the 2 major functions (complex renaming and moving files to links) are the only things I use ZotFile for. Even just having the complex renaming feature would allow me to adopt Zotero 7.
  • Advanced renaming will be in Zotero 7, yes.
  • edited June 22, 2023
    So of @adamsmith's above list of Zotfile functionality that needs to be available in Zotero v7, we now know that complex renaming will be in v7, and we already knew that that an Android app should be coming at some point (soon ?).

    So Zotfile users who rely on Android functionality could *possibly* delay a move to Zotero 7 until the Android app is available - depending on how long that will be - and *assuming* it covers their workflow well enough (and that other critical features of Zotfile are also made available in some way under Zotero 7). But from what some Zotfile/Android users have said above, the Android app may not cover their needs. If not, another way of doing Android send/get would be required.

    Of the other features still requiring a solution ...

    '2. Moving files to links'

    I'm guessing that Zotfile's auto-moving/linking of attachments as they are downloaded is probably the key 'mission critical' feature for most users. It allows a workflow and file organization system that is vital to the way that people need to work, and is not achievable in any other way. An addon that did nothing but this would be good (eg a 'cutdown' v7-compatible Zotfile ?); better would be to have that functionality within Zotero itself (consistent with its wide use).

    '1. Attaching recently added files'

    This is also very useful IMHO. It shows the most recently-dated file in a designated folder (that the user has deposited there - for example when Zotero's web connector could not download/add it automatically from an obscure website), and asks the user if they want to attach it to the item. If so, the file is renamed/moved to a pre-designated Custom Location folder and added to the item as a linked file. So it's a variation on Zotero's right click Add Attachment\Attach Link To File (which requires a few more clicks; but does it rename too ?). Which would presumably make it relatively easy to add to Zotero (importantly it's not a 'watch folder', to which I believe Zotero developers are averse). But the fact that it can be done in another way by Zotero 'as-is' possibly makes it non-critical.

    So we're perhaps talking mostly about attachment auto-moving/renaming that would have to be made available in Zotero v7. And Android send/get if there is no other way of doing that.
  • Hi

    as expressed by many others here, I think the availability of zotfile (or zotfile-like functionality) would be make or break for my transitioning to zotero 7. and as someone above has already said, it's made for people like me who don't have the skills to code, but love to tweak our files.

    that said, the version 6 PDF reader/annotation functions are a step up from what's available on Zotfile, and given that history I am hopeful that all the Zotfile functions that are lost (and I'm sure each of us have favourites) will get substituted either by functionality within zotero itself, or with new add-ons.

    given that this is going to be a bump that most of us will come up against, I wonder if it would be worthwhile compiling a how-to for zotfile users separately, to soften the blow? not up to the developers obviously, but if we can put this together I think it would be very useful.
  • Hi everybody,

    I quickly slapped together a Zotero 7 plugin called ZotMoov. It can be found at https://github.com/wileyyugioh/zotmoov. All it does is move added files to a directory and change it to a linked file. It would be nice to get some feedback to polish it a bit more.

    It should be noted that I'm not planning on adding any features--I want to keep it simple.

  • Hi everybody, I know that developpers do not give details on if and when certain functionalities are introduced, but I am wondering whether the zotfile situation would not call for a change of this practice. To me this sounds like a mess, and it leaves users (and plugin developers) in a difficult situation. It is obviously great that wileyyu develops ZotMoov. But wouldn't it be much more useful if the devs could clarify the following:
    -Will the android app be released at the moment when zotero 7 will be officially released?
    -Or at least: Will the android app be released at a time before zotero 6 will become entirely defunct (or will zotero continue to offer version 6 for download after the official introduction of zotero 7)?
    -Which parts of zotfile will be included into zotero at some point and what is the order of their inclusion?
    -Which parts of zotfile will never be included into zotero?

    This would at least give users and plugin developers some kind of planning safety. Otherwise there may be plugin developers wasting their time on developing a plugin whose functionality will be included into zotero/the android app soon.

  • Due to the fact that Zotfile doesn't work, I have switched back to version 6. It seems a great pity to me. Could someone modify it to work on 7, or is the extension too complicated? Being able to sort attachments into folders in my storage (and have it synchronized throughout multiple devices) is a very key thing for me.
  • Please keep in mind that the official version is Zotero 6 with all plug-ins work as expected. Zotero 7 is still in beta so don't expect everything to work yet. All plug-ins are developed by volunteers and if you're wondering if the extension is complicated: yes it is, so please consider donating. Others also have posted alternatives as well. You can try and share your experience.
  • edited July 7, 2023
    The underlying code modernised;
    Internal Pdf reader with amazing annotations capabilities;
    with hundreds of translator (scrappers) from the web;
    Betterbibtex almost ready to be released;
    Zotero is about to be the most potent reference manager in the world.

    Zotfile is the only weakness we have.
    Please, somebody, a generous soul, with programming skills: it would really be great if someone can help us to update Zotfile.
  • Just wanted to add another voice to the usefulness of Zotfile's functions.

    The 'Attaching recently added files' is something I use all the time, since I have lots of references that are documents, reports, and others that Zotero does not pickup automatically.

    The 'send/retrieve file from tablet' is something that it is also useful to me even though I use the iPad app since the beta was released, because sometimes I need to review a paper on another app and that really simplifies the workflow.

    Also, I don't currently use the linked files feature since I pay for Zotero storage, but it was a life saver when I was a broke college student.
  • For the broke college student: If you know how to set up a symbolic link (a single line command on a terminal), you can have, say, Dropbox (or the cloud storage your school pays for) to sync all your files seamlessly. As mentioned among the alternative solutions on the page https://www.zotero.org/support/sync You don't need ZotFile for that.
  • @enozkan of course we don't need ZotFile for that (in my case, that's what I am using it for, even though I have considered the slightly more complex route of setting up a WebDAV server), but having to run terminal commands still feels finicky and inelegant compared to a proper extension.

    Sadly, I will have to hold off on updating to Zotero 7 until an updated or forked version of ZotFile (or an alternative) becomes available, and I know I'm not alone in this. And I know Zotero developers have a vested interest on having people use their sync solution instead, but it does feel weird that a FOSS application doesn't have that basic functionality built-in. If Zotero 7 eventually supports linked files by default, boom, problem solved.
  • edited August 5, 2023
    Hi everyone! I completely understand that this is frustrating for people who have relied on zotfile for many years. Let me just clarify three things:

    1) Unfortunately, I am not going to update zotfile for Zotero 7 myself. I looked into it again but it would take a significant amount of time. Happy to support anyone who takes on that work. From my experience, that is unlikely and you should not expect zotfile to work in Zotero 7.

    2) In the long term, it is better to find alternative solutions for your workflow anyways. A combination of integrating features in Zotero and smaller plugins for specific tasks is much more future proof.
  • I can only agree with many of the other comments here : without Zotfile working, I will not update to Zotero7. I have 20 years old directory structure for my pdf collection, and I'm clearly not going to change this,especially that now Windows search has become to powerfull; and also because I use other programs to interact with pdfs of scientific papers, and all windows links would be invalid if I would change the directory structure for the pdfs. So the main feature I use from Zotfile is that during automated download from a webpage, the pdf is put into the correct directory, changed to the correct name and then linked to the Zotero reference. A plugin which would do this would lead me to upgrade to Zotero 7 (even though I would miss some of the other Zorfile features :-)
Sign In or Register to comment.