Automatic file renaming not reported properly in Zotero 7

cjb

ReportID: 266675166

When using Zotero connector to add a reference + PDF to Zotero, and when using a custom renaming pattern via extensions.zotero.attachmentRenameFormatString in Config Editor, the associated PDF is automatically attached but the attachment file is reported in the Zotero reference list as *not* automatically renamed, i.e. displays the publisher default (e.g. "SAGE PDF Full Text"). When selected, the info side panel is also titled with the publisher default name (e.g. SAGE PDF Full Text). However the actual file has been renamed as per the custom renaming pattern (e.g. Heyes (2023) Rethinking Norm Psychology.pdf) - which can be seen both the Filename field in the info side panel and in Windows Explorer.
All relevant checkboxes under general preferences are checked.

Steps to reproduce:

1. Start Zotero.
2. In Firefox, browse to a journal article page e.g. https://journals.sagepub.com/doi/10.1177/17456916221112075
3. In Firefox, click on Zotero Connector 'Save to Zotero' button.
4. Select attachment once saved to look at info panel, confirm renaming with "Show file" context menu entry.

Note: displayed name can be corrected using "Rename File from Parent Metadata"

adamsmith

Zotero has always distinguished between the attachment title (Full text PDF or the like), displayed in the middle panel, and the attachment file name (visible e.g. in the right panel.
That's by design and not new in Zotero 7. If this worked different from you before, that was likely due to using Zotfile for renaming

cjb

Not sure I'm understanding correctly, so asking for clarification to make sure we're talking about the same thing.

If it's working as intended then why does the manual "Rename File from Parent Metadata" function rename the file, the (right panel) title, and the middle panel attachment listing? What I'm seeing is that the automatic renaming only renames the PDF file itself, it leaves the info panel title and centre panel name alone. Is it really part of intended design that manual and automatic renaming work so differently? I assumed that the automatic renaming setting should just automate the manual renaming function?

adamsmith

It's not ideal that it's inconsistent, but the automatic rename gets the title from the translator (e.g. it's different for open access PDFs added from other sources). The manual rename doesn't have that information so falls back to the filename for the title

cjb

Still feel like we're talking past each other, apologies if it's just me being dense.

Can you give me an example of a URL where Zotero takes the title from the translator to give it a title and list name other than "SAGE PDF Full Text" or similar? Because that's all it's giving me for the title (right) and attachment name (centre) - the name of what the PDF file would have been if it wasn't being automatically renamed. Or is that the expected behaviour? (if so... why?)

adamsmith

The attachment title (i.e. what's displayed in the middle panel and the bold title on the right) is typically something along the lines of Full Text PDF. You can always verify the filename under "Filename" on the right.

The attachment title is a description of what the PDF is. E.g., when you add 10.1029/2004GL020892 you get "Submitted Version" -- i.e. not the full text from the publisher, but the submitted version as deposited by the author. The attachment title provides additional search functions (e.g., if you don't want any snapshots you can search for "snapshot", select all items and delete) and is quicker to read than the filename (which, in the context of the middle-panel, presumably consists of information available in other columns for the parent item anyway).

Again, this is expected behavior and has been that way for at least a decade, probably since the beginning of Zotero.

cjb

I see, thanks for taking the time to the explain. I guess I'd been ignorant of all that due to previously using zotfile as you say (and never using snapshots etc). Any chance this will be made tweakable in the upcoming renaming revamp I was reading about? My preferences here are mostly aesthetic I realise, but given that any necessary manual renaming (e.g. after making corrections) overwrites both filename and title, the intended behaviours introduce inconsistency.

adamsmith

I don't know (not something I'm involved in the decision-making on), but from past statements, I doubt they'll pref this (though they do take note of threads like this).

dstillman

We have no plans to change the title/filename distinction. One thing we could consider would be not changing the attachment title on a manual rename if the title is already different from the filename. So if both the title and filename are foo.pdf (because you just dragged the file in from the filesystem), a manual rename would change both, but as long as the title was different (e.g., "SAGE Full Text PDF), it would change only the filename. The main problem with that is that it wouldn't be as obvious that something happened, but the Filename field would change, and we could perhaps do something else to help indicate that something happened.

cjb

Got it. Thanks both of you for taking the time to explain. I see the point now with having the title carrying different information and will probably start using it that way. So yes, I have now flipped to desiring the update suggested! i.e. not changing the attachment title on a manual rename if the title is already different from the filename.

Goes to show that you can use Zotero every day, for the better part of 2 decades, and still not fully appreciate how subtle and useful it is.

dstillman

One thing we could consider would be not changing the attachment title on a manual rename if the title is already different from the filename.

We've made this change in the latest beta.

danielborek

thanks for explanation (I am coming from other thread)
Still, could this new behaviour of not manually renaming be opt-out, for example under flag in advanced settings? Not crucial thing, but small quality of life improvements

If the Item has "Full PDF" type pdf file, it indicated for me that I didnt processed it, and I should check this pdf as newly downloaded from internet (this is error-prone, I know), but it is useful heuristics for me, when I read/skim pdf I was changing the displayed name (without further tagging or moving to folders)

With Zotero 6, there is Zotfile, with Zotero 7 users doesnt have this plugin for renaming, I am renaming for asesthethical reason, "Full PDF" name as information is most oftern nor meaningful or helpful, a way of adding suffixes, custom names or similar thing on top of renamed pdf would be indeed beneficial (Both preprint and the published version can have name full pdf), but it can be really complicated to implement

dstillman

Still, could this new behaviour of not manually renaming be opt-out, for example under flag in advanced settings?

No. We explain the reasons for this above. The attachment title is meant to be different, it frequently provides important information (e.g., "Submitted Version"), and there's no reason to repeat metadata from the parent row in the attachment title. It was a bug that it was changed unnecessarily before.

With Zotero 6, there is Zotfile, with Zotero 7 users doesnt have this plugin for renaming

Zotero 7 has a new, advanced template system for filenames — we'll be adding a UI and documentation for it shortly. But that's for filenames, not the attachment title.

You can change the attachment title to whatever you want, but that has nothing to do with the filename. In an upcoming build we'll be adding continuous attachment file renaming as parent metadata changes, and at that point we'll be removing the manual Rename File from Parent Metadata option altogether as long as that setting is enabled.

tim820

@cjb if you are used to the way that Zotfile handles renaming in this case (as I am too), the new zotero-file addon should replicate that behaviour in Zotero 7.
https://forums.zotero.org/discussion/comment/439774/#Comment_439774

danielborek

Unfortunately it doesnt replicate old behaviour, as it probably rely on the mechanism that Zotero devs now changed :(

I created an github issue here
https://github.com/MuiseDestiny/zotero-file/issues/20

dstillman

@danielborek, it's worth being clear that you were using the attachment-title behavior as a hack for your own personal workflow. You weren't even actually renaming files, since Zotero has always automatically renamed downloaded files. So even making this about file renaming isn't particularly appropriate. If your goal is just to flag something as processed, there are plenty of other ways to do that — plugins that automatically add a tag to new items that can be cleared, clicking the attachment title in the right-hand pane and adding some indicator, whatever. Again, the actual files were already being renamed, so there's no reason you had to use Rename File from Parent Metadata.

For the reasons explained above, I'd strongly recommend that, if a plugin was going to customize something related to this, it just customize whether the items list shows the filename or attachment title instead of permanently overwriting attachment titles created by Zotero. That won't make it work for your specific workflow, but your workflow isn't about filenames anyway.

danielborek

@dstillman thanks for the clarification! Indeed, the reason is my personal workflow and aesthetic preferences (the redundant information is also useful for me, having it in several places help me remembering/better processing/retrival and attachment name "full pdf" is not useful for that);

So I would change this request to "Customize whether the items list shows the filename or attachment title instead", as it is not possible right now using advanced settings in Zotero to set it?

This would satfisfy our vocal small minority, but I understand that development might have other priorities :)

And thanks to whole Zotero team for the great work you put into improvement of Zotero, I am appreciating all fixes and new features, even those that breaks my workflow :)

tim820

Re "instead of permanently overwriting attachment titles created by Zotero" ...

Zotfile has *always* (over)written the attachment title with the same (renamed) name it has applied to the actual PDF filename. At least for the auto-renaming that Zotfile invokes during PDF download. For right-click Rename/Move, Zotfile can do some stuff that can result in the title being slightly different to the actual filename (ie the latter may have a '2' suffix added, while the title does not).

In any case it would seem reasonable that a Zotfile-coded addon for Zotero 7 would do the same as Zotfile does in Zotero v6.

@danielboek I am not able to test the operation in Zotero 7 beta. But just to be sure , do you have 'Use Zotero to Rename' set to OFF in zotero-file preferences ?

And if you do have it set to OFF, how exactly does the zotero-file behaviour differ from Zotfile's ? And does it do so for both auto-renames when downloading a PDF and also for right-click Rename/Move ?

dstillman

Zotfile has *always* (over)written the attachment title with the same (renamed) name it has applied to the actual PDF filename.

Right, and I'm saying it shouldn't overwrite something like "Submitted Version" with a redundant filename — that's throwing out important data. If people want to see the filename, a plugin can provide an option for that, but the title should be preserved.

tim820

I think we'll have to agree to disagree on this. I cannot recall a single instance where the often-arbitrary download file name applied by a journal publisher was worth preserving. Applying Zotfile renaming rules to both the title field and the actual filename has always seemed the most sensible option to me (albeit making double fields redundant; which is another way of saying separate title/filename fields are probably rarely useful).

If I look at my most recent download titles they are mostly meaningless things like ...
paper_221219.pdf
000118.pdf
10.1515_hf.2011.148.pdf
main file.pdf

I am sure there are instances where the download title from some organizations (eg government departments, legal documents) is very meaningful and important to preserve (and in those cases you would probably preserve the actual filename too ... so again, redundancy ?). So there must be a way of preserving those if needed. I just haven't come across it in my work, mainly with journal papers.

If nothing else, the 1000s of Zotfile users who are used to this behaviour will expect it to be preserved in a Zotero-7 compatible version of a Zotfile-emulating addon (ie @polygon 's zotero-file).

I understand that others have other preferences - for example the way that Zotero now handles this issue. Different preferences is why we have addons. If an addon didn't do something different to Zotero itself - like overwriting attachment titles - it wouldn't be an addon. ;)

dstillman

@tim820: You seem to have totally misunderstood this discussion. Zotero has never, in its entire existence, kept those sorts of useless names anywhere. Files from the web don't even always have filenames, so Zotero has, from day one, renamed downloaded files based on the creator, year, and title of the parent item. Those publisher filenames are not what we're talking about.

This entire discussion is about the attachment title, which comes from Zotero translators and Zotero code — titles like "Full-Text PDF", "Snapshot", and, more importantly, "Submitted Version" or "Accepted Version" for open-access files, to distinguish them from published versions. This entire thread is just about a bug fix to stop overwriting those unnecessarily.

What I'm explaining now is that, just as we fixed this in Zotero, there's absolutely no reason for a plugin to throw out that data. Most of the feedback to this change has just been from people who thought that file renaming wasn't happening anymore (and who perhaps thought that they had to run Rename File from Parent Metadata to rename downloaded files in the first place), but regardless, if people prefer seeing filenames in the items list, the correct solution — from Zotero or a plugin — would be to have an option to show the filename instead. The attachment titles would then still appear in the right-hand pane when clicking on the attachment, and they'd be searchable for various purposes (e.g., cleaning up snapshots, updating preprints).

If nothing else, the 1000s of Zotfile users who are used to this behaviour will expect it to be preserved in a Zotero-7 compatible version of a Zotfile-emulating addon (ie @polygon 's zotero-file).

There's no reason to reproduce destructive behavior in a new plugin due to some misguided idea about fidelity to ZotFile. I'm explaining how a plugin can provide the exact same experience people are used to — the filename appearing in the items list — without throwing out data that Zotero's built-in features and site-specific translators have been specifically coded to include.

tim820

OK, apologies if I've misinterpreted what I've been seeing/what Zotero does (not for the first time !). My use case with Zotfile is ... my use case. Thanks for clarifying the Zotero operations that I've not seen as often - because I have rarely used them.

As to my opinion that Zotfile's making the title and the filename the same is the most sensible option for me (and possibly other Zotfile users who are used to it) ... that remains the same. But I will try turning Zotfile's renaming off and see if the alternative makes sense.

dstillman

But I will try turning Zotfile's renaming off and see if the alternative makes sense.

You can — and this week we'll be sharing details on the new filename customization syntax in Zotero 7, which should mostly replace the need for filename customization via a plugin — but I just want to be clear about my point here, which is that, even if you prefer seeing the filename in the items list, the correct solution would still just be to actually show the filename, not to unnecessarily throw away data and have this field that looks like a filename but might not even be in sync with the actual file. There's just no argument to be made for the latter that's not based on some sort of misunderstanding.

We'll look into adding an option for showing filenames instead.

tim820

As to what the "whole" thread was about, it has obviously meandered. The OP's issue was basically why Zotero v7 no longer appeared to reflect what was actually a Zotfile operation (which does not run in v7). That is the issue that has been returned to several times in the thread (amongst others) and what I was referring to - that is, what Zotfile puts in the attachment *title* field, as it appears in the main pane under the item and at the top of the right pane when the PDF is selected.

I have just checked what Zotero itself actually does (in v6, without Zotfile), to refresh my memory. Zotero downloaded the PDF, put 'Full Text PDF' in the middle pane (so my memory was just wrong - I thought Zotero put the *actual* download filename as the title there), and it put the same title above the PDF URL in the right pane (along with the actual stored filename). I also tried an in-press pre-proof. Zotero put "ScienceDirect Full Text PDF" in the main/right panes for that.

So an argument that those titles used by Zotero are clearly more meaningful than having the title match the actual renamed filename (Zotfile-style) I feel is somewhat tenuous (esp given that the ScienceDirect one says nothing about the paper being a pre-proof). That was the accurate part of my memory of what Zotero does.

I have sometimes manually added my own suffix to the same-as-filename title from Zotfile, eg where the PDF file is a google translation from another language, or where I have attached several PDFs to an item and only one is annotated, or where I want to distinguish between different versions (presumably more accurately than Zotero can).

So all in all I think it still comes down to personal preference. And I still prefer how Zotfile does it. Part of that is familiarity/consistency (I have 4000 PDFs stored in the database that way). But it's also that "PDF Full text" and "ScienceDirect Full Text PDF" (for a pre-proof) don't seem very meaningful to me. But I will try to approach any advances in Zotero's new renaming scheme in the eventual v7 release with an open mind (compared to what the Zotfile-emulating addon zotero-file does, hopefully exactly what Zotfile does). ;)

dstillman

So an argument that those titles used by Zotero are clearly more meaningful than having the title match the actual renamed filename (Zotfile-style) is somewhat tenuous

We've given the relevant examples repeatedly. When Zotero retrieves an open-access file via Unpaywall, it sets the title to "Accepted Version" or "Submitted Version". Translators usually use something fairly generic such as "Full Text PDF" (which, as adamsmith says, is much quicker to read than a filename), but they have the option of putting other information in there. The ScienceDirect translator may or may not be able to tell from the site whether a given PDF is a preprint, but if it could, that would be an appropriate thing to put in the title. It's just a field where the code that actually knows what's being saved can add additional information, so we're asking that plugins not overwrite that by default.

I have sometimes manually added my own suffix to the same-as-filename title from Zotfile, eg where the PDF file is a google translation from another language, or where I have attached several PDFs to an item and only one is annotated, or where I want to distinguish between different versions (presumably more accurately than Zotero can).

I mean, OK, but I really don't see the sense in that. If it's only for your Zotero library, then that's the whole point of the attachment title — the parent item says what it is, so you can just make the title "English Translation" or "Annotated Version" or whatever. Or, if it's actually relevant to the filename, you should just rename the file so that you can search for it and see it outside of Zotero as well, and then the option I'm proposing to show the filename in the items list would suffice. I understand that people have gotten used to ZotFile's behavior. I'm just suggesting that there are better options, and that overwriting data is bad, which is why we fixed this bug in Zotero.

dstillman

Think about it this way: if you added a meaningful suffix to the attachment title, and then you corrected an author name in the metadata and wanted to rename the file again to match, you wouldn't want to lose your manual edits! That's why the attachment title is separate from the filename, and why overwriting it doesn't make sense. Even for automatic file renaming, it'd be better to populate some other field that could be used automatically to rename the file with that info preserved. In Zotero 7, you'll be able to put, say, {{ extra }} or {{ rights }} into the filename template to add some information to the filename automatically. To support renaming multiple child attachments, we could even add support for {{ attachmentTitle }}, and then the attachment title itself (or the attachment title with a particular prefix, etc.) could be the string that was added to the filename, even if you chose to display the filename in the items list.

dstark

@dstillman, regarding how

if people prefer seeing filenames in the items list, the correct solution — from Zotero or a plugin — would be to have an option to show the filename instead

is this something that is already available (e.g., in 7.0.0-beta.38+b79e0b3d7) or on the roadmap for v7?

Thanks so much for the clarifications about v7's change in behavior on this issue!

aaaaaaaaaaaaaaa

To support renaming multiple child attachments, we could even add support for {{ attachmentTitle }}, and then the attachment title itself (or the attachment title with a particular prefix, etc.) could be the string that was added to the filename

@dstillman, I am looking forward to the addition of the {{ attachmentTitle }} variable for renaming files. Some sites (ie. Project MUSE) offer PDF downloads for each individual chapter of a book, and the Zotero browser extension makes it easy to download them all at once, and very helpfully adds each PDF as a child nested under the same book entry. Currently, the translator automatically assigns each of those PDF files a title (not file name!) corresponding to the name of each chapter. After the addition of the {{ attachmentTitle }} variable, it would finally be possible to automatically transfer the chapter names (which Zotero displays in the attachment's Title field) to the PDF file names!

erazlogo

Use cases for having an option to match file name and attachment title:

1. The list view then makes clear to the user how to search for the file name in Spotlight. It's true that from Zotero you just need to use "Show file" option. But if you switch to another program and continue working, searching via Spotlight might be faster.

2. Advanced Search by Title searches Attachment title and there is actually no way via Advanced Search to search for Attachment file name. Why would I ever need to search for "PDF Full Text" to find all my imports from databases?

3. If I want to separate a preprint from a published version I can use tags. There is no need to provide any "useful information" in the attachment title if a user needs to click several times to find it.

I understand that my workflow is different from others' but all I am asking for is an Option that people like me can use.

adamsmith

1. would be covered by _showing_ the filename instead of the attachment title as mentioned by dstillman above. I understand that's planned as an option.
3. isn't a use case but a claim that there isn't a use case for the attachment title -- which there very much is: we have the different names for different green OA articles, for PDFs linked to from google scholar, we have helped people here on the forums remove unwanted snapshots by taking advantage of attachment titles of those. All of this happens automatically on import, so not comparable to manually added tags. It's fine if you find none of that relevant to your work, but it's just not true that there is 'no need.'

Which leaves 2. as a use case, but when are you actually doing that? I find it hard to think of a scenario that doesn't have an easier solution (like, searching by the actual item title). If anything I'd find the search hits for title searches here rather confusing.

aaaaaaaaaaaaaaa

@adamsmith, regarding your suggestion that displaying the _filename_ instead of the attachment title in scenario 1. --- here's a use-case that complicates things: Using the browser extension to import a book from ProjectMUSE adds unique child attachments for each chapter. Imported in this way, the _titles_ reflect the unique chapter name, but the _filenames_ are the same for every single chapter in the book. So opting to display the filename instead of the title would make it difficult to distinguish one attachment from the other within the library view, and it would probably require the user to tediously open and check each of the attachments with identical filenames in order to figure out what attachment they're looking at.