Zotero 7: Will renaming be as robust as ZotFile's?

dstillman

This discussion was created from comments split from: Automatic file renaming not reported properly in Zotero 7.

ria3k

Just curious — will the renaming features in Zotero 7 be as robust as Zotfile's?

For example, Zotfile allows regex changes to particular fields. You can also specify subfolders based on various criteria (collection, type, creator, etc.).

warguelles

You can also specify subfolders based on various criteria (collection, type, creator, etc.).

I was just wondering about that too, because with zotfile I not only rename the files, but also the subfolders where to save them. That allows me to have a logical-organizational structure of the DB.

With the last options included, I think that the module to rename files in Zotero 7 is extremely powerful (much more than zotfile!!!)... hopefully it can also be complemented with the possibility of predefining (and creating if necessary), the subfolders where they should be saved to.

ria3k

Call me crazy but being able to combine conditionals with "/" or "\" in a suffix parameter would be nice. Currently in the new renaming system slashes appear to be disallowed.

dstillman

@ria3k: I'm not totally sure what you're asking re: slashes, but slashes aren't valid in filenames on all platforms (and different slashes are invalid in different ways on different platforms), so as a cross-platform app that syncs, Zotero doesn't allow slashes in filenames. See this post and its comments for alternatives.

Zotero only creates stored files when saving, so I'm afraid folder creation is out of scope for Zotero itself.

ria3k

Understood.

I was referring to linked attachments, not Zotero-stored ones. Zotfile allows two kinds of "renaming": one template for the filename itself, another for the subfolder where you want the linked attachment to land (outside Zotero storage). This is useful for those who like to manage their own attachment structure.

One could imagine a preference such as "Specify subfolder for linked attachments?" with a subpreference for the subfolder destination, using a template syntax much like the existing one for renaming.

Of course, all this assumes an option for renaming AND moving attachments out of Zotero, which I realize is controversial. But judging by the many comments around attachment storage, it would be welcome.

dstillman

Right, we obviously know what ZotFile did, but that's not something we'll be supporting in Zotero. We specifically discourage the use of linked files — which wouldn't work in the web library, iOS app, or any group library, and which frequently cause problems that we can't provide support for — so we're not going to add a setting to save linked files by default.

northword

@dstillman Do the fields supported by the file rename function take into account what is in the EXTRA?

One of the users of my plugin wants to include journal abbreviations in the filename (the most simplified journal abbreviation, containing only the first letter of the full name of the journal, e.g. cem. eng. j. would be abbreviated as CEJ), and he expects to store the CEJ in the journal abbreviation field to make it easier for the file renaming feature to use it.

However, I would strongly discourage this, as I believe that the journal abbreviation field should always store the correct iso4 abbreviation, so I would expect it to be stored in extra, but the current file renaming feature does not support the use of the data in extra.

Any thoughts on adding this support please?

ria3k

Spent some time toying with the file renaming feature and developed this wish list.

Add regex find-and-replace parameter for all variables.

Add firstCollection variable, ideally returning the first and topmost collection name of the parent item.

Add urlHost variable, returning only the www.domain.com portion of the parent item's url.

Add truncatedTitle variable. Akin to shortTitle but returns a title stripped of any period, question mark or colon, plus anything after it, so the file name can use the equivalent of a shortTitle even where none has been entered in the parent item's metadata.

Allow use of logical operators (and, or, not, xor) in renaming syntax.

Allow if statements evaluating the number of creators to be compared with integers using equals, greater-than and less-than operators.

Allow parameter “join” to work with firstCreator variable, so users can set an "and" alternative.

Provide option for user-defined “et al.” string for use with with firstCreator.

Allow multiple variables in a single bracket, separated by an exclusive operator, for instance, " | ". In this case, the first variable in the list to match would prevail. This would reduce the need for lengthy "if ... elseif" structures.

Allow bracketed statements to be nested.

Provide additional option for simplifying special characters, e.g., removing curly quotes and apostrophes, diacritics on Latin letters, etc., to ensure valid, easily searchable file names across operating systems.

Provide option for suppressing suffixes where they would create a repeat character, e.g. two hyphens, two periods etc.

ria3k

Zotero obviously does create folders when saving stored attachments — just not any that users have direct control over. They have randomized eight-character folder names for every attachment. If you have 4,000 attachments, every computer you use has 4,000 folders for them under "storage," and you can't directly view your attachment files all in one place. This is major reason many users prefer to maintain linked attachments despite the risks, syncing them with third-party platforms. Those who do this appreciate that linked attachments are still allowed, even if discouraged.

dstillman

@ria3k:

If you have 4,000 attachments, every computer you use has 4,000 folders for them under "storage," and you can't directly view your attachment files all in one place.

The folders allow Zotero to reliably manage and sync files between devices, without any risk of files getting unlinked due to their being moved or renamed outside of Zotero. As we explain in the documentation I linked to, you can create a smart folder in your OS for a flat list of all files stored in Zotero, and your OS search should allow you to open files directly outside of Zotero (and now with much greater flexibility of filename to search for).

Spent some time toying with the file renaming feature and developed this wish list.

That's…quite a wish list! But we'll take a look — thanks. For the less-obvious ones, it might help to explain how you see yourself using them.

dstillman

@northword: All fields are accessible, including Extra. You can't currently extract key: value fields from Extra, but we can consider making that possible. (The regexp support suggested by @ria3k would be one option.)

ria3k

Happy to do that. Which ones appear less obvious? Generally speaking, these options would allow for powerful fine-tuning of renaming and consistency with a user's existing approach. For instance, for items with no creators, the filename can lead with the publisher, the URL hostname, etc., depending on what fields are present. Regex is at the top of the list because that alone could address a lot of the subsequent issues. (Though not the URL one because it appears the current renaming for that field, understandably, takes out all slashes, the colon, etc., so it could be nearly impossible to extract just the hostname.)

dstillman

Regex is at the top of the list because that alone could address a lot of the subsequent issues. (Though not the URL one because it appears the current renaming for that field, understandably, takes out all slashes, the colon, etc., so it could be nearly impossible to extract just the hostname.)

No, invalid filename characters are removed after the full string has been generated, before saving, so a regex option would have access to the full string. And that's the one I think we'd be most likely to do, since as you say it could be used for most other custom options.

Provide option for suppressing suffixes where they would create a repeat character, e.g. two hyphens, two periods etc.

I think we should probably just make this the default and only behavior.

ria3k

Ah, that's great.

jeremyvancleve

The file name renaming in Zotero 7 looks really nice @dstillman!

There are a few things I really like in Zotfile that it would be great to see in the built in renaming. Some of these are already covered by @ria3k and I'll just be seconding them.

1. authors, editors, creators should have an "etal" option for when the number of items exceeds the "max" value. This should also be customizable. Then, one could replicate the "firstCreator" variable with {{creators max=2 join=" and " etal="et al."}}

2. Zotero has a nice abbreviate function, Zotero.Cite.getAbbreviation, that can abbreviate titles of papers, journals, books, etc. I added that function in Zotfile so that I could use publicationTitle abbreviations in the file name (https://github.com/jlegewie/zotfile/blob/a6c9e02e17b60cbc1f9bb4062486548d9ef583e3/chrome/content/zotfile/wildcards.js#L14). Having an "abbreviate" function would be great since you could then not simply truncate, but abbreviate long titles.

3. regex or replace function. I like to have filenames without spaces and it would be nice to be able to remove/replace them or other characters in the file name components.

tcrow86

I second (or third or fourth?) a "native" solution to the "et al" issue depending on number of authors.

For now, I've found a workaround that adds "et al" when there are 2 or more authors but omits "et al" when there is only 1 author. For anyone interested, here is the Filename Template code, which allows me to replicate the functionality which I previously used ZotFile for (in Zotero 6):

{{ if firstCreator == creators }} {{ firstCreator }}
{{else }} {{ creators join = " & " max = "2" suffix=" et al" }} {{ endif }}

gndinh

I've added a PR to support regex replacements for arbitrary fields here: https://github.com/zotero/zotero/pull/3562

The main limitation is that only a single regex replacement is possible, so you can't perform multiple replacements in sequence. If there's interest I can try to figure out a way to add that feature.

ruijie

@tcrow86 zotfile's algorithm is a bit more complicated at least in my experience. If there are editors in the item, only the editors are used to rename the file, otherwise the authors are used. I took your formula and created the following to take the editors into account.

{{ if editors == "" }} {{ if {{ creators }} == {{ creators max="2" }} }} {{ creators max="2" join="+" suffix="#" case="snake" }} {{ else }} {{ creators max="1" suffix="_etal#" case="snake" }} {{ endif }} {{ else }} {{ if {{ editors }} == {{ editors max="2" }} }} {{ editors max="2" join="+" suffix="#" case="snake" }} {{ else }} {{ editors max="1" suffix="_etal#" case="snake" }} {{ endif }} {{ endif }} {{ year suffix="#"}} {{ title truncate="28" case="snake" }}

Edit as needed.

dstillman

Provide option for suppressing suffixes where they would create a repeat character, e.g. two hyphens, two periods etc.

Beta 101 will now suppress duplicate prefixes/suffixes by default.

jeremyvancleve

I see that regular expression functionality has been added, which works great! Thanks!

I think the "abbreviate" function would still be quite useful and easy to implement. I'd happy to take a stab too if you'd look at a pull request.