The title sorting problem after the recent update(special characters no longer identified)

jeaccuse

Please allow me to point out an issue that others might be experiencing as well following the new update: the change in the title sorting system.

Before the recent update, special characters such as asterisk, @, parentheses came before the alphabetical characters in the title sorting order. For example, if I wanted a certain item to be at the top of the collection, I was easily able to do it by adding an asterisk at the front of the title.

ex) (Title order before the update)

* Note on the subject
※ Items to look for
A Brief Analysis
Case Studies
The Causes

However, after the update, it appears that the new title sorting system no longer identifies the special characters and refers to the first alphabetical character of the item's title.

ex) (Title order after the update)

A Brief Analysis
Case Studies
※ Items to look for
* Note on the subject
The Causes

I believe there would be certain benefits from this change, but it also mixes up the item orders we have set beforehand for reasons. Especially for group works, the members need certain items to be exposed at the top of the order, and adding a special character at the front has been the easiest way to do so. After the update we are facing a need to rename numerous items for the aforementioned purpose.

I wonder if we could restore the previous system, or at least have an option to use the previous system. If there is a way, you would be most kind to inform us with it.

Thank you very much.

dstillman

No, sorry — this was intentional. Some punctuation was always stripped while sorting items, and Zotero 6 just expanded that to cover additional punctuation that had been left out.

Items contain metadata, which is meant for use in all kinds of places, so hacking the sorting with punctuation just isn't really appropriate. There's also no guarantee that someone will be sorting the library on a given column. (You can still do this in collections where neither of those is an issue.)

I'd recommend using colored tags to make items stand out. If there are particular use cases where being able to pin items to the top regardless of the sort order would make sense, let us know and we can think about proper solutions, but hacks on the sorting system aren't the way to do that.

jeaccuse

"If there are particular use cases where being able to pin items to the top regardless of the sort order would make sense, let us know and we can think about proper solutions"

→ A function to pin selected items to the top would be a fantastic addition to Zotero. Thanks for the great suggestion, and I would deeply appreciate it if the aforementioned function could be included in the updates to come.

dstillman

My question was what you would use it for.

jeaccuse

https://imgur.com/cvRzMf4
https://imgur.com/nvPOwhL
https://i.imgur.com/Hi2Vn4N.png

"My question was what you would use it for."

→ In addition to the metadata items added via Zotero, my colleagues and I have manually added an item in each collection that compiles our personal notes and reviews on the collection as a whole. (Please refer to the links above.) Due to the purpose, we have preferred that item to be pinned at the top of the sorting order, and we have been able to do that by adding a special character or punctuations at the beginning of the title.

We are well aware that adding a colored tag would achieve a similar result by making the items stand out, but we have been looking for ways to pin the item at the top of the collection to maximize its visibility and minimize the efforts to locate it.

(For that specific purpose, probably the easiest way to pin our personal item at the top would be to edit the title to begin with a punctuation that has not been covered yet within the sorting order.)

qqbb

In my opinion, Zotero should strip punctuation marks more moderately for sorting purposes. Perhaps only punctuation marks at the beginning of the Title field should be stripped? I would also suggest that stripping should only affect a limited set of punctuation marks that are used in normal writing. There's no clear distinction between punctuation marks and symbols in unicode. A symbol like "@" isn't used in normal writing without a meaning. I don't see why it shouldn't be considered when sorting items.

How about introducing new preferences for sorting:
- Ignore initial punctuation marks: « " (default: on)
- Ignore initial articles: A, The, ... (default: off)

This could be used for the items list only and wouldn't need to be applied to the collections tree. The ability to switch off these preferences could be useful, I think. You might, e.g., want to see all titles that start with quotes. Perhaps you want to fix them or remove the quotes.

Here are some issues with the new sorting that strips off all punctuation:

Expected sorting by Title:
A*B*@!@C*D 1
A*B*@!@C*D 3
ABCD 2
ABCD 4

Current sorting by Title:
A*B*@!@C*D 1
ABCD 2
A*B*@!@C*D 3
ABCD 4

Expected sorting by Title:
Why! Volume 1
Why! Volume 2
Why? Volume 1
Why? Volume 2

Current sorting by Title:
Why? Volume 1
Why! Volume 1
Why! Volume 2
Why? Volume 2

Expected sorting by Title:
"@" Sign. Its accidental history.
Sign and symbol
Sign language

Current sorting by Title:
Sign and symbol
"@" Sign. Its accidental history.
Sign language


@jeaccuse: I wouldn't recommend using hacks that affect metadata which might be used in citations. For personal use or in a closed community, using dummy items for organizational purposes shouldn't be a problem, though. You could perhaps consider using the item type Bill as a dummy, which will be sorted to the top when sorting by item type, see here. I also noticed that symbols like ⊕ (U+2295) are sorted to the top at the moment.

bolejj

I used the 'hack' quite extensively. Each collection/folder has a note which contained nothing but what you might call 'bread crumbs' to the collection. I have several hundred collections. So having a set of notes which contain nothing but the location of the collection (bread crumbs to) is handy for quickly locating that collection in a very complex tree of zot folders/collections.

For example...

All of my 'bread crumb' notes sorted to the top of the main window for the library like so in this example of 3

#Phil #Hegel #Being
#Phil #Hegel #Phen
#Phil #Meta #Maritain

In the list of over 200 bread crumb note files when sorted to the top I can quickly scroll down the list to find the philosophy section, then pick Hegel or Maritain, then by using the CTL key on that chosen note, I can find the location in the hierarchical/nested collection tree. This is a simplified example. My collection hierarchy is fairly complex.

Pinning them all to the top would work for me IF they then are sorted by ignoring all '#' characters.

Secondly,

My tagging identifiers were 'corrupted' by a known issue of some article imports or DOI imports forcing their tags into my custom tagging list. Fine, that couldn't be addressed programmatically by the zot Dev team, so I stopped using the zot tag process and created my own process which too uses the '#' character.

The '#" character for tagging is used in the following way. In an item's 'INFO' tab (right panel) there is a column for 'Language'. I don't import anything but English so that is a useless field for me. So I use that field for tagging, and in that field I put tags prefixed with '#' and then can search for all items with a prefixed tag such as #PHIL or #whatever.

All that said, it should be the users option to use a 'hack' or not.

Then again, the second issue can be addressed if the native zot tagging actually worked with the option to NEVER bring in tags from an import source. Again, a user option which seems like it is an option now, but no, some sources force their tags into the user's tags, making tags useless.

Using the # character is useful. #whatever is different than 'whatever' when tagging in the instance that use of 'whatever' alone would cause 'false hits' when doing a search.

The use of the Language field for tagging is again a 'hack'.

When you see your users 'hacking' around it is likely pointing out issues with usability.

Right?

dstillman

@bolejj:

My tagging identifiers were 'corrupted' by a known issue of some article imports or DOI imports forcing their tags into my custom tagging list.

I'm not sure what you're saying is a known issue. You can turn off importing of automatic tags in the preferences, and you can delete all existing automatic tags from the tag selector menu. If you're getting tags from a non-file import with that setting off, that would be a bug, and you should start a new thread with steps to reproduce. (For a file import Zotero has no way of knowing whether tags were manually created.)

A new feature that makes it possible to pin notes to the top of a collection regardless of the sort column/direction is something we can consider.

I also agree with @qqbb that internal punctuation shouldn't be stripped for sorting, so we'll look into fixing that.

But obviously the tagging system is meant for tagging, and the metadata fields are meant for bibliographic metadata, and if you're using those in other ways you shouldn't be surprised if you run into trouble.