Upgrade StorageAvailable for beta testing: Accent-insensitive searching and other search improvements
dstillman
Zotero Team
In the latest Zotero beta, we've made a huge number of changes to make searching more useful.
Most searching throughout Zotero now ignores accents, typographic characters, and formatting. Searching for "cafe" will find "café", "can't" will find "can’t", and "h2o" will find "H<sub>2</sub>O" and "H₂O", and vice versa. This applies to item fields, creators, tags, annotation text and comments, note content, and attachment content via the quick-search bar and advanced search, as well as filtering of collections and tags in the left-hand pane. (It doesn't yet apply to searching within a given note or document.)
Full-text content searches — the "Attachment Content" condition in advanced search and quoted phrases in the quick-search bar — are also now much faster, returning results nearly instantly instead of scanning files on disk.
Searching for Chinese, Japanese, or Korean text in the quick-search bar previously matched attachment content containing any individual character of the search terms unless the search was quoted. Unquoted CJK searches now match the full phrase, and do so quickly.
We've also overhauled how Zotero processes attachment content in the background, indexing new content faster and more reliably. If you're using syncing, be sure that "Sync full-text content" is enabled in the Sync settings, which allows you to search full-text content already indexed elsewhere even if a file isn't available locally (e.g., when using on-demand file downloads) and avoids unnecessary indexing work on multiple devices.
One change worth noting: searches of attachment and note content — in both "Everything" quick search and advanced search — now match text anywhere within a word, rather than only at the beginnings of words (e.g., "colonialism" also matches "postcolonialism"). This is consistent with how field search has always worked, but it means that short search terms can match many more items than before (e.g., "ion" will match any document containing "-tion" words). We're probably going to switch back to matching only at word beginnings for attachment content, which will also reduce the size of the full-text index.
When you update to the latest version, Zotero will need to reindex some data in your database. For most people, this shouldn't take more than a few seconds, or potentially a few minutes in very large libraries. Let us know if you see longer indexing times.
Any problems, let us know in a new thread!
Most searching throughout Zotero now ignores accents, typographic characters, and formatting. Searching for "cafe" will find "café", "can't" will find "can’t", and "h2o" will find "H<sub>2</sub>O" and "H₂O", and vice versa. This applies to item fields, creators, tags, annotation text and comments, note content, and attachment content via the quick-search bar and advanced search, as well as filtering of collections and tags in the left-hand pane. (It doesn't yet apply to searching within a given note or document.)
Full-text content searches — the "Attachment Content" condition in advanced search and quoted phrases in the quick-search bar — are also now much faster, returning results nearly instantly instead of scanning files on disk.
Searching for Chinese, Japanese, or Korean text in the quick-search bar previously matched attachment content containing any individual character of the search terms unless the search was quoted. Unquoted CJK searches now match the full phrase, and do so quickly.
We've also overhauled how Zotero processes attachment content in the background, indexing new content faster and more reliably. If you're using syncing, be sure that "Sync full-text content" is enabled in the Sync settings, which allows you to search full-text content already indexed elsewhere even if a file isn't available locally (e.g., when using on-demand file downloads) and avoids unnecessary indexing work on multiple devices.
One change worth noting: searches of attachment and note content — in both "Everything" quick search and advanced search — now match text anywhere within a word, rather than only at the beginnings of words (e.g., "colonialism" also matches "postcolonialism"). This is consistent with how field search has always worked, but it means that short search terms can match many more items than before (e.g., "ion" will match any document containing "-tion" words). We're probably going to switch back to matching only at word beginnings for attachment content, which will also reduce the size of the full-text index.
When you update to the latest version, Zotero will need to reindex some data in your database. For most people, this shouldn't take more than a few seconds, or potentially a few minutes in very large libraries. Let us know if you see longer indexing times.
Any problems, let us know in a new thread!
Just a question. Does it mean that we will not able to distinguish between lower and capital letters/match case? Couldn't be a setting to give the option to choose this?
I did not say that anything has changed, but given that searching inside a PDF can be case sensitive by setting "Match case".
https://s3.amazonaws.com/zotero.org/images/forums/u6735168/19ta1eq1hj7wvkhk4gxe.png
I asked for implementing this option for general search.
Thanks!