Accented character ů

somnambWl · May 16, 2016

Hello,
I tried to make bib file while using ISO encoding. Other characters are fine, š is {\v s} etc. but on the place of ů is just question mark.
Is this bug or am I doing something wrong?

adamsmith · May 16, 2016

we likely just don't have that covered. There's a virtually limitless range of characters and there's no way we can have a full list of bibtex escaping. If you're working with diacrits a lot, I'd look for a utf-8 capable bibtex implementation (or just work in biblatex).

noksagt · May 16, 2016

We likely don't replace that entity because it is less common & nobody has encountered it yet (many people use a unicode-enabled toolchain these days). The proper entity that likely needs to be added is: \r u

somnambWl · May 17, 2016

Well, I know how to fix it in .bib file, but as you have to admit, it is uncomfortable to fix .bib file every time you make one.

I would like to use biblatex, but I found only one style for bibtex in czech and for biblatex there is only one that is in progress. But I will try to deal with it somehow. Thank you.

Couldn't you make a discussion on forum and specify the form for redefinition of character, f.e. (I don't know how it is done)
\redef{ů}{{\r u}}
And if somebody finds an uncovered character, they would just add it? In a year you would have a list of letters to add and could just copy it.

noksagt · May 17, 2016

New character requests come infrequently enough that we have just added them as they are requested. There's a pull request for this one and the other Czech ringed characters.

Note also that various BibTeX reimplementations should be able to use your existing BST style and a UTF-8 .BIB file.

adamsmith · May 17, 2016

noksagt's pull request is now live. This should work in Zotero within 24hs or after clicking the Update Now button in the general tab of the Zotero preferences.

emilianoeheyns · June 4, 2016

(a little late to the game, and sorry for the plug, but BBT does cover this case and others)

adamsmith · June 4, 2016

If you have a longer list of special characters, emiliano, we'd take it as a PR (or do you handle this differently)?

noksagt · June 4, 2016

It'd take some minimal amount of work, but it's probably worth doing sometime:

https://github.com/retorquere/zotero-better-bibtex/blob/master/resource/translators/latex_unicode_mapping.coffee

emilianoeheyns · June 4, 2016

I have a very long list of special characters (3220 mapped, 6695 reversed), but the way I structure them is particular to how I use them in my code. That file that noksagt links to is generated however, and it would be fairly easy for me to output a list in a format that's useful for Zotero BibTeX's mappingTable and reversemappingTable.

I don't have the symbolic names for them however (such as "GREEK rho"), so those'd be missing. I also distinguish between characters that only have a proper representation in math mode, and those won't do well in the Zotero BibTeX translator.

The thing that stands between me and a clean patch is that I'd have to figure out which constructs are already handled by Zotero BibTeX, so that I only add those that you don't have yet. I've tried parsing BibTeX.js, but I'm currently stuck on multi-line comments. I don't really relish the thought of going through all of my mappings to cross-match with stock... I could easily generate a wholly new pair of mapping/reverseMapping, but it'd undo any choices you might have made there.

what I could do is generate a separate mapping table, and a few lines of code that patches (at runtime) (reverse)mappingTable if it doesn't find an entry. That should yield a clean result, but not really clean code.

The mapping is generated by https://github.com/retorquere/zotero-better-bibtex/blob/master/lib/unicode_table.rb#L33 BTW, and the results are cached in https://github.com/retorquere/zotero-better-bibtex/blob/master/resource/translators/unicode.mapping (which is an sqlite database). It is from that cache that I generate my table. The sources for that, in turn, are, in order (later sources override earlier sources)

* http://milde.users.sourceforge.net/LUCR/Math/data/unimathsymbols.txt
* http://www.w3.org/2003/entities/2007xml/unicode.xml
* http://www.w3.org/Math/characters/unicode.xml

plus some manual fixups in https://github.com/retorquere/zotero-better-bibtex/blob/master/lib/unicode_table.rb#L72

emilianoeheyns · June 4, 2016

Untested, but something like https://gist.github.com/retorquere/29f940e85c8e03b319a3346cdc2149c6 ought to work for the "generate a separate mapping table" approach.

emilianoeheyns · June 4, 2016

Turns out I did have access to the codepoint descriptions (for most anyway), so I've added them.

emilianoeheyns · June 4, 2016

I've issued PR https://github.com/zotero/translators/pull/1073. I've tried my best not to touch chars Zotero already had a decision on, but it's a pretty big change (9.5k lines), so it's a little hard to oversee. There's also *loads* of chars in that map that only happen in the wild very rarely.