Importing of BibTex with TeX-encoded characters

Robert Samal

Hi, I encountered this issue in two different settings:

1) when importing a bibtex file
2) when adding a reference from http://www.ams.org/mathscinet/

What's wrong is that bibtex uses TeX sequences to encode non-latin characters
(\'a is "aacute", \v{s} is "scaron", etc.) and only some of them are understood by Zotero.

I was able to locate this issue to the file scrapers.sql where a bibtex translator
is defined (and indeed, some characters are missing). I tried to mess with this
file and add more characters, but I couldn't get Zotero notice any changes.

As I cannot really use Zotero without this corrected, I'm willing to help in getting this
straight, but I need some guidance. I can provide the pairs of "unicode number" - regular expression (in what format? and is it really regular expression?).
I have no clue, however, how to feed Zotero with this new knowledge.

Any pointers will be appreciated.

(Btw. the issues 1 and 2 above are related: the scaper for mathscinet
uses bibtex translator ...)

pviton

Same thing's happening to me - zotero isn't recognizing the changes. My guess is that zotero uses a binary form of the sql files (= zotero.sqlite), so presumably you need to process the edited text file somehow, but I don't know how to do this or what tools are needed. Can anyone help?

pviton

OK, I think I've figured it out. You need to load the "scaffold" app (xpi file) available from elsewhere on this site. Once that's available in FF, you can start the app, load the bibtex exporter, change it, then save. I've successfully made the tiny changes that get rid of the "_"'s in the generated keys, changing the underscore to a hyphen.

pviton

Scaffold link:

http://dev.zotero.org/scaffold

Robert Samal

Thanks a lot, pviton!

Scaffold is a great tool. (By the way, anything would look great in comparison to editing sql-file with a text editor and then trying to force Zotero to "take it". But this works
really as it should.)

I have almost completed the list of more TeX encodings plus their Unicode interpretations. (It has to wait for completion till I return from a holiday, though - about
2 weeks, in case someone else is eagerly looking forward to it :) )

Do you possibly know, what's the right next step - I mean where to post it, so that it
can be included in the official version?

Matthias

Robert, I did also attempt to compile a comprehensive mapping between Unicode and LaTeX entities (see here) which may be helpful for comparison. The list of Unicode characters was adopted from transtab by Markus Kuhn.