Chinese characters missing from footnotes

xanise.c
Hello Zotero community,

I am using Zotero standalone 4.0.8 on Linux to write a paper that includes a good number of citations in Simplified Chinese.

When I create automatic citations with the OpenOffice add-in, some characters are being skipped. The same character disappears in different citations. However, if I manually export the citation from Zotero, everything prints out correctly.

For example, the following citation:

“国务院批转教育部2003—2007年教育振兴行动计划的通知,” Ministry of Education of the People’s Republic of China, March 3, 2004, http://www.moe.eol.cn/edoas/website18/17/info34017.htm.

becomes when I use the OpenOffice addin:

“国务批教2003—2007年教振兴动划的知,” Ministry of Education of the People’s Republic of China, March 3, 2004, http://www.moe.eol.cn/edoas/website18/17/info34017.htm.

Various characters are missing in the second string. The same characters missing here will be missing from all the other citations, so it looks like the add-in fails to render a set of characters.

Does anybody have any clue on why this is happening and how to correct it?

Thank you very much.
  • adamsmith
    I don't think this is a Zotero issue - these render correctly for me. My best guess would be that it might be the font issue in OpenOffice - so try a different font.
    Also try changing citation styles - unlikely to change anything, but good for troubleshooting.
  • xanise.c
    Thanks for your suggestion.

    I have changed the citation into different fonts (e.g MS SimSun, AR PL UKai CN, WenQuanYi Zen Hei). In none of these missing characters appear. Changing the citation style does not help either.

    I have noticed that in the yellow code that appears when I mouseover a citation in the footnotes, the citation is repeated twice. In the first instance (plainCitation) it is incomplete, but in the second (title) it is complete.

    Does anybody have any insight?
  • fbennett
    Let's take a closer look. Please export one of the items as Bibliontology RDF, save it to a file, open the file with a text editor, paste the content to http://gist.github.com, save it as a "Public Gist", and post the URL back here.
  • xanise.c
    edited May 15, 2013
    Everything looks fine in the Bibliontology RDF, but the corresponding citations are incomplete when I use the LibreOffice addon.

    Sorry I don't have github account. Instead, here is a Dropbox link.

    https://dl.dropboxusercontent.com/u/9413810/Exported%20Items.rdf

    Here also the screenshot showing the incomplete entries and the yellow mouseover code.

    https://dl.dropboxusercontent.com/u/9413810/Screenshot%20-%2005152013%20-%2006%3A41%3A30%20PM.png

    Thanks for your help.

    Versions:

    Zotero 4.0.8
    LibreOffice 4.0.3.3
  • fbennett
    edited May 15, 2013
    The data imports fine for me, and it renders fine, both in the Zotero style edit pane, and in a document. Here is a screenshot of the same "field codes" area that is shown in your screenshot:

    http://imageshack.us/f/694/screenshotfrom201305151.png/

    As you can see, everything is intact. So the cause must be some difference between our settings or environments.

    I tested with MLZ using LibreOffice on Ubuntu 12.04.
  • xanise.c
    edited May 15, 2013
    Thanks again for the reply.

    I have tried to delete and rebuild all Zotero and LibreOffice settings, but the problem persists.

    More interestingly, I have done a fresh install of Zotero 4 and LibreOffice 4 on a Windows 7 partition, and... citations are still incomplete.

    So it looks like this issue does not only affect (Arch) linux users.

    It is really strange. Does anybody have any further insight?
  • fbennett
    It is indeed odd, and something must be causing it. What is your language environment in Zotero?
  • adamsmith
    (works for me, too, btw. also on Ubuntu linux but with regular Zotero).
  • xanise.c
    edited May 16, 2013
    I am not entirely sure where I find information about the language environment.

    In about:config the general.useragent.locale is set to en_US, and intl.locale.matchOS is true. I have tried modifying these values, setting intl.locale.matchOS to false.

    Next, I have set extensions.zotero.export.bibliographyLocale to zh.CN and zh.TW

    The problem is still there.

    EDIT: I have tried installing Firefox with the Zotero addon but it does not make any difference...
  • fbennett
    Just to be sure, when you installed LibreOffice and Zotero on a Windows 7 partition, did you do your testing with Linux running as the OS (but reading from that partition) or under Windows 7 itself?
  • xanise.c
    I tested under Windows i.e. I rebooted and loaded Windows 7.
  • dstillman
    If you import the file you exported and cite that item instead of the original one, does that make a difference? (Or maybe that's what you did on Windows 7...)
  • xanise.c
    edited May 16, 2013
    Hi Dan,

    last time I had synced the bibliography from Zotero's server.

    Today, I exported my bibliography to Bibliontology RDF, uninstalled Zotero and removed all the profiles, reinstalled Zotero, then imported the bibliography from the file.

    Same issue.

    Thank you to all taking the time to understand this problem.
  • fbennett
    edited May 16, 2013
    The Zotero document fields contain the complete citation in two forms: formattedCitation and plainCitation. The latter is (supposed to be) the former, but with formatting markup stripped. In your screenshot, the formattedCitation form contains all of the characters (in RTF encoding). The characters are missing from the plainCitation form only.

    If I understand the Zotero integration code correctly, the plainCitation form is provided by the word processor plugin, while the formattedCitation form is generated Zotero-side.

    That seems to suggest that the problem lies either in the Zotero integration plugins, or in LibreOffice itself. The citation processor does not appear to be producing the truncated output.

    It is interesting that in three-byte UTF-8 encoding, the characters being dropped all begin with 0xe8 or 0xe9, which evaluate to è and é respectively if single-byte characters are assumed. A pattern-match or find-and-replace operation somewhere could be going awry.
  • xanise.c
    edited May 17, 2013
    So are you saying that this is a bug within the integration plugin or LibreOffice? What would you recommend doing in this case?

    Old documents that I produced with Zotero and LibreOffice a while ago do not present this problem so it must be a recent development.
  • fbennett
    That's another useful point of information.

    I know the citation processor code pretty well, so if the problem lay in the processor I could follow up. The next step would be for Simon to take a look at the integration plugin, I think. (We know that this works correctly on other similar systems, but the plugin might [for example] depend on a third-party library that has gotten out of whack on your machine.)
  • xanise.c
    Hi fbennet,

    Sorry for the late reply. I am not an expert, but if the issue is caused by a third-party library in my machine, how to you explain the fact that I have the same problem both in Windows and in Linux?

    Is there anything I could do to have this fixed?
  • isabel-h1
    Hello!

    I'm running Debian 7, Zotero 4.0m374 (multilingual version) and LibreOffice 4.0.3.3 (wheezy-backports).

    I don't know if you've solved your problem yet, but I had the exact samle problem with the above-mentioned setup. I did some troubleshooting, trying different versions of Zotero and LibreOffice and found out that the problem lies with LibreOffice 4.0.3.3. Following this guide (https://wiki.documentfoundation.org/Installing_in_parallel), I realized that only changing the LibreOffice-version to either 3.6 or 4.1 solved the problem (I used the same LibreOffice config folders).

    Best of luck,

    Isabel
  • fbennett
    @isabel-h1: Thanks for reporting this!
  • xanise.c
    Hi Isabel,

    Thanks for letting me know.

    Since I updated the LibreOffice recently, the problem seems to have been resolved.

    My LibreOffice version is 4.0.4.2.
  • isabel-h1
    Thanks for your suggestion, I also just tried it with the version you mentioned and it works for me, too. Looks like there is a solution at last!
Sign In or Register to comment.