Chinese and Japanese

lynnrich · May 26, 2007

Perhaps I am not using Zotero correctly, but I cannot see any way to include strings of Chinese and Japanese characters (titles, Authors, etc.) from on-lin records into Zotero libraries. Library record pages, which do include such characters are imported into Zotero records with all such non roman script characters omitted. Does Zotero only work in ASCII characters? As it is, such strings of non-roman script characters have to be added by cutting and pasting from library records as they are displayed in web pages, which is terribly tedious and time-consuming. Another way of asking the question: Does Zotero work in UTF-8 Unicode? What I want, of course, is all text from on-line library records brought over into Zotero created records with all characters intact, CHinese, Japanese and English/Roman.

dstillman · May 26, 2007

Zotero should support Unicode strings natively, but there might be incompatibilities with certain library catalogs. Can you provide an example of a record that doesn't import correctly?

lynnrich · May 26, 2007

here is the record I tried to enter into Zotero; the roman letter scripts came across; nothing of the Chinese showed up. The CHinese characters show up below because this is a past in from the library webpage

If this is a shortcoming of the HKU library site, which East Asian Libraries works properly with Zotero; anyone ever made up a list?

http://library.hku.hk/search/i?SEARCH=7805126577

HKUL > Dragon

Start Over Export MARC Display Limit/Sort Another Search

Limit results to available items
Record: Prev Next
Search HKALL Central Catalog
Other Resources
Z39.50 Search
TITLE Qing chu si Wang hua pai yan jiu lun wen ji.
清初四王畫派硏究論文集 / 朵云編輯部編.
IMPRINT Shanghai : Shanghai shu hua chu ban she, 1993.
上海 : 上海書畫出版社, 1993.
EDITION Di 1 ban.
第1版.
Permanent URL for this record=> http://library.hku.hk:80/record=b1548334

LOCATION CALL # STATUS STACK #
FPS Library [中] ND1043.5 .C5746 1993 AVAILABLE
DESCRIPT. 942 p. ; 21 cm.
BIBLIOG. 參考文獻: p.929-938
LC SUBJECT Painting, Chinese -- Ming-Qing dynasties, 1368-1912.
Show similar items
ALT AUTHOR Duo yun bian ji bu.
朵雲編輯部.
ISBN 7805126577
Record: Prev Next
Start Over Export MARC Display Limit/Sort Another Search

dstillman · May 27, 2007

The problem here is that the Chinese characters are in the 880 (Alternate Graphic Representation) MARC fields, which Zotero doesn't currently support (as far as I know—I didn't write Zotero's MARC support). Have a look at the raw MARC record to see what I mean.

While Zotero could automatically grab data from these fields, it's unclear exactly how it would go about deciding which version of the field to grab. A pref? Sticking the alternative values in a note? Other suggestions?

lynnrich · May 27, 2007

Chinese or Japanese characters follow the transcriptions of authors, titles, publishers, etc., so the strings involved are in those same fields. Zotero should keep the strings of non-roman characters exactly where they appear in the original on-line records, for they already all appear in "the correct fields" there should be no need to migrate them somewhere else. If it is simply a matter of Zotero not recognizing that they are there, is there a "simple" way to fix this so it does recognize their presence?

lynnrich · May 27, 2007

I just tried another on-line library: HOLLIS at Harvard. Zotero did include Chinese characters for authors, but only for authors and, in one case, a contributor. The rest of the fields only had romanized transcriptions; i.e., titles, publishers, series titles, etc. I shall try a few more libraries, including some in mainland China and Taiwan.

dstillman · May 27, 2007

Zotero uses MARC records to save data from online catalogs. If you'll look at the MARC record I linked to above, you'll see that the non-Roman strings are, in fact, in separate fields. The catalog has opted to concatenate them with the transliterations on the main record page, but that's a decision on their part separate from how the data is actually stored. Zotero could (and perhaps should) simply add the non-Roman strings onto the main field values, but that may or may not produce good results at all libraries.

When looking at other libraries, it's important to look at the MARC records (which are almost always available via a link on the main record page) and see how the data is actually stored, as how the data is formatted on the main display page is mostly irrelevant.

lynnrich · May 27, 2007

I got it to work perfectly (from my point of view) with WorldCat: all fields filled in with both romanized transcriptions and strings of Chinese characters; so this is probably the site I shall use most to compile records for my own personal library (10K + volumes).

Other observations: RLG Eureka connected via VPN/Proxy at the University of Toronto does not work: the book icon at the right of the url string does not appear; Zotero works with Red Light Green Light but omits all non roman characters.

National Union Library on-line in Taiwan does not work at all; Zotero does not seem to recognize webpages entirely in Chinese characters, so I assume it won't work for Mainland Chinese libraries either. I have not tried Japanese libraries yet.

lynnrich · May 27, 2007

Does not work with NACSIS Webcat, the National Union Catalog of Japan, since, I suppose record pages are almost entirely in Chinese and Japanese. There is some romanization for authors and titles among strings of Chinese and Japanese characters, but apparently this is not enough.

Tried a few other Japanese libraries; Zotero does not work with any of them--the same problem--can't recognize the record pages as library records. Too bad!

noksagt · May 27, 2007

lynnrich: read Dan's comments.

Zotero looks for metadata that it can harvest (whether this is embedded in the page, linked to from the page, or can be found in a predictable location (based on what webapp is being used as the catalog)).

It has nothing to do with the language of the page you're looking at.

Enabling Zotero to scrape those other pages may involve adding another site translator, enabling a pre-existing translator for particular URLs, and/or to ask those who run the sites to configure their servers to emit the bibliographic metadata in a standard way.

lynnrich · May 27, 2007

I think I have figured out why only WorldCat works properly with Zotero: The strings of Chinese and/or Japanese characters are placed before their romanized equivalents in all the appropriate fields; all other sites place the strings of non-roman characters after the romanized equivalents. If Zotero could recognize the content of fields in reverse order, perhaps all the other sites would work as well?

scot · May 28, 2007

The order may well be different in WorldCat from the non-working sites you mentioned, but that's not the cause, of their different ability to import, nor is the language of the page. See the above comments by Dan and noksagt.

What might help is to compare the use of MARC records by different libraries whose records are bilingual and see if they all use separate fields in the same way that the HKU library does (again this is in the MARC records we're interested in, *not* what you see on the catalog's record page, which is in most cases irrelevant to Zotero's import process). If there is a standard way of keeping that data in MARC records, there is a good chance at developing a standardized way for Zotero to treat it.

If HKU has done their own thing, then what would be needed is for someone to write a special MARC translator just for them.

Any other non working libraries must be treated on a case-by-case basis. Libraries which have developed their own catalog software will almost certainly need a custom import translator developed (note that this is the case will many libraries whose records are all in roman script as well). Alternately, if they can be persuaded to put their metadata into their record pages in standard machine-readable ways (like COinS tags), they make that job easy for Zotero, and a host of other newer bibliographic tools as well.