[mlz] default locale not working with twlaw after 4.0.14m420 update

mlwang

As titled. When trying to cite (in Word) an item of which the language field is left blank, I would get an error message that says:

Zotero experienced an error updating your document.
this.locale[lang] is undefined

No problem if I put "en" (or "zh" for that matter) in the language field. No problem either if set to use the almlaw style.

No output when tested in the CSL Editor pane or the CSL Preview pane.

fbennett

Well, that's not good. There seems to be a hole in the test suite that let this bug slip through on styles with multiple locale layouts. Will have a repair out soon.

fbennett

This should be fixed now (in MLZ 4.0.14m422).

There were a number of problems in the application of locales. I've tested the obvious problem cases (invalid locales and alternative region codes set in the item language field), and I'm pretty confident it will work reliably for you now.

Note that the default locale for "zh" is "zh-CH" in CSL. You can specify multiple locales on a layout. The layout will match any of the listed locales, and the first-listed locale will be applied when rendering terms.

Following that logic, to apply the zh-TW locale to all Chinese items, you should set the in-style locale definition to "zh-TW", and set the layout locales as "zh-TW zh". That should yield the "zh-TW" locale for all items that have "zh" as their base language code. Invalid item language codes and items with no language code should fall back to the style default.

I will need to wade into the logic for selecting sort-key locales again -- it currently falls back gracefully to the base locale of the item automatically, while the layout locales just fail to the style default. On reflection, I think the latter behaviour may be better (because explicit is better than implicit). Not sure though.

In any case, at some point I will have to bring the sort locale selection logic and the layout locale selection logic into line with one another. For now, though, the settings described above should get things working.

(Edited to show my current uncertainty over the correct locale selection behaviour.)

fbennett

(Well, that was quick. While doing some revisions on our house journal style, I ran across a bad failure when feeding a Japanese item to mlz-twlaw.csl. This proved that the explicit behaviour described above is the right one, so I've amended citeproc-js to use that logic, and made a fresh MLZ release. We're now on MLZ 4.0.14m423.)

mlwang

Fix confirmed. Thanks. I've changed the locale codes in twlaw per your instruction, though I must admit I don't fully understand yet.

With the new settings:
<style ... default-locale-sort="zh-TW">
...
<locale xml:lang="zh-TW">
...
<citation ...>
<layout locale="zh-TW zh" delimiter="；" suffix="。">
tw_layout_codes...
an item will be formatted according to "tw_layout_codes" if its language is set to "zh", "zh-TW", "zh-CH" or any other "zh" variant, right? I guess that's to be expected before a new layout section is in place for another zh locale.

Under my old settings:
<style ... default-locale-sort="zh-TW">
...
<locale xml:lang="zh">
...
<citation ...>
<layout locale="zh" delimiter="；" suffix="。">
tw_layout_codes...
the result should be the same, isn't it?

I guess I don't really get yet how the default locale works for a given language. I thought it would makes a difference only when both
<layout locale="zh-TW" ...>and <layout locale="zh-CH" ...>were defined, and neither claimed to be the default for "zh" by including "zh" in the "locale=" setting. An item whose language code is set to only "zh" should be formatted according to the "zh-CH" layout, for it's the default locale for zh.

What am I missing here? Maybe I'll test it out tomorrow. Anyway, thank you for the speedy fix.

fbennett

Yeah, locale management is tricky. A base locale is resolved in most cases to a specific regional locale, so "zh" is equivalent to "zh-CH".

With the old settings, an item set explicitly to "zh-TW" would (or should) render in the default locale (i.e. en-US), because it is not a strict match for the locale specified in the layout ("zh-CH" after resolution).

mlwang

Ah, so! Got it. Thanks.

fbennett

We probably do want to be able to specify "zh" as applying to all "zh" language tags at some point. Some of the European languages have bazillion local variants, only a few of which might affect the locale used for terms. In that case, specifying all of them in a style will be a pain, so smarter matching behaviour will be helpful. It will take awhile to figure out the requirements, though.

fbennett

I went ahead and implemented what I think is intuitive matching behaviour (in MLZ 4.0.14m425). A single-element language code ("zh") will now match all of its sub-locales, and when in first position on the @locale attribute to cs:layout, it will map to the CSL default locale term-set for the target language. Locale matches are also now correctly prioritized in cs:sort, so it will produce the same result as the cs:layout matches.

No changes are needed to mlz-twlaw.csl for the new code, but it will now work better with layouts that require (say) matching a set of language variants that require Latin characters, before falling back to Cyrillic for the main language. Or vice-versa etc.

The changes have been tested, but if you do run into problems give me a shout.