biblatex: both `langid` and `language` are needed?

I use BetterBiBTeX and find that `langid` doesn't "work". In the reference list generated by `biblatex`, we need a comment like "[in Japanese]", but this functionality is activated only when `language = {japanese}` is included in the bib entry. `langid` doesn't do this. I'm pasting a self-contained example below.

In several past threads in this forum, it seems to have been concluded that `langid` is the bibtex field for the Zotero field `language` to be mapped to. But, given the fact that the actual `biblatex` doesn't do the correct job with `langid`, shouldn't both `langid` and `language` be generated for the bibtex file?

\documentclass{article}
\begin{filecontents}[overwrite]{tmp.bib}
@article{UseLanguage,
title = {Title of UseLanguage},
author = {A. U. Thor Language},
date = {1986},
journaltitle = {Journal of Engineering},
volume = {123},
number = {28},
pages = {1--12},
language = {japanese}
}
@article{UseLangid,
title = {Title of UseLangid},
author = {A. U. Thor Langid},
date = {1986},
journaltitle = {Journal of Engineering},
volume = {123},
number = {28},
pages = {1--12},
langid = {japanese}
}
@article{UseLangidAbbrev,
title = {Title of UseLangidAbbrev},
author = {A. U. Thor LangidAbbrev},
date = {1986},
journaltitle = {Journal of Engineering},
volume = {123},
number = {28},
pages = {1--12},
langid = {ja}
}
\end{filecontents}
\usepackage[sorting=none]{biblatex-chicago}
%\usepackage[sorting=none]{biblatex}
\addbibresource{tmp.bib}

\begin{document}
\nocite{*}
\printbibliography
\end{document}

(By the way, what's the proper method to display a block of code in this forum? By trial and error, I've found <blockcode> kind of works, but it doesn't treat the block as a code. For example, it highlights only @article. The above code is pasted with the <code> tag, which is better but with the same highlighting problem.)
  • The two fields do different things. language gives the bibliographic information in which language the work is written. langid, on the other hand, tells babel or polyglossia which language to switch to for typesetting the entry.
  • > The two fields do different things.

    I know that. My point is: Generate **both** `langid = {japanese}` and `language = {japanese}` in the bibtex file from the **single** `language = japanese` field of Zotero.
  • Oh sorry, I misunderstood that.
  • Please report BBT issues on https://github.com/retorquere/zotero-better-bibtex/issues

    According to the biblatex manual, there is no field language, just langid/langidopts. language a bibtex field (or more specifically, a babelbib field). If you have a biblatex style that needs language, I can take a look at that.
  • edited January 19, 2024
    Oh wouldn't you know it, biblatex-chicago considers itself the standard biblatex, and that does have a language field edit: instead of the langid field apparently. Swell.
  • For the short term, you can put this in your preamble: \DeclareSourcemap{
    \maps[datatype=bibtex]{
    \map{
    \step[fieldsource=langid,fieldtarget=language]
    }
    }
    }
  • I’m sorry to disagree, @emilianoheyns, language is a biblatex field, listed on p. 23 of the current version of the biblatex manual and listed as optional field in most entry types.
  • Looks like you're right, apologies. Please open an issue on github.
  • Thanks!!!! I confirm that "export-language-as"="both" solves my problem: the generated reference list now has "[in Japanese]".

    Now, would there be any harm if "export-language-as" were set to "both" by default?

    Actually, that was the reason why I posted here instead of posting to the BBT github: in order for other users to learn from the discussion.

  • There's a discussion facility at https://github.com/retorquere/zotero-better-bibtex/discussions. It makes my life heaps easier if talk about BBT happens on github. I come here occassionally but I have no structured way to be notified that BBT-relevant talk happens on these forums, and the forum itself lacks features I need for my support.
  • Given they're both in the biblatex manual I've just made them fixed for biblatex and only configurable for bibtex. Will be in the next release.
  • edited January 22, 2024
    I disagree.

    Zotero's "language" field should be exported to the biblatex field "langid" only. Both fields are supposed to contain only language tags that control formatting, e.g. capitalization of titles and hyphenation.

    Biblatex's "language" field, by contrast, which has no equivalent in Zotero, is used to generate textual output in a formatted bibliography.

    Exporting Zotero's "language" field to the biblatex field "language" would result in what are now merely language-dependent formatting instructions all of a sudden turned into textual output as well, breaking virtually every style on the biblatex side.

    The OP might have noticed that Zotero itself never generates textual output from its own "language" field, so I do not think the expectation that this field can be used sensibly to populate biblatex's "language" field (which, again, serves a very different purpose) is justified at all.

    Information to be stored in a Zotero database and then exported to the "language" fields of biblatex must therefore be stored elsewhere, just not in the "language" field of Zotero.
  • edited January 28, 2024
    Which can be done using a tex.language entry in extra -- or by setting BBT's language-field handling to the non-default "both".
  • @nickbart
    > Exporting Zotero's "language" field to the biblatex field "language" would result in what are now merely language-dependent formatting instructions all of a sudden turned into textual output as well, breaking virtually every style on the biblatex side.

    What happens in practice? Can you elaborate on what you mean by "breaking"?

    Are you sure that the original providers of the metadata assume the "language" field should affect only the formatting of the reference entry?

    If the original writer of the metadata means merely to indicate the language the article is written in and if Zotero restricts its meaning to just formatting instructions, it's Zotero which has to be fixed.
  • If the original writer of the metadata means merely to indicate the language the article is written in and if Zotero restricts its meaning to just formatting instructions, it's Zotero which has to be fixed.
    I mean -- BibLaTeX has two language fields. Zotero has one. To the extent that two fields with different functionality are needed, yes, Zotero needs to get fixed. This is a bit of a tricky issue because Zotero's general approach to metadata is a bit different from Bib(La)TeX -- in BibLaTeX the general approach (though not without exceptions) is to only include things to be cited in the metadata. Zotero's approach is to generally prefer completeness.
    That's a challenge for something like "Language" which, typically, you only want displayed in some very select cases (you certainly wouldn't want every English publication to say "in English", say).
    This is also the reason this would 'break' biblatex styles -- they'd suddenly be displaying the language of pieces when, in most cases, that's not actually desirable.

    This rarely comes up in the Zotero context, so it's not, I think, a high priority to address, to be honest.
  • @adamsmith (This is a question not a criticism.) What about the case when most of the citations are to sources in a langu other than English? My users are mostly from small government agencies and NGOs that cannot afford the costs of subscriptions to the major commercial databases.

    By far, most of the users of my own online database service are from nations where English isn't the primary language.The users are writing their reports in languages other than English. My users requested that I always include an "en" tag when the citation is to an English language work.

  • I guess I don't understand the question? As I say above, Zotero is designed to always prefer complete data, so including an en tag is strictly better than not. It's just that this means mapping Zotero's language field to the biblatex field of the same name is problematic -- I'm going to assume this latter part is irrelevant for your users.
  • edited January 27, 2024

    Even if BBT continues to export Zotero’s language fields to biblatex’s langid fields, and never to language, which I maintain is the only sensible way, the biblatex langid tags can still be used to selectively generate language fields that produce the output desired by the OP, assuming the aim is to have only items tagged as Japanese, i.e., containing biblatex langid fields whose contents begin with “ja” (irrespective of case) receive the string “[in Japanese]” in the formatted output.

    In the following, biblatex's \DeclareSourcemap mechanism is used to generate such language fields at runtime. Details can be found in the biblatex manual.

    \documentclass{article}
    \begin{filecontents}[overwrite]{tmp.bib}
    @article{UseLanguage,
    title = {Title of UseLanguage},
    author = {A. U. Thor Language},
    date = {1986},
    journaltitle = {Journal of Engineering},
    volume = {123},
    number = {28},
    pages = {1--12},
    language = {japanese}
    }
    @article{UseLangid,
    title = {Title of UseLangid},
    author = {A. U. Thor Langid},
    date = {1986},
    journaltitle = {Journal of Engineering},
    volume = {123},
    number = {28},
    pages = {1--12},
    langid = {Japanese}
    }
    @article{UseLangidAbbrev,
    title = {Title of UseLangidAbbrev},
    author = {A. U. Thor LangidAbbrev},
    date = {1986},
    journaltitle = {Journal of Engineering},
    volume = {123},
    number = {28},
    pages = {1--12},
    langid = {ja-JP}
    }
    @article{en,
    title = {Title of Something Completetly Different in English},
    author = {Doe, John},
    date = {2024},
    journaltitle = {Journal of Whatever},
    volume = {321},
    number = {1},
    pages = {33--77},
    langid = {en}
    }
    \end{filecontents}
    \usepackage[sorting=none]{biblatex-chicago}
    %\usepackage[sorting=none]{biblatex}
    \addbibresource{tmp.bib}

    \DeclareSourcemap{
    \maps[datatype=bibtex]{\map{
    \step[fieldsource=langid, matchi=\regexp{^ja},
    final]
    \step[fieldset=language, fieldvalue=Japanese]
    }
    }
    }

    \begin{document}
    \nocite{*}
    \printbibliography
    \end{document}


    (To avoid misunderstandings, datatype=bibtex in the above code does not mean bibtex as opposed to biblatex, but bibtex including biblatex, as opposed to e.g. biblatexml).
  • @nickbart
    > Even if BBT continues to export Zotero’s language fields to biblatex’s langid fields, and never to language, which I maintain is the only sensible way,

    Please define "sensible". My question has been: What *practical harm* would result if BBT exports Zotero's `language` both to biblatex's `langid` and `language`? If there is no practical harm, then do it, until Zotero is "fixed". Zotero is just a tool. What matters is the result in practice.

    First of all, this is the workflow if you don't use Zotero:

    1. Download the bib database from the publisher's website.
    2. Use it with biblatex.
    3. The reference list includes the wording "[in Japanese]".

    If you use Zotero and BBT, then the reference list doesn't include "[in Japanese]". Obviously, it's Zotero+BBT which needs to be fixed.

    **Zotero is just a tool** to help the workflow. It's **not** a standard. That means, it's Zotero who is breaking the workflow.

    What Zotero *should* do is to have two language-related fields: one to indicate the language the article is written in and the other to indicate how the title should be formatted.

    Until that happens, BBT *should* export `language` both to biblatex's `language` and `langid`, not to break the workflow. This is a temporary workaround.

    ---
    1. There are too many metadata standards and publishers' attitudes toward standards are generally lax.

    2. The Zotero developers don't have the power to convince the publishers to use Zotero's strict standard.

    3. Therefore, Zotero or BBT has to adapt to this word's chaotic reality.
  • I thought I was very clear on the practical harm and nickbart went to some effort to give you a solution to your particular issue. I don't understand why you think that deserves this rather unfriendly lecture rather than a heartfelt thank you (for his effort, not mine, TBC)?
  • My question has been: What *practical harm* would result if BBT exports Zotero's `language` both to biblatex's `langid` and `language`? If there is no practical harm, then do it, until Zotero is "fixed". Zotero is just a tool. What matters is the result in practice.
    It seems to me @nickbart has described this at length above. It would cause output to appear in the rendered bibliography of which the user has not expressed intent it should be there.

    BBT, for better or for worse, already allows what you want, it's just not the default behavior.
    If you use Zotero and BBT, then the reference list doesn't include "[in Japanese]".
    what you are arguing is that import should make better choices. That is something we can talk about.
    Obviously, it's Zotero+BBT which needs to be fixed ... Until that happens, BBT *should* export `language` both to biblatex's `language` and `langid`, not to break the workflow.
    You talk about "obviously" and "the workflow" as if your view on this is a universal standard. Given that BBT has existed for 11 years, and this is just now being debated, it doesn't look like your opinion is widely shared.
  • If you use Zotero and BBT, then the reference list doesn't include "[in Japanese]".
    BTW it is not the primary target of BBT to have bib(la)tex round-trip perfectly through import-export. My target is to have import capture best it can the meaning of the bib(la)tex input to Zotero's datamodel, and to have export present best it can Zotero's datamodel to bib(la)tex. Given that these models differ in a great many places, during import and during export BBT makes lossy choices by necessity. This makes round-tripping a desideratum but not a requirement.

    Note that the behavior you want would not round-trip either.
Sign In or Register to comment.