Citation style disambiguation problem after Zotero upgrade(s)

donsushi

Hi everyone, I started experiencing a problem with my citation style(s) after upgrading Zotero.

I normally use a slightly modified version of the CMS citation style for my thesis where the bibliography is supposed to look something like this:
Mundell, Robert. 1973. Plan for a European Currency.
Mundell, Robert A. 1961. “A Theory of Optimum Currency Areas” ...

Both papers were written by the same author, even though his personal name is written slightly differently. After upgrading Zotero, however, I am getting this:
Mundell, R. 1973. Plan for a European Currency.
Mundell, RA. 1961. “A Theory of Optimum Currency Areas” ...

Only entries that require disambiguation are affected, other entries haven't changed. The same issue appears with the default CMS, APA and Vancouver style presets. I am not sure when exactly this began, but it seems to have started after upgrading Zotero's Firefox plugin to version 2.1.4. I am currently running Zotero 2.1.5 on an Ubuntu 10.10 (Maverick) Linux machine with OpenOffice 3.2.1 and the Zotero OpenOffice Integration plugin 3.5.a1.

Could someone please give advice on what might have caused the problem and if there is a workaround available? If possible, is there a place where I can download older Zotero versions and downgrade to a version <2.1.4 until this is fixed?

adamsmith

I'm pretty sure this is fixed on the trunk for the 2.1.6 release.
All old Zotero versions are for download using
http://www.zotero.org/download/zotero-z.z.z.xpi
where the zs are the version number
2.1.3, e.g. is
http://www.zotero.org/download/zotero-2.1.3.xpi
a downgrade to any 2.1.x version is no problem. 2.0.x versions are trickier.

fbennett

Definitely fixed for 2.1.6, as adamsmith says.

donsushi

Hi adamsmith, hi fbennett. Thank you for your quick response and the information.

Do you know when 2.1.6 will be released? I'll give the new version a try and see if that fixes the problem.

Cheers

adamsmith

soon - if you're in a hurry you can install the branch.xpi and then revert back to 2.1.6 once it comes out
http://www.zotero.org/download/dev/zotero-2.1-branch.xpi
should be safe, but back-up nonetheless.

donsushi

I updated to version 2.1.6 today and found that author names are displayed in full in the bibliography again now when using the CMS style (author-date format). Thanks for fixing this!

However, there seems to be an issue with disambiguation of citations where names aren't displayed in full. For example, where I would expect to see
(Robert Mundell, 1973)
I only get (R Mundell, 1973) with the CMS style (author-date format) or (R. Mundell, 1973) with the APA style (6th ed.). And where I would expect to see
(Robert A. Mundell, 1961)
I only get (RA Mundell, 1961) or (R. A. Mundell, 1961) respectively.

I have checked this with the most recent CSL style definitions (https://github.com/citation-style-language/styles) and the Zotero Test Pane (chrome://zotero/content/tools/csledit.xul). Switching through the various citation styles I noticed that all default styles (except for the CMS note-bibliography style, which doesn't use disambiguation) display only an abbreviation of the given name.

I am pretty sure that this hadn't been the case before (see this previous post: http://forums.zotero.org/discussion/16321/changing-the-citation-format-for-author-disambiguation/). Is this behaviour related to the update to CSL 1.0 and intended? If so, could you please give me a hint on how to change the CSL style definition file to achieve the desired disambiguation format?

fbennett

This is a bug that I thought we had fixed; the processor is not respecting the request for full-form in the bibliography when disambiguation has occurred in the citation. I'm currently in the midst of a complete rewrite of the names formatting code (not because it's broken for current functionality, but because the code is messy, and needs to be cleaned up to assure it can be maintained into the future). A fix for this disambiguation issue will have to wait until after this refactoring work is done (probably another couple of weeks, at the current rate of progress), but I'll keep this thread bookmarked, and test for this problem before the next release.

donsushi

Thank you for clearing this up. I downgraded Zotero to version 2.1.1 (http://www.zotero.org/download/zotero-2.1.1.xpi) to see if this makes a difference, but the problem still persists. I believe this first appeared during the upgrade to Zotero 2.1 & CSL 1.0.

Anyway, thank you for working on this problem. Please let me know if I can be of any help. I'd be glad to help wherever I can.

adamsmith

FYI, in APA that's intended behavior and coded into the style, but I can confirm the bug for CMoS (which also means it can't be changed for the APA style) with a recent trunk release.

jani_tampi

adamsmith are you sure that is the correct URL to access older versions of Zotero? I want to downgrade to 2.1.3 but can't find where to do it and your link doesn't seem to work.

thanks!

adamsmith

yes, I'm sure. My link works, just tested, if it doesn't, that's due to something on your side - maybe try downloading first (right-click -> save link as...) and then install by dragging the downloaded .xpi to Firefox.

dstillman

We pulled the 2.1.3 XPI because of a packaging problem. 2.1.4 is equivalent.

wallhalla

Hi everyone,

after upgrading to all the newest versions with word 2007, i became an error message: this.style is undefined. Same message in a new document, different styles worked. I reinstalled everything. Does not help.

In the mean time i did reset translators and styles, and now i cannot even find the style i was using. Do you have any idea what I might try or how to fix the problem?

adamsmith

well, what was the style you used? If you used a custom style that didn't validate properly, it would have broken the citations in Zotero 2.1 (even if it worked in 2.0) and you would have erased it by resetting translators and styles.
If it was a style from the repository you should be able to reinstall it from there.

fbennett

Scratch my earlier comment: thankfully this is not a regression in the earlier bug. But I want to be sure that I understand the issue.

However, there seems to be an issue with disambiguation of citations where names aren't displayed in full. For example, where I would expect to see
(Robert Mundell, 1973)
I only get (R Mundell, 1973) with the CMS style (author-date format) or (R. Mundell, 1973) with the APA style (6th ed.). And where I would expect to see
(Robert A. Mundell, 1961)
I only get (RA Mundell, 1961) or (R. A. Mundell, 1961) respectively.

In CMoS author-date, only the last name is given by default. In APA, the initial is included in the base form. Disambiguation will proceed from there, so if there is a clash in author names at the base level, the format will be moved up one step at a time, starting from the base level (from family-only to family+initials, then from family+initials to full-name), until the names resolve, or the names are found to be identical. If they are identical, the formatting returns to the base level. If they differ, the formatting level at which they were resolved is applied.

That's the way I remember the intended logic, anyway.

So cites requiring disambiguation in CMoS should appear either as family-name+initials, or a full-name, whichever is necessary. Cites requiring disambiguation in APA should appear as full-name.

Is that not what is happening? Or if it is, is some other behavior desired?

adamsmith

I'm not sure I understand you. But to clarify, both CMS and APA should be:
(Smith, 1776) - neither has initials in the base form.
With disambiguation, APA should do (and does!)
(A. Smith 1776; V. Smith 2003)
as should, as I've just found out, CMS (15.21):

where two or more works by different authors with the same last name are listed in a reference list, the text citation must include an initial (or two initials or a given name if necessary).

Text citations:

(C. Doershuk 2010)
(J. Doershuk 2009)

So for CMS we're just missing a period for initialize with in the style and we're good, I'll fix this asap.

This is not the behavior I would have expected, though - I'd have thought disambiguation just defaults to the long form of the author when called - i.e. initials when there is an 'initialize-with' term, full first name when there isn't. I'm pretty sure that's what it used to do in Zotero 2.0.x - I guess the question is if there are any styles that require full first name disambiguation where an initial would do - at least APA and CMS don't - hope that makes sense.

fbennett

Sorry, I misread the APA style. I think I was looking at the macro used for citation sort instead of the one used for rendering. Good to hear that it's in order.

The behavior could be shifted to jump to the full form, but if it's right for Chicago with the current algorithm I'll stand pat unless we hear complaints.

adamsmith

ok, fixed for CMS author-date on the csl 1.0 repository
https://github.com/citation-style-language/styles
(we've gotta get this repository situation solved - I feel like I'm juggling an increasingly untenable number of balls/citation styles)

donsushi

Fbennet, adamsmith, sorry for not replying earlier. I was away for the last week.

If I understand you correctly, this is not a problem of Zotero but intended behaviour by the APA and CMS (author-date) csl styles? I remember modifying the original APA style before upgrading to the new Zotero version (see http://forums.zotero.org/discussion/16321/changing-the-citation-format-for-author-disambiguation/) to display disambiguations as (Smith, John, 2010). If this is what broke, I am sorry for causing confusion.

One of the reasons I need to display the full name in disambiguations right away is because my database includes various works by Asian authors, whose names are written in Chinese and Japanese characters and can't really be abbreviated. Is there a way to force this behaviour with the most recent APA and CMS (author-date) csl styles from https://github.com/citation-style-language/styles ?

fbennett

The processor will automatically apply the correct word-ordering to names written with the Chinese and Japanese character sets. If I remember correctly, I've implemented the processor to apply abbreviations, though, because the first character of the given name is used in Japanese academic publishing to distinguish authors with the same family name, much as in Western publishing.

If you require more elaborate discrimination in the formatting of references containing Chinese or Japanese script, there are ways to accomplish that, but they are experimental and non-standard (at present). A recently posted screencast (by me, in my own wheezy voice) from Debbie Maron's Multilingual Zotero blog post provides an overview. The platform is experimental, but it provides pretty sound multilingual coverage.

donsushi

Thank you for the link, fbennett. I had no idea that you're the author of the Multilingual Zotero project. I read about it when I started using Zotero a couple of months ago. Very impressive. Keep up the good work!

However, Multilingual Zotero is still in beta and not intended for production use, like you said. So I am currently still using the default Zotero version and normally attach the author's full Chinese or Japanese name to his given name, like this:
Author's family name: Yu
Author's personal name: Yongding 余永定
Obviously, this is not a very clean solution and causes a few problems. But it's sufficient for what I am trying to do.

The above author shows up as Yu, Yongding 余永定 (2009) [...] in the bibliography, which is the formatting I need. When citing this reference I get "(Yu, 2009)", which is fine as well. However, when disambiguating between this and another author named "Yu", Zotero produces this: (Y. 余永定 Yu, 2009)

My university requires that I put the full name in this case, though, with the family name coming first. I would need (Yu, Yongding 余永定, 2009), but so far I have only been able to rearrange the name order and got (Yu, Y. 余永定, 2009). I still haven't been able to get Zotero to give me the full given name. Is this something I can change via the csl style definition?

fbennett

It depends on whether you need initials for the Western authors, or whether jumping from family-only to full-name for those as well is acceptable. If the latter, you can remove the initialize-with attribute from the name node, and disambiguation should jump to full form in one step.

donsushi

Just tried that on the Zotero test pane with the default APA csl style. To make it work, one more step is required, though.

By default, APA displays names in the bibliography as follows:
Yu, J. (2010). [...]
Yu, Y. 余永定. (2009). [...]
And disambiguations as:
(J. Yu, 2010)
(Y. 余永定 Yu, 2009)

First, I removed "initialize-with" from the author macro, which controls how names are displayed in the bibliography: <macro name="author"><names variable="author"><name name-as-sort-order="all" and="symbol" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/> ...
This changes the bibliography to:
Yu, Jie. (2010). [...]
Yu, Yongding 余永定. (2009) [...]
Disambiguation remains unchanged.

Then, I removed "initialize-with" from the author-short macro, which should have an impact on disambiguations: <macro name="author-short"><names variable="author"><name form="short" and="symbol" delimiter=", " /> ...
But this only changes disambiguations to
(J Yu, 2010)
(Y余永定 Yu, 2009)

Since this wasn't what I wanted, I modified the author-short macro a little more until the name tag looked like this:
<name form="long" and="symbol" delimiter=", " name-as-sort-order="all" sort-separator=", "/>
Finally, I removed the option disambiguate-add-givenname
<option name="disambiguate-add-givenname" value="true"/>
This finally gave me the disambiguation format I wanted:
(Yu, Jie, 2010)
(Yu, Yongding 余永定, 2009)

Thank you for your help with this, fbennett. This finally solved my problem. I hope this little guide will be helpful to someone.

fbennett

Yes, that would turn off disambiguation entirely, and use the long form only. I'm sorry about getting the behavior wrong; it was late at night when I wrote that last response, and I misinterpreted the behavior exhibited in one of my own tests.

It seems to me that the processor did once behave as I described, though. I'm not sure whether the use of abbreviations even when there is no initialize-with attribute is an accidental regression, whether it was changed to track what the CSL 2.0.9 processor did in that case, or whether my memory is just faulty. I'll take it up in the CSL group, and see whether it's possible to restore/add that (simpler and more flexible) behavior.

donsushi

You are right, this modification will turn of disambiguation altogether. It was also late when I modified the csl yesterday, so I didn't notice the long form was used by default now. Sorry about that.

I am pretty sure that Zotero did once give me the citations in the correct format, though, i.e. printing the full first name in case a citation required disambiguation, since this is the reason I actually started this thread.

Thanks for following up on this. I'll keep tracking this topic and try out new versions of Zotero and the csl definitions to see if this makes a difference.

fbennett

Great, thanks for getting back. I'll follow this up, and post back to this thread if I have anything definite. It's easy to adjust the processor to do this; it's just a question of whether the change would cause a ruckus among users who expect something else. (We have to be careful about imposing unnecessary hardship on adamsmith and others involved in style maintenance and development).

fbennett

Ha. Before posting to the CSL list I checked the specification, and the behavior you desire (a jump straight from short form to full name) is listed as the correct behavior when initialize-with is not set. It's right there in the disambiguation section of the spec (which I wrote), so there's no need for delay. I'll build the tests this evening and fix the processor. You should be able to take advantage of the corrected behavior in the next Zotero release.

fbennett

@adamsmith: You wrote:

This is not the behavior I would have expected, though - I'd have thought disambiguation just defaults to the long form of the author when called - i.e. initials when there is an 'initialize-with' term, full first name when there isn't.

It took me awhile to absorb the message, but you're exactly right; and that's exactly what the CSL 1.0 spec calls for. As noted above, it's now been fixed in the processor, and should work its way to the surface pretty soon.

adamsmith

thanks Frank, glad my csl intuition works ;-).

donsushi

Thanks for following up on this, guys. It's really great you're keeping track of these problems and respond to user input in this way. I'll let you know the results as soon as the new Zotero version is out. And please let me know if there is anything else I could do to help.

donsushi

I can confirm that the problem has been fixed in Zotero 2.1.8. On the Zotero test pane I get full-name disambiguation with the first name following the family name. My debug collection looks as follows:
(Mundell, Robert, 1973)
(Mundell, Robert A., 1961)
(Yu, Jie, 2010)
(Yu, Yongding, 2009)

To achieve this disambiguation style, the line "<name form="short" and="symbol" delimiter=", " initialize-with=". "/>" in the APA style needs to be changed to: <name form="short" and="symbol" delimiter=", " name-as-sort-order="all" sort-separator=", "/>

Thanks for fixing this, guys!