Error: year disambiguation in author-date styles

Hi -- I'm experiencing a year disambiguation issue in author-date styles (Zotero 2.1.8, Firefox 5.0, Ubuntu 11.4). The years get disambiguated in order of appearance in the text rather than order of publication date.

Then there's occasional issues with the alphabetical disambiguation running on from one year to the next; say, for instance, you have publications 2007a to 2007m by one author, and the references list will then start with 2008n for the same author the following year, rather than resetting to 2008a.

Thanks for any help.
«13
  • That's certainly odd. The year-suffix letters should be applied in bibliography order.

    Which style?
  • APA. I was going to generalise from the fact that Harvard Author-Date messes up as well, but I've just discovered that Chicago Author-Date handles my pared-down test file correctly.
  • For what it's worth: Chicago Author-Date handles a longish piece with a 100+ refs just fine, but when I switch it back to APA, it's all over the place again (almost all the references in this piece are date-stamped web postings, so the deviation from chronological order is virtually unmissable -- this would be far less noticeable if the references were predominantly books and journal articles).
  • If you have a minimal set of entries that show the bad behavior, could you export them as Zotero RDF, paste to http://gist.github.com/, save as a public gist, and post the address from the address bar back here? I'd like to get to the bottom of this.
  • APA seems to be missing an explicit year-suffix in one of the date macros, which could be a source of trouble. But I'll wait for the sample data before looking further.
  • Here's some random sample data tested and confirmed to misbehave in APA -- I suppose you could use any other set of entries with day-month-year datestamps. APA sorts the set in order of appearance in the text. Chicago Author-Date sorts it correctly.
  • edited August 1, 2011
    Okay ... the processor is performing correctly on the style. The sort keys for the bibliography come from the "author" macro, and from the "issued-year" macro. The actual keys used by the processor are the following, for the sample data:
    Item 1:keystring: (Brooker, C.)
    keystring: (Y20110000Y00000000S00000000000000000000)

    Item 2:keystring: (Brooker, C.)
    keystring: (Y20110000Y00000000S00000000000000000000)

    As the sort keys are the same for the two entries, they are ranked in document order in the bibliography. The year-suffix is missing in the bibliography for these entries, and that is a problem with the style. But things are coming out in the right order, and the year-suffix is being assigned correctly.

    Debate over whether APA should sort in this way is beyond my jurisdiction. :) But the processor is doing what it is asked to do. As far as I can tell, nothing strange is going on here.
  • I've filed a revision to the APA style that adds the missing year-suffix on periodical dates. You should be able to pick it up in an hour or two by reinstalling the style from the repository.

    When the year-suffix elements appear properly in the bibliography, it will be less confusing to the eye, and we will be able to sort through any remaining issues.
  • Thanks!

    No improvement, though.

    I went to the style repository, hit 'install' on American Psychological Association 6th Edition (rev of 2011-08-01 06:01:03), then restarted Firefox and LibreOffice and switched the test file back and forth between Chicago Author-Date and APA.

    Chicago Author-Date:

    Brooker, Charlie. 2011a. July 24.
    ———. 2011b. July 31.
    APA:

    Brooker, C. (2011a, July 31)
    Brooker, C. (2011b, July 24)
    Chicago Author-Date gets it right, APA gets it wrong. It used to work correctly for a long time, so something must have broken it.
  • You're now getting the year-suffixes, that's all I changed in the style. As I wrote above, the sorting is being done exactly as it is defined in the style. If you can quote the section of the APA style guide that requires the month and day to be included in the date sort for newspapers, it can easily be changed.
  • I go back a bit with Zotero and author-date style, and the year suffixes have always been there. I've been working with date-stamped web items throughout this period, and Zotero always used to sort the reference lists in chronological order according to the date-stamps.

    I noticed only very recently that this behaviour has changed. I first suspected I had broken the sorting through my habit of pasting Zotero references from older documents, some of which will occasionally be pre-Zotero 2.1.x.

    But it doesn't seem like anything is really broken (except for the disambiguation suffixes that now occasionally run on from one year to the next), and if the new sorting is a feature rather than a bug I'll probably manage to live with it, even though the old sorting seemed to make more sense. I don't have a copy of the APA manual nearby, and a few web searches didn't turn up the relevant rule, so I can't prove the chronological sorting by date-stamp is the right thing. It just feels right.
  • You're now getting the year-suffixes, that's all I changed in the style. As I wrote above, the sorting is being done exactly as it is defined in the style. If you can quote the section of the APA style guide that requires the month and day to be included in the date sort for newspapers, it can easily be changed.
    APA is an "author-date" style; "author-year" is a misnomer.

    I'd suggest the sorting should be by full date regardless of the type, rather than specifically changed for particular types.
  • edited August 1, 2011
    The style consistently sorts on year only at the moment, for all types. I'm happy to change it to sort on the full date, but I'd like to make the change just once. Does anyone have access to the APA 6th guide?

    (Re discriminating by type in the sort, I think you'd want at most to include the rendered date elements in the sort. If a book gives a specific date of publication, but shows only the year, it would be confusing to sort on the unseen date elements.)
  • I guess I'm to blame for this one:

    https://www.zotero.org/trac/changeset/4659/csl/apa.csl
    http://forums.zotero.org/discussion/7427/?Focus=33198#Comment_33198

    That change solved problems with sorting for bibliographic entries rendering only the year. But looking at the APA style, it's probably better to sort by the "issued" macro instead of the "issued-year" one.
  • A separate macro will be required, since the date is rendered separately twice in the "issued" macro. I can take care of that tomorrow.
  • That shouldn't be a problem, right?

    The spec states: "... if a date variable is called by the variable attribute, the complete date is returned. In contrast, macros return only those date-parts that would otherwise be rendered ..."

    As the month/day follow the year, sorting should be correct.

    We can also rewrite the macro to put the item type conditional at a higher level, which should eliminate your concern.
  • edited August 1, 2011
    https://github.com/citation-style-language/styles/commit/892a07fa385b457feb212a0d513ff3f9a22f99f8
  • edited August 1, 2011
    Sorry, had to reverse that commit, as it messed up the placement of the year-suffix for full dates.

    In my commit, the year-suffix is suppressed as it's explicitly defined for the no-date case. I could add a cs:text element calling "year-suffix", but this places the year-suffix after the full date ("2002-10-3a") instead of after the year ("2002a-10-3"), which might not be desired.
  • This gets back to that long, hairy, conversation about disambiguation and grouping.

    In my first citeproc + CSL prototype, this issue would never come up: sort the list by date, group by year, index year groups.

    Can that logic really not be represented here?
  • This is just a small issue with sort keys.
  • Any progress on this? Zotero's APA (6th Edition) style in its current state (rev of 2011-08-01 06:01:03) is not fit for purpose, whereas Chicago Author-Date seems to be working fine.
  • edited August 10, 2011
    Sorry for not getting back earlier. There was a limitation in the processor logic that we needed to remedy in order to get date sorting right. In the citeproc-js version that triggered this thread, the first cs:date node in whatever macro was used in cs:key was used for sorting -- so you always got either the full date or the year, with no possibility of discriminating on the item type.

    That wasn't right: a book might record the exact date of release in Zotero but display only the year, in which case it should be sorted by what is displayed. To get that working, we needed to recognize conditional logic inside date macros fed to cs:key. That's been done, and will find its way into Zotero in the next release.

    Once the new processor is available, we'll need to make a small change to APA to adjust the sort behavior. The change can be made early; I'll look at it later today.

    Thanks for your patience; we're nearly there on this one.
  • Thanks for the update! It's good to know that things are moving along.
  • I've upgraded Zotero 2.1.10 to APA 6th edition (rev 2011-08-23 15:22:34) and I still get totally random, non-chronological disambiguation, eg:

    (1993d, April 14)
    (1993e, January 13)
    (1993f, July 2)
    (1993g, January 16)
    (1993h, November 27)

    Also, an entry's disambiguation letter now gets inserted after the year of the access date if an entry has a URL.

    For a while I've been using Chicago Manual of Style (Author-date format) as an alternative, but this turns out to be faulty in other ways: its disambiguation, once it has run through the whole alphabet, just keeps on incrementing the disambiguation tokens in the double-letter range, without ever resetting back to single letters for a new year.

    Thus, eg:

    Smith, J. (1907ab)
    Smith, J. (1907ac)
    Smith, J. (1908ad)
  • The style code is still sorting on issued-year only in the bibliography (and in the citation itself). When that's changed, the processor will pick it up, and the sort will come out correctly. (I'll leave the actual change to others, since I don't know the APA well.)

    The disambiguation letter after the access date can be removed, as it's set explicitly wherever it appears throughout the style. (This was my own typo, but I'll leave it to be fixed when the sort key is addressed. The error is at line 144.)

    On the issue with bad increments to the year-suffix letter, I'll need sample data.
  • Frank - could you clarify? What needs fixing in the style/why is sorting by issued-year not correct?
  • Apparently for some types the full date should be included in the sort. anark gives an example above. It's not safe to just turn on full-date sorting for all items, since that would sort books by full date as well.

    I'll make what I think is the right change now and file it as a pull-request against the main repo.
  • I don't know if the publication date of books (if available) should be factored into the sort -- probably not. But I'd assume that in author-date styles newspaper articles and blog posts would need to be sorted by their full date rather than just their year -- magazine and journal articles as well, probably.

    Thus, instead of:

    (1993d, April 14)
    (1993e, January 13)
    (1993f, July 2)
    (1993g, January 16)
    (1993h, November 27)

    They would need to come out as:

    (1993d, January 13)
    (1993e, January 16)
    (1993f, April 14)
    (1993g, July 2)
    (1993h, November 27)
  • edited October 10, 2011
    I'll take back my comment on the bad increments to double-letter disambiguation suffixes in Chicago Author-date. I've just converted an article to Chicago Author-date and it gets the following transition right, amongst several others:

    ———. 1995af.
    ———. 1996a.

    I'm fairly sure I wasn't hallucinating, though, and will get back to you when I have a reproducible issue...
This discussion has been closed.