Some citations crash Firefox

anark
In one particular word-processed file of about 8,000 words and heavy citation, some of my Zotero citations cause Firefox to crash: Firefox will then hog all CPU and needs to be force-quit.

The offending citations work fine in any other document, and one temporary solution to the problem consisted in breaking the document into two parts, both of which were then fine. To merge the two parts again, I needed to identify and remove the offending citations.

So here's a group of citations that will crash Firefox if inserted into this document. The same citations work fine if inserted into any other document, or indeed the troubled document broken up into two parts.

A database check reveals no corruption.

Thanks for any suggestions on how to fix this issue.

Vital stats: Zotero 2.0.3, OOo 3.1.1, Firefox 3.5.9, Ubuntu 9.10
«1
  • Simon
    This will probably be fixed in Zotero 2.1, which implements citeproc-js to handle citation generation and processing, but it would be great if you could email the smallest document fragment that produces this issue to support@zot...org.
  • adamsmith
    edited June 5, 2010
    note that there is a thread on Zotero & Ooo not working stably with Bookmarks on Ubuntu (and Ubuntu only) - don't have time to look for it right now.
  • anark
    Adamsmith, I'm using ReferenceMarks rather than Bookmarks.

    Simon, as mentioned, when I break the document into two parts, the problem goes away, so to come up with the smallest document fragment that produces the problem, I would probably need to remove all the copy and leave the citations in. I'd also need to send you the database for you to reproduce the problem, as the crash occurs when I'm trying to insert any one of a small number of citations that will misbehave in this particular context.

    As a test, I chose one troubled citation in the final few paragraphs of the chapter, then selected everything further up in the document, dropped it into a text editor and then copied and pasted it back into the document. With all of the prior citations thus removed, I tried inserting the citation that would otherwise crash Firefox, and it went in without a hitch.

    Creating a fresh Zotero entry for a misbehaving citation, then going back to the full unmodified document and trying to insert the fresh instance of the citation instead of the known miscreant resulted in another crash.

    I then decided to log a crash incident in debugging mode. The log turned out to have reams and reams of "SELECT A.itemId" and "SELECT N.itemID" towards its end:


    (5)(+0000000): SELECT A.itemID, value AS title FROM itemAttachments A NATURAL JOIN items I LEFT JOIN itemData ID ON (fieldID=110 AND A.itemID=ID.itemID) LEFT JOIN itemDataValues IDV ON (ID.valueID=IDV.valueID) WHERE sourceItemID=? AND A.itemID NOT IN (SELECT itemID FROM deletedItems)

    (5)(+0000000): Binding parameter 1 of type int: 9579

    (5)(+0000001): SELECT N.itemID, title FROM itemNotes N NATURAL JOIN items WHERE sourceItemID=? AND N.itemID NOT IN (SELECT itemID FROM deletedItems)

    (5)(+0000000): Binding parameter 1 of type int: 9579

    (5)(+0000000): SELECT A.itemID, value AS title FROM itemAttachments A NATURAL JOIN items I LEFT JOIN itemData ID ON (fieldID=110 AND A.itemID=ID.itemID) LEFT JOIN itemDataValues IDV ON (ID.valueID=IDV.valueID) WHERE sourceItemID=? AND A.itemID NOT IN (SELECT itemID FROM deletedItems)

    (5)(+0000000): Binding parameter 1 of type int: 7275

    (5)(+0000000): SELECT N.itemID, title FROM itemNotes N NATURAL JOIN items WHERE sourceItemID=? AND N.itemID NOT IN (SELECT itemID FROM deletedItems)

    (5)(+0000000): Binding parameter 1 of type int: 7275

    (5)(+0000000): SELECT A.itemID, value AS title FROM itemAttachments A NATURAL JOIN items I LEFT JOIN itemData ID ON (fieldID=110 AND A.itemID=ID.itemID) LEFT JOIN itemDataValues IDV ON (ID.valueID=IDV.valueID) WHERE sourceItemID=? AND A.itemID NOT IN (SELECT itemID FROM deletedItems)

    (5)(+0000001): Binding parameter 1 of type int: 12481


    The Debug ID is D1240430265.
  • anark
    Discovery: changing the date field of a miscreant Zotero entry solves the issue, technically, but running with incorrect publication dates is obviously not a solution.
  • anark
    Changing the date field in another miscreant Zotero entry did nothing to solve the problem.
  • anark
    I think I've isolated the culprit: it's year suffix disambiguation. I requested the feature some time ago and have been happily citing authors with more than 26 references in any given year. I'm now trying to cite an author with more 52 references in one particular year; my References stop at 1997az, and when I try to slip in another reference for the same year, the year suffix disambiguation doesn't flip over to 1997ba. Instead, Firefox crashes.

    To test the hypothesis that year suffix disambiguation lies at the bottom of this, I removed a bunch of the author's citations from that year, then attempted to add the formerly miscreant citation from the same year -- which promptly went in.

    So this is beginning to look like a bug report. Year suffix disambiguation doesn't scale up beyond YYYYaz. Could you fix this?
  • adamsmith
    I don't think this will be fixed in the current plugin, as a new one is being rolled out currently for Zotero 2.1 - https://www.zotero.org/trac/timeline -
    maybe someone can test this on the new processor?

    For the time being it might make sense for you to switch to a style without disambiguation to prevent crashes.
  • RintzeRintze
    Mea culpa. While the isolated code (below) generates "ba" when the previous suffix was "az" when run in a javascript interpreter (http://www.mochikit.com/examples/interpreter/), I now see that the "i" variable is global, which probably is messing things up. I haven't tested it, but it's probably the culprit.

    var lastDisambiguate = "az";

    var newDisambiguate = "";
    var charCode = lastDisambiguate.charCodeAt(lastDisambiguate.length-1);
    if(charCode == 122) {
    newDisambiguate = lastDisambiguate.replace(/z+$/, "");
    var consecutiveZs = lastDisambiguate.length-newDisambiguate.length;
    if(newDisambiguate.length >= 1) {
    var nonZCharCode = lastDisambiguate.charCodeAt(newDisambiguate.length-1);
    newDisambiguate = newDisambiguate.substr(0,newDisambiguate.length-1);
    newDisambiguate += String.fromCharCode(nonZCharCode+1);
    for(i=0;i<consecutiveZs;i++) {
    newDisambiguate += "a";
    }
    }
    else {
    newDisambiguate = lastDisambiguate.replace(/z/g, "a");
    newDisambiguate += "a";
    }
    }
    else {
    // next lowercase letter
    newDisambiguate = lastDisambiguate.substr(0,lastDisambiguate.length-1);
    newDisambiguate += String.fromCharCode(charCode+1);
    }
  • anark
    adamsmith:
    > I don't think this will be fixed in the current plugin, as a new one is
    > being rolled out

    Is there a target release date for the new plugin or for a public beta of it?

    Rintze:
    > Mea culpa

    Well, never mind. Do you share Adamsmith's assessment that there won't be a maintenance release of the old plugin with this bug fixed?
  • adamsmith
    anark -
    there is a (n almost) daily public trunk release for testing (and testing only!)
    http://www.zotero.org/support/dev/svn_and_trac_access

    Zotero devs have in the past either not given target release dates or have been off significantly - for 2.1. the former is the case and I wouldn't expect any reliable eta - from how things look on trac my guess would be sooner rather than later, but that's about as reliable as you're going to get.

    Devs have said that the current csl implementation is no longer developed&maintained, so I'm pretty sure I'm right about that, especially since yours is a very, very rare usage case and not a crucial fix.
  • anark
    > yours is a very, very rare usage case and not a crucial fix.

    Fair enough.

    Thanks.
  • ajlyon
    I went ahead and tried to replicate this using the Zotero trunk, which is using the new citeproc-js engine, and OpenOffice crashed on me.

    I'll try to get some debugging information, but since the insert multiple citations dialog doesn't allow me to select multiple items, it's a bit of a pain.

    If the fix in the 2.0 branch is just adding a line "var i;", then perhaps that could still get committed to the 2.0 branch. The old processor is not being actively maintained, but that doesn't mean that user-submitted patches will be automatically rejected. Then anark could use the 2.0 development XPI and keep on working.
  • RintzeRintze
    edited June 7, 2010
    Do you share Adamsmith's assessment that there won't be a maintenance release of the old plugin with this bug fixed?
    There is a 2.0 branch, so it's possible that there will at some point be a 2.0.4 release. But if you don't want to wait for that (and want to test whether my proposed fix really works), you're probably better off trying the change in your local Zotero 2.0.3 copy:

    Instructions:
    http://www.zotero.org/support/dev/modifying_zotero_files

    The line to change:
    https://www.zotero.org/trac/browser/extension/branches/2.0/chrome/content/zotero/xpcom/csl.js#L2461
    Original line
    for(i=0;i<consecutiveZs;i++) {
    New line
    for(var i=0;i<consecutiveZs;i++) {
  • anark
    Rintze wrote:

    > you're probably better off trying the change in your local Zotero 2.0.3 copy

    Let's see, I edited /content/zotero/xpcom/csl.js to:

    if(newDisambiguate.length >= 1) {
    var nonZCharCode = lastDisambiguate.charCodeAt(newDisambiguate.length-1);
    newDisambiguate = newDisambiguate.substring(0,newDisambiguate.length-1);
    newDisambiguate += String.fromCharCode(nonZCharCode+1);
    for(var i=0;i<consecutiveZs;i++) {
    newDisambiguate += "a";
    }

    Then rebooted the system and tried to insert another citation into the big document. Firefox crashed.
  • RintzeRintze
    In that case I don't know what's giving the problem.
  • RintzeRintze
    As a bit more debug info, with Zotero 2.0.3, csledit.xul gives:

    52 items (correct):
    (Zelle et al., 2008a, 2008b, 2008c, 2008d, 2008e, 2008f, 2008g, 2008h, 2008i, 2008j, 2008k, 2008l, 2008m, 2008n, 2008o, 2008p, 2008q, 2008r, 2008s, 2008t, 2008u, 2008v, 2008w, 2008x, 2008y, 2008z, 2008aa, 2008ab, 2008ac, 2008ad, 2008ae, 2008af, 2008ag, 2008ah, 2008ai, 2008aj, 2008ak, 2008al, 2008am, 2008an, 2008ao, 2008ap, 2008aq, 2008ar, 2008as, 2008at, 2008au, 2008av, 2008aw, 2008ax, 2008ay, 2008az)
    53 items (problem already starts after 2008b):
    (Zelle et al., 2008a, 2008b, 2008zb, 2008zc, 2008zd, 2008ze, 2008zf, 2008zg, 2008zh, 2008zi, 2008zj, 2008zk, 2008zl, 2008zm, 2008zn, 2008zo, 2008zp, 2008zq, 2008zr, 2008zs, 2008zt, 2008zu, 2008zv, 2008zw, 2008zx, 2008zy, 2008zz, 2008aaa, 2008aab, 2008aac, 2008aad, 2008aae, 2008aaf, 2008aag, 2008aah, 2008aai, 2008aaj, 2008aak, 2008aal, 2008aam, 2008aan, 2008aao, 2008aap, 2008aaq, 2008aar, 2008aas, 2008aat, 2008aau, 2008aav, 2008aaw, 2008aax, 2008aay, 2008aaz)
  • bdarcus
    Irrespective of the bug, I've got a bigger question: why do you have a situation where you have so many colliding references? It seems one should never have this situation to begin with.
  • anark
    edited June 8, 2010
    The references don't "collide" in any meaningful sense that I'm aware of. After a certain number is exceeded, Zotero 2.0.3 just stops handling them correctly.

    In a world where monographs and research papers are the smallest citable entities, 52 refences for the same author within the same years does seem excessive and one "should never have this situation", I agree.

    However, we don't live in that world any longer. In my particular case, the 52+ references are mostly blog posts.
  • adamsmith
    but is an author-date citation style really the way to go then?
    It seems like a terrible thing to do to your reader.
    Any Chicago-style (it seems in your case note would be better than full note) will handle this without problem and the reader actually has an idea what the citations mean - the rational behind author-date, after all, is that a reader from the same field will often know what a citation like "Smith 1776" refers to - clearly that's not the case for what you're writing.

    Even if you have to use author-date for whatever reason, you should consider adapting your citation practice (e.g. by referring to individual blogposts by date in-text or so).
    After all, if you're living in an entirely different world it seems appropriate to adjust your citation practice to that alternate reality.
  • anark
    Rintze wrote:
    > 53 items (problem already starts after 2008b)

    Erratic reference sorting has been a problem before. Maybe the fix that was adopted at the time doesn't play well with the upper reaches of year suffix disambiguation?
  • anark
    Adamsmith -- author-date isn't negotiable and works reasonably well. I'm afraid you'd need to be familiar with the specific work and its context to be able to advise me on best citation practices. Thanks for the effort, though.

    Ajlyon wrote:
    > I went ahead and tried to replicate this using the Zotero trunk, which is
    > using the new citeproc-js engine, and OpenOffice crashed on me.

    Have you filed a bug report?
  • ajlyon
    Have you filed a bug report?
    Frank is kind enough to follow all such discussions, and he's already working on a fix to citeproc-js (http://groups.google.com/group/citeproc-js/t/32151fd28b4d7129).
  • anark
    Excellent, thanks!
  • bdarcus
    However, we don't live in that world any longer. In my particular case, the 52+ references are mostly blog posts.
    So what is the issue here? That they don't have authors, or that you're really citing 50 or so blog posts from the same author in the same year?

    If it's the former, then a style that substitutes the post title might be appropriate?

    I almost exclusively use author-date styles, and I also often work with historical manuscripts, and I've never had more than 10 or so duplicate author-year, even in a dissertation or book.

    YMMV of course, but just thought I'd ask the question.
  • fbennett
    edited June 12, 2010
    @anark, ajlyon, everyone,

    Thanks for raising this. In fact, I had largeish year-suffixes in mind when I wrote the code that generates them in the new processor. It's a tricky problem, actually, because a system of counting by letter-sequences doesn't have a zero. It seems like nothing (!), but it becomes an issue when rolling over to the next digit. Not sure whether that's what's affecting the current processor, but the citeproc-js suffix generator will work with three or four hundred items without any problem. However ...

    Identifying groups of entries that need a suffix assigned to them is another story. A 52-item test case built to mimick anark's requirements turned up a bug in the citeproc-js disambiguation code, and when that was fixed, the test took over a minute to clear. Ouch.

    This was clearly unacceptable, so I put some time in yesterday to rewrite the processor's disambiguation code (which had grown extremely wooly and hard to manage). That's now done, all existing test cases clear, and the 52-item citation clears in about 2 seconds on my (very slow) netbook.

    It will take awhile for these revisions to work their way into the development trunk, but when 2.1 arrives it should handle use cases like this just fine.
  • anark
    Bdarcus -- I'm citing more than 52 Web posts by the same author within the same year. My original feature request some time ago resulted in an implementation of year suffix disambiguation that currently supports up to 52 citations by the same author in the the same year. The number 52 is two times 26, i.e. two times the number of letters in the alphabet. Zotero currently handles the first 26 citatations as, for instance, 2009a to 2009z, and then the next 26 citations as 2009aa to 2009az; the 53rd citation should result in 2009ba, but that's the point at which Zotero causes Firefox to crash and freeze.

    Fbennett, thanks for fixing this in the new processor. I really appreciate the effort!
  • laufie7
    I also have an author with more than 26 references of one single year. The citation worked until suffix "y". When I add article nr.26, the "NAME YEARa" switches to "NAME YEARb" etc. so the last one is "NAME YEARa". When I add article nr. 27 from this author, it is also called, wrongly, "NAME YEARa".
    How can I tell the programme that it should work like "aa,ab,...,az,ba,bb,...,bz,...,za,...zz" relating the suffixes?
    Thanks for any help!
  • anark
    Thanks to fbennett's work on this, I haven't had any problems with large citation counts per author per year, and I quite often deal with documents that cite in excess of twenty-six items per author per year. Are you using an unusual author-date citation style? (I mostly use APA in LibreOffice on a recent-ish Ubuntu operating system.)

    For the developers to be able to reproduce the issue, you might need to state what exactly your setup is.
  • laufie7
    "Zeitschrift für Soziologie" in Microsoft Word is the concerning citation style. And even if it would be easier to use another, I have to use it because the document is part of a project many people are involved and using this citation style too.

    Do you would like to have a look into the citation style editor? Or do you need any more information?
  • anark
    I'm not a Zotero developer and can't speak for them. It's for them to say what information they need.

    What happens when you switch from "Zeitschrift für Soziologie" citation format to one of the default author-date formats that ship with the English version of the software, such as American Psychological Association (APA)? Does the problem persist?

    As mentioned above, using APA I haven't had any problems with any of this for years. If you can get your citations to work in APA as well, the bug is likely to be in the "Zeitschrift für Soziologie" citation style rather than in Zotero's rendering engine.
Sign In or Register to comment.