Word Plugin very SL-O-O-W

It's taking 4 and a half minutes to generate a single footnote using the Word plugin. That includes at least three error messages from Word about a stuck script (hit CONTINUE and it goes on) and then three erroneous messages from Z about "this entry no longer exists in the database, delete it?" (hit NO and it continues).
Oh, yes, and the database is corrupt, but restoring from the most recent backup doesn't help (it's corrupt also). Reindexing doesn't cure the corruption. But despite the corruption it generates footnotes and bibliography OK (as far as I can tell). At this point, with 320 entries in the database and 350 footnotes (of which 250 are done) I can't back out and try Plan B (3x5 cards) -- but I suspect the cards would be faster. I'm using Word XP on a fast new AMD64 HP machine with Vista Ultimate. Oh yes, error message number 761819423.
  • error messages from Word about a stuck script (hit CONTINUE and it goes on)
    http://www.zotero.org/documentation/unresponsive_script_warning
    three erroneous messages from Z about "this entry no longer exists in the database, delete it?" (hit NO and it continues)
    They're not erroneous. You have three citations (possibly hidden ones) that no longer exist in Zotero. Additional feedback from the plugin on the missing citations would obviously be helpful, but with only three citations it shouldn't be too hard to figure out which three were removed (for example, using Undo/Redo).
    the database is corrupt
    Upgrade to Firefox 3, which fixes database instability problems in Firefox 2, and then use the DB Repair Tool.
    350 footnotes (of which 250 are done)
    You may be hitting the limits of the current beta version of the plugin. We're in the process of completely rewriting the word processor plugins, which should help with performance.
  • Thank you for the helpful comments.
    OK, I upgraded to FF3.
    I modified the config settings.
    I uploaded the .sqlite file to the DB repair tool. What happens next? Does the DB repair tool return the cleaned up file to me, somehow? If so, how?
  • Does the DB repair tool return the cleaned up file to me, somehow?
    It should provide download links as soon as you've uploaded the file.
  • Upgrading to FF3 and resetting the config is a BIG improvement in working time. The long delays are gone and I can do a footnote in seconds instead of minutes. I should have come here for help earlier! On the other hand I have some lovely music to listen to while waiting so it wasn't total torture.

    As for the db repair tool, it did not give me any response other than graying out the upload button. Maybe it's because I navigated away from the page while waiting. I'll try it again. The db is only 9 mb.
  • As for the db repair tool, it did not give me any response other than graying out the upload button.
    That just means it's uploading. Depending on your network upload speed, it could take a while.
  • Hmmmm. Does this
    You may be hitting the limits of the current beta version of the plugin. We're in the process of completely rewriting the word processor plugins, which should help with performance.
    mean that I'm going to have trouble in the 200-citation range? How close are you to a plug-in release that will address that issue?

    I'm really loving the speed/performance improvements in FF3 - thank you!
  • Does this mean that I'm going to have trouble in the 200-citation range? How close are you to a plug-in release that will address that issue?
    There's really no way to say what the performance will be with the current plugin on your system and with your database, but our goal is to release a new Python-based plugin in the next couple months.
  • OK, I figured out the db repair tool, and I've got a cleaned up database. Thanks for the help. Things are running much better.
  • Much faster now -- 20 seconds for a footnote, including a few seconds for me to find the reference on the long list. Considering the number of references -- now up to 331 -- this is very acceptable performance.

    But there's still three allegedly deleted entries in the database -- how in the world am I going to find which ones they are? Do I have to hand check them?
  • OK, I muddled through, using ctrl-z and alt-return to find one entry that was allegedly deleted from the zotero db, but was really there. I deleted the footnote reference to it and then regenerated it, and somehow that seemed to fix the problem. No more "deleted entry" error messages.

    In an ideal world, Z would mark or pinpoint the entry it thinks is missing. But I'll take this, for now, it's working great. I might even finish before dinner. Thanks for a very useful product.
  • I did finish before dinner!
    But now I have another question.
    I'm using end notes in MS Word, and those have to go into a new MS Word section; so basically the whole body of the book is in Sec. 1 and the endnotes are in Sec 2. Now I want to generate the Z bibliography after the endnotes. No can do! Dialog pops up and says bibliography can only be put in "main body" which means Sec. 1. So my book now has the bibliography ahead of the endnotes -- not the usual order.
    Moreover, Word will not let me insert another section break after the endnotes ... hmm ... what does this all mean?
  • Hmmm,

    Not necessarily an elegant solution, but you could flatten the bibliography field (select it then press CTRL-SHFT-F9). That converts it to regular text. Then you could cut and paste it after the endnotes.

    I'm not sure what the problem might be regarding not being able to make a new section. Maybe you don't really need to. Just paste it.
  • see also here:
    http://forums.zotero.org/discussion/3159/reference-list-after-endnotes/

    You may have to paste the bibliography into the last endnote rather than after the endnote section (at least in OOo).

    I do think that there should be a more elegant solution to this. After all it's standard to have the endnotes section before the bibliography. But from what I gather, this is a problem with the word processor?
  • Oops, new headache.

    When clicking on the Z toolbar in Word, the button for Refresh, things hum a little bit and then up pops a Visual Basic error message:

    "Run Time Error 91: Object variable or With block variable not set."

    I've rebooted both apps and the whole system and it still occurs.

    So, no can refresh. Ideas?
  • Hmm ...having posted this three minutes ago, and added another footnote, now the Refresh button appears to work -- no VB error message ...
  • A tip for first time users:

    Check for and eliminate duplicate entries in the db BEFORE generating footnotes. If you eliminate the duplicates afterward, there is a risk of orphaning the footnotes that refer to the duplicate entry, and this is a pain to hunt down.

    (Hint to developers: a routine to flag probable duplicate entries in the db could be helpful)
  • I would LOVE such a feature - one of my favorite things about RefWorks was the way it would create a folder (or a saved search - I don't really remember the process) that showed anything it thought was a duplicate. Super handy.
  • Hmm ... pasting the bibliography after the endnotes (in actuality, as the last of the endnotes, but not numbered) means that the Table of Contents routine in Word does not pick up or show the bibliography. Similarly, other back-of-the-book end matter -- e.g. suggested reading, appendices, etc.-- all gets subsumed under endnotes and does not get picked up or displayed in the Table of Contents.

    I would respectfully ask for a fix to this problem. Thank you.
  • Uh-oh ... another problem, and this one is scary.

    When I ask Z to generate a fresh bibliography, I get a Visual Basic error No. 6, "Overflow." And no bibliography. I've rebooted everything twice, even tried moving the notes to a fresh (and much smaller) file, no luck.

    I hope to get some emergency help on this ... the book manuscript is due at the publisher's by the end of this week.
  • The error code on my preceding post is 638906848
    Using MS Word from Office XP with MS Vista Ultimate.
  • The overflow problem is not due to file size. I have an earlier draft of the same document with only half a dozen fewer citations and this problem does not occur. Same setup on same machine. Baffling.
  • Well, it's only clear that it's not due to size if you've previously generated a bibliography at least that big. In a copy of the document, try removing a dozen or so references and see if the problem goes away.
  • The problem seems to be in the very last endnote. When I delete the last endnote (No. 339) everything works fine.
    Hypothesis: After I generated the bibliography originally, I moved it from its position in the body of the text (before the MS Word Section Break) to a new position at the end of the endnotes. This is to conform the relative positions of the notes and the bibliography to standard practice.
    In the new position, the bibliography would not update automatically as new endnotes were added. I therefore deleted the bibliography and attempted to generate a new one in the original position, before the section break. This is when the overflow occurred.
    I suspect that the final endnote become corrupted or mingled or merged with the bibliography when I moved it to the end of the endnotes, leading to an endless loop or recursion and a stack overflow of some kind. (However, the database does not show as "corrupted" in the test in the Preferences array.)
    I'm experimenting with different options to get my final endnote back. It's possible that the database entry for that endnote became corrupted, because I'm finding that inserting a different endnote reference for No. 339 does not create problems. Stay tuned, I hope to have an answer shortly.
  • Well, forget the above. The problem is in the number of bibliography entries.
    Deleting and re-introducing the bibliography entry to which the last endnote pointed did not fix the problem. Pointing the last endnote to an already existing bibliography entry created no problems.
    When I deleted a couple of (not strictly necessary) bibliography entries, and then re-inserted the final endnote (No. 339) pointing to the correct database record, the system worked.
    Conclusion: I've hit a ceiling on the maximum number of bibliography entries that the Word plugin will handle. I'm not sure where I can find a count of biblio entries (short of counting them by hand). Also not sure whether it's an item count ceiling or a file length ceiling.
    In any event, I've got a finished manuscript, so off to the publisher it goes.
    Thank you for what is (within its limits) a marvelous piece of software, and the price is right.
  • Well I guess that was a good point to end your book...

    I'm not sure what the ceiling is you're hitting—I believe VBA in Word is supposed to support strings up to 2 billion characters (whereas in OO it's 65K characters), so it might be an integer limit—but, regardless, this should be fixed in the next version of the plugin, which should be out within the next couple months.
  • Unfortunately, I'm getting the same error :'/
    It works tho by splitting the text into multiple files... which is very annoying if you are having cross references ...

    I hope it will be solved soon!
  • See http://forums.zotero.org/discussion/4107/word-overflow-when-inserting-bibliography/#Item_10 for the solution to the overflow problem.

    I also find Zotero slow. I've removed the bibliography ... it takes too long to update with 300+ entries. Adding a new citation is fine, but editing an old citation -- ''strewth, it takes over 19 seconds to bring up the dialogue box. That's too long. I wonder if the problem is the fact the integration is achieved through VB.
Sign In or Register to comment.