zotero very slow in long document

iselim

Hi,

I am about to finish a long project. Initially I had separate documents for all the chapters. Now that I've put them together (about 300 pages) zotero is very slow in updating or adding new citations. It takes up to two minutes and sometimes Zotero gets stuck.

In shorter documents citations are still added or updated in a blink. I tried reinstalling the zotero word for mac integration, but it didn't help. Any ideas what might cause this problem?

(I'm using Word for Mac 2011, Firefox 22.0, and Zotero 4.0.8 on a macbook air with a 1.7 GHz Intel Core and 4GB memory, running on OS X 10.8.4)

Thanks,
Markus

Simon

There are several places where things can be slow.

The first place is in actually getting the citations out of Word. This is reasonably fast in LibreOffice and Word for Windows, but in Word for Mac the amount of time to retrieve information about each individual citation scales with document length (not actually the number of citations), which can make things much slower. I have an idea for a potential workaround here; it involves changing the way Zotero communicates with Word in the hope that this performance issue doesn't affect every way we can get citation information out.

The second place is in collecting information about references. This would typically cause slowness when inserting the first citation after restarting Firefox or pressing the Refresh button, but shouldn't be a problem when adding new citations.

The final place is updating the document itself. If you use a numbered style and insert a new citation in the middle of the document, Zotero has to go through and update all subsequent numbers, which can be slow.

With a debug ID, I can provide you with a better idea of what's happening, but it's likely that the only presently available solution is to break up your document into smaller pieces.

iselim

Thanks for this information. Apparently long docs do present a problem (thanks for the link, Gracile). The second and final reasons that Simon points out don't seem to play a role in my case. Following the other thread on this topic, I removed the bibliography, which did not help, though.

As I mentioned, I had different docs for my chapters and maybe I should have kept it so until the very last changes have been done. But then I also wanted to get it together in one doc to avoid last-minute formatting issues and in order to get cross-references etc. ready.

Here is the debug ID, D2137337520.

If there is no immediate solution, I will simply extract the bits that need revision and insert them again when they are done. At this stage of my work (only revision) this is not a big issue. Any help is still welcome, of course. :)

Thanks!

iselim

Just an update considering the recent information provided in the other thread: in my case it is not re-pagination that causes the problem. Switching it off did not change the time zotero takes to update/add citations and during the whole time the zotero status bar is busy.

Simon

Yes, in your case the problem is definitely the length of the document combined with Word for Mac:

(3)(+0042633): Integration: Retrieved 1123 fields in 45.357; 24.759133099631807 fields/second

(This is ~17x slower per field than Word for Windows with an approximately comparably sized document, at least going by the other thread.)

iselim

Oh boy, that sounds terrible! -1 for Word for Mac...

If you happen to figure out the workaround you mentioned, I'd be happy to try it. Otherwise I'll just continue extracting bits.

francois_thibeault_UdS

I have experienced the same problem as you, with my PhD thesis.
https://forums.zotero.org/discussion/22622/inserting-citations-is-very-slow/#Item_13

The best solution was to work, in LibreOffice, with a master document and chapter documents. You then need to edit your in-between chapters crossreferences manually, though. But it's feasible.

francois_thibeault_UdS

One trick is to open Zotero preferences panel, in "Quote/Cite" tab, check "Use Zotero classic insert panel". That will accelerate things. And it's faster to sort your references in alphabetical order, and scroll down the page to the references you want to cite (rather than using the search bar). When references are sorted in alphabetical order (creator field), clicking on any reference, and then pressing any letter will bring you down automatically to references that begin with this letter in the library.

Simon

At least in principle, the classic insert citation dialog shouldn't be any faster than the new insert citation dialog. It switches where the delay happens (with the classic insert citation dialog you have to wait longer before the dialog opens), but it will be slower overall, since the new insert citation dialog can start loading references from the document as you're deciding what citation to add.

If you're sure the classic insert citation is faster, it's a bug, and a debug ID might help us figure out what's going on.

francois_thibeault_UdS

Well, I could not say that I am objectively convinced or sure that classic insert is faster than the new insert citation dialog... For sure, I did not see any difference for small files, with about 50 references and a corresponding number of citations. However, my thesis was huge by normal standards, and maybe the master document with subdocuments option made a bigger difference than simply changing the new insert dialog to the classic insert window.

francois_thibeault_UdS

By the way, do you know if adamsmith proposition will become real anytime soon?
Quote:
"There are, obviously, better ways to work with large, demanding documents than WYSWYG processors like Word and LO - markdown, re-structured Text, LaTeX, all of which can be integrated with Zotero in one way or the other.
At some point in the future - but not anytime very soon - Zotero may also get a better RTF scanning feature, which would allow you to just enter simple codes and then have Zotero fill in the references at the end. "
see https://forums.zotero.org/discussion/22622/inserting-citations-is-very-slow/#Item_13

Rintze

There is. See http://zoteromusings.wordpress.com/2013/05/06/announcing-rtfodf-scan-for-zotero/

qenghis

A comment for Simon re: Word for Mac. I've found that the most maddening issue on Word for Mac is the "Cited" autocomplete list. It refreshes with every keystroke and takes forever to populate on a long document. While it's populating, the "My Library" autocomplete list--which loads quickly--is unusable.

So an easy workaround (I assume) that would save me from immense frustration would be to create a preference option to disable the Cited list.

Please respond, if only to let me know that this suggestion won't fly, as I have already suggested it twice with no response from anyone:
https://forums.zotero.org/discussion/22622/inserting-citations-is-very-slow/#Item_14
https://forums.zotero.org/discussion/30293/option-to-disable-cited-items-in-word-plugin/#Item_1

Thanks.

Simon

qenghis: Can you provide a debug ID?

qenghis

D1761425397

Thank you, Simon.

Simon

qenghis: That looks like a bug. Zotero is updating the session data every time it does a search, which isn't actually necessary to produce the cited items list and shouldn't be happening. Searching the cited items is itself fast. I'll fix this.

qenghis

Great, I'm so glad that you figured it out. I look forward to the fix. Will it be anytime soon?

Simon

The fix will be available in the next version of Zotero.

nmarantz

I am in the process of consolidating a number of chapters into a single document. I need to change a few references, and Zotero standalone has become very slow. If any solutions other than breaking up the document have emerged since this thread began, please let me know. Thanks.

Debug ID: D1763696038
Mac OSX 10.9.2
Zotero Standalone
Word 2011 14.4.1

adamsmith

no, and Word for Mac is slower than other word processors, unfortunately.

velovite

"no, and Word for Mac is slower than other word processors, unfortunately."

Adamsmith: what processors are the best with Zotero? Is LibreOffice that much faster?

adamsmith

It's Word for Mac specifically. On Windows, Word and LibreOffice are awash, I believe, Word may even be a tiny bit faster. (see Simon's comment from last June above - it's a factor 17 or so between Word for Mac and Word for Windows to retrieve the citations in a document for processing).

Other advantage of LibreOffice is that it works nicely with our ODF-scan add-on http://zotero-odf-scan.github.io/zotero-odf-scan/

But there are obviously advantages to using Word, so I'm not telling anyone "just use LibreOffice."

ajstroff

I'm currently working on a document that contains less than 30 single-spaced pages and experienced a slow appearance of the zotero popup when trying to add or edit a reference.

After deleting my list of references (bibliography) at the end of the document, things got much more speedy. So consider deleting yours (if you have one already). Insert the bibliography just before document submission. I hope that helps others.

megabert

Would there be a difference between libreoffice and neo office in speed? I’m on neooffice and by now it’s become really slow (235 pages..) Deleting the Bibliography seemed to helped a bit, but it’s still slow.. just 1-2 minutes between making a change and refresh being finished. I can work on the doc in the meantime, but not on other citations.

adamsmith

no there wouldn't be. I strongly recommend not working with the full document until the very last minute. In a 200+ pages document with PhD-type referencing, what you're seeing is entirely expected, regardless of word processor.

megabert

Yea, I kind of assumed that.. Other than the speed, having it all in one document is extremely helpful for me since I jump around a lot in this phase, cross-reference, move paragraphs to an out-takes-document with the same structure etc. So if I had to split the chapters up again I think I’d lose the time elsewhere. I guess I’ll be in the very last minute for the coming month, if you will :) Very non-pc question but is Zotero slower than other ref managers? (Have never used anything else and have to emphatically add that I love Z with a passion apart form the speed at this stage of my project :) Was just wondering.) And thanks for all the help.

adamsmith

I think Endnote is faster. Mendeley would be the same. That's about all I know.
You could work with the ODF-scan markers I link to above, but you might find them too unwieldy. But obviously no lag in a document that has just markers.

megabert

I looked into the ODF-scan thing but couldn’t find how to convert what I have now to just markers - how would that work?

adamsmith

You just do the reverse of what the instructions focus on: Install ODF-scan, go to ODF/RTF Scan, then select your document and select ODF-scan (to markers).

This is risk free, it will save a copy of your document converted to markers.

megabert

Got it now - thanks, but, as you say.. too unwieldy for me. Thanks anyway. Any hope it will get faster in the future?

adamsmith

nothing I'm aware of, no.