I see, that nobody found a solution for that common problem. Just admit, that Zotero is not capable of supporting doc with more than 1000 citations. Unticking automatic updates will not solve the problem, because at the end You have to make update, and then problem comes back (progress bar in infinite mode).
I'm seriously dissappointed. I will never use this app. Now I have to make manual changes into my phd dissertations. I'm angry, but this is free software so... good luck everyone.
There are many threads here that are pretty clear about the speed limitations of Zotero with Word for Mac, but it absolutely works for large documents in LibreOffice on all platforms and in Word for Windows (which you can also use for the final edit of a work otherwise written on Mac). If that's not what you are seeing, maybe just ask for help, which you never did?
Disabling autoupdate is not good solution. Firstly, it will change the layout, because it will insert at least two lines (or more) in footnote (if You are using full-note style - like me). Secondly, refreshing with more than 1000 citations can take more than 3 hours - that was my case.
I've enabled debug panel checked what's going on. It seemed, that when I splice file into smaller sections, the process is still quite long (about 2 minutes for adding quote), but update is very quick during which the lines are changing so fast, that You can barely see what's going on. But when I opened my whole document auto-update was very long. The lines where changinging in aprox. -12 sec each at the time. That's not normal.
If this app could be still usefull for anything this bug should be fixed.
My OS is Linux Ubuntu 20.04 LTS, LibreOffice 7.6.2.1. Zotero 6.0.27, but I've also tested version 6.0.23. After downgrading the problem still occurs.
The forums are not a place to vent frustration or anger.
Not everybody shares your priorities, sometimes you're the only person experiencing a particular problem, or maybe you just didn't discover the best way to perform a task. Avoid generalizing statements like “it's obvious that without this feature, Zotero is useless for anybody”.
Zotero is used by millions of people, and most people are obviously using word processor integration without trouble. If you're having trouble with something, just report the problem politely like everyone else in these forums manages to do and we'll do our best to help. But your tone here is totally inappropriate. Please stop.
@michal.biedziuk as adamsmith above pointed out, Zotero with LibreOffice is very fast even with extremely large documents in our experience, although it will only be as fast as your computer and configuration. I believe we have had some users report performance issues with LibreOffice with some versions of Java, so that would be one way to troubleshoot the issue. After installing a different version you should make sure you select it in LibreOffice preferences.
About two weeks ago inserting or changing citations in my phd dissertation became nightmare. My work has 600 pages in A5 size, over 1200 citations, over 300 positions. Updating document can last for one hour or two. It happens in every single operation, so it doesn't matter if I quote some book for the first time, or make repeating of existing citation. I've tried changing quotation style from Chicago 16 full-note (slightly modified by myself ) - the same result.
I don't want to turn off autoupdates, because it makes mayhem in layout. Before this issue autoupdate was relatively quick (at first modification after opening file it lasted for about minute or two, but then almost immedietely).
I've made debugging and I noticed strange thing. Sometimes the lines are short, but mostly they are insanely long. Excerpt from debug:
(3)(+0000054): Integration: updateDocument complete in 3231.737s
[…]
(3)(+0000054): Integration: OpenOffice-refresh complete in 3453.125s
Why are you ignoring us and creating a new thread? We've responded to you here. This is where you need to respond. I've moved your post back into this thread, but you should actually respond to what Adomas said to you.
And you shouldn't include raw debug output here. That's what Debug IDs are for. We'll excerpt parts if they're at all relevant or unusual.
I was unable to create debug ID, because generated file was too big. I've received error few minutes after I've pressed 'Submit' button. That's why I've posted only excerpt. I've made copy in html file which is 19 MB big.
About what @Adomas said: Few weeks ago there was no problem. I've got old versions of the work from that time and the problem occurs also with them. I've also installed Java Runtime Environment 17. No improvement.
You're running Zotero 6.0.23 from March. The current version is 6.0.27. So there's no way a recent change on our end could've affected this for you. Unless you for some reason switched from some other version of Zotero to an outdated version, any slowdown would be caused by some other change on your system that you'll need to identify.
Also, though, just saying you don't want to use the feature that exists specifically for faster citing in large documents isn't going to be that compelling. You should test with automatic updates disabled and see — and say — how the performance compares. And obviously if you had turned off automatic updates previously (which Zotero prompts to do when updates are slow) and recently turned them back on, that would explain the slowdown.
I've tried different versions of Zotero with the same result, i.e. 6.0.23 - 6.0.27. I've even tried the newest 7 beta. I've also tried it in Windows 10 in virtual machine. The same result: after adding any quote or pressing 'refresh' button Zotero starts indexing everything, which in my case last for few hours.
But yesterday I've almost solved the problem, and I want to share with You my solution (I hope that empathy is not forbbiden in Your netiquette).
I've exctracted every chapter into separate file and performed update in each of it. It was relatively quick. In largest chapter in lasted about 1,5 minute, but after that it was almost instant with autoupdate. Then I've pasted each chapter into new file, and - a miracle! - it worked just like month ago. Autoupdate was almost instant (over 1200 quotations!) and the same was during adding new quote or modifing existing one.
Everything worked fine until I've added citation that was already cited, but before that place. Everything is now lagging again.
Looking at this Debug ID I see that the integration is indeed uncharacteristically slow. If you installed a different version of Java, you should make sure you have it selected in LibreOffice Options -> Advanced. It is still the most likely culprit.
Otherwise, if switching Java versions doesn't help, and you're unable to troubleshoot other reasons why your system/LibreOffice may be slow, we cannot recommend much but to split your document into multiple chapter files and work on them separately, recombining in the end, or disabling automatic citation updates.
I've installed JRE 17 and properly switch it in LibreOffice. The full parameters are:
openjdk version "17.0.8.1" 2023-08-24 OpenJDK Runtime Environment (build 17.0.8.1+1-Ubuntu-0ubuntu120.04) OpenJDK 64-Bit Server VM (build 17.0.8.1+1-Ubuntu-0ubuntu120.04, mixed mode, sharing)
Before that I have JRE 11.0.20.1. Maybe newest version is not the best? Which one is recommended for Zotero?
I don't want to split my document into chapters, because my dissertation is basically finished. I'm making just minor edits. Beside that there are many internal links inside which will not work when the document will be in parts. I also don't want to turn off autoudpates, because it makes mayhem in layout, because I'm using Chicago 16 full-note style.
Are you saying you can produce two copies of the document, one in which you can add new citations quickly and another (after adding a citation that was already cited?) where adding new citations becomes slow?
If you're willing to share those privately, please send them to support@zotero.org with a link to this thread.
When document is split into separate chapters there is no problem while working with them. When I merge those chapters again into one big file (manually, i.e. by pasting whole content of the smaller files into big file) firstly there is no problem and autoupdate works almost instantly. Problem comes back when in this big file I add citation that was already cited before the place that is was cited (e.g. first citation originally was on page 314 and I want to quote it at page 100). Then Zotero autoupdates for few hours.
You didn't answer my question. Can you produce two documents, one that's reliably slow for inserting new citations and another that isn't? And can you share those two documents?
Yes, I can create such documents. I've made copy of newly merged file which worked normal before I added old citation which was cited later in the document. Should I send those file to support@zotero.org ? Or just debug output?
If you can make a document that is fast, and then becomes slow again, do send it to support@zotero.org and include the steps which make it slow.
As for automatic citation updates, it really shouldn't do much to your layout, as it only replaces your bibliography (which is presumably at the end of your paper anyway), and only newly added citations are "wrong" and get updated upon refresh. Other citations that you do not update without automatic citation updates should not change and cause issues with your layout.
I've send email over two weeks ago (19th October), and still have no answear. In the meantime I've bought new PC. I've installed Ubuntu 22.04, newest Zotero (6.0.30). I've tried pasting smaller portions of my document into a new file and make update after each paste. At the begining updates was quite fast, but the larger document bacame the update time was growing to unreasonable rates (over 300 seconds in the middle of the document).
Update of whole document lasted for 1239 seconds (20 minutes). It lasts so long after every single change (even minor, like deleting one citation). DebugID: D1831578357
Edit2. Another update lasted for 1042 seconds. I've removed only one citation. DebugID: D1011365556
Sorry, your email got lost due to Gmail triggering some sort of security block. We run our own support email servers, but the forward to the relevant developer (me) at a gmail account lead to this issue.
I've looked and tested over your file and unfortunately I do not have good news for you. You have over 1200 citations, and your document is quite complex, and when my system is under heavier load, even LibreOffice slows down just scrolling over it. I thought maybe older versions of LibreOffice were faster and there is a recent slowdown, so I tried LibreOffice 6, but it didn't help (I could not get LO5 to run on Ubuntu 22.04, but I believe we've tested with LO6 with excellent results in the past). I also tested on Windows to check if it's some sort of Linux library issue, but it updated at about the same rate there.
I think you've simply reached the upper limits of speed for this type of document, and I imagine any other word processor, like Word, wouldn't be much faster for a document of this scope. In that case, we cannot offer much but the advice you've got before -- split your document into chapters and only recombine at the very end.
So my first hypothesis (which was highly criticize here) was correct: Zotero cannot handle such big documents. I'd used it especially because I knew, that my dissertation will be big and complex. I needed some reference system that will handle it. Now it's obvious, that Zotero is bad mean for this end.
At this stage this is "the very end" of my work, so I want to have it in one volume. Anyway, I cannot work on separated parts, because - as You can saw - there are many interior references in the document. Apart from that I'm using Chicago Manual Style full-note, so when parts are splitted the layout changes significantly (also when auto-update is disabled), because very often one citation takes two or even three lines in the footnote. I want to see the final layout at this stage. That's the reason why I used Zotero. I spent almost half a year especially for exchanging my manual citations with Zotero. Now it's clear that I wasted my time.
Is there a way to integrate Zotero datebase into built-in LO bibliography manager? I have to finish my PhD before spring.
Thanks for You effort. I know that this app is freeware, but if I'd knew, that Zotero is not capable of reaching goals for which it was created, I would have rather used some paid reference system in connection with MS Word.
I am sorry that we cannot offer much here. Just to note, the slowness is on the LibreOffice end of the equation, where we use LibreOffice provided programming API to manipulate your document. LibreOffice own bibliography management may be faster, but not necessarily.
You could also try transferring your document to Microsoft Word on Windows, that may indeed work faster with Zotero than LibreOffice, as our Microsoft Word on Windows integration is very fast, even in a VM.
Do note that while Zotero is free to use, we have spent a lot of time making our plugins as fast as possible on every platform as we understand that Zotero is a time-saving tool and a slow integration plugin can counteract any time-saving effected at other parts of writing. Having said that, while we are not aware of any comparisons of citation manager performance, given anecdotal evidence on various forums and feedback pages you'll find that users of paid software often have to deal with many technical issues, including slowness, with little technical support, so there isn't necessarily a way to pay your way out of this.
Either way, sorry for not being able to provide a fix here, and for others not taking your initial guess seriously -- we've really never seen anything like this before, and usually have managed to fix poor performance by checking Java versions and the like for LibreOffice, but you truly have a huge document. I hope using Zotero in other parts of your writing was helpful.
This may be due to you using A5 paper size. This could result in libreoffice doing a lot of work reformatting the page length after each citation addition. You could try (on a copy!) with a larger page size e.g. A4 or even A3 just to help isolate the problem.
I know that you are limited by budget and the resources you have but could this be related to the power and memory of your computer?
Folks in my shop recently completed 2 VERY large documents: one was 500 pages with about 1100 references and the other 600 pages with about 900 references. Using a Mac with 16 MB of memory, after concatenating chapters, producing a bibliography required only a few minutes and after an edit of the whole document (including the addition of one new reference and repositioning a couple of paragraphs) the creation of the bibliography required about an hour. This was using LibreOffice. The 900 reference document was in Vancouver style the 1100 reference doc used APA.
edit: actually, one machine, a (not especially new) MacBook pro had 24MB of memory. Both machines have Intel processors. Both have "combo-drives".
@DWL-SDCA thanks for the tip, but I tried refreshing the document with my fast laptop and it is about as fast as OP originally reported. I think it's a matter of many figures, tables and also using footnote citations that is the culprit. Using A5 for page size may also make it slower, causing a lot of reformatting.
1. Original version of my work was in A4 format. I've tried to make changes on that version, and it also take a lot of time.
2. I've bought completely new PC. It has AMD Ryzen 3 4-core Processor, 16GB ram, M2 ssd drive. I don't think that it is to weak. On the old PC Zotero was working with the same speed.
3. I know issues with other reference system. E.g. EndNote is not working in Linux at all. JabRef has very limited support for style. Mendeley... I don't want to speak about it...
4. Built-in LO bibliography manager is very poor. I'd rather put all references manually than use LO function.
But I have some good news.
A) I've again splited document into few parts. I've made update in each of the parts, and then paste them into the first part to recreate whole document. After making one change in one citation updating lasted only for few seconds. A miracle! I hope, that it will work in whole document, but I'm quite pessimistic.
B) I've also exchanged all my 4 charts (OLE objects in navigation pane) with png files. Document load faster and maybe it affects also Zotero plugin.
C) I've deleted all my comments. I've noticed, that during update the field 'comments' in navigation pane was constantnly blinking. I thought, that maybe Zotero is using comments in some way and my comments are making mess. You should check it. Maybe this cause the slowliness.
UPDATE 24.11.2023
After initially working normal Zotero is again updating citations very slow (500-800 seconds, or more). Editing existing qoutation was no problem, but when I added new one (no matter if it was used before or brand new) update lasted for ages.
Upgrade Storage
I'm seriously dissappointed. I will never use this app. Now I have to make manual changes into my phd dissertations. I'm angry, but this is free software so... good luck everyone.
If that's not what you are seeing, maybe just ask for help, which you never did?
Disabling autoupdate is not good solution. Firstly, it will change the layout, because it will insert at least two lines (or more) in footnote (if You are using full-note style - like me). Secondly, refreshing with more than 1000 citations can take more than 3 hours - that was my case.
I've enabled debug panel checked what's going on. It seemed, that when I splice file into smaller sections, the process is still quite long (about 2 minutes for adding quote), but update is very quick during which the lines are changing so fast, that You can barely see what's going on. But when I opened my whole document auto-update was very long. The lines where changinging in aprox. -12 sec each at the time. That's not normal.
If this app could be still usefull for anything this bug should be fixed.
My OS is Linux Ubuntu 20.04 LTS, LibreOffice 7.6.2.1. Zotero 6.0.27, but I've also tested version 6.0.23. After downgrading the problem still occurs.
OS: Ubuntu 20.04
LibreOffice 7.6.2.1
About two weeks ago inserting or changing citations in my phd dissertation became nightmare. My work has 600 pages in A5 size, over 1200 citations, over 300 positions. Updating document can last for one hour or two. It happens in every single operation, so it doesn't matter if I quote some book for the first time, or make repeating of existing citation. I've tried changing quotation style from Chicago 16 full-note (slightly modified by myself ) - the same result.
I don't want to turn off autoupdates, because it makes mayhem in layout. Before this issue autoupdate was relatively quick (at first modification after opening file it lasted for about minute or two, but then almost immedietely).
I've made debugging and I noticed strange thing. Sometimes the lines are short, but mostly they are insanely long. Excerpt from debug:
(3)(+0000054): Integration: updateDocument complete in 3231.737s
[…]
(3)(+0000054): Integration: OpenOffice-refresh complete in 3453.125s
[additional output removed — D.S.]
And you shouldn't include raw debug output here. That's what Debug IDs are for. We'll excerpt parts if they're at all relevant or unusual.
About what @Adomas said:
Few weeks ago there was no problem. I've got old versions of the work from that time and the problem occurs also with them.
I've also installed Java Runtime Environment 17. No improvement.
Also, though, just saying you don't want to use the feature that exists specifically for faster citing in large documents isn't going to be that compelling. You should test with automatic updates disabled and see — and say — how the performance compares. And obviously if you had turned off automatic updates previously (which Zotero prompts to do when updates are slow) and recently turned them back on, that would explain the slowdown.
But yesterday I've almost solved the problem, and I want to share with You my solution (I hope that empathy is not forbbiden in Your netiquette).
I've exctracted every chapter into separate file and performed update in each of it. It was relatively quick. In largest chapter in lasted about 1,5 minute, but after that it was almost instant with autoupdate. Then I've pasted each chapter into new file, and - a miracle! - it worked just like month ago. Autoupdate was almost instant (over 1200 quotations!) and the same was during adding new quote or modifing existing one.
Everything worked fine until I've added citation that was already cited, but before that place. Everything is now lagging again.
Debug ID: D1158570850
Otherwise, if switching Java versions doesn't help, and you're unable to troubleshoot other reasons why your system/LibreOffice may be slow, we cannot recommend much but to split your document into multiple chapter files and work on them separately, recombining in the end, or disabling automatic citation updates.
openjdk version "17.0.8.1" 2023-08-24
OpenJDK Runtime Environment (build 17.0.8.1+1-Ubuntu-0ubuntu120.04)
OpenJDK 64-Bit Server VM (build 17.0.8.1+1-Ubuntu-0ubuntu120.04, mixed mode, sharing)
Before that I have JRE 11.0.20.1. Maybe newest version is not the best? Which one is recommended for Zotero?
I don't want to split my document into chapters, because my dissertation is basically finished. I'm making just minor edits. Beside that there are many internal links inside which will not work when the document will be in parts. I also don't want to turn off autoudpates, because it makes mayhem in layout, because I'm using Chicago 16 full-note style.
If you're willing to share those privately, please send them to support@zotero.org with a link to this thread.
As for automatic citation updates, it really shouldn't do much to your layout, as it only replaces your bibliography (which is presumably at the end of your paper anyway), and only newly added citations are "wrong" and get updated upon refresh. Other citations that you do not update without automatic citation updates should not change and cause issues with your layout.
Update of whole document lasted for
1239 seconds (20 minutes). It lasts so long after every single change (even minor, like deleting one citation).
DebugID: D1831578357
Edit2.
Another update lasted for 1042 seconds.
I've removed only one citation.
DebugID: D1011365556
I've looked and tested over your file and unfortunately I do not have good news for you. You have over 1200 citations, and your document is quite complex, and when my system is under heavier load, even LibreOffice slows down just scrolling over it. I thought maybe older versions of LibreOffice were faster and there is a recent slowdown, so I tried LibreOffice 6, but it didn't help (I could not get LO5 to run on Ubuntu 22.04, but I believe we've tested with LO6 with excellent results in the past). I also tested on Windows to check if it's some sort of Linux library issue, but it updated at about the same rate there.
I think you've simply reached the upper limits of speed for this type of document, and I imagine any other word processor, like Word, wouldn't be much faster for a document of this scope. In that case, we cannot offer much but the advice you've got before -- split your document into chapters and only recombine at the very end.
At this stage this is "the very end" of my work, so I want to have it in one volume. Anyway, I cannot work on separated parts, because - as You can saw - there are many interior references in the document. Apart from that I'm using Chicago Manual Style full-note, so when parts are splitted the layout changes significantly (also when auto-update is disabled), because very often one citation takes two or even three lines in the footnote. I want to see the final layout at this stage. That's the reason why I used Zotero. I spent almost half a year especially for exchanging my manual citations with Zotero. Now it's clear that I wasted my time.
Is there a way to integrate Zotero datebase into built-in LO bibliography manager? I have to finish my PhD before spring.
Thanks for You effort. I know that this app is freeware, but if I'd knew, that Zotero is not capable of reaching goals for which it was created, I would have rather used some paid reference system in connection with MS Word.
You could also try transferring your document to Microsoft Word on Windows, that may indeed work faster with Zotero than LibreOffice, as our Microsoft Word on Windows integration is very fast, even in a VM.
Do note that while Zotero is free to use, we have spent a lot of time making our plugins as fast as possible on every platform as we understand that Zotero is a time-saving tool and a slow integration plugin can counteract any time-saving effected at other parts of writing. Having said that, while we are not aware of any comparisons of citation manager performance, given anecdotal evidence on various forums and feedback pages you'll find that users of paid software often have to deal with many technical issues, including slowness, with little technical support, so there isn't necessarily a way to pay your way out of this.
Either way, sorry for not being able to provide a fix here, and for others not taking your initial guess seriously -- we've really never seen anything like this before, and usually have managed to fix poor performance by checking Java versions and the like for LibreOffice, but you truly have a huge document. I hope using Zotero in other parts of your writing was helpful.
Folks in my shop recently completed 2 VERY large documents: one was 500 pages with about 1100 references and the other 600 pages with about 900 references. Using a Mac with 16 MB of memory, after concatenating chapters, producing a bibliography required only a few minutes and after an edit of the whole document (including the addition of one new reference and repositioning a couple of paragraphs) the creation of the bibliography required about an hour. This was using LibreOffice. The 900 reference document was in Vancouver style the 1100 reference doc used APA.
edit: actually, one machine, a (not especially new) MacBook pro had 24MB of memory. Both machines have Intel processors. Both have "combo-drives".
1. Original version of my work was in A4 format. I've tried to make changes on that version, and it also take a lot of time.
2. I've bought completely new PC. It has AMD Ryzen 3 4-core Processor, 16GB ram, M2 ssd drive. I don't think that it is to weak. On the old PC Zotero was working with the same speed.
3. I know issues with other reference system. E.g. EndNote is not working in Linux at all. JabRef has very limited support for style. Mendeley... I don't want to speak about it...
4. Built-in LO bibliography manager is very poor. I'd rather put all references manually than use LO function.
But I have some good news.
A) I've again splited document into few parts. I've made update in each of the parts, and then paste them into the first part to recreate whole document. After making one change in one citation updating lasted only for few seconds. A miracle! I hope, that it will work in whole document, but I'm quite pessimistic.
B) I've also exchanged all my 4 charts (OLE objects in navigation pane) with png files. Document load faster and maybe it affects also Zotero plugin.
C) I've deleted all my comments. I've noticed, that during update the field 'comments' in navigation pane was constantnly blinking. I thought, that maybe Zotero is using comments in some way and my comments are making mess. You should check it. Maybe this cause the slowliness.
UPDATE 24.11.2023
After initially working normal Zotero is again updating citations very slow (500-800 seconds, or more). Editing existing qoutation was no problem, but when I added new one (no matter if it was used before or brand new) update lasted for ages.