Zotero is slow
Hi,
Zotero is my favorite application. The developers have demonstrated a tremendous vision. I have been using it as database for all of references as well as my web browsing.
However, Zotero has one huge problem: it is painfully slow. I typically have to wait 15-20 seconds for Zotero to add or delete an entry. (If I accidentally enter anything into the auto search box, Firefox hangs for 30 seconds.) My database is only of modest size ( ~3000 entries).
My problem is not unique. I have installed it on several machines, and started with fresh databases. There are numerous other posts complaining of the same problem. Here are just a few:
http://forums.zotero.org/discussion/13822/slowtero/
http://forums.zotero.org/discussion/2218/
http://forums.zotero.org/discussion/2314/
http://forums.zotero.org/discussion/14080/too-slow/
http://forums.zotero.org/discussion/7728/way-too-slow/
The developers have posted some responses that the 3.0 version of firefox would improve the speed issue. Firefox is now at 3.6 and the issue remains. Even the new improvements to the Javascript engine for firefox 4.0 would not improve the speed sufficiently to reduce a 30 second delay to something acceptable.
Simply put, Zotero has the potential to be a superb aggregator of information, but the current implementation is crippling.
It is frustrating that the developers will not engage the community in a frank discussion of the speed issue. Is the underlying database too slow? (Sqlite?) Dan Stillman mentioned something a couple of years ago about dom and javascript being the fundamental problem? Does the entire application need to rewritten from scratch? Smallish database of several thousand entries should be much faster.
Zotero is my favorite application. The developers have demonstrated a tremendous vision. I have been using it as database for all of references as well as my web browsing.
However, Zotero has one huge problem: it is painfully slow. I typically have to wait 15-20 seconds for Zotero to add or delete an entry. (If I accidentally enter anything into the auto search box, Firefox hangs for 30 seconds.) My database is only of modest size ( ~3000 entries).
My problem is not unique. I have installed it on several machines, and started with fresh databases. There are numerous other posts complaining of the same problem. Here are just a few:
http://forums.zotero.org/discussion/13822/slowtero/
http://forums.zotero.org/discussion/2218/
http://forums.zotero.org/discussion/2314/
http://forums.zotero.org/discussion/14080/too-slow/
http://forums.zotero.org/discussion/7728/way-too-slow/
The developers have posted some responses that the 3.0 version of firefox would improve the speed issue. Firefox is now at 3.6 and the issue remains. Even the new improvements to the Javascript engine for firefox 4.0 would not improve the speed sufficiently to reduce a 30 second delay to something acceptable.
Simply put, Zotero has the potential to be a superb aggregator of information, but the current implementation is crippling.
It is frustrating that the developers will not engage the community in a frank discussion of the speed issue. Is the underlying database too slow? (Sqlite?) Dan Stillman mentioned something a couple of years ago about dom and javascript being the fundamental problem? Does the entire application need to rewritten from scratch? Smallish database of several thousand entries should be much faster.
We're always trying to speed up Zotero, and we hope to make significant performance improvements in future versions, but there can be completely different and unrelated reasons for performance issues in separate components. And beyond that, different people have different experiences based on the particularities of their databases, systems, and usage patterns. Actually, as those threads show, we've responded in some depth on quite a few performance issues, but they mostly don't have anything to do with one another. So there's no single "speed issue" to discuss.
But if you'd like us to look into problems you're experiencing personally, we'd be happy to help.
So:
1) What are your full-text indexing stats in the Search pane of the Zotero prefs?
2) Is there a speed difference for adding items between the tag selector in the bottom left being open or closed?
3) Provide a Debug ID for adding an item, and another (after clearing the output) for deleting.
Fair enough. It may be particular to my usage.
Here are the indexing stats:
Indexed: 5707
Partial: 15
Unindexed: 965
Words: 345240
The speed problem existed (at least) < 3000 indexed entries.
Anecdotally, it seems slower when the tag selector box is shown. (There is no longer a tag selector box icon. I use the grippy to show or hide the tag selector box.)
I'll upload debug info shortly.
unfortunately I have the same problem. It now takes me about 20-30 seconds to add an entry.
Might it have something to do with the size of our libraries? My library now contains about 1.5 GB of pdf-files (around 900), and the zotero.sqlite file is 61 MB. And if I understand the programm right, this is the index and has to be rewritten everytime I add a new file.
Nevertheless, I still love the programm and keep advertizing it to everyone I meet since I just successfully finished my PhD thesis with Zotero without any major problems and it has saved me soooo much time, so thanks to evereyone who made this possible.
Thanks.
As I've noted in other threads related to this, we're planning to switch to SQLite's full-text indexing system (which wasn't available in Firefox when we first added full-text indexing to Zotero), which should alleviate this problem. That will happen for either 2.1 or the following major release.
Ok - that seems to work. To test, I added three or four references with and without indexing. The import time drops from ~20 seconds to ~2 seconds. Is the underlying problem the full text index size (~345,000)? I have several large PDFs in the library.
It's great to hear about the new SQLite full text indexing. For now, can I just disable full text indexing and manually rebuild the index every week or so? (It might be useful to include a caveat about speed repercussions on the indexing page until the new indexing is incorporated. I searched the forums several times but somehow missed this.)
Thanks very much. I had been dealing with this the last couple years and had tried installing Zotero on several computers, disabling other extensions and so forth. Frustratingly, it kept reappearing.
Zotero is a great piece of software. I'll keep singing its praises.
I have similar experiences. When it works as intended Zotero is easily the best reference manager I've used. I've found it slow on two kinds of occasions. When I start Zotero the dialogue box offering to stop the script or continue it appears many times. I can send you a debug file if you want to give me an address to send it to but it's rather big to paste here. Then, once Zotero is started, when I try to add an item to the data-base the GUI appears to hang although the debug konsole shows lots of activity taking place behind it. I can send this debug file too but in brief it contains hundreds of lines like this:
[unrelated output removed — D.S.]
The start-up delay is only once per session but the add-new-record delay makes a big difference to the usability of Zotero.
Preferences > Sync
Uncheck the buttons 'Sync automatically' and 'Sync attachment files in group libraries using Zotero storage'.
Normal service is then restored (at least, for the local PC). The problem seems to have been the size of data - too much to synchronise on-line. For the time being I'll have to synchronise using files on usb-sticks.
this is particular when, for example, I do a search, find the reference i want, and then drag it into a folder. at that point I get a long hangup - well, long or indefinite.
any suggestions? or do I need to start a debug number?
By comparison, Mendeley handles my database with a breeze and syncing works just fine. I am in the process of looking for alternatives to Endnote, but Zotero does not work for me right now.
Please let me know whether I can provide you with info that can help you addressing these problems.
Disable auto-sync in the Sync pane of the Zotero preferences to make sure you're not just seeing repeated failed syncs (though "syncing does not work" is not a valid bug report).
Opening up Zotero takes 30 secs during which Firefox is frozen and the rest of the computer slowed down due to excessive CPU usage. The same happens when I switch between the Trash and my database.
Btw, I just tried to insert a citation into a doc in Libre Office (3.3.0) and I got first the error message "Action not supported - Invalid procedure call", and then after repeated tries "Open Office could not communicate with Zotero - please check whether Firefox is open". Of course Firefox and Zotero were open.
But opening the Zotero pane will also trigger an auto-sync. Have you disabled auto-sync in the Sync pane of the Zotero preferences to make sure the slowness you're experiencing isn't related to syncing? If it is, start a new thread.
Switching between collections within Zotero should not be slow unless you have either a massive library (are you including child items in your item count?) or a slow computer. But as stated above, we'd need a Debug ID for a particular operation (clearing output immediately before) to debug it. Start new threads for unrelated issues.
The 9000 items count refers to articles/book chapters and hence does not include child items.
I always have Firefox open, so the way Zotero is intended to work should be perfect for me.
And yes, I had already disabled the sync function.
My full item count is about 18000 (I don't now whether the exact number can be displayed).
Anyway I cleared the index and rebuilt it to include fewer pdf pages (15 instead of the default 100). It seems dramatically improved.
Is it possible for the index to become corrupted?
This time I cleared the index before I rebuilt it, which took a while but worked. Last time I was problem-solving re: slow searches but I couldn't get the index to rebuild. I ended up clicking the 'reindex item' button on a bunch of the individual files by hand, which helped. When I initialized a full rebuild, Firefox would go off on its own and the process just wouldn't finish. I left it overnight a couple of times to no avail.
What's the latest report on whether an improved indexing subsystem will come out soon?
Hi guys,
I've got some similar speed issues. I'm running Zotero 2.1.8 in Firefox 4 on Windows 7. My database only has around 2000 entries, many of them with one or more attachments.
I already turned of sync and auto-index, as someone suggested in some previous discussion. That solved part of the problem, although it's a shame that I had to turn off all those nifty advanced features. Three problems remain:
- The quick search still bothers me. I know, you can stop the instant search by starting each query with ", but if you're in the middle of an urgent paper, you don't always remember that. Wouldn't it be possible to create a toggle for instant search so that people who want to can just search on enter?
- The trick with the double quotes does not appear to work in the OpenOffice plugin, so when you try to insert a quotation, you still get a delay. Is there anything I'm overlooking?
- Sometimes, especially when adding new entries or attachments or when changing the value of certain fields (often the author fields, it seems), Zotero just hangs for 5~20 seconds. If your going through 20 or 30 citations and you're on a tight schedule, that can get a bit annoying after a while.
I agree with Ragan and some of the others that these speed issues start to work on your nerves after a while, especially if you use Zotero for heavy-duty tasks (writing a thesis, doing intensive research). And frankly, for some people it might actually be a dealbreaker.Dan, I know it must be annoying for you to get all these complaints while you're working your ass of to give everyone free software, so I understand you react a bit cranky once in a while. However, I think it would be a bad idea to dismiss them as coming from a whining minority (as you did above). Lots of newbies might bump into these speed issues when they first try Z and then just give up.
As for your specific points - the first two are being worked on - I'd recommend completely ignoring the quick search fields in both Zotero and Ooo plugin until then - I assume you usually know the author or title of the reference you want to cite, so you can just sort by that column and type the first couple of letters.
Your third point is too unspecific to be useful - attaching files can take a little longer because of indexing, but changing field content really shouldn't, if you can provide steps to reproduce that reliably that would be helpful.
Sometimes, especially when adding new entries or attachments or when changing the value of certain fields (often the author fields, it seems), Zotero just hangs for 5~20 seconds. If your going through 20 or 30 citations and you're on a tight schedule, that can get a bit annoying after a while.
In my case though it's often longer - up to 2 minutes or so in some cases.
I've reproduced an example at debug ID D2115085388.
Running - Zotero 2.18 in Firefox 5, Windows 7 Home Premium SP1 fully patched, integrated with MS Word 2007 (fully patched). Intel Core 2 Quad Q6600 @2.4ghz, 3GB RAM.
1248 parent items, 2619 total items in the library - many large PDFs.
I'd very much appreciate any thoughts / comments.
Thanks in advance.
If this happens with all extensions disabled, upload your database to the DB Repair Tool and e-mail the Upload ID to support@zot....org with a link to this thread. Don't post the Upload ID publicly. Also don't worry about the download link the tool provides, at least until we take a look to see if it would help.
@Dan I'll try that and revert - many thanks.