Slow tag loading in Zotero 5.0

dstillman

This discussion was created from comments split from: [5.0 Beta] Slow performance.

aka.sps

Hi,

I'm using Zotero 5.0.7 also and I having the same problem. Full library list view take about 1 minute.
Of course, I have about 30000 references but I do not have this problem with Zotero 4.

The Debug ID is D77483814.

IMHO problem is here:

(4)(+0000001): SELECT DISTINCT name AS tag, type FROM itemTags JOIN tags USING (tagID) WHERE itemID IN ([...41414])
(3)(+0051689): Refreshing tags selector

Possible, It is due to "pass ids directly to SQLite" (commit a80f1309974e5e36e9d7bd96c5e54414e0961282).

Typical following SQL request (please, add all need indexes for FOREIGN KEY REFERENCES fields) take about 80 ms. "pass ids directly" - 51689 ms.

WITH Q AS (SELECT itemID FROM items WHERE libraryID=1)
SELECT DISTINCT name AS tag, type FROM itemTags JOIN tags USING (tagID) WHERE itemID IN (SELECT itemID FROM Q)

BR
Pavel

bwiernik

Initial startup in Zotero 5 is somewhat slower than in Zotero 4, but you should find that other actions after are consistently much faster. The developers are working to try to speed up the initial loading process.

You might find it helpful to leave Zotero open in the background, rather than loading it each time you want to use it.

aka.sps

Hi,

You can see in log (Debug ID is D77483814) that each navigation to collection with big number of items take significant time.

Navigation to root of library take 1 minute. Each time.
Note: It is impossible do not show full list. For example double click on item of "Advanced search" results try to show item in full list, etc.

Of course, I had enough patience to wait "initial loading" but my patience is not enough to wait for this every time.

From your answer, I did not understand whether the bug will be fixed or I need to restore version 4 from the backup copy?

Thank you,

Pavel

bwiernik

I will let @dstillman take a look at the logs and see if there is something unusual happening with your system. After a library loads the first time, navigating between collections and other views should be very fast.

aka.sps

Debug ID D1648264797
Problem is not fixed. Too slow navigation.

(4)(+0000001): SELECT DISTINCT name AS tag, type FROM itemTags JOIN tags USING (tagID) WHERE itemID IN ([...44904])
(3)(+0067273): Refreshing tags selector

Possible, it is a result of old @dstillman commit a80f1309974e5e36e9d7bd96c5e54414e0961282 "Avoid temporary table when getting tags for current view".

It is a critical issue. Zotero 5 - almost unresponsive.

github issue#1290 (https://github.com/zotero/zotero/issues/1290) - w/o actions.

Last time I had to restore Zotero 4 from the backup.
At least this time, can I hope for support?

aka.sps

Fix:

CREATE INDEX IF NOT EXISTS "itemTags_tagID_type_idx" ON "itemTags" ("tagID" ASC, "type" ASC);

Adding the index significantly reduces the query execution time (~200 times)

(4)(+0000001): SELECT DISTINCT name AS tag, type FROM itemTags JOIN tags USING (tagID) WHERE itemID IN ([...45064])
(3)(+0000274): Refreshing tags selector

dstillman

That's not a fix, and it's not something people should run, but the latest 5.0 Beta should fix this for anyone experiencing it. More details on the GitHub issue.