On my work computer, Zotero 5 Beta is significantly slow. I am seeing delays in populating the rightmost panel when I click an item and significant delay when I start to type a tag name. With 4.0.29.x, the tag matches happen instantly and as I type. With 5.0 beta, I have typed the entire first word and it is still matching tags related to the first alphabet. A full match can take up to 30 seconds.
@dstillman Overall, large variability in speed. Operations are not consistently fast or slow. For example, time to update the middle pane with contents of a RSS feed could vary from 2 seconds to 10 seconds. I get a feeling that the speed is either sensitive to past events or the order of events or both.
Time to load library (~2300 items) at startup: 4.0.29.x: About 5-7 seconds 5.0 Beta: About 10-15 seconds, but gets down to the 5-7 second range after multiple starts
----
Debug ID for item selection delay: D1533732054 (~ 6 seconds) Debug ID for tag addition delay: D2018505036 (over 50 seconds)
----
Just FYI (I should probably create another thread for this issue) Hit tag delete button but it would do nothing. Clicked on another item and then returned to the previous one and this time the tag is gone. Tried re-adding the deleted tag and it won't show up as an option when I start typing, hinting that the tag is still associated with the item? Probably same behavior when I add a tag - need to toggle item selection to see changes to the tags.
@dstillman Just FYI, I am still seeing a large variability in speed and at some point Zotero will crash requiring the program be closed and restarted . For example, when I clicked on "Unindexed Files" it took about 140 seconds for the items to be displayed in the center pane. If I close and restart Zotero, it opened in the same selection but took only 7 seconds to display the items. I then clicked on various saved searches and collection, and then when I came back to "Duplicate Items", it crashed.
Report ID: 798451416
Zotero 5.0-beta.180+de7b56b on Windows 7 Enterprise 64-bit.
The debug output shows high activity in full-text indexing. @dstillman will need to look into this, but as a temporary non-solution you can try disabling it by setting the number of characters to index to 0 in Preferences -> Search and seeing if that helps.
Could we see another Debug ID while it's being slow? Your latest Report ID does suggest that this is still related to full-text indexing, but that was also the case when you posted in March. It might be expected to be a little slow shortly after a sync of many new items (though Zotero should only index in the background while you're not using it), but after Zotero has had time to index those it shouldn't really affect things.
What are your index stats in the Search pane of the prefs?
Note that my indexing stats have been the same for a while because the un-indexed files are non-OCRed PDFs. Once I have added new PDFs, I change my page setting, rebuild the index to include non-indexed items, and them change the page setting back to zero.
Once I have added new PDFs, I change my page setting, rebuild the index to include non-indexed items, and them change the page setting back to zero.
Why do you do that?
The delays you're seeing are in processing of synced full-text content, which are independent of the Search settings — if you have full-text syncing enabled, downloaded content will be processed in the background. But the delays you're seeing for that aren't normal or expected.
Because in the past I noticed Zotero would slow down when I attach or import a PDF. And I tend to bulk attach, so I want to just complete my operations quickly and let the indexing happen when I am away from my desk.
I am surprised the 5.0 Beta is still syncing full-text content. I haven't made any substantial changes to my full-text content and indexing in few weeks now. So, it should be fully synced with only very small incremental changes.
I have unchecked the "Sync full-text content" setting to see if that makes a difference.
BTW, am I right in thinking that if I set pages to index to zero, I am effectively preventing any indexing operations?
I'm still looking at this, but I've fixed an issue in the latest beta that was preventing downloaded full-text content from being properly processed.
BTW, am I right in thinking that if I set pages to index to zero, I am effectively preventing any indexing operations?
Full-text content that it pulls down will be indexed, regardless of the length settings (because they control how much is extracted from the original file, and that already happened on the other computer).
When I tested last week, I had the full-text sync turned off and it seemed to have made a difference in speed and crash errors. I'll be back in office next week and will test the latest beta.
@dstillman The 193 beta looks much faster. The only place where it seemed slow (but still faster than past betas) was "unindexed files". I will continue to test and report back.
My settings: Full-text sync is off. Maximum characters and pages to index, per file, are set to zero.
I'm using Zotero 5.0.7 and I having the same problem. When I open Zotero, it take about 4 seconds to be read. This time is to show the middle panel. I have about 950 references and I do not have this problem with Zotero 4. This is some parts of my debug output:
version => 5.0.7, platform => Linux x86_64, oscpu => Linux x86_64, locale => en-US, appName => Zotero, appVersion => 5.0.7, extensions => ZotFile (5.0.3, extension), Zotero LibreOffice Integration (5.0.6.SA.5.0.7, extension)
(3)(+0000002): Loaded collection relations in 3 ms
(3)(+0000009): Loaded item relations in 10 ms
(3)(+0000034): Loaded primaryData in My Library in 34 ms
(3)(+0000015): Loaded childCollections in My Library in 16 ms
(3)(+0000028): Loaded childItems in My Library in 28 ms
(3)(+0000009): Loaded relations in My Library in 10 ms
(3)(+0000000): Loaded collections in My Library in 91 ms
(3)(+0000003): Loaded primaryData in My Library in 4 ms
(3)(+0000001): Loaded conditions in My Library in 2 ms
(3)(+0000001): Loaded searches in My Library in 7 ms
(3)(+0000389): Loaded primaryData in My Library in 393 ms
(3)(+0000334): Loaded creators in My Library in 336 ms
(3)(+0000057): Loaded itemData in My Library in 857 ms
(3)(+0000000): Loading note for items in My Library
(3)(+0000016): Loaded note in My Library in 47 ms
(3)(+0000001): Purged data tables in 1794 ms
(3)(+0000022): Loaded childItems in My Library in 66 ms
(3)(+0000078): Loaded tags in My Library in 79 ms
(3)(+0000049): Loaded collections in My Library in 49 ms
(3)(+0000058): Loaded relations in My Library in 59 ms
(3)(+0000000): Loaded items in My Library in 1890 ms
(3)(+0000072): Sorted 1235 items in 74 ms
(3)(+0000013): Refreshed open parents in 13 ms
(3)(+0000012): Set tree for items view tkHhZKkN in 158 ms
(3)(+0000000): User interface ready in 3521 ms
(3)(+0000802): Cached 515 translators in 1540 ms
(3)(+0000005): Cached 22 styles in 97 ms
I do not know if it is related with unindexed items. As I said, I do not have problem with Z4. I have only 9 items unindexed and I can not find it. I tried enable debug, rebuild unindexed items e check debug output, but it not shown witch item is unindexed. This is debug output:
(3)(+1219789): Unverifying storage
(3)(+0000018): Username not set before getting Zotero.Sync.Storage.WebDAV.password
(4)(+0011442): SELECT itemID FROM itemAttachments WHERE linkMode!=3 AND itemID NOT IN (SELECT itemID FROM fulltextItems WHERE indexedChars IS NOT NULL OR indexedPages IS NOT NULL)
(4)(+0000012): Beginning DB transaction 3PVHdBbx
(4)(+0000023): DELETE FROM fulltextItemWords WHERE itemID IN (SELECT itemID FROM itemAttachments WHERE linkMode!=3 AND itemID NOT IN (SELECT itemID FROM fulltextItems WHERE indexedChars IS NOT NULL OR indexedPages IS NOT NULL))
(4)(+0000005): DELETE FROM fulltextItems WHERE itemID IN (SELECT itemID FROM itemAttachments WHERE linkMode!=3 AND itemID NOT IN (SELECT itemID FROM fulltextItems WHERE indexedChars IS NOT NULL OR indexedPages IS NOT NULL))
(4)(+0000006): Committed DB transaction 3PVHdBbx
(4)(+0000007): SELECT COUNT(*) FROM fulltextItems WHERE (indexedPages IS NOT NULL AND indexedPages=totalPages) OR (indexedChars IS NOT NULL AND indexedChars=totalChars)
(4)(+0000006): SELECT COUNT(*) FROM fulltextItems WHERE (indexedPages IS NOT NULL AND indexedPages<totalPages) OR (indexedChars IS NOT NULL AND indexedChars<totalChars)
(4)(+0000004): SELECT COUNT(*) FROM itemAttachments WHERE itemID NOT IN (SELECT itemID FROM fulltextItems WHERE indexedPages IS NOT NULL OR indexedChars IS NOT NULL)
Upgrade Storage
Overall, large variability in speed. Operations are not consistently fast or slow. For example, time to update the middle pane with contents of a RSS feed could vary from 2 seconds to 10 seconds. I get a feeling that the speed is either sensitive to past events or the order of events or both.
Time to load library (~2300 items) at startup:
4.0.29.x: About 5-7 seconds
5.0 Beta: About 10-15 seconds, but gets down to the 5-7 second range after multiple starts
----
Debug ID for item selection delay: D1533732054 (~ 6 seconds)
Debug ID for tag addition delay: D2018505036 (over 50 seconds)
----
Just FYI (I should probably create another thread for this issue)
Hit tag delete button but it would do nothing. Clicked on another item and then returned to the previous one and this time the tag is gone. Tried re-adding the deleted tag and it won't show up as an option when I start typing, hinting that the tag is still associated with the item? Probably same behavior when I add a tag - need to toggle item selection to see changes to the tags.
Report ID: 798451416
Zotero 5.0-beta.180+de7b56b on Windows 7 Enterprise 64-bit.
What are your index stats in the Search pane of the prefs?
I thought I had auto-indexing disabled by setting my pages to index to zero. Is that not the case?
Screenshot of my indexing settings: https://1drv.ms/i/s!Atcr8aCyjBrulLQ6Xlu7ObxxIqdlEQ
Note that my indexing stats have been the same for a while because the un-indexed files are non-OCRed PDFs. Once I have added new PDFs, I change my page setting, rebuild the index to include non-indexed items, and them change the page setting back to zero.
The delays you're seeing are in processing of synced full-text content, which are independent of the Search settings — if you have full-text syncing enabled, downloaded content will be processed in the background. But the delays you're seeing for that aren't normal or expected.
I am surprised the 5.0 Beta is still syncing full-text content. I haven't made any substantial changes to my full-text content and indexing in few weeks now. So, it should be fully synced with only very small incremental changes.
I have unchecked the "Sync full-text content" setting to see if that makes a difference.
BTW, am I right in thinking that if I set pages to index to zero, I am effectively preventing any indexing operations?
My settings: Full-text sync is off. Maximum characters and pages to index, per file, are set to zero.
I'm using Zotero 5.0.7 and I having the same problem. When I open Zotero, it take about 4 seconds to be read. This time is to show the middle panel. I have about 950 references and I do not have this problem with Zotero 4. This is some parts of my debug output:
version => 5.0.7, platform => Linux x86_64, oscpu => Linux x86_64, locale => en-US, appName => Zotero, appVersion => 5.0.7, extensions => ZotFile (5.0.3, extension), Zotero LibreOffice Integration (5.0.6.SA.5.0.7, extension)
(3)(+0000002): Loaded collection relations in 3 ms
(3)(+0000009): Loaded item relations in 10 ms
(3)(+0000034): Loaded primaryData in My Library in 34 ms
(3)(+0000015): Loaded childCollections in My Library in 16 ms
(3)(+0000028): Loaded childItems in My Library in 28 ms
(3)(+0000009): Loaded relations in My Library in 10 ms
(3)(+0000000): Loaded collections in My Library in 91 ms
(3)(+0000003): Loaded primaryData in My Library in 4 ms
(3)(+0000001): Loaded conditions in My Library in 2 ms
(3)(+0000001): Loaded searches in My Library in 7 ms
(3)(+0000389): Loaded primaryData in My Library in 393 ms
(3)(+0000334): Loaded creators in My Library in 336 ms
(3)(+0000057): Loaded itemData in My Library in 857 ms
(3)(+0000000): Loading note for items in My Library
(3)(+0000016): Loaded note in My Library in 47 ms
(3)(+0000001): Purged data tables in 1794 ms
(3)(+0000022): Loaded childItems in My Library in 66 ms
(3)(+0000078): Loaded tags in My Library in 79 ms
(3)(+0000049): Loaded collections in My Library in 49 ms
(3)(+0000058): Loaded relations in My Library in 59 ms
(3)(+0000000): Loaded items in My Library in 1890 ms
(3)(+0000072): Sorted 1235 items in 74 ms
(3)(+0000013): Refreshed open parents in 13 ms
(3)(+0000012): Set tree for items view tkHhZKkN in 158 ms
(3)(+0000000): User interface ready in 3521 ms
(3)(+0000802): Cached 515 translators in 1540 ms
(3)(+0000005): Cached 22 styles in 97 ms
I do not know if it is related with unindexed items. As I said, I do not have problem with Z4. I have only 9 items unindexed and I can not find it. I tried enable debug, rebuild unindexed items e check debug output, but it not shown witch item is unindexed. This is debug output:
(3)(+1219789): Unverifying storage
(3)(+0000018): Username not set before getting Zotero.Sync.Storage.WebDAV.password
(4)(+0011442): SELECT itemID FROM itemAttachments WHERE linkMode!=3 AND itemID NOT IN (SELECT itemID FROM fulltextItems WHERE indexedChars IS NOT NULL OR indexedPages IS NOT NULL)
(4)(+0000012): Beginning DB transaction 3PVHdBbx
(4)(+0000023): DELETE FROM fulltextItemWords WHERE itemID IN (SELECT itemID FROM itemAttachments WHERE linkMode!=3 AND itemID NOT IN (SELECT itemID FROM fulltextItems WHERE indexedChars IS NOT NULL OR indexedPages IS NOT NULL))
(4)(+0000005): DELETE FROM fulltextItems WHERE itemID IN (SELECT itemID FROM itemAttachments WHERE linkMode!=3 AND itemID NOT IN (SELECT itemID FROM fulltextItems WHERE indexedChars IS NOT NULL OR indexedPages IS NOT NULL))
(4)(+0000006): Committed DB transaction 3PVHdBbx
(4)(+0000007): SELECT COUNT(*) FROM fulltextItems WHERE (indexedPages IS NOT NULL AND indexedPages=totalPages) OR (indexedChars IS NOT NULL AND indexedChars=totalChars)
(4)(+0000006): SELECT COUNT(*) FROM fulltextItems WHERE (indexedPages IS NOT NULL AND indexedPages<totalPages) OR (indexedChars IS NOT NULL AND indexedChars<totalChars)
(4)(+0000004): SELECT COUNT(*) FROM itemAttachments WHERE itemID NOT IN (SELECT itemID FROM fulltextItems WHERE indexedPages IS NOT NULL OR indexedChars IS NOT NULL)
(4)(+0000004): SELECT COUNT(*) FROM fulltextWords
Is this load time normal?
All the best!
Mario