Upgrade StorageBetter BibTeX 5.2.x for large libraries
I'll not make a habit out of doing advertisements here, but if you have a large library and experienced Zotero lockups during BBT (auto) exports, I'd love to hear about your experiences with BBT 5.2.x.
Exports now happen on a background thread ("worker"); they won't be any faster (in fact the workers currently don't use the cache so they are probably slower), but they shouldn't interfere with the Zotero UI anymore. This should in principle make auto exports feasible even for the packrats among us with 30k+ items (you know who you are). I've tested with 24k items set up for auto export and it's insane -- auto-exports spike one core for me for a minute and a half, but Zotero itself behaves as if it sat idle.
Pretty happy with this one. I'm pondering whether to add caching to the worker (which should make it faster at the cost of more memory use) or just scratch the caching system entirely (which would make BBT more robust -- as it is said, "there are two hard things in computer science: cache invalidation, naming things, and off-by-one errors").
This does *not* work for drag-and-drop, so for those, the pre-5.2 behavior is active. Don't drag-and-drop 24k items as bibtex (maniacs).
Exports now happen on a background thread ("worker"); they won't be any faster (in fact the workers currently don't use the cache so they are probably slower), but they shouldn't interfere with the Zotero UI anymore. This should in principle make auto exports feasible even for the packrats among us with 30k+ items (you know who you are). I've tested with 24k items set up for auto export and it's insane -- auto-exports spike one core for me for a minute and a half, but Zotero itself behaves as if it sat idle.
Pretty happy with this one. I'm pondering whether to add caching to the worker (which should make it faster at the cost of more memory use) or just scratch the caching system entirely (which would make BBT more robust -- as it is said, "there are two hard things in computer science: cache invalidation, naming things, and off-by-one errors").
This does *not* work for drag-and-drop, so for those, the pre-5.2 behavior is active. Don't drag-and-drop 24k items as bibtex (maniacs).
With a hot cache though that takes 15 seconds. Damn.
5.2.7 adds caching for workers, and the 24k export now takes 11-15 secs with a hot cache, running up to 35-40 secs if I start 4 of those in parallel. And heating up the cache is tolerable now for those numbers -- I used to heat the cache by exporting it first in small slices of the whole, so the UI would get its time and then repeated the full export from the hot cache, that is now no longer necessary, as a worker on a cold cache uses about the same (AFAICT) as foreground export, and now it only needs to be done once, while Zotero remains usable.