Huge decrease of zotero.sqlite file size. Something wrong?

cadudesun
Hi,

A couple of days ago my Zotero library had:
4200 parent items, 13GB total size, 350MB zotero.sqlite

Now:
4150 parent items, 11.5GB total size, 43MB zotero.sqlite (I am still having a 350MB zotero.sqlite.1.bak)

I am concerned with this huge decrease of zotero.sqlite file size. I did some operations in the last days and I am wondering if it would be expected the reduction in size, or if something can be going wrong. I would appreciate help to understand what is going on.

Operations I did:
- Delete around 50 duplicated items
- Delete around 150-200 snapshots (that type which come automatically when importing literature from sites as Ebsco, Jstor, ScienceDirect, Googgle Books, etc.)
- PDF indexing (maximum characters and pages) set to 0 + clear index operation (before I had thousands of PDFs indexed and I thought that it could be causing performance deterioration).
- Sync: reset - restore to Zotero Server - in order to clean my on-line items, substituting them by local files.

Thanks in advance for any clarification,
Cadu
  • dstillman
    - PDF indexing (maximum characters and pages) set to 0 + clear index operation (before I had thousands of PDFs indexed and I thought that it could be causing performance deterioration).
    It was that, most likely. The full-text index is what makes the database large. Improvements to full-text indexing are planned for Zotero 4.1 or 4.2 that should improve performance and possibly decrease database size.
  • wepierce
    Hello all.

    I see this is from a while ago...

    But has anyone else cleared the index of a LARGE sqlite datebase (500MB+)? How long did that process take? 10 mins? An hour? I'm wondering if I should force quit and start again or if I'd risk corrupting my database...

    Any help'd be great!

    Will
  • dstillman
    I would strongly discourage killing it. How long has it been?
  • wepierce
    Maybe 20 mins?

    TOP reports that it's using CPU, but "stuck". Which could just mean "doing its job", right?
  • dstillman
    Yeah, I'd give it more time.
  • wepierce
    edited June 11, 2015
    Thanks. I'm being patient.

    The fact that the "Bytes Read" and "Bytes Written" under disk usage in Activity Monitor for Zotero are changing is a good sign I think!

    :)
  • wepierce
    Success! Hallelujah!

    It took about 3.5 hours to clear an index of ~3000 items, ~2 million words, 500MB+ sqlite database.
  • dstillman
    OK, great. (The current full-text search system doesn't scale well and is being replaced.)
Sign In or Register to comment.