Problem with full-text sync

jpintar
I've recently moved my library from Mendeley to Zotero, and I'm experiencing some syncing problems. While the metadata and PDF attachments are syncing fully across my Windows 10 machines, the full-text index is not.

The "sync full-text content" option is checked on all machines. Manually triggered sync completes without any reported errors on all desktop clients, but the zotero.sqlite files remain of very different size, and the Index Statistics report different numbers of indexed files. I've tried clearing and rebuilding the index on the primary machine (followed by a re-sync), but this did not help.

Given that no errors are reported, I'm at a loss for what to do, apart from letting the index be rebuilt separately on each machine, which is not ideal. I suspect the size of my library may be contributing to the issue (it contains over 14,000 PDFs), but I have no direct evidence of this.

Does anyone have suggestions for how to fix the problem?
  • jpintar
    This is with Zotero 6.0.2, by the way.
  • dstillman
    How recently did you sync? Indexing happens in the background when Zotero is idle and can take a while.

    Also make sure you're actually using Everything mode to search in the toolbar.
  • jpintar
    The last thing I tried was to clear and rebuild the index on one computer, which took about ten hours, and to prompt a sync after the re-indexing was completed. I confirmed that the re-indexing was in fact done by looking at the number of indexed files under Index Statistics and checking that Zotero was no longer writing anything to the hard drive over a sustained period. I've since re-started Zotero and re-synced on all machines several times, with no change and still no errors logged.

    I am searching using the Everything mode, and getting a different number of results on the different machines, in addition to the Index Statistics showing a different number of indexed files.
  • dstillman
    edited March 23, 2022
    The primary purpose of "Sync full-text content" is to sync full-text content between devices even when using "as needed" file sync mode and files aren't available locally. If you cleared and rebuilt the index, that's no longer relevant, and the index would just be based on the files you have locally. You can check the indexing state of a given item from the right-hand pane after clicking on an attachment.

    Generally, you should completely ignore the Search pane of the prefs — messing with it or worrying about the stats mostly just causes problems, and it will likely be removed in a future version.
  • dstillman
    edited March 23, 2022
    Rebuilding the index on one computer also causes Zotero to try to resync the full-text content from that computer to all other computers, which, again, happens after a period of inactivity and can take a while. Don't do it.
  • jpintar
    My concern is precisely that the full-text content doesn't seem to have re-synced, leading to different full-text search results across the computers (on the machine on which the index was rebuilt, many more results are returned).

    When I enable logging and view the output, I come across the following on the machines where the index looks to be incomplete:

    (3)(+0000000): Starting full-text syncing
    (3)(+0000001): Starting full-text sync for My Library
    (4)(+0000000): SELECT version FROM version WHERE schema=? ['fulltext_1']
    (3)(+0000001): Library version hasn't changed -- skipping full-text download

    So is there anything more I can do to diagnose the problem? Or do I just have to resign myself to waiting for the indexing to run separately on all the machines (perhaps forced with "Index Unindexed Items")?
  • dstillman
    edited March 23, 2022
    You should just ignore it for a while. Syncing isn't the same as processing of the synced content.
  • dstillman
    (When it downloads full-text content, it will create .zotero-ft-unprocessed files — hidden by default on most systems — in the storage directories, which will get indexed later when the computer is idle and turned into .zotero-ft-cache files when added to the DB.)
  • jpintar
    I followed your advice and let Zotero idle on one of my computers for over 48hrs (with nothing else running), and afterwards it's still the case that most of my library on that machine hasn't been indexed. When I search the storage folders, over 10,000 .zotero-ft-unprocessed files are still there. Is there anything I can do to prompt Zotero to actually process them?
  • calvinke
    I have the same question here. How long will it take for unprocessed attachments with a .zotero-ft-unprocessed file to get processed?
Sign In or Register to comment.