Bug in v7 Beta: attachments not appearing (ID: 1693066995)

scottbot

Upon opening Zotero, I found most (but surprisingly not *all*) of my zotero refs no longer have any accessible or visible attachments. As far as I can tell, those attachments all (or at least mostly) still exist in the zotero storage folder; they're just unlinked to / unseen by the application.

This issue followed my attempting to reindex several PDFs manually (via right click), since they weren't indexing automatically. I then restarted Zotero and my computer, potentially in the middle of a re-index.

The issue continues to be present after closing and opening Zotero, and after restarting my PC.

Steps to reproduce:

1. Start Zotero.
2. Look at any reference; see there is (in almost all cases) no attachment.

Any thoughts?

Thanks,
Scott

dstillman

Can you provide a screenshot of what you're seeing (here or in an email to support@zotero.org with a link to this thread)?

You still have many gigabytes of attachments in your online library.

scottbot

Huh, in trying to find some screenshots for you, I've discovered that the attachments have all just moved to a random folder, now disconnected from the original reference record.

https://s3.amazonaws.com/zotero.org/images/forums/u366438/8uiql4stm00rpl89qinc.png

https://s3.amazonaws.com/zotero.org/images/forums/u366438/2vamu3aw6yd4z2qd2mwn.png

In fact, as best as I can tell, every record and attachment has somehow ended up in the 'juggling' folder, but even in there, the attachment and the record are no longer connected as they once were.

dstillman

Did you have any plugins installed when this occurred? It looks like about 4,000 attachments were modified right before you posted. I don't think we've ever seen anything like this. If you didn't have any plugins, can you explain again exactly what you were doing when this occurred?

But first, before opening or closing Zotero, you should immediately check your Zotero data directory to see if you have a zotero.sqlite.bak or zotero.sqlite.1.bak file from before noon ET on the 8th. If so, make a copy of it, and then see Restoring From the Last Automatic Backup.

If you have Zotero installed and syncing on another computer, you can also look for an automatic backup on there before opening Zotero.

Otherwise, you'd want to restore from some other backup of your Zotero data directory from before this occurred and then overwrite synced changes.

scottbot

Nooo, I was worried you'd say that. I did make a manual backup on the 7th, but I added so many new references with attachments since then. *sigh*

To answer your question, no, there were no plugins installed. I was getting frustrated that so many of my attachments were not getting indexed automatically or through various iterations of "Rebuild Index" that I started throwing a bunch of pasta at the wall to see what would stick. I believe one such approach was using a regular expression to find all files in my ~8k attachment library, and seeing if I could figure out a way to "manually" (through right-click) re-index each of them. Because it's such a big library the search was taking forever, as was the reindexing. I believe I force quit zotero a few times, and reset my machine a few times. I wish I could give you a better reconstruction of the process, but honestly, I was being sloppier than I should have been without expecting it would lead to ill effect.

adamsmith

You can export the new references to Zotero RDF and re-import after restoring -- might want to tag for collections if those are important because they won't restore to the right collection when you import them.

scottbot

Currently working on remarrying attachments to reference records in order to do just that, but I'm glad to hear confirmation from you that my approach is reasonable.

scottbot

Alright, while I'm here, can I ask about the indexing issues that led me to this error? Or should I make a new thread?

A whole bunch of PDFs seem to be unindexed or queued, despite any combination of clearing and/or rebuilding and/or indexing unindexed items. Any ideas why or pointers to fix the issue?

https://s3.amazonaws.com/zotero.org/images/forums/u366438/epj5snkiefyiiadws6zc.png

dstillman

Queued items should be indexed on idle.

What happens if you click the Reindex button (the arrows)?

scottbot

The reindex button does work as intended.

But the idle point may be the issue here: what does idle mean in this context? I usually close Zotero when not using it (even though "using it" could be several hours at a time of research/writing). Could that be the culprit?

scottbot

Alright, left Zotero on overnight and kept the computer from sleeping, and I'm still getting unindexed PDFs. Whether or which old PDFs get indexed seems to be somewhat random (or at least I can't discern a pattern), but I do see a trend in newer attachments. When I add a file on another machine running the old Zotero, then sync it to the machine running Zotero 7, the new synced attachment doesn't appear to index automatically. Nor does it sync when I go to settings > advanced > rebuild index > index unindexed items.

When I manually sync it, it does work.

I would (begrudgingly) sync *all* unsynced PDFs manually, but it's difficult to find them. I've attempted the regex method (Attachment Content --> does not contain --> . [with regex enabled] // Attachment File Type --> is --> PDF), but the saved search always hangs/fails (presumably because my library is too large).

Thoughts?

https://s3.amazonaws.com/zotero.org/images/forums/u366438/a3j7hjifkhqswsebwiuk.png

Edit: On the old zotero / old computer, the stats are 6696 indexed and 1137 unindexed, for reference.

migugg

A long time ago I had a similar question regarding what exactly indexing does and why it does not index certain files,and how the index count works, but I have never received an answer. It would be great if the devs could shed some light on this.
see here: https://forums.zotero.org/discussion/102853/auto-index#latest
Atm, I have 12 000 items in the library, but the count says 8000 are indexed, 53 partial, and 1300 unindexed, with roughly 2700 unaccounted for.

scottbot

@dstillman, I'm getting a strange sync message. Seeing it, I dimly recall this same thing happened before my library last exploded, and then I'd forgotten about it so had not mentioned it in this thread.

Pressing "Migrate to new note" and "choose this version" appears to do nothing. I cancelled the sync (which worked), opting not to press the mysterious empty buttons, and am currently backing up the latest database from another PC just in case.

Any advice?


https://s3.amazonaws.com/zotero.org/images/forums/u366438/fsshi9v576e9vjgmtybi.png