Zotero stops working and/or erases meta data

upena

Hi,

after importing a PDF I tried to change some of the meta data when suddenly Zotero stoped working and the program announced that it has to be restarted again.

Error report:
_______________

[JavaScript Error: "The connection to wss://stream.zotero.org/ has terminated unexpectedly. Some data may have been transferred." {file: "chrome://zotero/content/xpcom/streamer.js" line: 155}]

[JavaScript Error: "WebSocket connection closed: 1006 "]

[JavaScript Error: "operation timed out"]

[JavaScript Error: "late cancellation observer"]

[JavaScript Error: "late cancellation observer"]

[JavaScript Error: "late cancellation observer"]

[JavaScript Error: "late cancellation observer"]

version => 5.0.45, platform => MacIntel, oscpu => Intel Mac OS X 10.11, locale => de, appName => Zotero, appVersion => 5.0.45, extensions => Zutilo Dienstprogramm für Zotero (2.0.4, extension), ZotFile (5.0.7, extension), Zotero Word for Mac Integration (5.0.14.SA.5.0.45, extension), Zotero LibreOffice Integration (5.0.13.SA.5.0.45, extension, disabled)
_______________

I tried again just from scratch: Created a new entry manualy, filled in all the missing meta data. Again, after a while the data was erased by Zotero and fields were filled with crap data.

See Debug-Id: D2002876658


Any help welcome.

dstillman

User interface ready in 640765 ms

How many items (including child items) do you have in your library and in the trash? Zotero seems to be taking 10 minutes to start up for you, suggesting that things are taking a very long time.

upena

Starting time is taking quite some time therefore I try to keep Zotero running over longer periods.

Ups, seems to be a little above 200.000 items all in all. Plenty of newspaper clippings and webpages with lots of data in each folder.

dstillman

Yikes. OK, so that's big.

Could I see a Debug ID for just a single edit to a single field, without moving on to another field?

upena

I did two things.

First I imported the PDF with the missing meta data and changed the author name:
D1026166701

Second I changed the field "Publication". Strangely when starting to type in this field names of authors are shown contiously, also not connected to the letters i typed. When I fill in author names it never happend in my nearly 10 years of use of Zotero that I got a list of author names:
D1215725796

dstillman

OK, so I think most of the problem here is autocomplete. In a library the size of yours, each autocomplete query is taking a very long time, which is causing various other problems. That includes your seeing the wrong results for a given field (e.g., because it finally finished with results for 'G*' even though you've since clicked on a different field from where you typed 'G') and also timeouts saving changes to fields because the database is so busy performing autocomplete queries.

You obviously shouldn't have to do this, but I'd guess that if (after a restart to clear any bad state) you pasted in a field value instead of typing it, or if you tried in a field that wasn't autocompleted (e.g., "Date"), you'd be able to save a field properly.

The quickest workaround we could provide here is the ability to disable autocomplete. Beyond that, we'll need to build a sample database of this size and try to reproduce some of this behavior. I've created an issue to track this, though it might be a little while before we get to it.

I wouldn't expect any of this to be a new problem, though. With a database of that size, I'd think you'd have been having problems like this for a while. Is that not the case? Have you recently added many more items?

upena

A autocomplete function for the "author field" did never show up when using Zotero, even when my database was consiting only of some hundred items. Autocomplete for "publication" worked since ever ok.

I'm importing data on a daily base. Mostly i don't have to change the meta data. But especially with (German) newspaper clippings i often have to fill in meta data, with no problems until now.

Importing this single PDF (D1026166701) started all the trouble with disappearing meta data! After pasting – not typing – the correct author name it just returned after a while to the automatically collected – wrong – data. As written before, i never experienced autocompletion in the fields.

I tried it manually as mentioned before: Created a new entry, copy&pasted all the missing meta data in the required fields. Suddenly there was a "screen refresh" and fields were filled with crap data or data was erased.

Aside of the really very long starting time i actually don't remeber much problems. Only sometime while importing several journal itemes simultaneously – like from Wiley, which still doesn't function up to now - Zotero "forgot" to import anything and the procedure had to be repeated.

Importing right now another PDF worked without problems and the meta data stayed in place.

Remains a bit of a mystery for me.

Thanks anyhow for your help!

dstillman

A autocomplete function for the "author field" did never show up when using Zotero, even when my database was consiting only of some hundred items.

I suspect you're misremembering that. Zotero has always had author autocomplete. And it's definitely searching for author names to autocomplete for you, and finding a huge number of them — if it's not showing them, that's likely just because of the large number.

Suddenly there was a "screen refresh" and fields were filled with crap data

I don't know what you mean by this. Can you be more specific, or upload a screenshot somewhere and provide a link?

Keep in mind that because of your database size, once things go wrong for you, they might be in a weird state until you restart, so it's only really meaningful if you reproduce a problem after restarting Zotero (though I know takes a long time for you). Otherwise it might just be a side effect of an earlier problem. So, e.g., if you restart Zotero and then paste a new field value into a field without making any other manual edits, I'd be surprised if you had trouble. But once you trigger autocomplete, I suspect things are going to go astray. We'll look into what we can do to improve that (even if it's automatically disabling autocomplete at databases this large).