Some collections empty after Mendeley sync

stefanollinger

I just installed Zotero 6 and synced all articles from Mendeley via Zotero's Mendeley sync.

After some investigation I found that most of the collections seem to be ok, but some collections are empty. The articles themselves exist, but only in the main "My Library" collection.

Any ideas? Should I retry a Mendeley sync to an empty Zotero data folder?

Regards,
Stefan

dstillman

Can you provide a Debug ID for trying to import again, and provide the name of an empty collection? (Can provide name here or via email to support@zotero.org, but we'll respond here.)

You don't need to delete anything, and it shouldn't duplicate anything.

stefanollinger

The Debug ID is: D1274970972.

The collection information has been sent via email.

dstillman

Sorry for missing this. I'm afraid that debug output is filled up almost entirely with Better BibTeX output. Can you disable all third-party plugins and generate another Debug ID for an import?

stefanollinger

The new Debug ID is: D1657358750

This was done with Better BibTeX disabled.

dstillman

Sorry, and I'm afraid that one just shows syncing of the data. Can you also temporarily disable auto-sync in the Sync pane of the preferences and provide another that just shows the import?

stefanollinger

I disabled the sync.

This is the new Debug ID: D1099453397

dstillman

Did you do anything to annotations in the library since the last import attempt? The debug output shows lots of annotation creations (which are also pushing the collections operations out of the log), which shouldn't happen unless annotations in the database were removed since the last import or more were added in Mendeley.

stefanollinger

After the import I did not create any annotations.

This is what I did:
- I disabled Zotero Sync and Better BibTeX
- Re-started Zotero with debug logging enabled
- Started the Mendeley import

Right after the import was finished I sent the debug log and posted the debug id here.

dstillman

OK, sorry, so it turns out there was a bug that was causing Mendeley annotations to be duplicated on import if Zotero has synced since the last import, and your debug output is what helped us discover that. We've fixed that in the latest Zotero beta, and the fix will be included in 6.0.5.

If you look, I'm afraid you'll find that you have multiple sets of annotations for each PDF. We might be able to provide a script that can clean those up automatically.

This bug wasn't related to your issue, except that the extra output from the redundant annotation creation has been pushing any relevant lines about collections out of the log. If you haven't synced since the last attempt, though, another import attempt shouldn't duplicate annotations, so another Debug ID for another run might give us what we need to figure out what's going on with the collections.

dstillman

@stefanollinger: Zotero 6.0.5 is out now with the fix for annotation duplication issue.

stefanollinger

@dstillman What is the easiest way to delete the online library?

Then I would do a new import and sync, avoiding the duplicate annotations.

dstillman

If you want to delete everything and start again, you wouldn't do that online. You can just either 1) delete everything in your local Zotero, empty the trash, and sync, or 2) close Zotero, delete your entire Zotero data directory, restart Zotero with an empty library, and do Replace Online Library from the Sync → Reset pane of the preferences.

stefanollinger

I deleted the data directory, did "Replace Online Library" and re-imported from Mendeley.

After the re-import I could not "Submit Output". Clicking leads to a short UI freeze and then closes the Help menu ( https://i.ibb.co/Qvm19cH/image.png ).

I repeated this process three times.

Any ideas?

dstillman

Does it still show 500K lines as logged?

Can you provide a Report ID?

stefanollinger

Yes, after clicking on "Submit Output" it still shows 500K lines.

The Report ID is: 303142851

stefanollinger

This is probably related:

[JavaScript Error: "allocation size overflow" {file: "chrome://zotero/content/xpcom/debug.js" line: 212}]

dstillman

Try going to the Config Editor in the Advanced pane of the preferences and reducing extensions.zotero.debug.store.submitSize (10 MB by default) little by little until it submits.

stefanollinger

I reduced it step by step down to 1000.

Submitting shows still the same behaviour (short UI freeze, then no dialog).

dstillman

Sorry, it'll need to be more than 1000. Try, e.g., 9500000 or 9000000.

stefanollinger

I reduced it by taking halves. 9500000 and 9000000 also show the same behaviour.

dstillman

Oh, sorry, I misread — you tried reducing it to 1000, not by 1000.

OK, we'll have to fix that, but a couple options:

1) Right-click and reset debug.store.submitSize and change debug.store.limit to, say, 450000 or 400000 (don't halve, because once it goes through the log is cleared), and then scroll and click around in the items list until that shows as the number of lines logged in the Debug Output Logging menu. (It's set to reduce it to the current limit roughly every 1000 lines.) Then try submitting.

2) I just realized that you're running Linux, so you could just generate a log file outside of Zotero, though you'd have to do the import again (but it shouldn't generate duplicates this time).

Thanks for your patience here, and rest assured that the vast majority of problems are not this hard to debug! (And if I had realized you were on Linux, I could've saved you a lot of time, so sorry about that. It's just Windows where we have to rely on the in-app logging. At least we fixed an import bug in the process.)

stefanollinger

Your help is appreciated :)

I sent the debug log to support@zotero.org

dstillman

Can you confirm that the items actually belong to these collections in Mendeley online (not Mendeley Desktop)? We're not seeing anything in the output that suggests there's a problem, so the most likely answer here is that the Mendeley API is simply not showing the items as being in that collection.

stefanollinger

Yes, you are right. The collections are empty on Mendeley online as well.

Since the majority of collections seem to be ok, I would keep the library as it is then.

Thanks a lot for your support!