Problem importing a large dataset (500MB) - Report # 1783265735
We are building a large bibliographical dataset of multiple 10s of 1000s of scholarly publications in our field. We have merged and deduplicated (based on a synthetic key and retaining the most promising field for each record from across the dataset) exports from Dimensions, Google Scholar, The Lens, OpenAlex, Scopus and Web of Science. If we try to import the full dataset, the system becomes irresponsive and stays so for hours (I've let it run for 11 hours) - meaning I also cannot submit a debug output log. I did make a few screenshots, though - see here and here. When I try to import a small subset (like here) then it does work. But even in this case, the Progress Bar does not build up during 'Importing' until completion.
Does anybody have any ideas how I could still get the entire dataset into Zotero or how to debug this better? I do appreciate that Zotero has been improving over the years in its handling of large files, but it would be nice if this could work better