Problems importing RDF file

IPCL · March 29, 2018

I have a large database (more than 20 000 references) transfered from Reference Manager via XML, but in the transfer very many accented characters got changed to two letter codes starting with Ã. To change these individually would take a lot of time and as there is no global edit yet in Zotero I ‎tried exporting the database to RDF to edit there.

I am well aware that exporting to RDF and reimporting is deprecated, but I can see no better way to do this editing. But the problem is that when I try to import the RDF file it fails after about 3000 references, both the origianl RDF file and the edited one, just with the message that "An error was encountered... Check the file ...".

Can anyone suggest why the import fails and how I can fix this? Or does anyone have a better idea for doing this editing?

noksagt · March 29, 2018

Reference Manager via XML, but in the transfer very many accented characters got changed to two letter codes starting with Ã.

Could you fix them in Reference manager or the XML file it generates? It seems semi-likely that the encoding of that file is not what Zotero expected, so I'd troubleshoot starting with this.

I am well aware that exporting to RDF and reimporting is deprecated

Not deprecated; it is still supported, but has never been intended to round trip data.

but I can see no better way to do this editing.

If you no longer have the XML file (which seems to be the best way to edit), I'd probably use PyZotero or similar before trying to work with Zotero RDF.

But the problem is that when I try to import the RDF file it fails after about 3000 references, both the origianl RDF file and the edited one, just with the message that "An error was encountered... Check the file ...".

You may be able to get more info from debug output or by submitting a debug id.

But the DIY approach would be to export ca. 10 files that each had ca. 2000 references in them & see if one fails & try to trace it to a particular reference.

adamsmith · March 29, 2018

Agree with all of this. Just to add to the last point, importing RDF is a very memory heavy operation, so it's possible you're just seeing an out of memory problem which splitting this up would solve.

IPCL · April 12, 2018

Following adamsmith's comment I moved the import process to a workstation with 64 GB memory. It still failed after 3000 something entries.
Unfortunately when it fails Zotero is unresponsive and I have to kill the program so even though I enabled debug reporting I do not get any debug output. Should there be a debug log stored somewhere that I can access?
(Just a thought - is there any difference between the Windows and Linux versions of Zotero? The workstation has dual boot with Ubuntu 16.04, I could try that if you think it would make any difference).

I have realised that I am using a sledge hammer to crack a nut. By searching for Ã I can export only the affected references, edit them and reimport them. There are only about 800 of them (if I ignore the abstracts, which are not critical) so hopefully the import will work.