Upgrade StorageImport and Export issues
Hello,
I am currently trying ways to edit a Zotero database on a broad scale.
A very useful way to do this would be some kind of Search/Replace function, but I didn't see that in the new beta. Any idea whether this could this be implemented at some point?
The other solutions I thought of all relied on exporting and importing back all data in Zotero. There are several issues here which I present here, maybe some of them could be easily solved (documentation on the matter can be found here: https://www.zotero.org/support/kb/importing and here: https://www.zotero.org/support/dev/data_formats):
1. the biggest issue when exporting and importing back a reference, in whichever format, is that the identifier gets lost in translation, so all links to references in Word e.g. are broken (see documentation here: https://www.zotero.org/support/getting_stuff_into_your_library#importing_from_other_tools). This is probably the least manageable issue. Notwithstanding this problem:
2. Zotero RDF seems the best export/import option. All fields, tags and files remain. The only issue I noticed was that the content of the Extra field disappeared on import (except if Abstract field is empty, then it goes in that field).
3. BibTex: seems to work fine on export, but on importing this type of file, the content of Extra goes into a note (XML MODS does this too). For BibTex, this means that both "note" and "annote" are interpreted as notes on import (having used this format to work out a large import two years ago, I am pretty sure this wasn't the case in previous versions of Zotero).
4. CSV format is really practical, but it cannot be used as an import format. Couldn't it be made one?
5. If not, converting a CSV into a JSON file can be done easily using free online tools, but JSON doesn't seem to manage tags anyway. On top of that, when directly exporting from Zotero to JSON, notes related to an item disappear, only independent notes remain. When importing back, these become Documents.
6. I didn't try in details the other options because: Bibliontology RDF doesn't export notes and files ; Refer doesn't export Extra and Notes ; RIS and TEI don't export Extra.
Thanks for any feedback or suggestions on this :)
I am currently trying ways to edit a Zotero database on a broad scale.
A very useful way to do this would be some kind of Search/Replace function, but I didn't see that in the new beta. Any idea whether this could this be implemented at some point?
The other solutions I thought of all relied on exporting and importing back all data in Zotero. There are several issues here which I present here, maybe some of them could be easily solved (documentation on the matter can be found here: https://www.zotero.org/support/kb/importing and here: https://www.zotero.org/support/dev/data_formats):
1. the biggest issue when exporting and importing back a reference, in whichever format, is that the identifier gets lost in translation, so all links to references in Word e.g. are broken (see documentation here: https://www.zotero.org/support/getting_stuff_into_your_library#importing_from_other_tools). This is probably the least manageable issue. Notwithstanding this problem:
2. Zotero RDF seems the best export/import option. All fields, tags and files remain. The only issue I noticed was that the content of the Extra field disappeared on import (except if Abstract field is empty, then it goes in that field).
3. BibTex: seems to work fine on export, but on importing this type of file, the content of Extra goes into a note (XML MODS does this too). For BibTex, this means that both "note" and "annote" are interpreted as notes on import (having used this format to work out a large import two years ago, I am pretty sure this wasn't the case in previous versions of Zotero).
4. CSV format is really practical, but it cannot be used as an import format. Couldn't it be made one?
5. If not, converting a CSV into a JSON file can be done easily using free online tools, but JSON doesn't seem to manage tags anyway. On top of that, when directly exporting from Zotero to JSON, notes related to an item disappear, only independent notes remain. When importing back, these become Documents.
6. I didn't try in details the other options because: Bibliontology RDF doesn't export notes and files ; Refer doesn't export Extra and Notes ; RIS and TEI don't export Extra.
Thanks for any feedback or suggestions on this :)
Indeed, export/import isn't the recommended way of doing that and I'm not sure we'll ever be able to guarantee 100% roundtrip of information in any format (even leaving asiden the internal identifier, which doesn't have a solution given the existing import/export infrastructure).
The recommended way of handling batch processes in the meantime is locally via the javascript api:
https://www.zotero.org/support/dev/client_coding/javascript_api#batch_editing
or on a synced database with the server api:
https://www.zotero.org/support/dev/web_api/v3/start
Obviously this isn't at all user friendly, hence the need to have something built in.
Even not using them for batch editing, Import and Export could be drastically improved (and made apparently quite reliable) by just working on points 2. and 3. mentioned above. Considering how transparent RDF and BibTex formats are, I have a feeling the improvement could be quite easily made... but this is way beyond my capacities, so I will let this for developers to decide!
Thanks again :)