Endnote XML export/import?

talazem

As part of my tests as to whether to use Zotero 3, I am testing its ability to import and export into other software (Bookends, Papers2, etc.). I have run into a problem.

When exporting out of Zotero using RIS, then importing into Bookends, the metadata transfers pretty well (and can be fine tuned within Bookends); however, attachments do not come through. It seems that RIS is not a strong format for such a use. The Bookends developer, however, recommended using Endnote XML, which contains no ambiguity, and thus allows for attachments to carry over. It seems that Endnote XML is an industry standard, of sorts (I see it supported in the import and export of many pieces of software, such as Bookends, Papers2, etc.).

May I enquire as to why Endnote XML is not supported, and whether there are any plans to do so? Many thanks.

Gracile

Partial answers in these threads:

http://forums.zotero.org/discussion/8614/import-from-zotero-to-endnote-with-pdf-attachments/#Item_9

http://forums.zotero.org/discussion/21931/import-bookends-to-zotero/#Item_4

adamsmith

I'm still interested in supporting Endnote XML in general, but It'd be nice to have some specs.
Endnote claims they share them freely:
http://blogs.plos.org/mfenner/2010/08/03/endnote_interview_with_jason_rollins/

bibutils has a conversion tool that we could look at:
http://sourceforge.net/p/bibutils/home/endx2xml/
(and that could be used as an intermediary in the meantime, since Zotero supports MODS).

Anyone has any thought on where/how to find the Endnote XML specs?

Rintze

http://drupal.org/node/1661390#comment-6165976

Edit: that leads to a 404, sorry about that. Working links can be found at http://archive.endnote.com/support/ensupport.asp, see "EndNote / EndNote Web XML DTD".

adamsmith

thanks, unfortunately http://endnote.com/support/helpdocs/endnote.zip
which I had also found linked to from Martin Fenner's blog, is a 404...

edit: got it, thanks.

talazem

Thank you both. Some of the discussions in the links above indicate that Endnote XML has not been implemented due to lack of public specs and/or instability ("moving target)". I asked Jon, the developer of Bookends, about this as he has implemented. He replied (I cite with his permission):

"The EN XML schema has been stable for years (since EN 8), at least there have been no changes that affect Bookends export/import. RIS actually has changed more. Also, with XML text style info is transferred, which is not possible (ever) with RIS, which is plain text.

Supporting EN XML takes some work initially, but once done there isn't much maintenance."

I hope that is of help.

DWL-SDCA

Last year, my university received a letter from Thompson-Reuters questioning the use of EN XML as an export format for records from my online literature database. We removed that format. Apparently, only ris and enw formats are OK. At least, that is the way it seemed to the lawyers.

Unlike what you were told, we found it quite simple to implement an EN XML export option. As I remember it was easier to do than the tagged formats (ris, enw).

adamsmith

color me shocked ;-)
That said, it seems like it's fine with other ref managers, so doing this in Zotero should be safe (although I did note that Endnote XML is always referred to in their documentation as "Endnote's proprietary XML format"...)

adamsmith

I got a reply from Endnote with the updated link:
http://help.thomson.com/default.asp?portalid=TSCINT&article=122577

Rintze

That link doesn't work for me (time out).

adamsmith

update link above (I pasted what was in the URL bar, the above is what they sent me & it works)

talazem

Hello. Just wondering if this is looking any more likely, now that the specs have been found. Many thanks for your time.

adamsmith

more likely yes, but no specific timeframe.

aurimas

so I looked through the spec and it's not very strict at all, which is not very helpful. Essentially each record can contain every possible field in EndNote. This means that we'll essentially have to do the same thing we do for RIS and figure out which fields map to what for each item type. There should be less variation from item type to item type. Titles and authors are going to be a pain again.

RE formatting, afaict we might be able to transfer bold, italics, and maybe small caps. I'll have to see what EndNote can actually support. Doesn't look like super/subscripts are supported. This is a still a nice improvement.

adamsmith

aurimas and I have implemented Endnote XML now, should be working well. That said, while the specs are indeed stable, it's very much not clear what should go where in the format and we're mainly relying on what Endnote is doing to get this right, I'm not sure how that works for interoperability with other ref managers.

cjb

Question about Endnote XML. Exporting from Zotero and importing to Endnote, I can't seem to get Endnote (X4 and X6) to completely import a Zotero XML export *with files*. The references import nicely and have the paperclip 'attachment' icons next to them, but the attachments themselves don't open and when I open the references themselves there is a red cross over the attachment icon in there. Clicking 'open attachment' gets an error message like this:

https://dl.dropboxusercontent.com/u/30366587/error.png

Can anyone (adamsmith perhaps?) let me know if this is problem on Zotero's end, something on Endnote's end, if it's something that's just happening to me, or if it's just because I'm doing something wrong?

thanks,
Carl

adamsmith

It's hard to talk about whose fault this is on the file export end. Endnote does weird stuff with file export and Zotero does its best to mimic that, so I'm inclined to say it's EN's problem
Zotero is able to import attachments from Endnote and from roundtrip Zotero import/Export via Endnote XML.

You could try exporting with the file box unchecked - Zotero then exports absolute links, maybe Endnote does better with that.

cjb

Thanks, that actually works locally but of course it creates absolute references to the PDFs in my Zotero library, and for whatever reason Endnote won't convert them into relative ones (I just get "no links converted" when I try), or in any other way import or copy them as relative or local to itself. Endnote's 'compressed libraries' won't carry them along either because (stupidly) it's just a compressed version of the library + whatever is in its data folder.

I went so far as to export what I needed in RDF, start a whole new Zotero database *inside* the Endnote library's data folder and export-import in XML from that, but of course the links remained absolute to that location and with Endnote's refusal to relativise the links manually it's stuck where it is.

The goal was a regular Endnote library (complete with PDFs) of a large publications audit we did, to send to my boss for her to deal with and store etc where she needs it. But it looks like this is still not possible with the current Endnote XML implementation in Zotero given that Endnote sucks so very very much.

Has anyone else had luck with this sort of thing? Does Endnote-to-Endnote XML transfer even work properly? I'd try myself but I'm sick of the sight of it.

adamsmith

Endnote puts an internal-pdf:// before the "file/" in Zotero - you could try doing that with search&replace (i.e something like replace "file/" with "internal-pdf://file" in the endnote.xml file (which you can just open in any text editor) maybe that works?

cjb

Thank you! With that advice it looks like I've got what I want now, with an extra step required.

Procedure, for the record:

1) I did a relative ('include files' box checked) Endnote XML export from Zotero.
2) In the XML file, replaced all "files/" mentions with "internal-pdf://files/" mentions.
3) Imported this XML into Endnote. This produced an Endnote library that expected to see the relative linked attachments (the paths after the "internal-pdf://files/" mentions) inside a 'PDF' folder inside the Endnote library's data folder. Therefore...
4) Make a 'PDF' folder inside the Endnote library's data folder, and copy the export file's 'files' folder into that folder.

The result looks like what I was after: an Endnote library + data folder that can be copied about at will.

Thanks again for the help.

aurimas

Thanks for testing this out. We updated the export translator to do this by default. It will also create a PDF directory that you can just copy over to your EndNote library directory.

You can update the translator via Preferences -> General -> Update Now