Hi: Thanks for a great tool.

How do I import my Endnote libraries into Zotero?

Thanks.

--Aaron

I'm getting the "no translator could be found for the given file." It's EndNote 6. Not sure it'll read the thing + I can't seem to be able to download (or find) the RIS export tool at the EndNote site. Any suggestions?

@aaronwilcher:
http://tinyurl.com/y5ycwd

@Dan:
Just a note that the RIS style isn't always installed and can sometimes be removed. It sounds like this might be the case for aaron.

Zotero can import a variety of formats. Two such formats which you should be able to export from Endnote (using the zip file for Endnote 4-7 linked to on that page) are Refer and BibTex.

(Slightly off-topic, but Word DOCs are proprietary, minimally documented, and binary. They don't have a standard way of storing bibliographic citation information. The new MS OpenXML format will have a "standard" way to do this & Zotero may use this when they add the ability to work with MS Office. If Zotero is made to write to this format, I suppose it might be a reasonable feature request to import it too. But none of your current Word files are likely to be in the correct format.)

I have my references in an Endnote 8.0 file, and my Endnote application is not working well (I cannot open my notes). When I follow the sequence - RefMan (RIS) Export - Style (Endnote: Output-Style; File>Export>Text) - I find that the new file has zero kb. Is there some way of translating my Enl file directly?

A little persistence pays off. Turns out that if you set the output style in Endnote X to RefMan, the export the file as a text (txt) file, Zotero can upload the references flawlessly. Note of caution. If you Endnote library is large, then this can freeze up the browser, so I found it necessary to carve up my library into smaller units for importing. Hope this helps others.

Hi
I am trying to import a .txt file in RIS output style from Endnote X running on Win XP into Zotero running on Ubuntu 7.0.4. The process appears to start but is terminate shortly after with the message "An error occurred while trying to import the selected file. Please ensure that the file is valid and try again."
Any ideas on how to overcome this?
Thanks

Another alternative would be to do a find & replace in the RDF file that Zotero will generate for me, but this seems complicated. The notes are presented as separate items in the file, with their own identifying item number cross-referenced to the original citation item. So without knowing the rules for generating these item numbers, which ones might be in use, etc., a find and replace seems like it's not going to fix it. Any thoughts out there?

Feel free to send files that don't import to support@zot....org, and we'll take a look at them.

I have been trying to import my libraries from Endnote (V9 for Mac) with partial success. I am not experienced in this kind of thing, so I compared imports using 4 formats: RefMan, BibTeX, Endnote Export and Refer.

Most of the basic info came through with all of them. However:

In RefMan, some of the dates of publication came through incorrectly! I think it may have become confused with dates or 4-figure numbers which occurred in other fields.

In Endnote (Export), there was some muddling of fields.

In BibTeX, there was also some muddling of fields, and URLs were not picked up.

In none of the above were my extensive research notes picked up. They all just ignored them, except for the Refer export format. This picked up a portion of the research notes and put them into the 'Extra' field. However, it cut off after a certain amount of text. It doesn't recognise URLs, but this still seemed to me to export the most data successfully and correctly.

I am not impressed at not being able to transfer my notes. A list of references is all very well, but it is not reasonable to have to copy across pages of notes by hand (as someone has suggested).

(I still like the look of Zotero so far, though, and am hoping to switch across properly.)

@noksagt: There is a workaround for this using the current BibTex exporter. @Silverfin: Here's how you can get your Endnote research notes and URLs into Zotero. This is based on my Mac experience, but I don't see any reason the Windows procedure would be different since Endnote seems to work pretty similarly on the two platforms.

Endnote 9 has two fields where I have stored my own notes. One is called "Notes" and the other "Research Notes." The BibTeX exporter only includes the data from Notes (if you select "BibTex Export.ens" as your Output Style in Endnote and open the preview pane, you can see exactly what it will output for each of your records). It ignores "Research Notes," and also ignores URLs. Anything that is in the BibTex Note field, when imported into Zotero, goes into the Extra field (just above Date Added at the bottom of the Info tab).

So, if you need your research notes and URLs in Zotero, you can use the following procedure (I strongly suggest making a copy of your Endnote library and doing this on the copy):
1. In Endnote, under the References menu, select Change and Move fields
2. Use the dialog to ask it to move everything from Research Notes to Notes (you can insert after, before, or overwrite current material there as you wish).
3. Use the same facility to move info from your URL field to the Notes field after or before the current material as you wish.
4. Use "Export" under Endnotes "File" menu to export your library.
5. Use "Import" under Zotero's gear icon to import your library. (Be prepared to wait a few minutes.)

All your research notes and URLs will be in the Extra field for the associated records. So far as I can tell, the Extra field will hold an arbitrarily large amount of material although I cannot promise this. But my longest notes seem to be complete.



As I noted above, it would be great if there were some way to convert this material to associated notes for the relevant records, rather than filling the Extra field. Could the BibTex importer be modified to give this as an option?

Do you have small excerpts from your RIS file that demonstrates these issues?

There is a suggestion to understand what's wrong with Endnotes import.

I try many times importing Endnotes files to Zotero using BibTeX, RIS, Refer Export even with EndnotesExport, with very different results and problems: fields not imported, wrongly placed, transfered in "Extra" or in "Notes", etc. But it's very difficult to analyse what is wrong with real bibliographic data.

To understand these problems and compare import formats, I made a Endnotes file with 32 "faked items" corresponding to the 32 items types in Endnotes 9 (wich I use now). After, for each item, I put the name of the each field in the field. For exemple, for item type "BooK, field Publisher = "Publisher", field Pages = "pages", etc. For fields TITLE, I put "title + item type name", for Book, Title = "Title_book", for Manuscript, Title is "Title_Manuscript", etc.

So, if the importation is right, the content of the field will be the same that the field's name, if not, that means that a field is misplaced in Zotero's field. An empty field in Zotero means:
- a field not imported,
- placed in an other field
- a field that does'nt exist in Endnotes (or with a different spelling)....

After, I export this Endnotes file of 32 items in 4 differents import format: BibTeX, Refman(RIS), Refer Export, and EndnotesExport. Now, I can compare exactly the differences and understand the problems. My impression is that the "best" one (for now) is RIS where the fields not rightly import in Zotero are transfered as "Note". At least, no information is lost. In Refer Export, these fields are put all together in Zotero's "Extra".

For me, it's a very good way to see and understand what is wrong with import from Endnote. Some fields, as "language" or "short title", are never or rarely (?) imported.
I don't understand why it's like that, and I can't change it. But if someone like to work to improve Endnotes import to Zotero, I can send my 4 small files in text file to import in Zotero. It's a good tool to anlayse the problem, I think.

(Sorry for my english, I hope it's clear..)

Dan: I'll send my 4 files at support...

An other problem is that some Endnotes items types are not well imported in Zotero. For exemple, with RIS, the item type "Manuscript" in Endnotes is recognized as "Web page" by Zotero.

If the complete importation is not possible, the idea the put the Endnotes "not recognized" fields with RIS as "Notes" in Zotero is very good. But, to improve it, is it possible to create a tag to these notes with the name of the field and the content of the field in the note itself.

For exemple, for item type "Magazine Article", the field "Reviewed item" in Endnote is put in a note in Zotero. If we create the tag "Reviewed Item" to this note (with the text in it), it will be much more useful. Not only the info is not lost, put it's tagged, and much more searchable...

Noksagt: I understand that create a complete "translator/importator" is not easy. At least, I try to understand wich fields are well imported for each item type. There is many "secondary" fields, not very essential to have and rarely used in a style.

And, if I found 2-3 items types and fields very badly imported, I could change it for a more "generic" item type or move-merge infos in a field to an other one. That way, I'll really migrate to Zotero, but for now, I have to many items in Endnotes (more than 10 000) to adopt completely Zotero, a very excellent tool.

Now, import is always a (bad) surprise! We don't know what will be well or quite well imported. If we can know that, we can manage to "adapt-prepare" Endnotes files for importation. It won't be perfect but useful.

Just a general comment here: I think EndNote users having trouble exporting their data should also report these issues to Thomson/Reuters.

Some of the most common issues seem to be due to shaky implementation of various export formats on EndNote's end. There is nothing the Zotero folks can do about that (except for trying to be helpful). If Thomson/Reuters gets enough support requests and complaints, one would hope that they try to make some improvements.

Contacting Thomson/Reuters to talk about how I can export my data and stop using their product is unlikely to solicit a response.

Follow up questions...

1. Is their a functional limit to how many references a 'collection' can have?
2. Does increasing the number of references increase the load on memory?
3. Is their an ideal format for importation into Zotero. I notice in the export dialog their is a Zotero RDF option. What format is this in? Why isn't their a comparable option in the import dialog?
4. Endnote allows you to create an output style. Is their documentation anywhere of the 'ideal' import format (that is a 1:1 list of data/type:fields/type)so such a style can be created? Once this style was created then issue of importing Endnote bibliographies would disappear.

Thanks for the background. I have been playing around with Zotero and came to the conclusion that the only export format that imports entirely is Zotero RDF.

I created a book and book section item in an empty library then exported and reimported the two dummy references. The data in the references were just descriptions of the field name. I was quite surprised.

Quickly...
MODS Export -- lost data in fields # of Volumes, Language, Short Title, Repository and Extra, and lost all the attachments (URLs to files or websites).
REFER/BIBLX -- Totally corrupted the records with an extra webpage reference being created from the book reference.
RIS Import -- lost data in fields series number, # of Volumes, edition, language, call number, loc. in archive, repository, rights and extra, and lost all attachments (URLs to files and websites)
Zotero RDF -- author fullnames were transfered to lastnames, lost URLs to local files, URL to websites retained.

On inspection of the RDF file created, it is obvious that it is not going to be easy for anyone outside Zotero to create a translator of any type.

I am wondering if a modified RIS format could be accepted as an import option (called Zotero RIS), which has the missing tags present to make a complete Zotero record. This information could then be used to make a modified Endnote Output Style. For example, RE - for repository. RI - for rights, etc.

Something else worth noting different between Endnote and Zotero is how the data is stored. In Endnote if you change between reference types, say from a book to article, fields are just renamed and the data stays intact. Change it back and all the original data returns. In Zotero, if you change to a reference type with less fields you get a warning about the potential loss of data in the fields now not required.

For the record: I am using XP Pro SP3, FireFox 3.0.3 and Zotero 1.0.7.

I agree with your comments. Personally I am just trying to transfer 1000s of references to Zotero in a way that avoids inserting unknown errors. Once done I will give EndNote the flick, and my transition to OpenOffice will be complete.

In regards to the actual number of references that can be imported in a shot. I have successfully imported 1000 without generating an error. So I will be cutting my EndNote library up into smaller bits for importation. Thanks everyone for your input.

First you might try using RIS instead of BibTeX. Otherwise you might check to make sure that you have the most up to date version of the BibTeX export format from Endnote.

The next step beyond this would be to open up the exported file in a text editor and make sure it looks OK. If you are still having trouble with the file there is a good chance you can massage it a little bit with a few thoughtful uses of find and replace.

Lots of Zotero users have transferred over from Endnote, many with thousands of references in their collections. Generally if you are willing to fuss with it a little bit you can get a very clean transfer.

This is a summary of how I eventually imported my Endnote Library into Zotero, and some general observations on the use of Zotero after using Endnote for many years.

General observations...
1. The generic 'Refman (RIS) Export' file provided with EndNote 5 does not result in a clean import into Zotero (I suspect different version of the RIS Format Specifications have existed over time but these versions have not been labelled as such). I searched the Endnote Style repository on the internet for the latest version of the file and found that this style is now not available. I proceeded to modify the file I had based on the latest RIS Format Specifications.
2. After doing this and conducting some trial imports, I found that not all the tags specified in the RIS Format Specifications are recognised by Zotero. I then started generalising until I obtained the maximum data transfer into Zotero Fields that made sense to me. Issues included retaining my repository data (e.g. library, reference collection, project file) and accession codes (e.g. W2345, R459, Y236 respectively) and importation of my keyword data and abstracts. I found that the keyword -> tags was unsatisfactory and messy and pushed this data, along with the abstracts into Zotero's Notes Section. I used the Zotero Abstract Field to store the RepositoryLocation:AccessionCode combination for the lack of an alternative field that imported RIS data could be sent. As Zotero functionality improves I am hoping that a field copy function will be developed that will allow me to transfer this data into the repository field available in Zotero (but which RIS Import data can not be directed). Personally I would like to see a Zotero 'text-based' RIS-like import file format where their is a TAG available for each field and reference type. People can then send their data to this text-based file using anyone of a multitude of programs e.g. endnote, databases (e.g. foxpro, access), notepad, word, etc.
3. I found that my Endnote library was best cut up into blocks of about 1000 references. Even with a reasonably fast computer this import took about 20 minutes.
4. With such a large dataset I find the use of the 'search' field very irritating. This field acts more like a dynamic filter; type 'a' and it filters the data to only show those with 'a', type 'ab' it only shows references with 'ab'. Type a name and the system shuts down for 5-10 minutes until the filter has caught up. The 'advanced search', represented by the magnifying glass is much better. This facility appears to search various indexes and produces good results in seconds and allows you to store the results as a type of collection.
5. Compared to Endnote, Zotero is a breeze at getting reference data available on the internet into its database. Coupled with a NewsReader to let me know when the latest issue of the key journals that I read are released - it has become a quick and easy routine of checking my feeds followed by reviewing the reference lists that appear and clicking on the icon in the navigation field for any reference that I wish to save. It is as easy as that. Of the various places I visit I have only encountered problems importing data this way 5% of the time. In most cases the problem centres around importing collections - if I drill down to the individual reference the import works.
6. Endnote Styles appear to provide the user with greater control of the output (combination of the huge variety of scientific styles available plus the availability of an inbuilt style editor). CSL styles are OK but biased towards the humanities and social sciences Very few scientific styles are available. Of four journals I intend to publish articles in 2008 and 2009, I am having to create CSL style sheets in XML for every journal. This adds an unexpected overhead to paper preparation. CSL/XML also represents a huge learning curve for the uninitiated.
7. For people trying to get their Endnote Data into Zotero I have provided the format of my Endnote Style Sheet below for the key reference types recognised by Zotero Bibliographic Style Sheets.


Generic
`TY - `GEN|`
AU - `Author|`
PY - `Year|`
BT - `Secondary Title|`
ED - `Secondary Author|`
CT - `Title|`
CY - `Place Published|`
PB - `Publisher|`
T3 - `Tertiary Title|`
A3 - `Series Editor|`
ET - `Edition|`
SP - `Pages|`
Y2 - `Date|`
SN - `ISBN/ISSN|`
N1 - `Notes|`
N1 - `Abstract|`
N1 - `Keywords|`
N2..- `Accession Number|`
VL - `Volume|`
UR - `URL

Journal Article
`TY - `JOUR|`
AU - `Author|`
PY - `Year|`
TI - `Title|`
SP - `Pages|`
JF - `Journal|`
VL - `Volume|`
IS - `Issue|`
N1 - `Notes|`
N1 - `Abstract|`
N1 - `Keywords|`
N2 - `Location|: Accession Number|`
UR - `URL

Book
`TY - `BOOK|`
AU - `Author|`
PY - `Year|`
BT - `Title|`
CY - `City|`
PB - `Publisher|`
SP - `Number of Pages|`
T3 - `Series Title|`
ED - `Series Editor|`
ET - `Edition|`
VL - `Volume|`
Y2 - `Original Publication|`
SN - `ISBN|`
N1 - `Notes|`
N1 - `Notes|`
N1 - `Keywords|`
N2 - `Location|: Accession Number|`
VL - `Volume|`
UR - `URL

Book Section
`TY - `CHAP|`
AU - `Author|`
PY - `Year|`
BT - `Book Title|`
ED - `Editor|`
CT - `Title|`
CY - `City|`
PB - `Publisher|`
ET - `Edition|`
VL - `Volume|`
T3 - `Series Title|`
SP - `Pages|`
N1 - `Notes|`
N1 - `Notes|`
N1 - `Keywords|`
N2 - `Location|: Accession Number|`
VL - `Volume|`
SN - `ISBN|`
UR - `URL


BTW, some problems with Endnote export are a consequence of really old (and AFAIK unfixed) bugs in Endnote.

Bruce,

You stated...

I'm biased, but I'm not the only one who believes CSL is much better designed than Endnote's style system purely from the styling standpoint.

At present, I am currently unclear of my opinion of CSL. As you may have noticed I stated "Endnote Styles appear to provide the user with greater control" - empahasis on "appear". I am still fiddling with XML creating a range of new styles and reserve judgement until I have fully explored CSL functionality.

One thing I have noticed is that most styles in Zotero only have 3 output styles - BOOK, CHAPTER and DEFAULT (formatted to present as a JOURNAL entry). This is very limited in my view. I have been told in other parts of the forum that this system captures most situations but I am yet to be convinced. I have a wide variety of webpages, reports, conferences, unpublished manuscripts, maps, CD Software, Computer Programs and Legislation that have quite specific data that needs to be presented in a set way in a bibliography. Having all these items presented like a journal is limiting. The problem though does not appear to be in the ability of CSL/XML to render the references only that people don't wish to program for all these reference types.

Simple Styles, as found in the Zotero Style Repository, already appear complicated, especially without any explanatory comments in the code. How is a style with all types of references accounted for going to look and be maintained? Questions I am sure that would spark a debate in other sections of the forum.

Suffice to say I have not finalised my judgement on this part of Zotero until I have fully explored its functionality. I will post my views in the appropriate spot once I have finished and submit my styles to the repository when they are complete.

One thing I have noticed is that most styles in Zotero only have 3 output styles - BOOK, CHAPTER and DEFAULT (formatted to present as a JOURNAL entry). This is very limited in my view. I have been told in other parts of the forum that this system captures most situations but I am yet to be convinced.

My argument is that the best, most robust, styles are typically not going to have any type conditional logic in the citation or bibliography elements, but will restrict those to macros.

I have a wide variety of webpages, reports, conferences, unpublished manuscripts, maps, CD Software, Computer Programs and Legislation that have quite specific data that needs to be presented in a set way in a bibliography.

I'd urge you to reassess that assumption. I know there are exceptions where one does need to tweak output for some types, but I don't think it's common. A good macro can go a really long way. And macros are a feature that Endnote's styling system doesn't have the last I recall.

See some design notes of mine from awhile back.

Having all these items presented like a journal is limiting.

It's not that simple. In CSL, all types have one of three fallback types: article, book, and chapter. These base types correspond to the structural characteristics I describe in that link above. So when, say, Zotero sees a record for which it does not see any particular CSL logic, it maps it to the corresponding base type. This is why for many case, you don't need definitions for much more than those three types.

Aside: with macros, this feature doesn't really need to be here, but has remained for what I'd call legacy reasons.

The problem though does not appear to be in the ability of CSL/XML to render the references only that people don't wish to program for all these reference types.

The more code, the longer it takes, and the more buggy it potentially is. So it makes sense to code for the common cases, and address problems as they arise.

Simple Styles, as found in the Zotero Style Repository, already appear complicated, especially without any explanatory comments in the code. How is a style with all types of references accounted for going to look and be maintained?

As I hinted above, in order to understand the design of CSL, you need to stop thinking about citation formatting through the lens of reference types. If you have styles that are a collection of smartly designed macros, with simple citation and bibliographic definitions, it becomes quite easy to maintain those styles. Indeed, that's a big part of the idea behind the macro system.

Yes, you are on the right track.

But, I'd go a little further. Take a look at the bibliography section for the APA style. It is only a series of macro and variable calls (though I'm a little confused about why there's both a "container-contributors" and a "secondary-contributors" macro; they ought to be the same thing).

Or, this more complex Chicago style also shows the basic idea. There you have fragments like:

<text macro="contributors"/>
<text macro="title"/>
<text macro="description"/>
<text macro="secondary-contributors"/>
So the code here is all generic, and if there's any type or data-specific logic, it happens in those macros.

Also, on your last question, it depends. In some cases, you'd be more likely to be making tweaks to the macros.