Contacting Thomson/Reuters to talk about how I can export my data and stop using their product is unlikely to solicit a response.

Follow up questions...

1. Is their a functional limit to how many references a 'collection' can have?
2. Does increasing the number of references increase the load on memory?
3. Is their an ideal format for importation into Zotero. I notice in the export dialog their is a Zotero RDF option. What format is this in? Why isn't their a comparable option in the import dialog?
4. Endnote allows you to create an output style. Is their documentation anywhere of the 'ideal' import format (that is a 1:1 list of data/type:fields/type)so such a style can be created? Once this style was created then issue of importing Endnote bibliographies would disappear.

Thanks for the background. I have been playing around with Zotero and came to the conclusion that the only export format that imports entirely is Zotero RDF.

I created a book and book section item in an empty library then exported and reimported the two dummy references. The data in the references were just descriptions of the field name. I was quite surprised.

Quickly...
MODS Export -- lost data in fields # of Volumes, Language, Short Title, Repository and Extra, and lost all the attachments (URLs to files or websites).
REFER/BIBLX -- Totally corrupted the records with an extra webpage reference being created from the book reference.
RIS Import -- lost data in fields series number, # of Volumes, edition, language, call number, loc. in archive, repository, rights and extra, and lost all attachments (URLs to files and websites)
Zotero RDF -- author fullnames were transfered to lastnames, lost URLs to local files, URL to websites retained.

On inspection of the RDF file created, it is obvious that it is not going to be easy for anyone outside Zotero to create a translator of any type.

I am wondering if a modified RIS format could be accepted as an import option (called Zotero RIS), which has the missing tags present to make a complete Zotero record. This information could then be used to make a modified Endnote Output Style. For example, RE - for repository. RI - for rights, etc.

Something else worth noting different between Endnote and Zotero is how the data is stored. In Endnote if you change between reference types, say from a book to article, fields are just renamed and the data stays intact. Change it back and all the original data returns. In Zotero, if you change to a reference type with less fields you get a warning about the potential loss of data in the fields now not required.

For the record: I am using XP Pro SP3, FireFox 3.0.3 and Zotero 1.0.7.

I agree with your comments. Personally I am just trying to transfer 1000s of references to Zotero in a way that avoids inserting unknown errors. Once done I will give EndNote the flick, and my transition to OpenOffice will be complete.

In regards to the actual number of references that can be imported in a shot. I have successfully imported 1000 without generating an error. So I will be cutting my EndNote library up into smaller bits for importation. Thanks everyone for your input.

First you might try using RIS instead of BibTeX. Otherwise you might check to make sure that you have the most up to date version of the BibTeX export format from Endnote.

The next step beyond this would be to open up the exported file in a text editor and make sure it looks OK. If you are still having trouble with the file there is a good chance you can massage it a little bit with a few thoughtful uses of find and replace.

Lots of Zotero users have transferred over from Endnote, many with thousands of references in their collections. Generally if you are willing to fuss with it a little bit you can get a very clean transfer.

This is a summary of how I eventually imported my Endnote Library into Zotero, and some general observations on the use of Zotero after using Endnote for many years.

General observations...
1. The generic 'Refman (RIS) Export' file provided with EndNote 5 does not result in a clean import into Zotero (I suspect different version of the RIS Format Specifications have existed over time but these versions have not been labelled as such). I searched the Endnote Style repository on the internet for the latest version of the file and found that this style is now not available. I proceeded to modify the file I had based on the latest RIS Format Specifications.
2. After doing this and conducting some trial imports, I found that not all the tags specified in the RIS Format Specifications are recognised by Zotero. I then started generalising until I obtained the maximum data transfer into Zotero Fields that made sense to me. Issues included retaining my repository data (e.g. library, reference collection, project file) and accession codes (e.g. W2345, R459, Y236 respectively) and importation of my keyword data and abstracts. I found that the keyword -> tags was unsatisfactory and messy and pushed this data, along with the abstracts into Zotero's Notes Section. I used the Zotero Abstract Field to store the RepositoryLocation:AccessionCode combination for the lack of an alternative field that imported RIS data could be sent. As Zotero functionality improves I am hoping that a field copy function will be developed that will allow me to transfer this data into the repository field available in Zotero (but which RIS Import data can not be directed). Personally I would like to see a Zotero 'text-based' RIS-like import file format where their is a TAG available for each field and reference type. People can then send their data to this text-based file using anyone of a multitude of programs e.g. endnote, databases (e.g. foxpro, access), notepad, word, etc.
3. I found that my Endnote library was best cut up into blocks of about 1000 references. Even with a reasonably fast computer this import took about 20 minutes.
4. With such a large dataset I find the use of the 'search' field very irritating. This field acts more like a dynamic filter; type 'a' and it filters the data to only show those with 'a', type 'ab' it only shows references with 'ab'. Type a name and the system shuts down for 5-10 minutes until the filter has caught up. The 'advanced search', represented by the magnifying glass is much better. This facility appears to search various indexes and produces good results in seconds and allows you to store the results as a type of collection.
5. Compared to Endnote, Zotero is a breeze at getting reference data available on the internet into its database. Coupled with a NewsReader to let me know when the latest issue of the key journals that I read are released - it has become a quick and easy routine of checking my feeds followed by reviewing the reference lists that appear and clicking on the icon in the navigation field for any reference that I wish to save. It is as easy as that. Of the various places I visit I have only encountered problems importing data this way 5% of the time. In most cases the problem centres around importing collections - if I drill down to the individual reference the import works.
6. Endnote Styles appear to provide the user with greater control of the output (combination of the huge variety of scientific styles available plus the availability of an inbuilt style editor). CSL styles are OK but biased towards the humanities and social sciences Very few scientific styles are available. Of four journals I intend to publish articles in 2008 and 2009, I am having to create CSL style sheets in XML for every journal. This adds an unexpected overhead to paper preparation. CSL/XML also represents a huge learning curve for the uninitiated.
7. For people trying to get their Endnote Data into Zotero I have provided the format of my Endnote Style Sheet below for the key reference types recognised by Zotero Bibliographic Style Sheets.


Generic
`TY - `GEN|`
AU - `Author|`
PY - `Year|`
BT - `Secondary Title|`
ED - `Secondary Author|`
CT - `Title|`
CY - `Place Published|`
PB - `Publisher|`
T3 - `Tertiary Title|`
A3 - `Series Editor|`
ET - `Edition|`
SP - `Pages|`
Y2 - `Date|`
SN - `ISBN/ISSN|`
N1 - `Notes|`
N1 - `Abstract|`
N1 - `Keywords|`
N2..- `Accession Number|`
VL - `Volume|`
UR - `URL

Journal Article
`TY - `JOUR|`
AU - `Author|`
PY - `Year|`
TI - `Title|`
SP - `Pages|`
JF - `Journal|`
VL - `Volume|`
IS - `Issue|`
N1 - `Notes|`
N1 - `Abstract|`
N1 - `Keywords|`
N2 - `Location|: Accession Number|`
UR - `URL

Book
`TY - `BOOK|`
AU - `Author|`
PY - `Year|`
BT - `Title|`
CY - `City|`
PB - `Publisher|`
SP - `Number of Pages|`
T3 - `Series Title|`
ED - `Series Editor|`
ET - `Edition|`
VL - `Volume|`
Y2 - `Original Publication|`
SN - `ISBN|`
N1 - `Notes|`
N1 - `Notes|`
N1 - `Keywords|`
N2 - `Location|: Accession Number|`
VL - `Volume|`
UR - `URL

Book Section
`TY - `CHAP|`
AU - `Author|`
PY - `Year|`
BT - `Book Title|`
ED - `Editor|`
CT - `Title|`
CY - `City|`
PB - `Publisher|`
ET - `Edition|`
VL - `Volume|`
T3 - `Series Title|`
SP - `Pages|`
N1 - `Notes|`
N1 - `Notes|`
N1 - `Keywords|`
N2 - `Location|: Accession Number|`
VL - `Volume|`
SN - `ISBN|`
UR - `URL


BTW, some problems with Endnote export are a consequence of really old (and AFAIK unfixed) bugs in Endnote.

Bruce,

You stated...

I'm biased, but I'm not the only one who believes CSL is much better designed than Endnote's style system purely from the styling standpoint.

At present, I am currently unclear of my opinion of CSL. As you may have noticed I stated "Endnote Styles appear to provide the user with greater control" - empahasis on "appear". I am still fiddling with XML creating a range of new styles and reserve judgement until I have fully explored CSL functionality.

One thing I have noticed is that most styles in Zotero only have 3 output styles - BOOK, CHAPTER and DEFAULT (formatted to present as a JOURNAL entry). This is very limited in my view. I have been told in other parts of the forum that this system captures most situations but I am yet to be convinced. I have a wide variety of webpages, reports, conferences, unpublished manuscripts, maps, CD Software, Computer Programs and Legislation that have quite specific data that needs to be presented in a set way in a bibliography. Having all these items presented like a journal is limiting. The problem though does not appear to be in the ability of CSL/XML to render the references only that people don't wish to program for all these reference types.

Simple Styles, as found in the Zotero Style Repository, already appear complicated, especially without any explanatory comments in the code. How is a style with all types of references accounted for going to look and be maintained? Questions I am sure that would spark a debate in other sections of the forum.

Suffice to say I have not finalised my judgement on this part of Zotero until I have fully explored its functionality. I will post my views in the appropriate spot once I have finished and submit my styles to the repository when they are complete.

One thing I have noticed is that most styles in Zotero only have 3 output styles - BOOK, CHAPTER and DEFAULT (formatted to present as a JOURNAL entry). This is very limited in my view. I have been told in other parts of the forum that this system captures most situations but I am yet to be convinced.

My argument is that the best, most robust, styles are typically not going to have any type conditional logic in the citation or bibliography elements, but will restrict those to macros.

I have a wide variety of webpages, reports, conferences, unpublished manuscripts, maps, CD Software, Computer Programs and Legislation that have quite specific data that needs to be presented in a set way in a bibliography.

I'd urge you to reassess that assumption. I know there are exceptions where one does need to tweak output for some types, but I don't think it's common. A good macro can go a really long way. And macros are a feature that Endnote's styling system doesn't have the last I recall.

See some design notes of mine from awhile back.

Having all these items presented like a journal is limiting.

It's not that simple. In CSL, all types have one of three fallback types: article, book, and chapter. These base types correspond to the structural characteristics I describe in that link above. So when, say, Zotero sees a record for which it does not see any particular CSL logic, it maps it to the corresponding base type. This is why for many case, you don't need definitions for much more than those three types.

Aside: with macros, this feature doesn't really need to be here, but has remained for what I'd call legacy reasons.

The problem though does not appear to be in the ability of CSL/XML to render the references only that people don't wish to program for all these reference types.

The more code, the longer it takes, and the more buggy it potentially is. So it makes sense to code for the common cases, and address problems as they arise.

Simple Styles, as found in the Zotero Style Repository, already appear complicated, especially without any explanatory comments in the code. How is a style with all types of references accounted for going to look and be maintained?

As I hinted above, in order to understand the design of CSL, you need to stop thinking about citation formatting through the lens of reference types. If you have styles that are a collection of smartly designed macros, with simple citation and bibliographic definitions, it becomes quite easy to maintain those styles. Indeed, that's a big part of the idea behind the macro system.

Yes, you are on the right track.

But, I'd go a little further. Take a look at the bibliography section for the APA style. It is only a series of macro and variable calls (though I'm a little confused about why there's both a "container-contributors" and a "secondary-contributors" macro; they ought to be the same thing).

Or, this more complex Chicago style also shows the basic idea. There you have fragments like:

<text macro="contributors"/>
<text macro="title"/>
<text macro="description"/>
<text macro="secondary-contributors"/>
So the code here is all generic, and if there's any type or data-specific logic, it happens in those macros.

Also, on your last question, it depends. In some cases, you'd be more likely to be making tweaks to the macros.

I have no problem importing that record in Zotero 1.5. What error do you get in "Report errors..." when you try?

Rintze's idea is a good one. In addition, Zotero should give the record number and a few pieces of identifying information (e.g., first two lines).

Sean, I'm not sure what you mean. All I get is the text I quoted in my earlier post. Is there some place to look for more detailed information?

Or perhaps Zotero could spew out a new RIS-file with the records that couldn't be parsed. That would probably be more helpful than a log.

I've read through all of the posts in this thread hoping to effect a smooth transfer from Endnote X (for Mac) to Zotero. I've tried exporting my Endnote references using both RefMan (RIS) and BibText, but in both cases when I go to import into Zotero I get the error message "no translator could be found for the given file." Wondering if I've missed the resolution to this issue, as I notice aaronwilcher brought it up in October. Suggestions much appreciated.

For a, you should be able to load EndNote styles via the Style Manager found in Zotero 1.5 beta (Zotero 1.0.x doesn't support EndNote styles). More info here: http://www.zotero.org/support/styles

For b, you will have to locate your Zotero library and copy it to the Firefox profile on your other computer. More info here: http://www.zotero.org/support/zotero_data

Whoops. Forgot you were using Endnote halfway writing my post. You will need to convert your EndNote library to a format Zotero can understand (RIS seems to work best). That will require a working copy of Endnote though.