Importing EndNote files into Zotero

ElisabethSvensson

Hi,

I'm trying to import a test library from EndNote to Zotero using these instructions. Since I don't really know anything about programming I'm unsure what parts of the script I need to modify to get it to work properly. I have exported my End Note library to a .ris file (C:\Users\Elisabeth\Desktop\testing.ris) and created the .vbs file and entered the path to the pdfs (I hope...).

' RIS link fixer for windows
' transforms Endnote internal-pdf:// links into file:// links

Dim FileName, objDialog, boolResult
Dim regEx, FileContents, modFileContents
Dim objShell, pdfFolder

'choose RIS file
Set objDialog = CreateObject("UserAccounts.CommonDialog")
objDialog.Filter = "RIS (*.ris)|*.ris|All Files (*.*)|*.*"
objDialog.FilterIndex = 1
boolResult = objDialog.ShowOpen
If boolResult = 0 Then
WScript.Quit
Else
FileName = objDialog.FileName
End If

'regexreplacement
Set regEx = New RegExp
regEx.Pattern = "internal-pdf://.*/"
FileContents = GetFile(FileName)

'choose folder containing PDFs
set objShell = CreateObject("Shell.Application")
set pdfFolder = objShell.BrowseForFolder(0, C:\Users\Elisabeth\Desktop\testing.Data\PDF, 0)
If (not pdfFolder is nothing) then
ReplaceWith = "file://" + pdfFolder.self.Path
If (Right(ReplaceWith, 1)<>"\") then
ReplaceWith = ReplaceWith & "\"
End If
regEx.Global = True
modFileContents = regEx.replace(FileContents, ReplaceWith)
If modFileContents <> FileContents Then
WriteFile FileName, modFileContents
Wscript.Echo "The RIS file has successfully been modified."
Else
Wscript.Echo "No EndNote attachments have been detected in the RIS file."
End If
End If

function GetFile(FileName)
If FileName<>"" Then
Dim FS, FileStream
Set FS = CreateObject("Scripting.FileSystemObject")
on error resume Next
Set FileStream = FS.OpenTextFile(FileName)
GetFile = FileStream.ReadAll
End If
End Function

function WriteFile(FileName, Contents)
Dim OutStream, FS
on error resume Next
Set FS = CreateObject("Scripting.FileSystemObject")
Set OutStream = FS.OpenTextFile(FileName, 2, True)
OutStream.Write Contents
End Function

aurimas

you are missing quotation marks around the path you are putting into the script.

Edit: also you will need to change backslashes (\) to double backslashes (\\). Note that the import procedure will be greatly simplified in the next Zotero release (due out within a week) and will no longer require a script.

ElisabethSvensson

Thanks!

But now I got another error message that ActiveX cannot create "UserAccounts.CommonDialog" Line 9, char 1

ElisabethSvensson

Sorry missed your edit there while I was writing. I didn't know about the new update, so perhaps I should just wait a week...

adamsmith

@aurimas - any reason Elisabeth couldn't just use the branch - which won't change much until release? It'd be nice for someone to give the new endnote import feature a test ride.

noksagt

For whatever it is worth: UserAccounts.CommonDialog is a file picker that was meant to make it so you wouldn't have to modify the script. But they removed this in Windows 7. There hasn't been interest in updating the script, as most people have either used an older copy of Windows or one of the *nix scripts or a text editor to accomplish it & there's even less motivation to update it with the upcoming version.

aurimas

sure, we could have her test it.

Elisabeth, if you are using Firefox, or you don't mind using Firefox, you can get the "beta" version of Zotero from http://www.zotero.org/support/dev_builds#zotero_for_firefox_development_xpis Open the link in Firefox and click on the link on that page that says "Install Zotero 3.0 Branch XPI"

Once Zotero is installed, you can import your RIS file as follows:

1) Copy the RIS file to your EndNote data directory ( C:\Users\Elisabeth\Desktop\testing.Data)
2) Use Zotero's Gear menu -> Import... feature to import the RIS file you just moved.

Let us know how it goes.

adamsmith

if you do what Aurimas describes - and I recommend you do - you should go back to "regular" Zotero once the 3.0.9 version comes out, likely this weekend. Just re-install it from zotero.org.

While there are stern warnings on the download page aurimas links to, the branch xpi is very stable and there is no risk installing it as it's about to be released as the next Zotero version.

ElisabethSvensson

Ok so I tried the branch and figured out how to import the file. In my test library I had some references with URLs and some without and the PDFs are only imported if there is no URL available. This is probably due to teh fact that in the Refman Journal Article template, L1 is set to 'URL' instead of File attachment as it is in e.g. the Generic template. I changed my Refman style so that also the Journal article template had L1 set to File attachemnt, but then I get an error message when I try to import the .txt file to Zotero "Unsupported format".

ElisabethSvensson

Thanks for the input noksagt. I do use Windows 7 and I have no idea what a *nix script is, but it's good to know what went wrong. This script was the method that came up when I google this and it had a fairly recent input which made me think that this was the way to go.

adamsmith

Can you post below two individual RIS examples that don't import correctly - one with File Attachment and URL and one that produces the "Unsupported format" error? (Open the .txt file with notepad and just copy & paste here.)

edit: *nix stands for all unix-family operating systems, i.e. unix, linux, and Mac OS - as you'll have seen on that page this is much easier on those systems, requiring only one line on the commandline.

ElisabethSvensson

Doesn't import PDF if URL is present:

TY - JOUR
AB - Stable carbon isotopes and C/N ratios of particulate organic matter (POM) in suspended solids, surficial sediments and sediment cores were used to define the spatial and temporal variability of POM in a poorly flushed, urbanized, eutrophic tropical estuary (Hunts Bay, Kingston Harbour, Jamaica). C/N Variation in the sediment surface POM is a function of initial suspended particulate organic matter (SPOM) (or other POM) type and the alteration of CM ratios in the water column or on the sediment surface. The delta(13)C(SPOM) (- 20 to - 25 parts per thousand) values suggest that this material is a mixture of: (1) in situ phytoplankton organic matter; (2) terrestrial river-borne SPOM; (3) terrestrial river-borne bottom sediment POM; and (4) sewage. Downcore variation in organic carbon content, C/N and delta(13)C is attributed mainly to change in the supply rate and type of organic matter. In the NE of Hunts Bay, down core variation in sedimentology and geochemistry are consistent with a change fi om fully marine to freshwater runoff-dominated sedimentation with increasing organic matter input from sewage in recent times. Despite large overlaps in the C/N and delta(13)C(org) 'end-members' pollutant POM, in this case sewage, was the only source which could account for the amount of POM deposited, the surface sediment C/N and delta(13)C(org) values and the trajectories of evolution in C/N and delta(13)C(org) values in cores. The data show that the combined delta(13)C and C/N successfully identifies the source, fate and history of POM even in a poorly-mixed estuary. (C) 1998 Academic Press Limited.
AD - Andrews, JE
Univ E Anglia, Sch Environm Sci, Norwich NR4 7TJ, Norfolk, England
Univ E Anglia, Sch Environm Sci, Norwich NR4 7TJ, Norfolk, England
Univ W Indies, Dept Chem, Kingston 7, Jamaica
AN - ISI:000073906200012
AU - Andrews, J. E.
AU - Greenaway, A. M.
AU - Dennis, P. F.
DA - May
J2 - Estuarine Coastal Shelf Sci.
KW - organic matter
sewage
carbon isotopes
carbon
nitrogen
c/n ratios
estuary
kingston harbour
jamaica
nitrogen stable-isotope
marine-phytoplankton
macrotidal estuary
amazon river
north-sea
photosynthesis
fractionation
diagenesis
transport
sediments
L1 - ://000073906200012
LA - English
IS - 5
N1 - Zq820
Times Cited:45
Cited References Count:39
PY - 1998
SN - 0272-7714
SP - 743-756
ST - Combined carbon isotope and C/N ratios as indicators of source and fate of organic matter in a poorly flushed, tropical estuary: Hunts Bay, Kingston Harbour, Jamaica
T2 - Estuarine Coastal and Shelf Science
TI - Combined carbon isotope and C/N ratios as indicators of source and fate of organic matter in a poorly flushed, tropical estuary: Hunts Bay, Kingston Harbour, Jamaica
UR - ://000073906200012
VL - 46
ID - 15
ER -

ElisabethSvensson

Unsupported format:

^TY - JOUR
AB - Stable carbon isotopes and C/N ratios of particulate organic matter (POM) in suspended solids, surficial sediments and sediment cores were used to define the spatial and temporal variability of POM in a poorly flushed, urbanized, eutrophic tropical estuary (Hunts Bay, Kingston Harbour, Jamaica). C/N Variation in the sediment surface POM is a function of initial suspended particulate organic matter (SPOM) (or other POM) type and the alteration of CM ratios in the water column or on the sediment surface. The delta(13)C(SPOM) (- 20 to - 25 parts per thousand) values suggest that this material is a mixture of: (1) in situ phytoplankton organic matter; (2) terrestrial river-borne SPOM; (3) terrestrial river-borne bottom sediment POM; and (4) sewage. Downcore variation in organic carbon content, C/N and delta(13)C is attributed mainly to change in the supply rate and type of organic matter. In the NE of Hunts Bay, down core variation in sedimentology and geochemistry are consistent with a change fi om fully marine to freshwater runoff-dominated sedimentation with increasing organic matter input from sewage in recent times. Despite large overlaps in the C/N and delta(13)C(org) 'end-members' pollutant POM, in this case sewage, was the only source which could account for the amount of POM deposited, the surface sediment C/N and delta(13)C(org) values and the trajectories of evolution in C/N and delta(13)C(org) values in cores. The data show that the combined delta(13)C and C/N successfully identifies the source, fate and history of POM even in a poorly-mixed estuary. (C) 1998 Academic Press Limited.
AD - Andrews, JE
Univ E Anglia, Sch Environm Sci, Norwich NR4 7TJ, Norfolk, England
Univ E Anglia, Sch Environm Sci, Norwich NR4 7TJ, Norfolk, England
Univ W Indies, Dept Chem, Kingston 7, Jamaica
AN - ISI:000073906200012
AU - Andrews, J. E.
AU - Greenaway, A. M.
AU - Dennis, P. F.
DA - May
J2 - Estuarine Coastal Shelf Sci.
KW - organic matter
sewage
carbon isotopes
carbon
nitrogen
c/n ratios
estuary
kingston harbour
jamaica
nitrogen stable-isotope
marine-phytoplankton
macrotidal estuary
amazon river
north-sea
photosynthesis
fractionation
diagenesis
transport
sediments
L1 - internal-pdf://Andrews et al 1998-3490059532/Andrews et al 1998.pdf
LA - English
IS - 5
N1 - Zq820
Times Cited:45
Cited References Count:39
PY - 1998
SN - 0272-7714
SP - 743-756
ST - Combined carbon isotope and C/N ratios as indicators of source and fate of organic matter in a poorly flushed, tropical estuary: Hunts Bay, Kingston Harbour, Jamaica
T2 - Estuarine Coastal and Shelf Science
TI - Combined carbon isotope and C/N ratios as indicators of source and fate of organic matter in a poorly flushed, tropical estuary: Hunts Bay, Kingston Harbour, Jamaica
UR - ://000073906200012
VL - 46
ID - 15
ER -

adamsmith

The first one doesn't actually have a file path in it, so there's nothing Zotero can do about that.

The second one should work, though. Is the ^ before the TY from the original? That would cause Zotero to fail.

ElisabethSvensson

No there's no file path as the Refman style puts the URL in that spot instead and doesn't have the File attachment by default for the Journal Article template. Which is why I changed the style so that L1 is File attachment also in the Journal Article template. But then I got the unsupported format warning.

The indent was there after export but removing it didn't make any difference. Still unsupported format.

But now it's midnight in my part of the world so further test will have to wait until tomorrow.

ElisabethSvensson

Got it to work first though :). I had only removed the indent but not the ^ that was there too. Removing that and the file was imported properly with PDFs. I did get 3 extra notes on each record though.

867ZP (this changes for each record)

AD (author addresses in a chunk)

ID and a number (think this is the EndNOte record number)

adamsmith

excellent! The notes are there on purpose: Many users complain when they lose data on import, so we pack anything that's not supported by the RIS specs (or doesn't have a corresponding Zotero field) into a note.

ElisabethSvensson

Well that was not too complicated. Just that the Refman style needed some updating. Tomorrow I'll try it on the real EndNote library.

Could all that data be consolidated into one single note instead of 3?

adamsmith

See the discussion on one vs. multiple notes here:
https://github.com/zotero/translators/pull/449#issuecomment-8288514
input welcome, this is very easy to change, but we'll have to agree on one or the other - too many options are a mess from the usability side.
You can comment here, no need for a github account.

ElisabethSvensson

Personally I don't use all those other fields and would like to see just one note, however I do see the point of having one for each for people who really do use them. If there was an easy way to delete multiple objects in multiple records this wouldn't be an issue at all. Could you add a select all option to search?

adamsmith

To delete _all_ attached notes:

1. Use advanced search (magnifying glass icon)
2. Search for
Item Type - is - Note
(Users who are adding an import to an already existing library with some notes they want to keep can add another search condition:
Date Added - is after - [date right before import])
3. Create a Saved Search
4. go into the saved search, click on any note, then press ctrl+a (for select all).
5. Note how all notes are selected, but not the items they are attached to or other attachments.
6. Right click and select "Delete Selected Items from Library" and confirm.

dstillman

7. Empty the trash, particularly before you try to sync.

aurimas

This is probably due to teh fact that in the Refman Journal Article template, L1 is set to 'URL' instead of File attachment as it is in e.g. the Generic template.

You should update your style from the endnote website. In the most recent version L1 is missing altogether (so it suggests your style is outdated or modified). I didn't realize this before, but, as you mention, you would have to modify the style and insert
L1 - `File Attachments|`
In the Journal Article template. Their style was updated in June, so it may have changed then.

adamsmith

@Elisabeth - the import will change to one note with all missing fields - so if you have a large library you may want to consider holding off on importing. On the other hand - my instructions above will work either way to delete notes - whether there is one or three per item.

ElisabethSvensson

@adamsmith I tried ctrl+A in the first search window as I didn't realize you first had to make a saved search. But that seems simply enough to be usable. With that information you could keep the multiple notes after import. Which you have already decided not to do :).

@aurimas Yes my EndNote is a bit out of date as I'm using Zotero since ~3 years. I'm actually trying to move someone else from EndNote to Zotero. It could be a good idea to put that information (about the updated style) in a manual somewhere though, so that others who like me haven't updated their styles in a while can easily figure it out.

aurimas

It could be a good idea to put that information (about the updated style) in a manual somewhere though, so that others who like me haven't updated their styles in a while can easily figure it out.

Done. Also updated the entire page for the upcoming Zotero 3.0.9 release. Please do let us know if something on that page is not clear or does not work.

ElisabethSvensson

I just finished converting a ~470 reference EndNote library to Zotero and it was easy, fast and flawless. There were 2 references that appeared to have been imported wrong as the last name had been replaced by the initials. However this was easily fixed by switching the author field from two fields to one field. Then I double checked with the EndNote library and it turned out that the references were entered wrong there already (no comma between last and first name). This issue is a pain to fix in EndNote as you manually have to add a comma to each author. In Zotero it took a few clicks.I am really impressed!

Now, deleting the notes turned out to not work so well though. I tried the saved search method but for some reason the Saved Search show up empty even though the Search found them all. Any ideas?

adamsmith

Great to hear on the import!

In the middle panel does it say "loading item list" or is it completely blank?
(Btw. - we're talking about further steps to make it easier to delete the notes - won't help you much now and this should work, I just thought I'd mention it in case you'll help other people move from Endnote).

ElisabethSvensson

Completely blank.

adamsmith

odd.
When you right click on the saved search and select
"edit saved search" - does everything still look right (you won't see the search results).

My suggestion would be to just try this again - create another saved search. They are easy to delete, so that can't hurt.