F and ff characters not recognized at import from Endone X1

raverdugo

Thanks for producing this nice free software. I am hoping to completely leaving Endnote for Zotero.

I am having problems importing records from Endnote X1 using bor RIS and BibTEX formats.

With BibTEX I get weird characters every time that there is a capital F or two contiguous fs:
F -> ℉
ff -> ff
Also the dashes in page rages get deleted (e.g. 34-40 -> 3440) and Notes are not imported.
In both cases the result is a single character. The ff looks like two f's but it is really a single character.

When using RIS format this problem is gone and all characters are imported fine. However, it also imports a bunch of notes for every reference that don't make any sense. My guess that this is caused by the way EndNote tags the fields in RIS format. Maybe I could fix this if i new how Zotero expects each field to be named. For instance, this is the template for journal Articles in RIS format in Endnote X1.

`TY - `JOUR|`
AU - `Author|`
PY - `Year|`
TI - `Title|`
SP - `Pages|`
N1 - `Date|`
JF - `Journal|`
JO - `Alternate Journal|`
VL - `Volume|`
IS - `Issue|`
SN - `ISSN|`
N1 - `Type of Article|`
N1 - `Short Title|`
N1 - `Call Number|`
N1 - `Accession Number|`
N1 - `Label|`
N1 - `Notes|`
M3 - `DOI|`
N1 - `Original Publication|`
RP - `Reprint Edition|`
N1 - `Reviewed Item|`
N1 - `Research Notes|`
L1 - `File Attachments|`
N1 - `Caption|`
Y2 - `Access Date|`
N1 - `Last Modified Date|`
N1 - `Translated Author|`
N1 - `Translated Title|`
N1 - `Name of Database|`
N1 - `Database Provider|`
N1 - `Language|`
KW - `Keywords|`
N2 - `Abstract|`
AD - `Author Address|`
UR - `URL|

The result is that date and accession number are not imported in their respective fields but as notes, along with the real notes. BTY, all the actual notes are imported as a single line note (hard to read). They should be imported as separate notes (delimited by new-line characters in RIS format).

Am I missing something? Is there a fix for this? It seems like a lot of people would like to export from EndNote to Zotero so maybe I am not following instructions correctly but I haven't been able to figure this out.

Thanks in advance for your help.

Ricardo.

PS: Here is an example of a BibTEX record badly imported
@article{
Author = {Futschik, M. and Crompton, T.},
Title = {Model selection and efficiency testing for normalization of cDNA microarray data},
Journal = {Genome Biol},
Volume = {5},
Number = {8},
Pages = {R60},
Note = {1465-6914
Journal Article},
Abstract = {In this study we present two novel normalization schemes for cDNA microarrays. They are based on iterative local regression and optimization of model parameters by generalized cross-validation. Permutation tests assessing the efficiency of normalization demonstrated that the proposed schemes have an improved ability to remove systematic errors and to reduce variability in microarray data. The analysis also reveals that without parameter optimization local regression is frequently insufficient to remove systematic errors in microarray data.},
Keywords = {Algorithms
Bias (Epidemiology)
Cell Line, Tumor
Colonic Neoplasms/genetics/pathology
DNA, Complementary/*genetics
Dyes
Fibroblasts/drug effects/metabolism
Gene Expression Profiling/methods/standards
Humans
Lymphatic Metastasis/genetics
Male
Middle Aged
Nucleic Acid Hybridization
Oligonucleotide Array Sequence Analysis/*methods/*standards
RNA, Messenger/analysis/genetics
Reference Standards
Reproducibility of Results
Research Design},
Year = {2004} }





PS2: Here is an example of a RIS record
TY - JOUR
AU - Bolstad, B. M.
AU - Irizarry, R. A.
AU - Astrand, M.
AU - Speed, T. P.
PY - 2003
TI - A comparison of normalization methods for high density oligonucleotide array data based on variance and bias
SP - 185-93
N1 - Jan 22
JF - Bioinformatics
VL - 19
IS - 2
N1 - A comparison of normalization methods for high density oligonucleotide array data based on variance and bias
N1 - 12538238
N1 - 1367-4803
Evaluation Studies
Journal Article
Validation Studies
KW - *Algorithms
Calibration
Comparative Study
Models, Genetic
Molecular Probes
Nonlinear Dynamics
Oligonucleotide Array Sequence
Analysis/*instrumentation/*methods/standards
Quality Control
Sequence Analysis, DNA/*methods/standards
Stochastic Processes
N2 - MOTIVATION: When running experiments that involve multiple high density oligonucleotide arrays, it is important to remove sources of variation between arrays of non-biological origin. Normalization is a process for reducing this variation. It is common to see non-linear relations between arrays and the standard normalization provided by Affymetrix does not perform well in these situations. RESULTS: We present three methods of performing normalization at the probe intensity level. These methods are called complete data methods because they make use of data from all arrays in an experiment to form the normalizing relation. These algorithms are compared to two methods that make use of a baseline array: a one number scaling based algorithm and a method that uses a non-linear normalizing relation by comparing the variability and bias of an expression measure. Two publicly available datasets are used to carry out the comparisons. The simplest and quickest complete data method is found to perform favorably. AVAILABILITY: Software implementing all three of the complete data normalization methods is available as part of the R package Affy, which is a part of the Bioconductor project http://www.bioconductor.org. SUPPLEMENTARY INFORMATION: Additional figures may be found at http://www.stat.berkeley.edu/~bolstad/normalize/index.html
AD - Group in Biostatistics, University of California, Berkeley, CA 94720, USA. bolstad@stat.berkeley.edu
UR - http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=12538238
ID - 2
ER -

dstillman

Please try upgrading to the latest BibTeX translator by clicking Reset Translators and Styles in the Advanced pane of the Zotero prefs. (This is only necessary for BibTeX and only in Zotero 1.0.* releases.)

raverdugo

Great! That fixed the characters problem. By Changing the BibTEXT template in Endnote a little I was able to also import ISSN and URL. However, I still haven't been able to import PDF links. I can make endnote export the name of the file, but I don't know how to make Zotero recognize this as a relative link. Any help would be appreciated.
This is my BiTEX template for articles in case it is useful for others
@article{
Label,
`Author = {`Author`},`
`Title = {`Title`},`
`Journal = {`Journal`},`
`Volume = {`Volume`},`
`Number = {`Issue`},`
`Pages = {`Pages`},`
`Note = {`Notes`},`
`Abstract = {`Abstract`},`
`Keywords = {`Keywords`},`
`ISSN = {`ISBN/ISSN`},`
`URL = {`URL`},`
`L4 = {`File Attachments`},`
`L4 = {`Figure`},`
`Year = {`|Year|`} }`

PD: i read in an website that L4 is recognized as attachment by Zotero, but it didn't work. In Endnote, I have PDFs in the Figure or the Attachment (only a recent feature in EndNote) fields.

sean

To import PDFs, add a PDF field specifying the full file path to the attachment, e.g.:

PDF = {C:\somedirectory\anotherdirectory\file.pdf}

or

PDF = {/Users/raverdugo/Documents/file.pdf}

etc.

raverdugo

Thanks very much for your quick replies. I just tried what was suggested but Zotero didn't import my PDF field. This is an example of a record exported by EndNote. It full path is included, and I know it is correct.

@article{
Author = {Tsai, Jennifer and Sultana, Razvan and Lee, Yudan and Pertea, Geo and Karamycheva, Svetlana and Antonescu, Valentin and Cho, Jennifer and Parvizi, Babak and Cheung, Foo and Quackenbush, John},
Title = {RESOURCERER: a database for annotating and linking microarray resources within and across species},
Journal = {Genome Biology},
Volume = {2},
Number = {11},
Pages = {software0002.1 - software0002.4},
Note = {Disclaimer: Whilst every effort is made by the authors, editors and publishers to ensure that the software performs as described, they accept no liability whatsoever for the consequences of downloading or using it.},
Abstract = {Microarray expression analysis is providing unprecedented data on gene expression in humans and mammalian model systems. Although such studies provide a tremendous resource for understanding human disease states, one of the significant challenges is cross-referencing the data derived from different species, across diverse expression analysis platforms, in order to properly derive inferences regarding gene expression and disease state. To address this problem, we have developed RESOURCERER, a microarray-resource annotation and cross-reference database built using the analysis of expressed sequence tags (ESTs) and gene sequences provided by the TIGR Gene Index (TGI) and TIGR Orthologous Gene Alignment (TOGA) databases [now called Eukaryotic Gene Orthologs (EGO)].},
ISSN = {ISBN/1465-6906},
URL = {http://genomebiology.com/2001/2/11/software/0002},
PDF= {/Users/rav/Documents/PDF/Tsai_2001.pdf},
Year = {2001} }

This is a mac, that's why the format of the path.

Also, Is it possible to use relative paths in Zotero (relative to the location of the Zotero library). That way, if I ever need to change my username in a future computer, or, for instance, I won't break the PDF links in Zotero (just trying to think head).

Thanks again.

sean

What you have done should work. Can you send the debug output generated during import to support@zot...org?

Since Zotero will actually copy the PDF into its own storage directory during import, there's no need to worry about relative/absolute paths.

raverdugo

Zotero didn't give me any error. It imported the records just fine, but it didn't attach any PDF to them. I also check the console and there were no errors there. Is tat stored in some file within zotero?

Also, I have a lot of PDF so I don't want to duplicate that in my hard drive. I did that mistake with EndNote already and I didn't want to repeat it. I was looking for a way to create relative links to the PDFs instead of copying them into the library. This is advantageous for other reasons too. I tend to edit PDFs in Acrobat Prof., making notes and highlighting paragraphs. I would lose track of them if I have multiple copies. It is better to edit only one. My PDF folder has all the files with descriptive names (FirstAuthorLastname_year.pdf) that makes it easy to find them without the need to start any reference manager if I am doing a quick look (with finder or Spotlight for instance). For all these reasons, I wouldn't like my PDFs to be moved, copied, or renamed. My ideal reference manager application could also use relative links, and export the PDFs when necessary. It would also grab them automatically just like Zotero does from the websites and place them in a custom folder with custom filenames if desired. I know Zotero cannot do that, but it is getting close and much better than anything else I have seen. Other people may function the same way I think.

Thanks.

raverdugo

If it is not possible to use relative or absolute links instead of copying the PDF file, I will still like to do the latter.

How can I get the debug output out of Zotero?

Thanks again for your help.

Ricardo.

dstillman

How can I get the debug output out of Zotero?

Debug output

raverdugo

After something got updated in zotero, I am now able to import the PDF along with my references exported from EndNote in BibTeX format. Thank!

As mentioned above, Zotero copies the PDF to is one storage hierarchy, which for many people is the desired behavior. I was wondering, is it possible to include also the option at import to choose importing only the link, without a copy pf the PDF?

Also, and you do set a link to a PDF using the "Link to File..." button, does it store it internally as an absolute or relative link? Would it be possible to add an option to choose among these to options? It would be useful for people, like me, who like to keep all the PDF in a centralized location, than can be easily accessed with multiple tools.

Thanks a lot for such an amazing software. I hope Zotero become the standard and that scientific journals start giving insensitive use it as a reference manager and submitting articles (as now they do for using Endnote in many cases).