Call numbers in RIS format

fiacha
I am migrating a collection of about 500 records from RefWorks to Zotero, and I am encountering a little snag. The RefWorks BibTex export output I initially tried did not contain many of the fields in the original records, including call number, so I exported them to a RIS format instead. This worked much better, but unfortunately my Call Number fields were apparently dumped to a generic note field, as in this example:

TY - BOOK
ID - 69
T1 - The New Oxford Annotated Bible, New Revised Standard Version with the Apocrypha
Y1 - 2001
KW - Bible--Commentaries
N1 - M2: 3; M2: Book; Date of Input: 8/29/2007; Date Modified: 8/29/2007; Priority: Normal; My Library
A2 - Coogan,Michael D.
PB - Oxford University Press
CY - New York
A3 - Anonymous
SN - 019528478X
U1 - 01-20-06 Amazon.com
N1 - BS191.5 A1
M1 - Book, Whole
ER -

The result is that call numbers end up in notes instead of Zotero's designated call number field. Since I categorize my own home library by LOC call number, this is a bit annoying. Is there some workaround for this? I could do a find and replace (carefully, since almost all my records have more than one N1 field) to rename the N1 call number field to something else, but I'm guessing if the RIS standard doesn't even support a call number field, that won't really help me. I'd appreciate any suggestion that would allow me to avoid manually re-entering these call numbers in the appropriate field.
  • noksagt
    Can you export to the so-called Endnote format (Refer/BibIX)? If so, then call number are tagged with %L.

    http://en.wikipedia.org/wiki/Comparison_of_reference_management_software#Export_file_formats

    suggests that you can't, but it may or may not be up-to-date.



    If you can't, you can use 'bibutils' to convert to MODS XML and rename the branch "classification" and/or can then export endnote format & use "%L"

    Note to devs: it'd be nice if MODS importer (and possibly exporter) allowed call numbers to be used in the 'identifier' branch.
  • fiacha
    You are correct that I cannot export to the Endnote format.

    I appreciate your suggestion of using bibutils, but I cannot seem to make them work. I downloaded the appropriate version (bibutils_3.40_win.zip), unzipped it, moved the appropriate .exe (ris2xml.exe) to c:\, moved my formatout.ris file to c:\, and then attempted to run the program and enter the command according to the syntax provided on the site (ris2xml ris_file.ris > output_file.xml), but the program doesn't seem to respond to anything I do. The documentation on the bibutils site is pretty basic, and I don't do much in Windows command line these days. There's evidently something basic I'm missing, but I don't know what it would be; the program doesn't even respond to me when I enter the flags for help or version. I have no idea what I'm doing wrong and I can't find any more detailed documentation for the program or a non-Linux GUI for it. If anyone can point me towards more detailed (read: for Dummies) documentation for bibutils or to a Windows-compatible GUI for it, I'd be very grateful.
  • noksagt
    edited May 28, 2008
    then attempted to run the program and enter the command according to the syntax provided on the site (ris2xml ris_file.ris > output_file.xml), but the program doesn't seem to respond to anything I do.
    Bibutils outputs to standard out and the '> output_file.xml' is just to redirect to a file. Omit this redirection for debugging purposes.

    Try 'ris2xml ris_file.ris' and 'ris2xml --verobse ris_file.ris' to see what is output.
    non-Linux GUI
    'refbase' or other tools that use 'bibutils' might be used, but many will concatenate the notes together, etc.

    BibTranslator might be worth a try, although it runs a year old version of bibutils & may have limits on the filesize.
  • fiacha
    edited May 29, 2008
    Thanks for those further suggestions. I still must be doing something wrong with the ris2xml program. I ran the program, entered "ris2xml formatout.ris" and hit return and it just sits there, the blinking cursor mocking me. I did it again, including the --verbose operator as you suggested; still nothing. Then I tried both versions of the command including the full path for the file, which is just c:\formatout.ris, just in case. Same result.

    I tried BibTranslator, and it did accept my entire file and translate it, but when I copied and pasted the resulting translation into a text file and then renamed it with the .xml extension Zotero wouldn't import it; I received the error message "An error occurred while trying to import the selected file. Please ensure that the file is valid and try again." This occurred trying to import it specifically as a MODS file and also without specifying the file type.

    I imagine refbase would work, but since I haven't a spare Apache web server to install it on, that isn't a practical option for me.

    It appears that I will end up transferring the call number information from the notes field to the call number field manually for this set of records. I can think of worse things.
  • noksagt
    edited May 29, 2008
    ran the program, entered "ris2xml formatout.ris" and hit return and it just sits there, the blinking cursor mocking me.
    How many references do you have? Have you tried a file with just one reference to test it? Also: bring up the task manager (ctrl-shift-escape] and see if the process is actually using your CPU.
    when I copied and pasted the resulting translation into a text file and then renamed it with the .xml extension Zotero wouldn't import it;
    You might look for bugs when you import it. The MODS that bibutils makes is simple & clean, so I don't know why it wouldn't work. Did you check the file at all e.g. that it at least has the last closing tag that it needs, if not actually running it through a validator)?

    Have you tried the Endnote output from that app?
    I imagine refbase would work, but since I haven't a spare Apache web server to install it on, that isn't a practical option for me.
    Feel free to use a public install, but again this might not be easy if the call number is appended onto the same notes field that other actual notes are on (but this same problem is likely to happen many places, since RefWorks uses 'N1' for multiple fields).

    JabRef might also be worth a try for the conversion. It does not use 'bibutils,' but should run on your machine & should be able to do the needed imports and exports.
  • fiacha
    Thanks again for all the suggestions. I don't appear to be getting anywhere with any of them, however, and I also realized that due to the limitations either of RefWorks or of the RIS format, I will have to manually edit almost all the records anyway. The RIS file does not properly handle the distinctions between author, editor, and translator, nor apparently does it handle original titles of translated works correctly. Since my field is medieval history, a large proportion of the 500 records will require manual editing to properly designate editors and/or translators. It is disappointing that RefWorks provides so few export options, but I suppose it's hardly surprising given that the ability to export one's records properly facilitates switching to a competing program.

    Just in case you are interested in how a few of the other solutions worked out, I tried testing just one reference in ris2xml with no difference. I do not see any trace of the program in my running processes when I use it. It appears that the xml format generated by the BibTranslator site is so full of errors, an xml validation utility quit partway through because there were too many to continue. I have no idea what that's about. I am not familiar with the proper syntax for xml files, so I can't tell anything looking at the file, except that when I rename it with the .xml extension and it opens in a web browser, it quits about 1/4 of the way through with an error. I did not try the Endnote format. I could not figure out how to use the public install of refbase to convert anything. I tried JabRef but it collapsed all notes fields into one and furthermore mangled all my accented characters, of which there are a great many.

    It appears to me that, since RefWorks has evidently severely hobbled their BibTeX output capability, the least painful solution at this point is to hand-edit. I appreciate all your suggestions.
  • noksagt
    Sorry you didn't have more success. I can try to convert the file for you if you send it to donatetofoss@gmail.com (please put "ZOTERO" in the subject line), but I won't be able to edit the various fields for you--I'd just send you the Endnote format & the MODS format so that you could edit them yourself & start off slightly better than having to do everything from within Zotero.
  • Matthias
    In case of refbase, you'd do the following:

    1. export your data from RefWorks using the RefWorks tagged format (which is supported by refbase)
    2. login to http://beta.refbase.net/ and goto http://beta.refbase.net/import.php
    3. Import your data
    4. After import, click the Zotero folder icon in the URL bar to fetch the data directly from refbase, or export your data to MODS/RIS/Endnote (see bottom of the results page) and import the file to Zotero
  • noksagt
    But please note the refbase (like JabRef and most other programs) has a single notes field, so all of the N1s in RefWorks (call numbers or not) will be concatenated.
  • fiacha
    I decided to e-mail tech support at RefWorks and was told that although their export format for BibTeX is set, one can also customize their pre-provided BibTeX format as a bibliography output style. I think this would solve my problem, except that I need to know how to tag the fields for "translator," "original title," and "call number" so that Zotero can figure out what they are and map them to the appropriate fields. I poked around in the documentation for Zotero a bit, but couldn't find any explanation of how the import filters translate different tags into Zotero's fields. Any suggestions?
  • fiacha
    Actually, I did finally find the .sql file with Zotero's list of fields, so I used "callNumber" for call number and "translator" for translator in my custom BibTeX file, figured out how to get the exported data into a text file and rename it .bib, and imported it into Zotero. Zotero recognized it as a valid BibTeX format, but failed to pick up the translator and callNumber fields I included, which I'm now assuming is because the BibTeX converter isn't set up to look for them or I'm not using the field names the converter is looking for. Which comes down to almost the same thing. Getting closer, but still need some help.
  • noksagt
    I can tell you that the bibtex translator did not import call numbers when I last checked & it may not import the translator field. Neither of these is a core bibtex field and there doesn't seem to be strong consensus in third party packages to support them.

    However, if you say that the exporter already outputs these fields, I don't see why the importer shouldn't import them too (it just doesn't at this time) & should be a straight-forward patch.

    Feel free to look into it. I may or may not be able to do this later.
  • fiacha
    Where might one take a look at the source code for the BibTeX translator to see how difficult it would be to modify for this functionality? I do very little coding, but if it's just a matter of adding a few new lines to the existing translator it oughtn't to be that difficult.
  • noksagt
    I think I misunderstood you--Zotero doesn't currently export the callnumber & exports translators as authors. Because of this & because there's no sort of consensus on these fields, I won't be the one to make this change.

    Translator source is in the scrapers.sql file. I tend to manually hack it, but you can use scaffold. More info on translators at:
    http://dev.zotero.org/creating_translators_for_sites
  • ias
    Thank you for the above. I am trying to get out of Refworks into Zotero, but am having similar problems.

    - I have used the fields 'user 1', 'user 2', etc. for taking notes. With a lot of fiddling, I can probably add these fields to a BibTex like output file by using the create bibliography feature in Refworks: I choose the BibTex citation format, which I then edit to include those fields. It seems I have to add the field to every output type (book, article...) But then how do I transfer these into notes that can be imported?

    - Why does it seem that the date of creation and viewing of webpages is dropped in importing. Does Bibtex not use those fields? Just guessing, haven't investigated.

    - A lot of my notes and abstracts have some sort of rich formatting (I assume html). (Even titles, when copy pasted from a e.g. bold text, end up in bold etc. -- which I don't need). How can I have Zotero import the User1 etc fields with formatting?

    Thank you very much in advance.
  • ias
    This is a problem. I will loose all my notes if I switch. There must be a way.
  • Tjowens
    What do the notes look like in the exported file? Is Refworks including the HTML markup in the file it generates?
  • ias
    The html seems to disappear. That is bad, but it is more important to figure out how to get the notes in. Here an example record with the User field denoted as U1(no U2, U3.. in this case)

    TY - ICOMM
    ID - 311
    A1 - Ковалев,Андрей
    T1 - АРХИВ 19.04.2004 Артпериодика
    Y1 - 2004
    Y2 - 19.04.2004
    IS - 06.10.2005
    UR - http://www.artinfo.ru/ru/news/main/ArtArchive/2004-19-04-ap.htm
    PB - ARTINFO
    U1 - http://www.artinfo.ru/ru/news/main/ArtArchive/2004-19-04-ap.htm <<NYTimes об успехах московских галерей Айдан и XL на Armory Show - " XL, the painting team of Vladimir Dubossarsky and Alexander Vinogradov contribute an image of a bald, blue-eyed Barbie, very heroic, very eyes-to-the-sky. They're cool. Aidan Gallery, another newcomer, has a Photo-Realist painting of a SoHo facade by Sergey Ossovski, pure 1970's but done in 2001, and an intriguing unstretched painting titled "Baghdad Bombing" by Timur Novikov, who died of AIDS last year. With its image of rockets falling in decorative patterns through a night sky, the painting was done in 1991 at the very start of the Persian Gulf war". Как видите, не обошлось без "умершего от СПИДА">
    M1 - Web Page
    ER -
  • Tjowens
    You can generally remap fields to match the RIS spec with careful use of the find and replace functions in your favorite text editor. In this case if you find all of your "U1 -" and replace with "N1 -" you should be able to pull in all the notes. The same general principle applies to any other custom user fields.
Sign In or Register to comment.