Importing from Freecite

pscheytt
Hello,

I am using Freecite (http://freecite.library.brown.edu/welcome) to parse references from PDFs. Freecite creates OpenURL results like this:

<ctx:context-objects xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='info:ofi/fmt:xml:xsd:ctx http://www.openurl.info/registry/docs/info:ofi/fmt:xml:xsd:ctx' xmlns:ctx='info:ofi/fmt:xml:xsd:ctx'><ctx:context-object timestamp='2009-12-31T05:37:48-05:00' encoding='info:ofi/enc:UTF-8' version='Z39.88-2004' identifier=''><ctx:referent><ctx:metadata-by-val><ctx:format>info:ofi/fmt:xml:xsd:journal</ctx:format><ctx:metadata><journal xmlns:rft='info:ofi/fmt:xml:xsd:journal' xsi:schemaLocation='info:ofi/fmt:xml:xsd:journal http://www.openurl.info/registry/docs/info:ofi/fmt:xml:xsd:journal'><rft:atitle>Acute Myocardial Infarction in the Medicare population: process of care and clinical outcomes</rft:atitle><rft:spage>2530</rft:spage><rft:date>1992</rft:date><rft:stitle>Journal of the American Medical Association</rft:stitle><rft:genre>article</rft:genre><rft:volume>18</rft:volume><rft:epage>2536</rft:epage><rft:au>I S Udvarhelyi</rft:au><rft:au>C A Gatsonis</rft:au><rft:au>A M Epstein</rft:au><rft:au>C L Pashos</rft:au><rft:au>J P Newhouse</rft:au><rft:au>B J McNeil</rft:au></journal></ctx:metadata></ctx:metadata-by-val></ctx:referent></ctx:context-object></ctx:context-objects>

I tried to import them into Zotero using the Import from Clipboard function, but couldn't get it work (Error message: No translator found). I went through the forum and found some entries that said that Zotero works with OpenURL, so I must be doing something wrong... Any ideas or is there an easier way altogether?

Thanks a lot,
Peter.
  • dstillman
    That's not a format that Zotero supports. (It supports COinS in web pages, but that's different.) See http://www.zotero.org/support/kb/importing for a list of supported import formats.
  • ajlyon
    Peter--
    After looking at the project, I'm playing with the idea of writing a translator for that format. Since it's just OpenURL encapsulated a little differently, it should be quite easy, and the service is useful enough. I'll write a site translator, so it still wouldn't translate from the clipboard, but that's still a start.
  • dstillman
    ajlyon: If you write an import translator (following the pattern of the translators for RIS, BibTeX, etc.), it can be called from other site translators.
  • ajlyon
    I just finished a translator for FreeCite and other systems that use the XML encapsulation of OpenURL ContextObjects. To try it out, download it from http://zotero-dev.googlegroups.com/web/CTX.js and place it in the translators folder of your Zotero installation.

    Please post any problems you have.
  • jbonito
    Thanks for doing that! A problem and comment:

    Problem: Journal name (Psychological Review) was placed in the "Journal Abbr" field in Zotero rather than in the publication field.

    Comment: imports just one reference at a time even if multiple citations are simultaneously parsed.

    Joe
  • ajlyon
    I've addressed the comment, but I can't replicate the problem. Can you provide an example input?

    My solution to the Comment is imperfect; it doesn't prompt you to select which entries to import when you click on the folder icon, but it's a start. Download http://github.com/ajlyon/zotero-bits/raw/master/CTX.js and put it in the translators directory of your Zotero data directory and you should be able to download multiple citations.

    I couldn't think of an elegant way to get it to prompt for saving, but this should tide you over.
  • jbonito
    Hi Ajlyon,

    That was fast!!! Yes, it imports multiple citations properly. Thanks again. Now the interesting part--the second citation has the journal title in the right place but the first doesn't. Weird. Here are the citations:

    Aamodt, M. G., & Kimbrough, W. W. (1982). Effects of group heterogeneity on quality of task solutions. Psychological Review, 50, 171–174.
    Abbey, D. S. (1982). Conflict in unstructured groups: An explanation from control-theory. Psychological Reports, 51, 177–178.


    Put them into Freecite then import. I have a screenshot of Zotero but can't attach here. It's probably a parse issue with Freecite.

    Joe
  • ajlyon
    It's a Freecite parsing issue; if you look at the ContextObject code for the first one, you'll see:
    <rft:stitle>Psychological Review</rft:stitle>
    The stitle means "short title". Nothing we can do here. Just make sure you mark it as imperfect parsing, so their algorithm can improve.
  • jbonito
    Odd. But thanks again for your help. One last question, if you don't mind: does the translator have an upper limit on the number of citations it will process? It will do 4 at a time but not 10, or at least the folder icon does not appear in the Firefox toolbar with 10 citations.

    Joe
  • ajlyon
    In my quick testing, it will copy at least 12 without a problem, so I'm not sure what's causing the issue you're seeing.

    There should be no limit on how many citations it will process.
  • jbonito
    I think you're right--there shouldn't be a limit. But when I paste in the following block of citations, the folder icon for the translator does not appear in the location bar. Not sure why.

    Thanks again for your help and for writing the code.

    Joe

    Aamodt, M. G., & Kimbrough, W. W. (1982). Effects of group heterogeneity on quality of task solutions. Psychological Review, 50, 171-174.
    Abbey, D. S. (1982). Conflict in unstructured groups: An explanation from control-theory. Psychological Reports, 51, 177-178.
    Abele, A. E. (2003). The dynamics of masculine-agentic and feminine-communal traits: Findings from a prospective study. Journal of Personality and Social Psychology, 85, 768-776.
    Abele, A., Gendolla, G. H. E., & Petzold, P. (1998). Positive mood and in-group—out-group differentiation in a minimal group setting. Personality and Social Psychology Bulletin, 24, 1343-1357.
    Aberson, C. L., Healy, M., & Romero, V. (2000). Ingroup bias and self-esteem: A meta-analysis. Personality and Social Psychology Review, 4, 157-173.
    Abougendia, M., Joyce, A. S., Piper, W. E., & Ogrodniczuk, J. S. (2004). Alliance as a mediator of expectancy effects in short-term group psychotherapy. Group Dynamics: Theory, Research, and Practice, 8, 3-12.
    Abraham, A. (1973a). Group tensions as measured by configurations of different self and transself aspects. Group Process, 5, 71-89.
    Abraham, A. (1973b). A model for exploring intra and interindividual processes in groups. International Journal of Group Psychotherapy, 23, 3-22.
    Abraham, A. (1974-1975). Processes in groups. Bulletin de Psychogie, 28, 746-758.
    Abraham, A., Geffroy, Y., & Ancelin-Schutzenberger, A. (1980). A method for analyzing group interaction: Development and application of a video observation grid. Connexions, 31, 145-166.
    Abramo, J. L., Lundgren, D. C., & Bogart, D. H. (1978). Status threat and group dogmatism. Human Relations, 31, 745-752.
    Abrams, D., Ando, K., & Hinckle, S. W. (1998). Psychological attachment to the group: Cross-cultural differences in organizational identification and subjective norms as predictors of workers’ turnover intentions. Personality and Social Psychology Bulletin, 24, 1027-1039.
    Abrahms, D., & Brown, R. (1989). Self-consciousness and social identity: Self-regulation as a group member. Social Psychology Quarterly, 52, 311-318.
    Abrams, D., Henson, M., Marques, J., & Bown, N. (2000). Pro-norm and anti-norm deviance within and between groups. Journal of Personality and Social Psychology, 78, 906-912.
    Abrams, D., & Hogg, M. A. (1988). Comments on the motivational status of using self-esteem in social identity and intergroup discrimination. European Journal of Social Psychology, 18, 317-334.
    Abrams, D., & Hogg, M. A. (1998). Prospects for research in group processes and intergroup relations. Group Processes and Intergroup Relations, 1, 7-20.
    Abrams, D., Marques, J. M., Brown, N., & Dougill, M. (2002). Anti-norm and pro-norm deviance in the bank and on the campus: Two experiments on subjective group dynamics. Group Processes & Intergroup Relations, 5, 163-182.
  • ajlyon
    edited May 26, 2010
    The issue is not that there are too many citations, but rather that one of them is being parsed incorrectly; the system can't figure out a type for it. You'll notice that one of the ContextObjects on the page is of type "info:ofi/fmt:xml:xsd:unknown"; I hadn't accounted for unknown types in the previous version of the code.

    I've patched the code to address this. Now it will skip such items, since Freecite doesn't provide structured data for them.

    Re-download and reinstall the translator from the link above and it'll start working again.
  • ajlyon
    And now it will prompt to select which of multiple citations to import. This is now complete, both as a site translator (for Freecite) and for importing files in this format.
  • jbonito
    Thank you again for your efforts. Sorry to have taken so long to respond, but I was out of town for the past week or so.

    The translator works very well, but could I trouble you for one more modification? Is it possible, in cases where the short title is populated but the publication name is not, to simply write a short title to the publication field? The bibliography that I need to parse and then import into Zotero consists entirely of journal articles and it seems as though FreeCite can't quite parse citations that seem perfectly formatted.

    Joe
  • adamsmith
    I agree that would be a good idea - I tested Free Cite a little and the fact that it considers about 2/3 of all journal titles short titles was one of the two consistent problems I found.

    (the other one - not fixable by translator - is that it doesn't get years past 2000 as years - I sent them a quick note on that)
  • ajlyon
    I'll work on this-- I can't patch it now, but perhaps tomorrow I'll post a new version. One issue is that I'm trying to make a general translator for ContextObjects in XML format, so I didn't want to litter it with FreeCite-specific hacks. That said, using Short Title in place of Title when Title is blank is a perfectly good heuristic in general.

    But please do mark the short titles as incorrect-- I'd like to see them improve their detection.
  • jbonito
    I will certainly mark them! And thanks again for your efforts--it's an incredibly useful translator.

    Joe
  • ajlyon
    The updated version should now work for you. Download and install again from the URL above.
  • jbonito
    It looks great! Thanks for doing that. BTW, you should be careful about setting expectations very high then meeting them so quickly!

    Joe
  • jbonito
    Still working well, though a couple of interesting issues. The first is that it pays to work through the parsed list to make sure formatting is correct. If not caught there then the import will fail at the ill formatted citation. That's what it should do. Another way to catch it prior to importing is at the "select all" drop down box. If a citation is broken, you'l see "undefined" in the drop down. Deselecting all undefined entries makes for an easy import. In some cases, the broken citation has a misplaced carriage return which is an easy fix, and in other cases the cause is not readily apparent. Even so, the translator makes life so much easier.

    Second, when finishing the import FF generates multiple versions of the same message: "Some of the requests to Google Scholar failed. Probably due to the large number of requests." FF then seems to click the "OK" button until all messages go away, but on a number of occasions FF froze, requiring a restart. All newly imported citations, however, are in the database.

    joe
  • ajlyon
    The second issue doesn't make any sense to me. This translator has nothing to do with Google Scholar.

    I'll take a look at the first issue-- can you provide a few examples of broken citations to test on?
  • jbonito
    Interestingly, the errors stopped as I kept plugging away. I finished translating the approximately 6500 references in just a couple of hours using Freecite and your translator. I ran about 400 at a time (20 citations per MSWord page, times 20 pages) and it performed flawlessly. Can't thank you enough.

    In the following 3 citations, the second one parses incorrectly. Put a carriage return anywhere in the other two and then see how it parses then translates into Zotero.

    Isenberg, D. J., & Ennis, J. G. (1981). Perceiving group members: A comparison of derived and imposed dimensions. Journal of Personality and Social Psychology, 41, 293-305.
    Islam, G., & Zyphur, M. J. (2005). Power, voice, and hierarchy: Exploring the antecedents of speaking up in groups. Group Dynamics: Theory, Research, and Practice, 9, 93-103.
    Islam, M. R., & Hewstone, M. (1993a). Dimensions of contact as predictors of intergroup anxiety, perceived out-group variability, and out-group attitude: An integrative model. Personality and Social Psychology Bulletin, 19, 700-710.
  • ajlyon
    I still don't see an "undefined" in the title selection window, and those all import into Zotero, even if I mangle them. Yes, there are cases where Freecite can't parse a citation, but it looks to me like Zotero is doing all it can at this point.
  • ajlyon
    By the way, the new translator has been committed. Manually update translators from the Zotero preferences to get it now, otherwise it will be automatically installed within 24 hours.

    Please continue to post errors that arise, but right now it looks like most errors are Freecite limitations, not Zotero errors.
  • jbonito
    I think you are right--Zotero and the translator are working well. Sorry not able to recreate the error but it happened early in the process and I didn't quite note all relevant characteristics.

    Joe
  • sdspieg
    Sorry for posting in this thread - but it is still the first one that comes up when you search for 'Freecite'. I'm trying to import a bunch of papers that were never officially published and therefore cannot be automatically downloaded from the usual places. I played around a bit with FreeCite and it seems to me it should do the trick - anything beats copying/pasting bits of citations into Zotero. But does anybody has a reference to what it actually parses and how? Even a list of examples for different types of sources (book, book chapter, etc.) would be great, as I could then make the changes that would allow me to then import the whole bunch into a format that I can then import into Zotero. Any help would be appreciated!

    -Stephan
Sign In or Register to comment.