Switching to Zotero from Papers

I've been interested in using Zotero for a while as a number of colleagues have rated it highly. However, I've been stuck on Papers 3 for some time and don't know what it would take to transfer my entire library—citations, PDFs, tags, etc—to Zotero. Is there a simple tool that allows users to import from a Papers library (sort of like the ReadCube Papers import tool, which mostly works fine)? Anything that requires manually importing each paper and recreating collections/tags seems like a huge barrier for those with large libraries who are coming from other reference managers.
«1
  • Thanks. Will this allow me to import all the tags from Papers as well?—the existing tags I have are critical to my workflow.
  • You may have to experiment with export formats from Papers some, but I think files and tags should work. Collections will definitely not work, though; they're not included in any standard exchange format.
  • Thank-you. I'll give it a try. Here's hoping that it works!

    (With regard to collections, I can probably just give all the items in the collection the same tag and later create the collections in Zotero based on the tag, assuming the import is successful and all the tags are transferred over as well).
  • Just wanted to leave a quick update, in case anyone is following a similar path in transitioning from Papers3 to Zotero...

    1. Exporting to an RIS file works, but it's less than ideal because some information seems to go missing while irrelevant/incorrect information is added. For example, if a book in Papers is listed as being published in 2019 (no month or day) and in its second edition, upon import the publication date given will be January 1, 2019 and the edition data will be gone. The same is true of journal articles. If a journal article only has year information (e.g., 2009), on import it will also have New Year's Day for that year listed as well (January 1, 2009).

    This probably isn't a huge issue, except for those who regularly use citation styles that require giving year, month, and day (if available).

    2. This problem doesn't seem to arise when exporting/importing with BibTeX. However, I've had problems with importing websites with BibTeX.

    3. This problem also doesn't arise when exporting/importing with XML. There are also no issues with importing websites when using XML. However, it's messy—for each item that gets imported a note is automatically generated in Zotero that reads "The following values have no corresponding Zotero field:label: r17379". That's every single item.

    Currently it looks like importing via BibTeX is the most promising solution. However, all of the methods above have allowed me to transfer my tags, which is great.
  • edited June 19, 2019
    You should use XML to import them. That will be the least lossy, as you've noticed. For the notes, they will all be tagged with "_Endnote XML". You can click on that tag in the tag select to filter your library to show the notehttps://www.zotero.org/support/getting_stuff_into_your_librarys, then select all of them at once using Ctrl/Cmd+A, then delete them all together.
  • However, it's messy—for each item that gets imported a note is automatically generated in Zotero that reads
    Those notes are all tagged with _Endnote XML so deleting them all at once is a single operations (click tag, select all, delete)
  • edited June 19, 2019
    Thanks! That hadn't occurred to me, but I'll do that. The only other issue with importing via XML is that it looks like apostrophes have been replaced with & apos;, so instead of "O'Reilly Media", the output is given as "O& apos;Reilly Media", and "Adams and Victor's Principles of..." is given as "Adams and Victor& apos;s Principles of...", etc.
  • To be clear, there is a space between the & and apos; ?
  • If there is, that's a problem with Papers, export and easy to fix.

    If there isn't, could you open the XML with a text editor and look at how the title looks there? If the XML has ' Zotero should import this just fine (just tested). If the XML has ' it won't, but again, that should be an easy fix.
  • edited June 20, 2019
    Some further updates:

    1. With the **XML method**: Seems to treat book chapters in edited books as edited books by omitting the book title (1a). Also, when dealing with an edited book by itself, converts editors to authors (1b). See examples below:

    (1a). Incorrect (APA): Gibb, R., & Kovalchuk, A. (2018). Brain development (R. Gibb & B. Kolb, Eds.). London, UK: Academic Press.
    Correct (APA): Gibb, R., & Kovalchuk, A. (2018). Brain development. In R. Gibb & B. Kolb (Eds.), *The Neurobiology of Brain and Behavioral Development* (pp. 3–27). London, UK: Academic Press.

    (1b). Incorrect (APA): Gibb, R., & Kolb, B. (2018). *The Neurobiology of Brain and Behavioral Development*. London, UK: Academic Press.
    Correct (APA): Gibb, R., & Kolb, B. (Eds.). (2018). *The Neurobiology of Brain and Behavioral Development*. London, UK: Academic Press.

    2. With the **BibTeX method**: Omits edition from chapters in edited books (2a). Also, does not import abbreviated journal names (2b). See examples below:

    (2a). Incorrect (APA): Baron, I. S. (2010). Maxims and a model for the practice of pediatric neuropsychology. In K. O. Yeates, M. D. Ris, H. G. Taylor, & B. F. Pennington (Eds.), *Pediatric neuropsychology: Research, theory, and practice* (pp. 473–498). New York, NY: Guilford.
    Correct (APA): Baron, I. S. (2010). Maxims and a model for the practice of pediatric neuropsychology. In K. O. Yeates, M. D. Ris, H. G. Taylor, & B. F. Pennington (Eds.), *Pediatric neuropsychology: Research, theory, and practice* (2nd ed., pp. 473–498). New York, NY: Guilford.

    (2b). Incorrect (AMA): Shapiro-Mendoza CK, Lackritz EM. Epidemiology of late and moderate preterm birth. *Seminars in fetal & neonatal medicine*. 2012;17(3):120–125
    Correct (AMA): Shapiro-Mendoza CK, Lackritz EM. Epidemiology of late and moderate preterm birth. *Semin Fetal Neonatal Med*. 2012;17(3):120-125.
  • (since we don't have Papers 3, for details such as this we'd almost always need to see the export to say anything helpful)
  • That's ok, I understand that neither method will be perfect—so I'll just have to opt for the one that produces the best result without requiring too much post-import editing. At the moment that looks like the XML method, since I only need to edit out the apostrophe problem in a text editor, change some authors back to editors, and add book titles to chapters in edited books. Based on your experience, would you say that there are any particular reasons to prefer the BibTeX method, or maybe some other method altogether? Since the RIS method adds extraneous (and incorrect) information, it is the least preferred.

    (An aside: I another difference I noticed that is the XML method will import author initials with periods, while the BibTeX method strips periods).
  • Periods on the author names generally aren't an issue one way or the other—Zotero's citation processor is smart about determining when single letters/multiple capital letters are initials versus a name.

    It's certainly frustrating that Papers is so inconsistent/lossy with its various export formats.
  • Generally, Endnote XML is the most standardized and complete of the widely used formats, so where well implemented, that's preferable.
  • Update: It's finally done. I just had to do some edits to the XML file, wait for the import to finish, and then modify the book chapters (of which there weren't too many) to include book titles. So far I'm quite happy with the switch. Papers became cumbersome to use long ago—so Zotero is a refreshing change. Looking forward to working with it going forward.
  • Great! Glad you got things working.
  • edited March 26, 2020
    Late to the party. I got annoyed with the XML-HTML formatting errors spewing out of Papers on export (e.g., &), so I made a little bash script to clean it up:

    [removed — D.S.]

    ETA: I failed to anticipate that this Zotero website would render some of this code as HTML! You can see the clean version of this script as a Github gist over at:

    https://gist.github.com/SiccarPoint/2729c64e4a5fa009d0e92fda68b95ddf
  • Maybe post it to http://gist.github.com and link here?
  • @bwiernik see above! Thx for spotting.
  • Hi I just used this approach, which did largely clean up my imported library from Papers3. However, I now have an error when trying to sync:

    JSON.parse: expected ',' or '}' after property value in object at line 1 column 1251 of the JSON data SyntaxError: JSON.parse: expected ',' or '}' after property value in object at line 1 column 1251 of the JSON data

    There are a bunch of other errors listed but that is listed first and a bunch of times in the Debug Output.

    Any idea what could be wrong and/or how to fix this?
  • edited July 12, 2020
    We can't help with manual modifications to the database, sorry. That's completely unsupported. Problems with the Papers export would ideally be corrected in the output file before importing, or if that wasn't an option for some reason you could do it via JavaScript — and not using SQL — in the Run JavaScript window.

    You should start from scratch with a fresh import. You can delete all the imported items by sorting by Date Added in the items list.
  • One doesn't need to use Papers export into XML and edit the files manually to move from Readcube/Papers to Zotero. I discovered that if you have Endnote installed already (I have v9 installed), that you can export readcube/papers to endnote (references and PDFs will appear in Endnote's imported references). Then you can export these references in an XML file, but not need to edit them. You can drag and drop them directly into Zotero. The PDFs will seem to transfer as well. I'm using Zotfile to help keep pdfs in cloud based folder.
  • In relation to rjain24's problem (July 12, 2020), the script from DanHobley replaces " by ", but I think this then messes up the JSON, which uses " as string delimiter. I changed the /"/ occurences in DanHobley's script by /\" /, so escaping these characters, and it seems to work now.
  • I just switched to Zotero from Papers3 and would like to add my experience here. From within Papers3, I went to File>Export>EndNote XML Library. I then exported my entire library using "EndNote 8 or higher" under EndNote version. I then opened the resulting XML file using TextEdit and replaced all instances of "'" with "'" as per adamsmith's suggestion. Lastly, within Zotero, I went to File>Import>A file and pointed to the edited XML file. Below is a list of what I observed after the library was imported:

    1. Some characters didn't import properly, notably "&", quotation marks, and the symbol ">".
    2. Some Greek letters imported properly, while in other instances they did not.
    3. Italics did not import properly.
    4. Some supplemental files did not transfer over. I know Papers3 sometimes removed the extension of supplemental files when importing them into Papers3. I wonder if this caused Zotero to fail to import them.
  • edited March 21, 2023
    I just made the switch from Papers3 to Zotero. I wasn't able to get the bash script to run on my system, and was also not able to access EndNote as an intermediate. I made a python script to clean up the Papers3 XML, and the cleaned XML imported into Zotero fine (including the associated pdfs).

    Script is here in case it's of use to anyone else: https://github.com/lastbroadcast/Papers3-to-Zotero

    *From Papers3, export as 'EndNote XML v8 or higher'
    *Customise the script to make the modifications you want. I did:
    -'%20' character changed to a space
    -I was using Parallels to run Papers3, so I needed to remove a specific string from all path names
    -several XML tags that are not recognised by Zotero are removed
    *Run script
    *Import into Zotero

    Note that folder/sub-folder structure is not retained by the XML export.


    Update:

    *I created some scripts to transfer the collections/folders from Papers3 to Zotero. Run these scripts after doing the XML export. See readme file for info on how to use them.

    https://github.com/lastbroadcast/Papers3-to-Zotero_collections

    *I found that Zotero and Papers categorise parent/child items differently. After using these scripts I found that papers were filed in parent collections if they were also present in child collections. If you want to eliminate papers from parent collections if they are also in child collections, use this script:

    https://github.com/lastbroadcast/Zotero_RemoveParents

    Note: back up your Zotero database before you use these. I am not a professional programmer, so things could go wrong!
  • dear lastbroadcast, I wanted to do the same as you : export as endnote xml but my application Papers3 quit any time I want to do so... Which are the other ways to switch from papers3 to zotero ? I have like 2MB of papers out of zotero. Thank you in advance.
  • I have the problem that Papers 3 stopped working properly- I cannot export (nothing happens). No way one could import the Papers file directly ?
  • edited March 11, 2023
    @mareb and @ReidarL:

    Are you trying to open papers using macOS Catalina or higher? If so, this is probably the issue. Catalina and higher only support 64-bit apps, and papers is 32-bit. The app might open OK, but won't function properly.

    I got around this issue by downloading Parallels and used the free trial period to install macOS Sierra (also downloaded for free) to open papers and export the library as xml.

    The whole thing was a faff - I still have a few more days left on the trial period, so if you're quick, you could try to send me your library (if it's not super large) and I'll export it and send it back to you.

    *Edit: send me a private message through the forum
  • edited March 13, 2023
    @lastbroadcast yes indeed I was trying to open papers with catalina or big sur...

    thank you for your suggestion and help ! I don't know what does mean "a faff" but I accept your help ! how do I send you a private message ?
Sign In or Register to comment.