Moving from Mendeley to Zotero

rdiaz02

[Update: Zotero now includes a direct Mendeley importer. — D.S.]

Dear All,

I am trying to move to Zotero from Mendeley. The basic approach is simple: get the bibliography as bibtex from Mendeley, and then import that into Zotero. And it does work great.

There are a few minor issues (such as Mendeley's storage of PDFs comments in a separate file) but, to me, the major stumbling block is what to do with what are called "Folders" in Mendeley, which are more or less like Zotero's collections.

The info for Mendeley's Folders is not exported in the bibtex file, but I guess I could add it into a custom bibtex file (since it is available from the sqlite db used by Mendeley). But I have no idea what I could do with that info: how to tell Zotero to create a bunch of Collections and put such-and-such entries in these or those Collections. I guess that is in the developers documentation, but my programming skills cover none of those languages ;-(.

I understand this is a messy problem (as explained in, e.g.,
http://forums.zotero.org/discussion/25981/zotero-and-mendeley/, or in Mendeley's forum: http://feedback.mendeley.com/forums/4941-general/suggestions/372255-zotero-2-way-sync). Note, however, that I am not interested here in two-way sync, but just in a one-shot thing. SO maybe somebody has a working solution.

Best,

adamsmith

the bibtex translator doesn't allow for collections, so that'd be a bit involved (since you'd also have to modify the translator) - the easiest would be to add the Mendeley Folders as keywords in the bibtex and then create collections in Zotero based on those tags, which isn't terribly hard to do.

adamsmith

out of curiousity - what are the export formats that Mendeley offers currently? I haven't checked for a while. If they had one that includes collections, we could certainly consider supporting that.

rdiaz02

Thanks for the suggestion. I'll give it a try and report back. However, you say "create collections in Zotero based on those tags, which isn't terribly hard to do. " I've looked at the docs quickly, and played with the interface, and if I understand correctly, it involves manually selecting the refs. that have the tag-that-defines-a-collection, and placing them in a previously created new collection? So how painful the experience is, will be proportional to number of folders, not number of references.


Export formats for Mendeley (v. 1.7, the newest as of a few days ago): Bibtex, RIS, and EndNote XML (v8, X1, or X2). I am not familiar with RIS or EndNote, but when playing with them I could see no obvious way of getting the Collection/Folder info in there.

adamsmith

all of those are flat formats and don't allow folder export (and hence no import) - that's really a rather terrible thing to do on Mendeley's part - they need to offer a lossless export format (for Zotero both Zotero RDF and the more neutral Bibliontology RDF are practically lossless and include collections). Even if they were to create their own format, we could still write an import translator.

As it is, the only way is for someone to write something based on their sql structure (not sure if that would even be legal - with all talk about how "open access" they are, you still sign a standard proprietary EULA), which would be quite involved.

As for my suggestion, yes you'd have to do this manually and how much work that is depends on the number of folders - in Zotero you can click on a tag to select all items with it, so the workflow would be:
Create collections --> click on tag --> select all and drag&drop to collection.
I don't see a better way, sorry.

dstillman

As it is, the only way is for someone to write something based on their sql structure

Yeah, the best bet here for a full import would probably be a Zotero plugin that read the Mendeley database directly. I would think that would be pretty simple.

rdiaz02

Thanks to both.

I see now how to create collections via tags.

DB access: I am not a lawyer, but I understand that access to the sqlite db on my machine by whatever means is legal. This was, I think, one of Mendeley's answers to the "this is not open source" criticisms: they argued that the user had full control over his/her records, as they were stored as plain sqlite, so the user retained full access (assuming the user knows or knows someone who knows sqlite) even if Mendeley run out of business or decided to start selling their program.


I know next to nothing about sqlite and nothing about how to write a Zotero plugin, but certainly the info is easily accessible in the DB: the text description of the Folders is the "name" field in the "Folders" table, and what Folder(s) each document belongs to is in the "DocumentFolders" table (where the documentId is the same key as id in the "Documents" table). For a simple hack, I was thinking about processing the csv versions of the tables to add the Folder(s) to the keyword entry in bibtex.

ajlyon

It would be pretty low for Mendeley to consider an SQLite-based import a form of infringement, since that's how they do their import from Zotero.

adamsmith

oh sure it'd be low and I don't think they'd complain, but as for legality, nowhere in the Zotero license does it require you to agree

4.7.5. not to disassemble, decompile, reverse engineer or create derivative works based on the whole or any part of our Software nor attempt to do any such thing except to the extent that (by virtue of section 296A of the Copyright, Designs and Patents Act 1988) such actions cannot be prohibited;

which was my point about the problems with Mendeley being proprietary.
I guess that figuring out how they store the data in the sqlite is probably not "reverse engineering" but one shouldn't need to wonder about such things...

seeholzer

I wrote a python/sqlalchemy script to import both the "date Added" and folders/collections from mendeley to zotero: https://github.com/flinz/mendeley2zotero

Very rough still, so be sure to back up your sqlite files before.
@Collections: it's not perfect yet, you still have to create the collections corresponding to your folders to Zotero manually - then my script assigns your items to these collections.

rdiaz02

I have done the moving from Mendeley to Zotero, preserving collections, tags, dates, annotations, pdf annotations, etc. I use seeholzer's https://github.com/flinz/mendeley2zotero for preserving the dates; the rest of the previous work is done modifying the bibtex file before importing into Zotero and taking advantage of the fact that the importer can recognize JabRef's collections.

All the code and explanations is available here: https://github.com/rdiaz02/Adios_Mendeley

This worked OK for me, but you'll probably need to modify it for your own needs.

cycoj

In addition to the script by rdiaz02 I've created a script that extracts the highlights and notes from Mendeley and adds them to the relevant PDF-files so that they can be read by any pdf reader.
Hope someone finds it useful:
https://github.com/cycomanic/Menextract2pdf

shicheng.guo

I think it would be better to combine or integrate Zotero and Mendeley. I prefer us mendeley to collect and read paper, however mendeley cannot insert citation in word while zotero could insert citation. I think, combination of Zotero and Mendeley would be trend.

aborel

> shicheng.guo

What do you mean "mendeley cannot insert citation in word"? Mendeley certainly has this functionality http://support.mendeley.com/customer/en/portal/articles/168756-installing-and-using-the-word-plugin-in-windows .

I don't see the integration you describe any time soon (but perhaps I'm wrong). What I would really like would be the ability to transparently share Word/LibreOfffice documents between authors who use Zotero and Mendeley.

bpiwowar

As I was not satisfied with import tools, I designed my own

https://github.com/bpiwowar/biblioruler/

where Mendeley (and Papers 3, but this is broken) is exported to Zotero RDF + annotated PDFs, so it can be imported without loss. At the moment, not all fields are properly handled, I focussed on titles (and conference/journal name), authors and notes.

fran_alba

Hello,

I am a long time Mendeley user and I would like to make the switch to Zotero...
Before trying and make a mess, I would like to know if all the methods discussed here are still valid for Zotero 5.

Thanks!

bwiernik

The Adios Mendeley tool seems to be the most commonly used and works well as far as I am aware
https://github.com/rdiaz02/Adios_Mendeley

Importing hasn’t changed much in Zotero 5–the key difficulty has always been getting the data out of Mendeley.

fran_alba

I am having a lot of trouble with those scripts :( do you know if the author rdiaz02 is still active on the forum?

Thank you for your help

bwiernik

You’re best bet of getting in contact would be to post an Issue on the GitHub page.

guangzhi

Hi all, I've created a python tool that exports a bunch of things from Mendeley, including a .bib or .ris document, the PDFs and notes, highlights. You can check it out here (https://github.com/Xunius/Menotexport).

I got a couple of feature requests that it would be better that the Zotero import could preserve the folder structures in Mendeley. I need some help on this. Is it possible to achieve this by formatting the .bib or .ris files in certain ways, or some sqlite database manipulation is required, or any other approach?

Currently, the python tool can save the pdfs to disk in a folder structure that mirrors that in Mendeley, and the file entry in .bib is an absolute path point to the pdf file.

Thanks

bwiernik

Neither .bib or .ris support Collections/Folders. Zotero RDF format does, so you could add that export format to your tool.

guangzhi

@bwiernik thanks for pointing it out. I've found only 2 places of the documentation of RDF: https://github.com/zotero/translators/blob/master/Zotero RDF.js and https://github.com/aurimasv/zotero-import-export-formats/blob/master/Zotero RDF.rdf.

Are there any other resources regarding the formatting of RDF?
Thanks

adamsmith

Unfortunately not, no, the format isn't as well documented as it should be. The only other place to look (and what I'd actually recommend) is actual export from Zotero into the format. That will, in particular, help you understand how collections are handled.

guangzhi

@adasmith yep it makes sense, trying out.