Available for beta testing: Mendeley import

edited August 10, 2018
[Update: The Mendeley importer is now available in the release version of Zotero.]

In the Zotero beta, it's now possible to import a Mendeley library directly into Zotero, preserving all item data, folders, tags, notes, linked and stored files, and extracted PDF annotations.

To import a Mendeley library, select "Import…" from the File menu in Zotero. If Zotero finds a Mendeley data directory in the standard location, you'll see a "Mendeley" option, and selecting that will show a list of available databases to import.

[Update: Mendeley very recently made changes that prevent this from working. If you've upgraded to Mendeley Desktop 1.19, see below for a workaround.]

If you have a backup of your Mendeley data directory stored elsewhere on your computer, you can choose the file option instead and just select the Mendeley .sqlite database wherever it's located.

Repeated imports will update and overwrite any collections or items (including notes) you've already imported from Mendeley rather than creating new copies. For collections, that only applies if you choose not to create a collection for the import, which is a new option during imports.

Once you've imported your Mendeley library with the beta version of Zotero, you can either stay on the beta track or switch back to the release version of Zotero.

Known issues:

- It's not currently possible to import items from group libraries. I'm not sure if we'll support this, since it's not clear that we can meaningfully improve on simply copying group items to a collection in your Mendeley library before importing. We could perhaps allow group-to-group mappings, but the group would need to be recreated on the Zotero side anyway.

- Mendeley stores PDF annotations and highlights in its database rather than storing them in the file where they would be accessible to other PDF readers, so we extract annotations to a note stored under the item, with links back to the original page. Highlights are not imported. I believe there's an export option in Mendeley to generate PDFs with annotations and highlights added back to the file, so it might be possible to automate replacing the imported PDFs with those exported versions down the line, but that's not something we've looked into.

- Mendeley seems to allow any field to be added to any type. When importing into Zotero, if a field isn't valid for a given item type, the field is placed into the Extra field. When possible, those will be used automatically in citations (e.g., Original Date), and future versions of Zotero will automatically convert those to any real fields that become available.

- When using the Zotero word processor plugins, document citations created with Mendeley won't currently be linked to imported citations in your Zotero database, but we're hoping to add support for that in a future version.

Let us know if anything doesn't come through how you expect or if you run into trouble.
  • Another feather in the cap. Great work, team Zotero!
  • Thanks for providing this feature. I am quite keen to find something better than Mendeley, so this has arrived at exactly the right time for me!

    However, I have tried to import my Mendeley DB on Linux. The import finds my Mendeley DB, but I then get the message "The selected file is not in a supported format." (Using Mendeley 1.19)

    I booted into Windows to try it there, but the 'Mendeley' option wasn't available in the import dialog (I checked that I was running the 5.0.49 beta).

    Any ideas?
  • @sam.c: That's no good. Can you provide a Debug ID for the import attempt?
  • D1212777753
  • OK, could you update to the latest version (5.0.49-beta.4) and generate another Debug ID?
  • I have the same problem. My debug id was D1947368303.
  • I've also submitted debug id D363841056
  • edited June 10, 2018
    @sam.c, @VitorCalistoSilva: Do you have Mendeley open when you're trying this? If so, does it help to close it?

    Your databases don't appear to be in standard SQLite format, at least from Zotero's perspective. I'm not sure if Mendeley is sometimes writing files in some other format (despite the .sqlite extension) or there's something else going on.

    Are you able to open the database in another SQLite tool? (E.g., if you open a terminal window, type sqlite3 (with a space at the end), drag in the .sqlite file from the filesystem, and press Enter, and then at the 'sqlite>' prompt type PRAGMA integrity_check; and press Enter, does it say 'ok'?)
  • Mendeley is not open when I tried the import. When I tried opening the .sqlite DB from the command line and run the integrity check, I get "Error: file is encrypted or is not a database"
  • edited June 10, 2018
    It seems like Mendeley might have changed their DB format very recently? I found a backup from a couple of weeks ago. It was a valid sqlite DB and the import gets a bit further with it (there was another error which I'll have to look into shortly). But the main DB definitely doesn't appear to be sqlite.

    Edit: With the backup file, it at least finds my folders, but then throws an error, which I've submitted as report ID 235993581.
  • edited June 11, 2018
    Oh. So it seems like Mendeley has just started encrypting the local database…

    That's presumably this entry in their changelog:
    We have improved security for users accessing Mendeley Desktop on a shared machine
    "Improved security" sounds like a good thing, until you realize that 1) encrypting a local database is a pretty rare thing for a program to do, since file protections are generally handled at the OS level by account permissions and full-disk encryption (and anyone using the same OS account or an admin account can already install a keylogger to get all your passwords), 2) it prevents you from accessing the database yourself using standard tools (the way that, say, Mendeley can import from the Zotero database), and 3) this apparently became a priority, after 10 years of not doing it, shortly after Zotero started publicly working on import functionality using the local database. But perhaps it's just a coincidence.

    Personally, if I were a Mendeley user, I would contact their support and ask why I can no longer access my own data with standard SQLite tools.

    We'll see what we can do to make this easier. In the meantime, if you have a Mendeley database you'd like to move to Zotero, and you haven't yet upgraded to Mendeley 1.19, you should import before upgrading. If you have upgraded to 1.19, you can either try to import from a recent backup or move your database out of the way in the Mendeley data directory, download 1.18, and do a fresh sync to pull down your data from the server, and then do the import into Zotero.

    Mendeley 1.18 installers:
  • @sam.c: The error you were seeing should be fixed in 5.0.49-beta.5.
  • Thanks very much for this. I will try again once I can downgrade Mendeley and resync my DB.

    To be honest, I wasn't 100% sure it was time to get rid of Mendeley, but now I am! Won't be using an app that wants to lock me in.
  • @dstillman Thanks for the comments. Following your advise i successfully downgrade the mendeley from 1.19 to 1.18. Since the encrypting database, I found an auto-backup database in ~\AppData\Local\Mendeley Ltd\Mendeley Desktop\backupSlot1, I use this backup to replace the encrypting database and install the 1.18 can solve the problem.

    However, I still cannot import the mendeley database to zotero. The report ID: 581059188
  • @JieQiao: Could we see a Debug ID (different from a Report ID) for the import attempt? (We'll generally need Debug IDs for problems here.)
  • @dstillman DebugID: D1556652374
  • edited June 11, 2018
    1) In regards to migrating group libraries in Mendeley. Is it possible to automate copying group library items to local library items prior to using Zotero for migrating? My entire Mendeley library resides as part of a group library, thus I was unable to use Zotero beta to migrate from Mendeley. Note: Although my entire Mendeley library is a group library, all my files are locally organized.

    2) Regarding v1.19 having encrypted database. Isn't it possible to decrypt the database? I mean for Mendeley itself to read the database, it needs to decrypt it.

    I am using Mendeley v1.17.13. I highly recommend others to also stick with v1.17.13 if you plan on migrating to Zotero in the future. I had weird issues with v1.18 and group libraries when using Mendeley. So I am guessing Mendeley started to trap users in starting v1.18 and most certainly v1.19 with encrypted database.
  • edited June 11, 2018
    @dstillman

    I was able to smoothly migrate from Mendeley to Zotero 5.0.49-beta.5+2831e8be3.

    Feature request: If a Mendeley item contains a "Star", is it possible to automatically add a tag to all items containing a star during migration? Maybe the tag could be named "mendeley-star". Migration of "Stars" from Mendeley would be amazing! With a huge library in Mendeley and few hundred "Starred" items, automatically tagging during migration would be amazing!
  • @qjgods: Thanks. That should be fixed in 5.0.49-beta.7, available now.

    @01baftb:
    Is it possible to automate copying group library items to local library items prior to using Zotero for migrating?
    Automate from the Zotero side? We wouldn't make changes to the Mendeley DB. But we'll likely add some options for importing from Mendeley group libraries.
    Isn't it possible to decrypt the database? I mean for Mendeley itself to read the database, it needs to decrypt it.
    Mendeley can decrypt it. Other programs can't — that's the point of the encryption. (It might be encrypted based on the account username and password, but since Mendeley is closed source and they haven't documented the encryption format, there's no way short of complex reverse-engineering to know that, and it'd still make the database very hard to use in most external tools for very little real security benefit (unless, of course, you consider that the security benefit).)
    Feature request: If a Mendeley item contains a "Star", is it possible to automatically add a tag to all items containing a star during migration?
    Good idea. I've added a star tag for those items in the latest beta. If you re-import it should update items to add tags. (Be aware that it will overwrite any changes you've made to those items since importing. Additional items you've added won't be affected.)
  • I will try to do the import at home again when I'm on Linux, but I am currently on my office PC (Windows), and still am not seeing the Mendeley import option in 5.0.49-beta.7
  • @dstillman

    I re-tested the migration with Zotero 5.0.49-beta.7+cfbb3d3d4, it worked very smoothly along with the star tags.
  • @sam.c: The latest beta should fix auto-detection on Windows. (You can also select the DB manually with the file option.)
  • edited June 12, 2018
    The import functionality is now available in Zotero 5.0.49, and we've created a documentation page with more information on the process. Thanks for helping to test this feature, and feel free to start new threads if you run into any other issues importing.
This discussion has been closed.