migrate without synching Mendeley desktop

Andy Stirling

I want to migrate to Zotero my local Mendeley Desktop (currently version 1.19.8) reference database of 135,211 fulltext files (700 GB). The size of the main encrypted Mendeley sqlite file is 231 MB. I have never synced this to Mendeley and do not want to.

Zotero help (here: https://www.zotero.org/support/kb/mendeley_local_import), says that in order to also transfer the links to my local fulltext file locations, even the ‘local method’ requires that I first sync my database and all files to Mendeley.

Please advise how I can export the full database with all local file links from my local Mendeley Desktop database to Zotero in such a way as to maintain linked file addresses but without requiring syncing of all files to the remote Mendeley servers.

dstillman

You can't — that's the point.

You can export to RIS or BibTeX, which might include links to files in a format we support (you can test with a single item), but as we explain, that's not a full export:

Direct access to the Mendeley database is the only fully local way to export the full contents of one’s own research. The export formats supported by Mendeley don’t contain folders, various metadata fields (date added, favorite, and others), or PDF annotations.

Andy Stirling

Huge thanks DStillman. I somehow missed that. It is very bad news. But it is at least good to know where one stands. Sadly not a surprise, but this corporate behaviour is toxic. The value of what Zotero does becomes even more clear!

I will try what you suggest: experimenting with RIS and BibTex exports. But I suspect that if this workaround were good, you would know about it? Otherwise, I need to gauge a strategy that is as robust as possible in the medium term.

Otherwise, since full text file searching is essential for me, it looks from what you say like the best of a bunch of bad bets, is to continue to use Mendeley Desktop locally pending a move to Zotero if and when a more favourable situation arises?

With apologies for this further query, then: does this thinking make sense to you? Are there possibilities that more favourable developments may occur? Or might this instead expose me to likelihood of even worse circumstances developing?

Your advice on these issues is invaluable. I greatly appreciate it.

dstillman

It's not going to improve, no. Mendeley Desktop isn't being developed anymore, and we have no way of getting data out of an encrypted database.

But just export a single item and see if a file path is included in the export (e.g., in an L1 field for RIS). It probably is, and Zotero should import it. You'll just lose the other data mentioned above.

Andy Stirling

Thanks DStillman,

It is fantastic to be getting this help. I made a few trials of BibTex export from Mendeley Desktop on single items and the file path transferred fine.

Encouraged by this, I exported all 135,211 items in three roughly equal groups by right-click selecting items in 'all documents' view ordered by 'date added'.

File paths seem to have transferred well overall. But an initial review of results shows other apparent problems. I'm sorry that these complicate this query trail.

First, the number of items successfully transferred in each group was massively smaller than the number of items selected. A total of only 8,500 transferred.

Second, it appears that contents of the 'tags' field have not transferred correctly. Zotero 'display all tags in library' shows very large amounts of extraneous text have imported into this field, including some but not all the tags that should be there.

I'm sorry if I've done anything wrong or misunderstand something. How can I sort the low number of items transferred and the mangling of tag contents?

Huge thanks again and all the very best
Andy

dstillman

You'd have to open the BibTeX files in a text editor (e.g., Notepad or TextEdit) and figure out 1) if the missing entries are present and 2) what the entries look like for the items where the tags are incorrect.

For (1), make sure you're actually looking in the library root for the items. If they're present in the exported file and not being imported, you'd want to cut the exported file down to try the smallest possible section that reproduces the problem (e.g., with an entry that imports and a subsequent one that doesn't). There's a good chance Mendeley is just misformatting something in a way that's causing subsequent items to be skipped.

Andy Stirling

Thanks DStillman,

Again: this help is hugely appreciated – especially so speedy and clear.

I’ll try this. And I also wonder if the sheer volume of export batches may be implicated in the problem? Even if only a small chance, a relatively graspable option for someone with my own limits, might be to export in smaller increments?

Otherwise, I’ll look at the BibTex files. But I fear as technicality rises, so risks grow of me mangling details of diagnoses or actions. As this approaches edges of my competence, a Plan B might be to pay someone trusted and more skilled.

For future reference just in case, would you have any recommendations DStillman, about the general idea or specific suggestions?

Huge thanks again for the fantastic assistance.

All best
Andy

dstillman

The number of items shouldn't matter for Zotero. I can't speak for Mendeley. But it's likely what I say — that one or more malformed entries are causing items to be skipped (e.g., because an entry isn't properly closed).

Andy Stirling

Many thanks again DStillman, for your hugely appreciated support. I've returned to the desk today to battle away with this further.

I fully understand you can't speak for Mendeley, but a first consideration on close examination, is an apparent anomaly that may relate to this problem.

The (in all respects standard) Mendeley Desktop folder I am trying to expert from registers in Windows Explorer with a contents of 63.7 GB. But the displayed sqlite files directly rooted in this total less than 1 GB. Other included folders add virtually nothing, aside from the cache folder (but this is still only 16.9 GB).

So, in a way that strikes my amateur brain on these matters as being odd, there appears to be a large amount of extraneous data here. I wonder if this may be related to the problem I am experiencing with the export?

I realise this is totally Mendeley and not a Zotero problem. But it is why I am trying to migrate and vital to me becoming a Zotero user. Both the embodied work and future utility of this database make the stakes very high for me. So I wonder if you might have any further guidance about what I might do (eg: delete cache folder?) to diagnose whether - or at least reduce risks that - this may relate to the possible malformed entries difficulty that you so helpfully pointed to.

Yours
Andy

damnation

I would assume that Mendeley folder includes all the . pdfs also. The SQlite you mention only contains the metadata information of all your items, but cannot include attachments.

Andy Stirling

Thanks Damnation. Good assumption! I understand what you say and why.

But this is definitely not the attachments directly, since the text file database is located elsewhere and anyhow (as mentioned in above stream) is 700 GB in size.

I wonder if this is some kind of residue of an index file? Maybe irrelevant? My main query is whether this might be implicated in the show-stopping block on migration?

Thanks again for your interest
Andy