Which link worked in JabRef? You've given a lot of different links.
When I use the stock Zotero to export a BibTeX containing both a stored file and a linked file, I get the following when I leave "Export Files" unselected: @book{osmith_stored_2019, title = {Stored copy of file}, author = {öSmith, John}, year = {2019}, file = {öSmith - 2019 - Stored copy of file.pdf:C\:\\Users\\noksagt\\Zotero\\storage\\7H24VG95\\öSmith - 2019 - Stored copy of file.pdf:application/pdf} }
@book{smith_linked_2019, title = {Linked copy of file}, author = {Smith, John}, year = {2019}, file = {ö.pdf:C\:\\temp\\ö.pdf:application/pdf} } I get the following when I select "Export Files": @book{osmith_stored_2019, title = {Stored copy of file}, author = {öSmith, John}, year = {2019}, file = {öSmith - 2019 - Stored copy of file.pdf:files/15/öSmith - 2019 - Stored copy of file.pdf:application/pdf} }
@book{smith_linked_2019, title = {Linked copy of file}, author = {Smith, John}, year = {2019}, file = {ö.pdf:files/4/ö.pdf:application/pdf} }All of these import successfully into Zotero. All of these are also links that JabRef is able to open.
Either style of link (the first is exported from BBT on Windows when you have "Export Files" unselected and the second is the style you said you edited manually), will not import into either stock Zotero. If you try to open either style of link in JabRef, it won't find the file. That is because the unescaped drive letter colon is understood by both programs to be a field separator. I created a BBT issue. Perhaps you can submit a Report ID and/or provide any other information in that issue.
BetterBibTex actually does the correct thing if "Export Files" is selected.
Either of these work in both Zotero and JabRef, though: file = {C\:\\temp\\ö.pdf} file = {:C\:/temp/ö.pdf:PDF}
I guess that also answers the question on why BBT doesn't import files with things like \"o in it. This is unfortunately not easy to change in BBTs bibtex parser. I'll give it some thought.
No — that's from Mendeley, right? I'm just saying, to reproduce the problem with extended characters in filenames, you can just export from Zotero as non–UTF-8 BibTeX and reimport. As I say above, the fix for the stock translator appears to be to use value instead of rawValue, but I don't know the other implications of that.
It'd be helpful if any discussion about BBT could move to the BBT GitHub tracker and we could keep this thread about the stock translator — this is getting too confusing.
In different messages, the OP pointed out what I would consider two different bugs in the way Mendeley and Better BibTeX were sometimes exporting links for files on Windows. emiliano has patched the latter.
The stock translator and BBT both export correctly formatted links on Windows now & either translator is able to import from itself or from the other.
Mendeley's exported links are still broken: file = {:D$\backslash$:/Users/Steffen/Documents/Literatur/pdf/Mendeley/Artikel/Oertel, Ruck - 2012 - Biostr{\"{o}}mungsmechanik(5).pdf:pdf},
Dan is correct that the current use of raw value mean that field is uncorrected. The transform leads to no changes: :D$\backslash$:/Users/Steffen/Documents/Literatur/pdf/Mendeley/Artikel/Oertel, Ruck - 2012 - Biostr{\"{o}}mungsmechanik(5).pdf:pdf For me, replacing rawValue with value fixes the umlaut and keeps the drive colon escaped, but does not seem to make a resolvable path: :D\:/Users/Steffen/Documents/Literatur/pdf/Mendeley/Artikel/Oertel, Ruck - 2012 - Bioströmungsmechanik(5).pdf:pdf Is this because of the forward slashes used?
@steffenrust I have a fix in BBT, I'd really appreciate it if you can drop by on github and help me test this. I don't have access to a Windows machine.
The stock translator and BBT both export correctly formatted links on Windows now & either translator is able to import from itself or from the other.
That's not the case, though, at least for the stock translator, until the value/rawValue thing is fixed. If you export from Zotero as Western, this is what you get:
I don't know if that's correct or not, but if you import that back, the file isn't included with the translator unless rawValue is changed to to value.
It's probably not (seeing as how the file field is verbatim) but it's also probably an acceptable compromise when targeting something that doesn't grok utf-8. For bibtex exports, this is going to be a compromise one way or the other.
BBT can now be told whether file should be treated as verbatim on import. I'm still pondering whether to just fix this behavior to "no", or what to set the default to if not fixed.
I also came across the problem with the tex-coding of special characters (umlauts and accents) in the file names when exporting a large Mendeley database to BibTeX and then trying to import these to Zotero. I tried both the stock importer and Better-BibTeX (Zotero 5.0.76, BBTex 5.1.145). As I use linux, I do not have the $backslash$ problem.
Is there a fix in the meantime?
By the way, the problem does not only affect the file field, but also the url field, where Mendeley for example encodes & as {\&}.
Importing shouldn't pose any problem for BBT, but in order to trigger the Mendeley-compatible behavior (which, to reiterate, is against spec, but I can envision a reason for that compromise) in BBT you either have to rename the file field to files or remove file from the hidden preference verbatimFields.
As to the way Mendeley exports that URL, that's just simply and inexcusably wrong. You can make BBT import it by likewise removing url from verbatimFields<rant>, but that one really shows the Mendeley team hasn't bothered to ever open the bib(la)tex manuals. Aside the fact that the url field is not supported by most bibtex styles so should be seen as out-of-spec for bibtex, and that in biblatex it is a verbatim field so should not be escaped, it is at the same time the dumbest way I've seen & exported, because even if it were a supported field, and not a verbatim field, the braces are simply unnecessary, and if they'd omitted that, BBT would already have imported it as you'd wanted to, because I skip backslashes in URLs (because too many bibtex files have them to escape stuff like this).</rant>
Thanks for the quick reply @emilianoeheyns. I have also contacted Mendeley support of course, but I fear that they will not have a fix soon. Obviously, they simply encode all fields in TeX-Syntax, which is complete bogus, without even thinking about it. Guess, why I am trying to convince a whole institute to switch to zotero and thus need to convert several large databases.
I removed file and url from verbatimFields, which seems to properly handle umlauts and accents.
However, he still cannot handle all filenames. For example, file = {:.../Barreau et al. - 2009 - Impact of {\{}Cu{\}}-rich growth on the {\{}CuIn{\_}{\{}1-x{\}}Ga{\_}xSe{\_}2{\}} surface morphology and related solar cells behaviour.pdf:pdf} fails. The actual filename is Barreau et al. - 2009 - Impact of {Cu}-rich growth on the {CuIn_{1-x}Ga_xSe_2} surface morphology and related solar cells behaviour.pdf so the extra curly braces are the problem here as well. Do you see a possibility to catch that?
In the url field, he now imports {\&} as <span class="nocase">&</span>.
Thanks, suppresnocase = True actually seems to have done the job also for the braces, which I realized when doing the debug for an BBT issue. Now, using also zotero's `Find available PDF` option, I have more PDFs in the database than previously with mendeley.
If you need some "dirty" BibTeX files for testing, let me know ;-)
The hidden options work for me in the meantime.
It's really incredible what Mendeley does both intentionally and through sheer ignorance to make interoperability with other programs a pain in the a*.
That would actually be helpful, specifically if the mess could be constrained to a one- or two-entry bibtex file, plus the attachment file. Preferably zipped and attached to an issue on github.
Upgrade Storage
When I use the stock Zotero to export a BibTeX containing both a stored file and a linked file, I get the following when I leave "Export Files" unselected:
@book{osmith_stored_2019,
title = {Stored copy of file},
author = {öSmith, John},
year = {2019},
file = {öSmith - 2019 - Stored copy of file.pdf:C\:\\Users\\noksagt\\Zotero\\storage\\7H24VG95\\öSmith - 2019 - Stored copy of file.pdf:application/pdf}
}
@book{smith_linked_2019,
title = {Linked copy of file},
author = {Smith, John},
year = {2019},
file = {ö.pdf:C\:\\temp\\ö.pdf:application/pdf}
}
I get the following when I select "Export Files":
@book{osmith_stored_2019,
title = {Stored copy of file},
author = {öSmith, John},
year = {2019},
file = {öSmith - 2019 - Stored copy of file.pdf:files/15/öSmith - 2019 - Stored copy of file.pdf:application/pdf}
}
@book{smith_linked_2019,
title = {Linked copy of file},
author = {Smith, John},
year = {2019},
file = {ö.pdf:files/4/ö.pdf:application/pdf}
}
file = {C:\\temp\\ö.pdf}
file = {C:/temp/ö.pdf:PDF}
Either style of link (the first is exported from BBT on Windows when you have "Export Files" unselected and the second is the style you said you edited manually), will not import into either stock Zotero. If you try to open either style of link in JabRef, it won't find the file. That is because the unescaped drive letter colon is understood by both programs to be a field separator. I created a BBT issue. Perhaps you can submit a Report ID and/or provide any other information in that issue.
BetterBibTex actually does the correct thing if "Export Files" is selected.
Either of these work in both Zotero and JabRef, though:
file = {C\:\\temp\\ö.pdf}
file = {:C\:/temp/ö.pdf:PDF}
@article{Oertel2012j,address = {Wiesbaden},
author = {Oertel, Herbert and Ruck, Sebastian},
doi = {10.1007/978-3-8348-8631-6},
file = {:D$\backslash$:/Users/Steffen/Documents/Literatur/pdf/Mendeley/Artikel/Oertel, Ruck - 2012 - Biostr{\"{o}}mungsmechanik(5).pdf:pdf},
isbn = {978-3-8348-1765-5},
keywords = {DOPPELT},
mendeley-tags = {DOPPELT},
publisher = {Vieweg+Teubner Verlag},
title = {{Biostr{\"{o}}mungsmechanik}},
url = {http://link.springer.com/10.1007/978-3-8348-8631-6},
year = {2012}
}
?
D$\backslash$:is simply wrong BTW.fileis a verbatim field so that literally just meansD$\backslash$:, notD\:.\"oin it. This is unfortunately not easy to change in BBTs bibtex parser. I'll give it some thought.filefield name intofiles?valueinstead ofrawValue, but I don't know the other implications of that.It'd be helpful if any discussion about BBT could move to the BBT GitHub tracker and we could keep this thread about the stock translator — this is getting too confusing.
In different messages, the OP pointed out what I would consider two different bugs in the way Mendeley and Better BibTeX were sometimes exporting links for files on Windows. emiliano has patched the latter.
The stock translator and BBT both export correctly formatted links on Windows now & either translator is able to import from itself or from the other.
Mendeley's exported links are still broken:
file = {:D$\backslash$:/Users/Steffen/Documents/Literatur/pdf/Mendeley/Artikel/Oertel, Ruck - 2012 - Biostr{\"{o}}mungsmechanik(5).pdf:pdf},Dan is correct that the current use of raw value mean that field is uncorrected. The transform leads to no changes:
:D$\backslash$:/Users/Steffen/Documents/Literatur/pdf/Mendeley/Artikel/Oertel, Ruck - 2012 - Biostr{\"{o}}mungsmechanik(5).pdf:pdfFor me, replacing rawValue with value fixes the umlaut and keeps the drive colon escaped, but does not seem to make a resolvable path:
:D\:/Users/Steffen/Documents/Literatur/pdf/Mendeley/Artikel/Oertel, Ruck - 2012 - Bioströmungsmechanik(5).pdf:pdfIs this because of the forward slashes used?
value/rawValuething is fixed. If you export from Zotero as Western, this is what you get:file = {F{\"u}bar.jpg:files/22/F{\"u}bar.jpg:image/jpeg}I don't know if that's correct or not, but if you import that back, the file isn't included with the translator unless
rawValueis changed to tovalue.filefield is verbatim) but it's also probably an acceptable compromise when targeting something that doesn't grok utf-8. For bibtex exports, this is going to be a compromise one way or the other.BBT can now be told whether
fileshould be treated as verbatim on import. I'm still pondering whether to just fix this behavior to "no", or what to set the default to if not fixed.$backslash$problem.Is there a fix in the meantime?
By the way, the problem does not only affect the
filefield, but also theurlfield, where Mendeley for example encodes&as{\&}.filefield tofilesor removefilefrom the hidden preference verbatimFields.As to the way Mendeley exports that URL, that's just simply and inexcusably wrong. You can make BBT import it by likewise removing
urlfromverbatimFields<rant>, but that one really shows the Mendeley team hasn't bothered to ever open the bib(la)tex manuals. Aside the fact that the url field is not supported by most bibtex styles so should be seen as out-of-spec for bibtex, and that in biblatex it is a verbatim field so should not be escaped, it is at the same time the dumbest way I've seen&exported, because even if it were a supported field, and not a verbatim field, the braces are simply unnecessary, and if they'd omitted that, BBT would already have imported it as you'd wanted to, because I skip backslashes in URLs (because too many bibtex files have them to escape stuff like this).</rant>I removed
fileandurlfromverbatimFields, which seems to properly handle umlauts and accents.However, he still cannot handle all filenames. For example,
file = {:.../Barreau et al. - 2009 - Impact of {\{}Cu{\}}-rich growth on the {\{}CuIn{\_}{\{}1-x{\}}Ga{\_}xSe{\_}2{\}} surface morphology and related solar cells behaviour.pdf:pdf}fails. The actual filename isBarreau et al. - 2009 - Impact of {Cu}-rich growth on the {CuIn_{1-x}Ga_xSe_2} surface morphology and related solar cells behaviour.pdfso the extra curly braces are the problem here as well. Do you see a possibility to catch that?In the
urlfield, he now imports{\&}as<span class="nocase">&</span>.For the problem with the extra braces I'll need an actual sample for diagnosis. You can open an issue on the BBT issue tracker for that.
suppresnocase = Trueactually seems to have done the job also for the braces, which I realized when doing the debug for an BBT issue. Now, using also zotero's `Find available PDF` option, I have more PDFs in the database than previously with mendeley.The hidden options work for me in the meantime.
It's really incredible what Mendeley does both intentionally and through sheer ignorance to make interoperability with other programs a pain in the a*.