Bibtex - Import pdf not found

Edisz

Hai everybody,

I am trying to move from Citavi to Zotero, via bibtex.

My bibtex-entries look like this:
@book{Abebe.2006,
author = {Abebe, Eyualem and Andrássy, István and Traunspurger, Walter},
year = {2006},
title = {Freshwater nematodes: Ecology and taxonomy},
keywords = {Freshwater nematodes;EBOOK},
address = {Wallingford and UK and Cambridge and MA},
publisher = {CABI Pub},
isbn = {0851990096},
institution = {{ebrary, Inc}},
file = {Freshwater nematodes_Abebe_2006:/media/sda7/Documents/Citavi 3/Projects/Komplett_Liste/CitaviFiles/Freshwater nematodes_Abebe_2006.pdf:PDF}
}

When I open the bibtex-file with JabRef the attached pdf are recognized and I can open them - everything fine.

When I import this bibtex file to zotero, it shows me the pdf-icon, however when I klick on it (to open the pdf) it says:

'The attached file could not be found.
It may have been moved or deleted outside of Zotero'


The debug-output of zotero says:

'Failed to load native module at path '/opt/Zotero_linux-x86_64/xulrunner/components/libmozgnome.so': (80004005) libnotify.so.1: cannot open shared object file: No such file or directory

[ Places Maintenance ]
> Integrity check
+ The database is sane
> Coherence check
+ The database is coherent

[JavaScript Error: "not well-formed" {file: "moz-nullprincipal:{52e7a1ef-eccf-4f87-bdc9-4927c8896323}" line: 1 column: 1 source: "@book{Nollet.2010,"}]

[JavaScript Error: "not well-formed" {file: "moz-nullprincipal:{55c5784b-d4f1-44a4-bec9-b27b7cbf2092}" line: 1 column: 1 source: "@book{Nollet.2010,"}]

[JavaScript Error: "not well-formed" {file: "moz-nullprincipal:{55c5784b-d4f1-44a4-bec9-b27b7cbf2092}" line: 1 column: 1 source: "@book{Nollet.2010,"}]

version => 3.0.3, platform => Linux x86_64, oscpu => Linux x86_64, locale => en-US, appName => Zotero, appVersion => 3.0.3

=========================================================

(2)(+0012486): Attachment file '/home/edisz/Documents/Literature/storage/TMCZDBXE/Freshwater nematodes_Abebe_2006.pdf' not found'

So Zotero looks in my datadirectory for the pdf, but cant find it there. One thing is perhabs that zotero does not copy the pdfs into the data directory, or it doesnot look in the path of the bibtex-file.

Any suggestions how to fix this?

adamsmith

I'd try to experiment a little to see what part isn't working - to start with - see if the folder '/home/edisz/Documents/Literature/storage/TMCZDBXE' exists and if it has a pdf file in it.
You could also see if the file-path or the name of the PDF matters.

Edisz

No there is no folder called TMCZDBXE. Perhabs also a permission issue, i'll check...

adamsmith

I vaguely remember that there might be an issue with moving the storage directory from it's default location on linux. Try to set it to it's default location - and try again.

Edisz

No, the same with the default data directory:

Attachment file '/home/edisz/.zotero/zotero/fx53y7ar.default/zotero/storage/VE23KPGB/Freshwater nematodes_Abebe_2006.pdf' not found

Also, the same when as root.

Edisz

Here is a snippet of the log when importing, perhabs has to do with this? That the only things that looked somehow strange in the log...

Thanks for your help!

(3)(+0000001): Translate: Initializing RDF data store

(2)(+0000001): Translate: Detect using Bibliontology RDF failed:
thrown exception => Translate: No RDF found
url => /home/edisz/Documents/Literature/citavi_db_export/ebooks_db_out2.bib
downloadAssociatedFiles => true
automaticSnapshots => true

(4)(+0000000): Translate: Parsing code for MODS

(2)(+0000006): Translate: Detect using MODS failed:
fileName =>
lineNumber => 1
string => SyntaxError: unexpected end of XML entity
url => /home/edisz/Documents/Literature/citavi_db_export/ebooks_db_out2.bib
downloadAssociatedFiles => true
automaticSnapshots => true

(4)(+0000000): Translate: Parsing code for Bookmarks

(4)(+0000002): Translate: Parsing code for CTX

(4)(+0000002): Translate: Parsing code for ISI Web of Knowledge

(4)(+0000003): Translate: Parsing code for MAB2

(4)(+0000002): Translate: Parsing code for MARC

(4)(+0000002): Translate: Parsing code for MARCXML

(4)(+0000001): Translate: Parsing code for NCBI PubMed

(3)(+0000002): Translate: Detecting Pubmed content....

(4)(+0000000): Translate: Parsing code for RDF

(3)(+0000003): Translate: Initializing RDF data store

(2)(+0000000): Translate: Detect using RDF failed:
thrown exception => Translate: No RDF found
url => /home/edisz/Documents/Literature/citavi_db_export/ebooks_db_out2.bib
downloadAssociatedFiles => true
automaticSnapshots => true

adamsmith

see here:
http://forums.zotero.org/discussion/6527/
I'm no bibtex expert, but it looks like Citavi exports in a jabref-specific fashion (which explains why jabref is happy...) - that's an old thread, but apparently no one ever took care of that.
Try if
pdf = {/media/sda7/Documents/Citavi 3/Projects/Komplett_Liste/CitaviFiles/Freshwater nematodes_Abebe_2006.pdf}
imports (or maybe with quotation marks?). If it does, this shouldn't be hard to fix with a good regex search and replace.

edit: actually - that's supposed to be fixed - try nevertheless. The output you posted is not relevant, no - that's just Zotero trying out different translators until it gets to bibtex - if you get the data imported, clearly Zotero recognizes this as bibtex and imports it as such.

Edisz

I tried with pdf = { and without :PDF at the end,
with pdf = { and without :PDF at the end,
with file = { and without :PDF at the end (gives me an error while importing),
with file = { and with :pdf at the end,
with pdf = { and with :pdf at the end.

And none of the worked :(

adamsmith

Provide a Debug ID (don't post the whole debug here, just the ID) for the whole process of importing a bbibtex with a PDF and then (unsuccesfully) trying to open it. Ideally just using one entry like the one above.
I'll see if I can replicate any of your problems later, but maybe the debug can help.

Edisz

The Debug ID is D777563174.

I have used this .bib - file:

@book{Nollet.2010,
author = {Nollet, Leo M. L. and Rathore, Hamir Singh},
year = {2010},
title = {Handbook of pesticides: Methods of pesticide residues analysis},
keywords = {Pesticide residues in food;Analysis;Pesticides;Agricultural chemicals;EBOOK},
address = {Boca Raton and Fla.},
publisher = {CRC Press},
isbn = {9781420082456},
file = {Handbook of Pesticides - Methods of Pesticide Residues Analysis_Nollet_CRC_2009:/media/sda7/Documents/Citavi 3/Projects/Komplett_Liste/CitaviFiles/Handbook of Pesticides - Methods of Pesticide Residues Analysis_Nollet_CRC_2009.pdf:PDF}
}

adamsmith

thanks - no need for you to do any more - this looks like it's a Zotero issue - I can replicate this even with bibtex exported from Zotero itself.

Edisz

No, thank you!

Hopefully gets fixed soon...

Edisz

Anything new?

adamsmith

Not yet - could you try moving a file to a different location - somewhere not on the media drive - change the respective filepath in the bibtex and try importing again? I've played with this a bit and I think there are two unrelated issues:
1. Zotero messes up the filepath - specifically the slash between media and sba7 - for your files. (that's why your import fails). I would like that confirmed and if I'm right maybe we can figure out why it does that - though for you the quickest fix might be to move all files and run a quick search&replace over your bibtex files.

2. Zotero doesn't deal correctly with directory relative paths on import (that's why it fails with files attached to its own bibtex).

Edisz

Moving the pdf to home path and changing the respective bibtex-file, gives still the same problem.

Edisz

And here is the DebugID for this: D191616002.

With this bibtex entry:
@BOOK{Chapin.a2005,
title = {Principles of terrestrial ecosystem ecology},
publisher = {Springer},
year = {2005},
author = {apin, Francis Stuart and Matson, Pamela A. and Mooney, Harold A.
and Chapin, Melissa C.},
address = {New York and NY},
edition = {[Nachdr.]},
file = {PrinciplesOfTerrestrialEcosystemEcology_Chapin_2002:/home/edisz/PrinciplesOfTerrestrialEcosystemEcology_Chapin_2002.pdf:PDF},
isbn = {0387954392},
keywords = {Ecology;Biogeochemical cycles;Biological systems;EBOOK},
price = {alk. paper}
}

adamsmith

odd - that works for me.
If you paste
file:///home/edisz/PrinciplesOfTerrestrialEcosystemEcology_Chapin_2002.pdf
into the URL bar of your browser, does that open the PDF?
Dan, Simon - anything you can make of the debug?

Edisz

Sorry, this works for me too! (Copied the wrong file to home).

So Zotero doesnot like 'media/sda7'...
Workaround would be to copy all the file to another place and then import.

Am I a right that Zotero will copy all the files into the storage-folder? So afterwards I could delete the citavi-files [I will not move back to Citavi]?

Edisz

OK, now it seems to work!
I just made a symbolic link pointing to /media/sda7... in my home-directory and edited the bibtex-files respectively.

Thank you adam for your time and help!

adamsmith

great.

Am I a right that Zotero will copy all the files into the storage-folder? So afterwards I could delete the citavi-files [I will not move back to Citavi]?

correct. You'll probably want to check whether things are working for a while before deleting the pdfs from citavi permanently, but once you're confident that they have imported reliably, yes, you can delete the citavi files.

I'd still be curious if Simon has any thoughts on fixing the issue with media/sda - what happens is that the filepath turns into
file:///medi℁da7/... on import and I don't know why.

Edisz

hmmm, looks like an encoding problem ?

adamsmith

ah yes - a/s is used in TeX to code ℁, so Zotero converts it - there are some other potential problems like c/u, c/o etc.. I've created an issue here https://github.com/zotero/translators/issues/279
this shouldn't be hard to solve.

adamsmith

once you update your translators (general tab of the Zotero preferences) this should now be fixed.