PubMed import doesn't work

SamInverso

Hi,

Importing a PubMed citation no longer works.
Here is an example:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=14283058

Zotero pops up a message:
"Could Not Save Item.
An error occurred while saving this item.
Please try again. If this error persists,
contact the translator author."

Importing from other sites does work, e.g. scholar.google.com

I'm using Firefox 2.0 and Zotero 1.0.0b2.r2

It's very frustrating as I predominately use PubMed and HubMed (which also doesn't work) to gather citations.

Thank you for any help,

Sam

dstillman

This is a known problem (mentioned in other threads and on the Known Issues page), though one without a known solution.

To be honest, we're not sure what's going on with this one, as we can't reproduce it and the error messages being produced are less than clear. We may try to rewrite the section of the PubMed scraper that seems to be causing problems, but it's hard to do without being able to see the problem ourselves.

A couple questions for people experiencing this:

1) Are you logged in to My NCBI when the scraping fails?

2) Are you using any other extensions that might affect parsing of the page? Greasemonkey, etc.?

SamInverso

Hi Dan,

Thanks for the quick response.
I am not logged in to My NCBI.
I disabled all other extensions and the same error appears.

neilfws

A lot of people are reporting this error with PubMed/HubMed. I decided to run Firefox from a command line and capture the error messages generated by a PubMed page. I hope this is useful to you for debugging.

Here are the messages generated when you go to the NCBI PubMed start page:

zotero(3): binding sandbox to http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?CMD=search&DB=pubmed

zotero(3): searching for translators for http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?CMD=search&DB=pubmed

zotero(5): SELECT detectCode FROM translators WHERE translatorID = ?

zotero(5): Binding parameter 1 of type string: "fcf41bed-0cbc-3704-85c7-8062a0068a7a"

zotero(3): executed detectCode for PubMed

zotero(3): executed detectCode for COinS

zotero(3): executed detectCode for Embedded RDF


If I do a search and click on a particular article, the abstract icon appears in the address bar. If I click on that icon, I get these messages:

zotero(5): SELECT * FROM translators WHERE translatorID = ? AND translatorType IN (4,5,6,7,12,13,14,15)

zotero(5): Binding parameter 1 of type string: "fcf41bed-0cbc-3704-85c7-8062a0068a7a"

zotero(3): translate called

zotero(3): binding sandbox to http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Retrieve&dopt=AbstractPlus&list_uids=16944959&query_hl=1&itool=pubmed_docsum

zotero(3): parsing code for PubMed

zotero(3): SyntaxError: missing ) after argument list in parsing code for PubMed
zotero(3): running handler 0 for done

zotero(3): translation using PubMed failed:
message => missing ) after argument list
fileName =>
lineNumber => 108
stack => doWeb()@:0
@:0
@:0

name => SyntaxError
url => http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Retrieve&dopt=AbstractPlus&list_uids=16944959&query_hl=1&itool=pubmed_docsum
extensions.zotero.cacheTranslatorData => true
extensions.zotero.automaticSnapshots => true

zotero(3): HTTP POST ids[]=fcf41bed-0cbc-3704-85c7-8062a0068a7a&lastUpdated=2006-10-23%2000%3A23%3A00&extVersion=1.0.0b2.r2&errorData=message%20%3D%3E%20missing%20%29%20after%20argument%20list%0AfileName%20%3D%3E%20%0AlineNumber%20%3D%3E%20108%0Astack%20%3D%3E%20doWeb%28%29@%3A0%0A@%3A0%0A@%3A0%0A%0Aname%20%3D%3E%20SyntaxError%0Aurl%20%3D%3E%20http%3A//www.ncbi.nlm.nih.gov/entrez/query.fcgi%3Fdb%3Dpubmed%26cmd%3DRetrieve%26dopt%3DAbstractPlus%26list_uids%3D16944959%26query_hl%3D1%26itool%3Dpubmed_docsum%0Aextensions.zotero.cacheTranslatorData%20%3D%3E%20true%0Aextensions.zotero.automaticSnapshots%20%3D%3E%20true to http://www.zotero.org/repo/report



Hopefully somebody can interpret that and address the issue.

dstillman

Thanks for posting that—alas, it doesn't help much, as we it's all standard output up until the error, which (as you can see) we receive via the POST to the repository when the notification pref is on.

The error, if we've done our math right, corresponds to an XPath line (doc.evaluate()) in the PubMed scraper's doWeb() method, which seems to suggest that there's something weird going on with parsing of the DOM—or the DOM itself that some people are receiving. (If someone experiencing the problem wants to (g)zip the HTML of an article page that fails and send it to support@zot....org, we can see if it's somehow different from what we're getting.) Simon, who wrote the scraper, is also going to try to rework it to avoid using the DOM this way, which may fix the problem. Otherwise, anyone seeing this error who wants to get their hands dirty can take a look and see if they can figure out what's going on—we'd appreciate any help.

dstillman

OK, we've solved the PubMed mystery.

It turns out the problem was due to a Firefox bug that causes DOM nodeValues longer than 4K characters to be split into multiple nodes. Any translators that had been updated via the repository with 'code' fields longer than 4K were getting truncated. We didn't notice this bug during testing of the repository update mechanism because most translators aren't that long.

I've put in a workaround for this in the code in the next release. In the meantime, I've managed to shrink the recently updated scrapers to less than the cutoff limit, so your copy of Zotero should auto-repair within the next 24 hours. (We'll have a manual update button in the next release. If you're particularly impatient you can close Firefox, run "DELETE FROM version WHERE schema='lastcheck'" in zotero.sqlite, and restart.)

Let us know if you continue to have problems after 1 a.m. EST Saturday.

SamInverso

Hi Dan,

Thanks it works now!

Best,

-Sam

neilfws

Great work guys - the "could not save" problem is now fixed for me.

So the next problem ;)

Has been posted in another thread, but was perhaps lost in the discussion of general PubMed problems. Zotero now saves from PubMed, but I don't get any page numbers in the citation.

Any thoughts?

dstillman

We've updated the repository with a version that supports PubMed page numbers. Your copy should auto-update by 1 p.m. EST Thursday.

neilfws

Page numbers from PubMed now working for me. Thanks guys - this is well on the way to becoming a must-have app.

andywingate

Hi I have a new issue with this citation

http://www.ncbi.nlm.nih.gov/sites/entrez?Db=pubmed&Cmd=ShowDetailView&TermToSearch=17494702&ordinalpos=1&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum

all the other ones i tried work fine?? Odd that this single citation wont save....

Regards
Andy

sean

Works fine for me.

andywingate

Hi,

Also worked for me just now.. so sorry about that!

regards
Andy

soaringbear

A year later on Zotero 1, firefox 2, vista, I'm finding erratic ability to import a batch of citations from pubmed. Single ones work but several sometimes works and sometimes fails. Right now I searched on samaderine and got 4 hits and they wouldn't import.

sean

Thanks for the heads up. We have a fix, and it will roll out shortly.

taitius

I can't get anything to work on pubmed! the little page icon doesn't show up on any of the article i pull up and I can't import from a text file either. I thought this program was going to be helpful, but all I've had is compatibility issues with everything!

sean

Could you provide a problematic URL?

theolis

I still have the problems mentioned in the first post, I am running Zotero 2.0rc5, here full description of the problem:

I am trying to import the citation data from http://www.ncbi.nlm.nih.gov/pubmed/10834981; http://www.ncbi.nlm.nih.gov/pubmed/15753260; http://www.ncbi.nlm.nih.gov/pubmed/8779555 by clicking either in the "folder" icon of the previous pubmed search page or for each abstract in the "page" icon at the right of the URL. It used to work fine for any article but now it gives me "an error occurred while saving this article" and some info about translator issues. I went to the translator issues page and verified my mozilla version (3.6), and performed the database integrity check (passed). What should I do?

theolis

got it,

I had 4 tabs more open while doing my pubmed search. They corresponded to sites of the journals from which I was downloading the articles. When I closed all tabs, reinitiated firefox and tried import it worked. Hope it helps to fix it. Best, Theolis

sidies

Hi, I'm using zotero 2.1.8 on windows vista. For some reason I can't import this reference from pubmed: PMID 21926981
Is this because the citation is new?

adamsmith

sidies -
update Zotero (current version is 2.1.10). See if the problem persists, then start a new thread.