Pubmed data not dowloading properly

mkostura

When saving a Pubmed page to Zotero, the translator appears not to be working properly. The citation is saved as a webpage, there is no snapshot and there are no associated tags. The authors are not listed or are listed as al. This just started happening for me today. It seems to be a generic problem and occurs with all references. I had looked into the forum and noticed that some people were having the same issue.

adamsmith

Run through https://www.zotero.org/support/troubleshooting_translator_issues and provide all the info requested under 12 (assuming 1-11 don't help), please.
Pubmed works fine for me, so this isn't a general issue (we've so far had one other user report this exact issue).

mkostura

I had gone through the debug issues. See uploaded Report ID: 1905241867

The Pubmed page is https://www.ncbi.nlm.nih.gov/pubmed/29220034

adamsmith

We'd want a Debug ID as per step 12, not an error report ID.
Instructions (also linked on the troubleshooting page): https://www.zotero.org/support/debug_output#zotero_connectors_firefox_chrome_and_safari

mkostura

The Debug ID is D660502874

dstillman

It looks like the PubMed API is being proxied, which won't work. Go to the Proxies pane of the Zotero Connector preferences, click on the proxy entry for your university, and look for an eutils.ncbi.nlm.nih.gov entry. Delete it and try again.

Not sure how this would've happened unless you clicked through to an eutils URL from your university library site, but @adomasven might have an idea what happened here.

dstillman

(Unless we're automatically proxying subdomains — which we shouldn't do — and you clicked through to an nih.gov page.)

mkostura

There was no such proxy entry for the university domain. I deleted all of the links to pubmed and NLM and started over. The only current link that I now have is www.ncbi.nlm.nih.gov. No success using that link as I have the same problems. Peculiar.

adamsmith

Are you saying that www.ncbi.nlm.nih.gov is still listed as a proxied site in the Proxy settings? Try removing that too. You could also temporarily disable automatic proxy redirection entirely to make sure that's the cause of the problem

mkostura

I deleted all links to ncbi and pubmed. I left the proxy redirection on. I then went to the Pubmed site without the proxy. I can download a reference to zotero with the proper translation. I then linked directly to a journal site through the proxy. I am not having difficulties there. In this case the I can pull down the abstract and the PDF. (but not at the same time; I have noticed that there is a failure now on most sites to bring down the PDF file. separate issue). So I conclude that something is amiss with the proxy settings and Pubmed.

adamsmith

Sorry, still confused.

I then went to the Pubmed site without the proxy. I can download a reference to zotero with the proper translation.

this sounds like it is working now?

mkostura

The Zotero connector and the pubmed site work without using the NCSU proxy. But that means I do not have a working search and reference retrieval since my access to all journals must go through the NCSU proxy. My work flow is to access the journals via Pubmed and the proxy. So I need to fix the PubMed proxy link. I am working with NCSU to see if there is something amiss on their end but so far they say nothing has changed.

adamsmith

OK, so just so I understand correctly --
https://www.ncbi.nlm.nih.gov/pubmed/29220034 isn't actually what you're seeing as the URL, right? You're looking at something like
https://www-ncbi-nlm-nih-gov.proxy.nscu.edu/pubmed/29220034 (I made the proxy string up, but same idea).

mkostura

You are correct. An actual link: https://www.ncbi.nlm.nih.gov/pubmed/28844391
A proxied version of that link: https://www-ncbi-nlm-nih-gov.prox.lib.ncsu.edu/pubmed/28844391. The former works (that is I can retrieve the informaton) and the latter does not.

dstillman

(3)(+0000001): Translate: Beginning translation with PubMed

(3)(+0000003): Translate: http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=PubMed&tool=Zotero&retmode=xml&rettype=citation&id=28844391

(3)(+0000001): Translate: resolving URL http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=PubMed&tool=Zotero&retmode=xml&rettype=citation&id=28844391

(3)(+0000001): Translate: resolved to http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=PubMed&tool=Zotero&retmode=xml&rettype=citation&id=28844391

(3)(+0000002): Translate: proxified to http://eutils.ncbi.nlm.nih.gov.prox.lib.ncsu.edu/entrez/eutils/efetch.fcgi?db=PubMed&tool=Zotero&retmode=xml&rettype=citation&id=28844391

(3)(+0000001): HTTP GET http://eutils.ncbi.nlm.nih.gov.prox.lib.ncsu.edu/entrez/eutils/efetch.fcgi?db=PubMed&tool=Zotero&retmode=xml&rettype=citation&id=28844391

(2)(+0000281): Translate: Translation using PubMed failed:
thrown exception => No Pubmed Data found - Most likely eutils is temporarily down

So what's happening here is that, since you're connecting to PubMed via the proxy, Zotero is using a proxied URL for PubMed's API (data service) as well, but your proxy server isn't handling requests to the PubMed API.

This could be something for your library to fix, but we could probably also blacklist the PubMed API from proxying, since as far as I know there'd never be any reason for that to be proxied. (The site itself doesn't need to be proxied either, of course, but as you say if you want to click through to proxied journals it can make sense to access it that wy.)

@adamsmith, do you know if anything on the PubMed site actually uses the API? If so, that would presumably be broken for anyone at @mkostura's institution accessing PubMed via the proxy and would be a good reason for them to fix it anyway, even if we also fix this on our end.

mkostura

I appreciate the details and will pass them along. I am a "patron" of the NCSU system and so I have different access controls for the library services than do faculty, staff and students. That has never been a problem before and I have used Zotero successfully for many years. For some reason access privileges to Pubmed were changed and that this difference may be root cause for the problem. The library has provided no reason for the change but they have investigated and indeed they agree there is an issue. When, or even if, that can be fixed is not clear.

adamsmith

@dstillman no, I'm pretty sure the website never uses the API -- at least I don't see any API calls, including when requesting XML and running export. So we just need to make sure eutils isn't being proxied for mkostura and this should work.

mkostura

I have had a conversation with the NCSU help desk and gave to them the text provided above. They find it very helpful. They been having some access issues with Pubmed and the problem I am having may be related. That said, how can the eutils site be blocked from the Zotero connector?

dstillman

The next version of the Connector, which should be out within a few days, will blacklist the PubMed API from proxying.

Gurdas_Sandhu

I wonder if the " Disable proxy redirection when my domain name contains*" could be used to prevent proxy redirection for the Pubmed site?
@mkostura , Go Pack! (I'm a patron, too)

adamsmith

It almost certainly would, but it'd obviously also remove this otherwise quite useful functionality.

mkostura

No need to make any adjustments to the connector. I spoke with people at the University library and they have made changes on their end that indirectly disabled functionality of the connector. I got quite an interesting education on how the libraries have to manage digital rights for the publishers and this problem was caused by those demands. Its all on how they have configured their proxy service. I now have a workaround that retains the functionality of Zotero.

dstillman

(The latest connector version, 5.0.35, skips the proxy for the PubMed API, in any case.)

Gurdas_Sandhu

Update: Actually, I might be looking in the wrong place. Where is the connector changelog?

@dstillman Just in case this was a documentation oversight, the changelog does not mention this:
https://www.zotero.org/support/5.0_changelog