Failing downloads of EBSCO PDFs

terber

Zotero bug report
Report ID: 1100051309

Unable to download PDFs for any papers hosted at EBSCO host
Steps to reproduce:

1. Start Zotero 5.0.59.
2. Start Firefox version 64.0, Windows 64-bit with latest Zotero Connector installed (as of 22.12.2018)
3. Use the library search tool of my university, which links to https://web-a-ebscohost-com.glos.idm.oclc.org/ehost/detail/detail?vid=1&sid=34ba0efd-c29f-4335-8bcd-0d879717a8ff@sessionmgr4010&bdata=JnNpdGU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#AN=4285493&db=bth ("Mobley, W. H. (1982). Some Unanswered Questions in Turnover and Withdrawal Research. Academy of Management Review, 7(1), 111–116. https://doi.org/10.5465/AMR.1982.4285493")
4. Click on Firefox' "Save to Zotero icon"
5. The Zotero icon shows "EBSCO Full Text" with a red cross, so the corresponding PDF is not successfully downloaded (while the bibliographic entry is correctly created)

I can download the PDFs manually without any issues or problems.

The problem is (for me) reproducible for all EBSCO host results. A restart of Firefox or Zotero or even Windows does not change the outcome in any way. No other Firefox add-ons were activated.

dstillman

A few people have reported this, but we haven't been able to reproduce it ourselves, so we'll need help to debug it.

See this post of mine for what we'd need. Unfortunately, it requires some unusually difficult troubleshooting steps, particularly on Windows, so if you're not able to help with that, we'd understand, and you can subscribe to that thread for updates.

terber

Dear "dstilllman",

in your recommended post at https://forums.zotero.org/discussion/comment/321733/#Comment_321733 you wrote:

"Open Terminal via Spotlight and paste in the command"

As far as I understand, "Spotlight" is only available on macOS, i.e. with Apple products. Do I assume correctly that the 'curl' command in question should be executed on Windows machines in a Windows Powershell instance?

dstillman

I've updated the instructions for Windows to explain how to get curl and use it with cmd.exe instead of using Cygwin. Alternatively, if you're familiar with PowerShell, at least from Chrome you can copy a PowerShell command instead and try following the same basic process of removing headers, but I don't really know anything about PowerShell.

terber

I tried to follow up the given instructions and observed a probably surprising result.
On my Windows 10 machine following curl version was already installed:

curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL

The full curl call as copied from Firefox:

curl "https://content.ebscohost.com/ContentServer.asp?T=P&P=AN&K=51827775&S=R&D=bth&EbscoContent=dGJyMNHr7ESeqLE4wtvhOLCmr1CeprRSs6a4Sq"%"2BWxWXS&ContentCustomer=dGJyMPGnr0qxprdNuePfgeyx44Dt6fIA" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H "Accept-Language: de,en-US;q=0.8,en;q=0.5,cs;q=0.3" --compressed -H "Referer: https://web-a-ebscohost-com.glos.idm.oclc.org/ehost/pdfviewer/pdfviewer?vid=2&sid=1458d58f-dd8e-4af5-9e40-f813f5ab0b08"%"40sdc-v-sessmgr04" -H "Connection: keep-alive" -H "Cookie: eis="%"5B"%"7B"%"22utm_medium"%"22"%"3A"%"22search"%"22"%"2C"%"22utm_source"%"22"%"3A"%"22www.google.com"%"22"%"2C"%"22utm_campaign"%"22"%"3A"%"22www.ebsco.com"%"2Fproducts"%"2Febscohost-platform"%"22"%"2C"%"22utm_content"%"22"%"3A"%"22undefined"%"22"%"2C"%"22eis_id"%"22"%"3A"%"22undefined"%"22"%"2C"%"22eis_strategy"%"22"%"3A"%"22undefined"%"22"%"7D"%"5D" -H "Upgrade-Insecure-Requests: 1"

I added following parameter to the curl call to save the resulting PDF:

--output "c:\dl\testUrlOut.pdf"

I then started to remove the "-H" options including their associated parameters one by one, waiting for the download to fail. This is the minimal command string which correctly downloaded the PDF:

curl "https://content.ebscohost.com/ContentServer.asp?T=P&P=AN&K=51827775&S=R&D=bth&EbscoContent=dGJyMNHr7ESeqLE4wtvhOLCmr1CeprRSs6a4Sq"%"2BWxWXS&ContentCustomer=dGJyMPGnr0qxprdNuePfgeyx44Dt6fIA" --output "c:\dl\testUrlOut.pdf"

This is surprising as the provided instructions stated:

"Try removing additional headers — -H and the entry after it — until it stops saying "Binary file (standard input) matches", leaving in -H 'Cookie: ezproxy=…' (since it definitely won't work without that one)"

Either I got something completely wrong or the last remark is incorrect. Any opinions?

dstillman

Ah, so when you click on an EBSCO PDF, it’s at an unproxied URL (content.ebscohost.com)? Or is there an intermediate proxied URL?

Could we see a Debug ID (different from a Report ID) from Zotero for a save attempt where the PDF doesn’t download?

terber

Just reproduced the problem. Debug ID is D1836032552.

dstillman

Thanks, but you didn’t answer my question, and that Debug ID is from the Zotero Connector rather than from Zotero.

terber

Here is the Zotero Debug ID: D999945248

terber

I was asked:

> so when you click on an EBSCO PDF, it’s at an unproxied URL
> (content.ebscohost.com)? Or is there an intermediate proxied URL?

I am on the following EBSCO result page:

https://web-b-ebscohost-com.glos.idm.oclc.org/ehost/detail/detail?vid=1&sid=377618ba-7ac1-4377-805f-811925bfe679@pdc-v-sessmgr05&bdata=JnNpdGU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#AN=51827775&db=bth

then I manually click on "PDF Full Text". The URL in the Firefox URL bar changes to the following:

https://web-b-ebscohost-com.glos.idm.oclc.org/ehost/pdfviewer/pdfviewer?vid=4&sid=377618ba-7ac1-4377-805f-811925bfe679@pdc-v-sessmgr05

Now the complete PDF is displayed in a Firefox frame, embedded in an EBSCO page

dstillman

OK, thanks, this was super helpful — I think I've figured out the problem. We should have a fix soon, and I'll let you know when it's available.

terber

Thank you very much for your efforts, dstillman!

In case further debugging from my side would be helpful, please do not hesitate to contact me.

dstillman

OK, can you try the latest Zotero Connector beta and see if that works? (If you try this within the next 24 hours, you might need to update translators from the Advanced pane of the Zotero Connector preferences.)

terber

Did not seem to work for me:

• Started Zotero 5.0.59
• Installed Zotero Connector 5.0.51beta3 for Firefox. Restarted Firefox.
• Hit "Update Translators" on "Advanced" tab of the Zotero Connector properties. As I did not observe any visible feedback I repeated this step once and waited for a half a minute. Restarted Firefox.
• Navigated to EBSCO page in question, clicked on Firefox' "Save to Zotero icon"
• Bibliographic entry was created but PDF was not downloaded (Zotero icon shows "EBSCO Full Text" with a red cross)

Connector Debug ID is D557993465. Zotero Debug ID is D1629391172

dstillman

Sorry, 5.0.51beta3 didn't properly include the update. Try 5.0.51beta4.

terber

Success! (with 5.0.51beta4)

Big thanks to you, dstillman, much appreciated.

dstillman

Great, thanks so much for your help. This fix is now available in Zotero Connector 5.0.52.