Getting references into Zotero from online databases (with PDFs)

drhardy
I typically use PsycInfo (an EBSCO online database) to search for articles/chapters. Using the Zotero web extension I can just click the button on my ribbon up top and get sources into the folder I want in Zotero. That's easy. The trick is getting it to also bring the PDF. Most of the time I have to download then attach the PDF. It's a pain. Sometimes I'm even on a webpage looking at the PDF and the web extension doesn't find it. When it opens the PDF in EBSCO it doesn't find it (even though I'm look at the PDF). When EBSCO takes me to the journal website and the PDF opens in the browser there, it works. Any suggestions? I really hate having to download and attach 3/4 of the time.
«1
  • adamsmith
    I haven't tested extensively, but works for me.
    - What browser, what version of Zotero?
    - are you accessing EBSCO from on-campus or off-campus?
    - does this happen only when you import directly from a search result or also after clicking through to the individual article on EBSCO?
    - In EBSCO, do you see the "PDF Full Text" icon under the search result/to the left of the article?
    - Can you provide a couple of permalinks (create at the right of EBSCO item)? They'll include your institution -- it'd be easier to keep that in, but you can xxx it out if you prefer.
  • drhardy
    Firefox browser

    Zotero 5.0.55

    This is off campus, but, I have to log in using my university credentials so it usually still knows my subscriptions. When I log into EBSCO there is a yellow bar up top with a message saying "Zotero automatically redirected your request to search.ebscohost.com through the proxy at erl.lib.byu.edu."

    Here is an example of one where the PDF Full Text icon was there, it opened the PDF in the EBSCO window, but didn't find full text.
    https://www-lib-byu-edu.erl.lib.byu.edu/cgi-bin/remoteauth.pl?url=http://search.ebscohost.com.erl.lib.byu.edu/login.aspx?direct=true&db=psyh&AN=2017-52850-001&site=ehost-live&scope=site

    Here is another example from a different journal:
    https://www-lib-byu-edu.erl.lib.byu.edu/cgi-bin/remoteauth.pl?url=http://search.ebscohost.com.erl.lib.byu.edu/login.aspx?direct=true&db=psyh&AN=2017-08314-004&site=ehost-live&scope=site

    I even tried using Cisco AnyConnect first, which is supposed to make my computer think I'm on campus. Still didn't work.

    It usually works when I have to go into the journal website to find the article, and it opens it in the browser from that point. Here's a link to one such example that worked:

    https://www-lib-byu-edu.erl.lib.byu.edu/cgi-bin/remoteauth.pl?url=http://search.ebscohost.com.erl.lib.byu.edu/login.aspx?direct=true&db=psyh&AN=2018-31820-001&site=ehost-live&scope=site
  • bwiernik
    Does it work if you use Cisco AnyConnect, but then don't use the proxy (right-click on the Zotero button in your browser, click Preferences, click Proxies, and uncheck the "Enable proxy redirection" box)?

    If you are on campus or using Cisco AnyConnect, the proxy redirection is unnecessary, and it is likely that the issue is with using EbscoHost in combination with the proxy, which the translator developers can look into.
  • adamsmith
    I don't think this is a proxy problem -- I have this working with proxy (and I think a mostly identical proxy setup to BYU) and including for the examples above.
    Could we please also get a URL exactly as you see it, i.e. copied from the URL bar as well as a debug ID from the Zotero connector for saving one of those item.
    https://www.zotero.org/support/debug_output#zotero_connectors_firefox_chrome_and_safari

    Also, do you get a red x in front of a PDF when it's trying to import, or is there no sign of a PDF at all?

  • drhardy
    URL for example 1 above:

    http://web.a.ebscohost.com.erl.lib.byu.edu/ehost/pdfviewer/pdfviewer?vid=8&sid=e78f73be-9c5c-4f5e-be08-a2f6160e6481@sessionmgr4009

  • drhardy
    Link to a Word doc with a screen shot of what I see when it doesn't work.

    https://www.sugarsync.com/pf/D644099_07382366_8239357
  • adamsmith
    the debug ID too, please
  • drhardy
    D679355317

    I logged myself doing the three examples above, the two that didn't work and the one that did.
  • adamsmith
    @dstillman could you check please? These are all working for me, both from the record page and from the PDF view page.
  • dstillman
    @drhardy: If you're getting a red X next to the PDF (that is, it's being found on the page but downloading just isn't working), we need a Debug ID from Zotero rather than from the Zotero Connector.
  • drhardy
    D1632184127

    Again, this is for the three examples above - two that didn't work and one that did.
  • dstillman
    (3)(+0000014): Downloading file from http://content.ebscohost.com.erl.lib.byu.edu/ContentServer.asp?[…]

    […]

    (5)(+0000002): CookieSandbox: Managing cookies for content.ebscohost.com.erl.lib.byu.edu

    (5)(+0000002): CookieSandbox: Added cookies for request to content.ebscohost.com.erl.lib.byu.edu

    (5)(+0000106): CookieSandbox: Managing cookies for content.ebscohost.com.erl.lib.byu.edu

    (5)(+0000000): CookieSandbox: No Set-Cookie header received for content.ebscohost.com.erl.lib.byu.edu

    (3)(+0000004): Getting contents of C:\Users\[…]\Zotero\storage\tmp-0HO5FJ\[…].pdf

    (2)(+0000006): Downloaded PDF was not a PDF

    (3)(+0000000): <H1>404 Not found</H1>


    (2)(+0000016): Error: Downloaded PDF was not a PDF
    So you're getting a 404 for the PDF download.

    The one that's working is using HTTPS, for what that's worth:
    I'll let others compare that to what they're getting for EBSCO when it works, but I'd guess this is a problem with the proxy. If you have Cisco AnyConnect, I would try using that without using the proxied URL, as bwiernik suggests.
  • drhardy
    When I used the Cisco AnyConnect without the Zotero proxy it worked. I just assume I have to get to the page that has the PDF. Ideally at some point Zotero will have something like Endnote's Find Full Text where Zotero goes to find the PDF, without me having to go all the way to the PDF page. But, at least it's working now when I'm on the PDF page. Thanks!
  • bwiernik
    That feature is in the current Zotero beta and will work when you are connected using Cisco.
  • drhardy
    Pardon my ignorance but by current Zotero beta do you mean the version I have or a version that is forthcoming? Not sure how it works with Zotero.

    Also, how do I use the feature? Does it happen automatically from the search result page in PsycInfo (EBSCO)? Also, if I enter a reference in Zotero by hand or another way, is there a way that Zotero will go find the full text (which is how the Endnote function works I think)?
  • adamsmith
    the current Zotero beta refers to https://www.zotero.org/support/dev_builds
    so you typically wouldn't yet have this installed, but it should land in regular Zotero in the near future, typically within a month or two, possibly sooner.

    As for the feature, it's a right-click option for any item with a URL or a DOI and without an attached PDF, whether it was automatically imported or manually added.
  • dstillman
    It will also find OA PDFs automatically when you add items from the browser and the PDF isn't available or accessible on the page.

    But that's somewhat unrelated to this. If you have access to the PDF on the page you're saving, Zotero should save it. If that's not happening, there's some problem in the interaction with the proxy server and/or VPN.
    if I enter a reference in Zotero by hand
    Note that you should pretty much never add items by hand in Zotero. Unless it's an archival item that's not on the internet, you should find it via the browser or using Add Item by Identifier.
  • dstillman
    @drhardy: We've had a couple other reports of this problem with EBSCO and proxies, but we're not able to reproduce it ourselves, so we're trying to debug it with others who've experienced it (though unfortunately we sort of need someone on macOS or Linux to do so). While I know this is working for you via VPN, if you're interested in fixing it via the proxy, you could check if someone from your university library is able to provide us with a temporary login to the proxy so that we can reproduce it, which should let us fix it quickly. If that's possible, they can email us at support@zotero.org.
  • drhardy
    I'm having this problem again. Not sure if Zotero changed, Ebsco changed, Cisco connect changed, or I changed a setting somehow. I'm contacting my library to see about your request above.
  • dstillman
    edited February 7, 2019
    We fixed the problem with EBSCO and proxies in Zotero Connector 5.0.52 in December. If you're seeing it now, before asking the library for anything, we'd want to see a Debug ID from Zotero for a save attempt that's not saving a PDF for you.
  • drhardy
    So, I shouldn't have to do through Cisco Connect anymore when off campus?

    When I go into Preferences for the Zotero Connector, and then Proxies, what should my settings be there?
  • drhardy
    And how do I know I have the latest version of the connector - does it update automatically or do I need to reinstall it?
  • dstillman
    edited February 8, 2019
    So, I shouldn't have to do through Cisco Connect anymore when off campus?
    Correct (though it should work via a VPN too).

    You generally don't need to worry about the proxy settings. It's all configured automatically.
    And how do I know I have the latest version of the connector - does it update automatically or do I need to reinstall it?
    It updates automatically. You can see your current version from your browser's Extensions pane.

    We'd need a Debug ID from Zotero for a save attempt to debug this further.
  • drhardy
    I'm worried about the proxy settings because I messed with them to try to see if I could fix the problem:)

    I enabled Debug ID. Here it is: D1566386452

    I did two tries in Ebsco, one that didn't work, and then one that did.

    Here is a link to a power point with screen shots of what I saw in each case.

    https://www.sugarsync.com/pf/D644099_07382366_9834393



  • drhardy
    I seem to still be having trouble with this off and on. I use our university library database, often from home. I typically am using PsycInfo, which is an EBSCO database. I have trouble getting Zotero to bring in PDFs, no matter what screen I'm on. Ideally when I am in the results list, and have the folder on the Zotero add-on, if I select a record it will go find the PDF. But even when I follow links down to the publishers website, to the page with the PDF, it often won't change the Zotero add-on to the PDF logo. I'm not sure what's wrong.
  • dstillman
    What do you mean by "the page with the PDF"? Do you mean the PDF itself, some sort of viewer page that shows the PDF in a frame, or a page that includes a link to the PDF? You would only see the PDF icon for the first and sometimes the second. On the article page, which is where you want to be saving from most of the time, you would just see an icon for the item type in question (e.g., journal article), and the PDF should show up in the save popup when you go to save.

    But if you're not seeing the PDF icon when viewing a PDF directly, we'd want to see a Debug ID from the Zotero Connector for loading the PDF and we'd want to know what it says when you hover over the Save to Zotero button.
  • drhardy
    This is still happening quite frequently, where I know my university has access to the PDF, and sometimes I'm looking right at the PDF, but Zotero won't bring it. Here is a Debug ID for one example. D1384943344. In this case, I was seeing the PDF from within the online viewer in PsycInfo (EBSCO). The Zotero Connector button looks like a piece of paper. When I click, it does import the record into my Zotero, but then when it tries to find the PDF, it just shows a red X (in the dialogue by the Zotero Connector). On the other hand, when I was in the PsycInfo search page, where the Zotero Connector looks like a folder, and I just checked the box for this reference, it worked. Here's the Debug ID for that run D2124930020. Shouldn't Zotero be able to find the PDF whether I am on the PDF page, on a viewer, or on the PsycInfo search page? I don't care so much as to what the Zotero Connector button looks like as whether or not it actually grabs the PDF reliably. Sometimes when on PsycInfo page it works, sometimes I have to follow the links to the journal page or something to get it to work, and sometimes it works on PsycInfo page but not beyond that (on a viewer or journal page).

  • dstillman
    In this case, I was seeing the PDF from within the online viewer in PsycInfo (EBSCO). The Zotero Connector button looks like a piece of paper.
    Can you provide a Debug ID from the Zotero Connector (not Zotero) for loading this page and saving?

    As I say above, we fixed this issue with EBSCO and proxies for most people back in 2018 after you and some other people originally reported it. For some reason it's still not working for you for some saves. The first one here is wrong:
    Importing attachment from URL http://content.ebscohost.com/ContentServer.asp[…]
  • drhardy
    Ok, here's a Debug ID from the Zotero Connector:

    986959460

    This was four attempts. The first two were through links like the following, that took me to results windows in Ebsco. So I was still in Ebsco, but looking at the PDF in the browser. In both cases they did not work (did not bring the PDF, even though I was looking at the PDF in the browser.

    https://www.lib.byu.edu/cgi-bin/remoteauth.pl?url=http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2020-61554-001&site=ehost-live

    In the third attempt, I clicked the following link, which took me to the publisher page. This worked.

    https://www.lib.byu.edu/cgi-bin/remoteauth.pl?url=http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2020-57447-001&site=ehost-live

    In the fourth case, I did a new search in Ebsco, and from the search results page, just selected a record from the Zotero Connector drop-down. This worked.

    Thanks,

    Sam
  • dstillman
    That's a Report ID, not a Debug ID.
Sign In or Register to comment.