Zotero causing continuous article-download request loop?
I received the following email from our electronic resources librarian this morning. Is Zotero aware of this issue or the setting that could be causing it?
...
Good morning.
Since late last week, I’ve received messages from some users and some vendors regarding unusual activity when attempting to access or download articles from some of the libraries’ online resources.
I am hearing from my colleagues at other institutions across the country that they are receiving similar reports. We are all still troubleshooting, but it appears that there may be some issue with Zotero.
For some users, Zotero is requesting permission to access via the proxy. If the user gets this message, they can accept it and it should permit access.
For some users, they are receiving blocks when trying to download articles. As best we can tell at the moment, it seems that Zotero sets off a loop of continuous attempts to download the article that then triggers an automatic block from the publishers’ site. Disabling Zotero or using a different browser should allow the user to access.
My colleagues and I, as well as some publishers, are investigating. I wanted to alert you in case a user contacts you with an access issue. Please send those reports to me so that I can assist them and gather information to help with the troubleshooting.
...
Good morning.
Since late last week, I’ve received messages from some users and some vendors regarding unusual activity when attempting to access or download articles from some of the libraries’ online resources.
I am hearing from my colleagues at other institutions across the country that they are receiving similar reports. We are all still troubleshooting, but it appears that there may be some issue with Zotero.
For some users, Zotero is requesting permission to access via the proxy. If the user gets this message, they can accept it and it should permit access.
For some users, they are receiving blocks when trying to download articles. As best we can tell at the moment, it seems that Zotero sets off a loop of continuous attempts to download the article that then triggers an automatic block from the publishers’ site. Disabling Zotero or using a different browser should allow the user to access.
My colleagues and I, as well as some publishers, are investigating. I wanted to alert you in case a user contacts you with an access issue. Please send those reports to me so that I can assist them and gather information to help with the troubleshooting.
I don't believe I have seen any reports of this here (though it's possible, if rare, that Zotero has received something privately), so any details you can find out would be helpful. The link to the proxy (is this a web or a system proxy?) seems very odd.
For clarification, when you say "those were for inaccessible resources," do you mean, when users tried to use the function to access something their university doesn't subscribe to/provide?
I'm pretty sure this would be about the Zotero Connector's proxy support, not "Find Available PDF" in the desktop app or automatic OA PDF downloading. The latest Zotero Connector release made some improvements to automatic proxy detection and redirection (separating the Login URL Scheme and the Proxied URL Scheme), and it's possible there's some bug or incompatibility there. But we'd need a more specific report, ideally with Debug IDs from the Zotero Connector and Zotero for reproducing it, to say more.
I'm beginning to ask patrons and colleagues to try and get a debug report of the issue. In the meantime, wanted to share the specifics of what we've seen so far.
"We spent last week working with various users and providers, as well as our colleagues at other institutions on this issue.
Apparently, recent web browser updates for Chrome and Firefox caused some problems with third-party plug-ins. We were seeing the issue with Zotero as that was the plug-in our users had installed but other institutions were seeing the same issue with other plug-ins as well.
The reports started tapering off at the end of last week and no new reports were received over the weekend. We will continue monitoring but the suspicious activity reports we received from vendors have been deemed resolved and the result of these browser issues rather than user action.
Please send users to me if they are experiencing any problems with access. While we must work with the vendors to clear the IP blocks, other errors this causes can be handled by the user clearing their cache and/or disabling the plug-in."
At your institution, with the Zotero Connector disabled, if you're on campus and you try to access a gated resource via your institution's proxy (
https://www-example-com.proxy.school.edu
), are you redirected back to an unproxied domain (https://example.com/
), or do you remain on the proxied domain even though the proxy isn't necessary for on-campus access?One theory we're pursuing is that this is a combination of 1) greatly improved proxy detection in the latest version of the Zotero Connector, causing automatic proxy redirection to be enabled for many more people and 2) a failure to detect a redirect loop if a proxy server redirects on-campus clients back to an unproxied domain. That could result in repeated requests to a vendor site for someone who previously used the proxy off-campus and then returned to campus. (It likely wouldn't result in repeated PDF downloads, but we suspect that's just imprecision in some of the reports and this is actually about webpage requests.)
We're working to add redirect-loop detection to the Zotero Connector (which it should have regardless, and used to many years ago), so if that is the cause, we'll have a fix out soon, but it'd be good to know if this is a plausible source of the current problem.As an example, while off-campus I can access this ebook through our proxy: https://ebookcentral-proquest-com.oca.ucsc.edu/lib/ucsc/detail.action?pq-origsite=primo&docID=4940559
Then, if I sign into VPN and gain an on-campus IP address, visiting that link again ultimately sends me here:
https://ebookcentral.proquest.com/lib/ucsc/detail.action?pq-origsite=primo&docID=4940559
I also have Zotero connecter installed but not configured at all. After arriving on the 2nd page I get this status message:
Zotero detected that you are accessing ebookcentral.proquest.com through a proxy. Would you like to automatically redirect future requests to ebookcentral.proquest.com through %h/%p?
As we suspected, the latest version of the Zotero Connector was failing to detect a redirect loop if someone who previously used a proxy later came onto campus (or used a VPN) and the proxy redirected back to an unproxied URL. We had code to deal with that previously, but the latest Connector version changed the proxy redirection process in a way that broke the loop detection.
We've now updated the Connector to properly detect these redirect loops. When a loop is detected, the Connector will suspend automatic proxy redirection for an hour. You can force it to continue redirecting by right-clicking on the save button and choosing Reload via Proxy or by clicking "Reenable proxy redirection" in the Proxies pane of the preferences.
The fix is included in Zotero Connector 5.0.104 for Firefox, available now, and it will be in version 5.0.104 for Chrome and 5.0.105 for Edge, which should be available later today, as soon as they're approved by Google and Microsoft.
Sorry for the trouble this caused, and thanks for helping us troubleshoot it.
And the good news is that, once these fixes are out, automatic proxy detection should work better in general at many more institutions, with many more people benefiting from automatic proxy redirection.