Zotero standalone should wait for proxy authentication

Sergio Callegari
Configuring zotero standalone to work with a proxy is already not easy, because of the lack of a proper interface to do so.

Now, the issue is that even when correctly configured, zotero standalone does not interact properly with proxies that require authentication.

Consider the following case:

1) You open Zotero standalone
2) You open your web browser
3) You navigate on the browser to a resource that you want to save on Zotero. Here there is something (e.g. a PDF document) that is only accessible through the proxy.
4) You press the Save to Zotero button
5) Zotero tries to get the document and to do so tries accessing the proxy. The proxy asks for authentication login and password. Zotero standalone shows the dialog to enter the authentication data. So far so good. However, at this point zotero standalone does not wait for the user to enter the data before attempting to download the document and consequently the download fails.
6) You get a half-downloaded entry in your collection with the "parent item" present and the inner stuff (e.g. PDF document) missing because of the failed download

at this point, you need to

7) Finish entering the authentication data so that zotero finally authenticates with the proxy
8) Erase the half-saved entry
9) Go back to the web browser and redo the save to zotero. At this time, Zotero standalone has already authenticated with the proxy and so the download succeeds

This is unnecessarily complex. When realizing that authentication to a proxy is needed, Zotero standalone should make the document download wait for it to succeed.

Note that having zotero work correctly with authenticated proxies is extremely important. Many universities can only provide to its stuff access to some resources they bought via authenticated proxies.
  • dstillman
    How did you "configure" the proxy in Zotero, and what settings is your proxy using? There's no UI because Zotero uses the system proxy settings by default. You can technically set the Mozilla proxy settings in about:config, but that's not a configuration we recommend or have particularly tested.

    Under normal operation, if extensions.zotero.triggerProxyAuthentication is true (which it is by default) at startup, Zotero checks with the networking stack to see if requests for several of the URLs specified in extensions.zotero.proxyAuthenticationURLs would use a proxy (i.e., for a general proxy or as specified in a PAC file with conditional rules). If so, it makes a foreground HEAD request to trigger an authentication dialog. So if you're not getting a proxy auth prompt at Zotero startup then something isn't behaving properly. We'd have to know how you've set up the proxy, in Zotero and/or on the system, to figure out why that is.
  • adamsmith
    (I'm a bit confused by the reference to gated materials -- those are typically accessed via suffix proxies like ezproxy, which are an entirely different topic--they don't require any configuration--but at least the ones I'm familiar with don't behave as described, either).
  • dstillman
    I was assuming it's about a PAC-based proxy setup — a PAC file has a set of conditional rules about which hosts need to go through the proxy, and all other requests are direct. So in that case you normally don't see the proxy auth dialog until you request a proxied site. But that's why we have the behavior I describe above to trigger this at startup. (That said, we haven't updated the list of URLs lately, and it's possible some of those aren't properly triggering the prompt.)
  • Sergio Callegari
    Sorry for the late answer.

    I have a manually configured proxy (no pac file), i.e. configured via about:config, by setting network.proxy.ssl, network.proxy.ssl_port
    and network.proxy.type = 1.

    I cannot use this proxy as a system proxy because it fails too often on some resources, but it is OK for papers. In firefox I generally use proxy selectors such as foxyproxy to select this proxy only for the resources that truly need it. On Zotero standalone, this should be unnecessary because I use Zotero standalone only to grab resources that work fine with the proxy.

    Indeed I could write a pac file, but I think that would be unnecessary if only zotero standalone could either:

    1) Ask for the proxy authentication at startup; or

    2) Check if downloads trigger a proxy authentication dialog, so that they are automatically retried after authentication is achieved.
  • dstillman
    Can you provide a Debug ID for Zotero startup, using the "Enable after restart" option? As far as I know this does work for most people. At startup, Zotero makes requests to hosts from a preconfigured list of gated domains, which should trigger proxy authentication both for general proxy setups (like yours) and for PAC-based proxies that specify those sites.
  • Sergio Callegari
    Sorry for the noise in the other thread...

    The Debug ID is D1359386684

    Relevant setup in about config is:

    network.proxy.http = localhost
    network.proxy.http_port = 8095
    network.proxy.type = 1

    This corresponds to a local stunnel instance (possibly unneeded today, but this is how my institution makes us work).

    With this, the zotero debug log says

    No general proxy or PAC file found -- assuming direct connection

    Hope this helps finding the issue.

    In the meantime, I'd like to thank you for looking into it.
  • dstillman
    edited August 6, 2016
    The URL Zotero checks at startup is HTTPS, so you probably need to configure network.proxy.ssl as well (using a separate stunnel port or connecting directly to the proxy server). More and more sites are using HTTPS, so you want to do that anyway if you need to use the proxy to access gated resources.
  • Sergio Callegari
    Great, with the network.proxy.ssl it works.

    But now I have one more question. This discussion made me curious, so I tried to get rid of the stunnel and foxyproxy in firefox by building a PAC file for my academic resources, since the PAC file is the only way to force firefox to access a proxy over SSL. This worked. Then I tried using the same PAC file with zotero standalone. I have:

    1) configured

    network.proxy.autoconfig_url

    to point to my pac file

    2) configured

    network.proxy.type

    to 2

    3) augmented my pac file to return the proxy to be accessed via ssl (marked as HTTPS host:port) also for urls like matching the pattern https://s3.amazonaws.com/zotero.org/*

    Unfortunately, in this way I cannot make zotero ask for the proxy authentication at startup.

    I wonder if you may have some hint at what I may be missing...
  • dstillman
    edited August 7, 2016
    Not sure — I'd have to see the new debug output. Note, though, that you shouldn't add S3 to the PAC file — that URL is used only for general proxy detection, and would cause your Zotero software updates, etc., to go through the proxy unnecessarily. For a PAC file it only needs to cover the domains in extensions.zotero.proxyAuthenticationURLs. (I don't recall the specifics off-hand, but it's possible it only checks those URLs and not the S3 address when it knows there's a PAC file, since the PAC file would never cover S3.)
  • Sergio Callegari
    Thanks for this advice, I have now sort of succeeded, but I'd like to make further experiments in the next days. In case I find something interesting, I'll post here (also for reference for anyone in similar situations as mine).
  • parthpatel
    @Sergio Callegari thanks a lot....you saved me... your network.proxy.type = 1 works...
Sign In or Register to comment.