Getting connector to use a static SOCKS 5 proxy

YGLvGa2NtMfci

I access my university remotely using a static SOCKS 5 proxy which tunnels all my internet traffic through its system. My config in Firefox is nothing more than "127.0.0.1:8000" in the SOCKS 5 field, and with that everything works perfectly.

I can directly access and download every single resource my university has access to this way in Firefox. My problem is that I just moved from the Zotero 4.x extension for Firefox to Zotero 5 standalone and the Firefox connector, and now when adding items through the connector, the full text PDF download inevitably fails.

I've tried adding my proxy to the connector's settings about a dozen different ways, but nothing I've tried works.

This is about the simplest use case imaginable: a single static proxy, and no need for prepending destination URLs to the proxy address, regular expressions to catch certain URLs or proxy detection. Also, no login or other interaction involved.

I simply need to get Zotero's connector to proxy everything through 127.0.0.1:8000.

How do I do this?

adamsmith

What operating system?

dstillman

Nothing to do with the connector — that's already using your browser's proxy settings. Just apply the same about:config settings in Zotero via the "Config Editor" button in the Advanced → Files and Folders pane of the preferences.

YGLvGa2NtMfci

Windows 10. Firefox 54.0.1. Zotero Standalone 5.0.11.

YGLvGa2NtMfci

@dstillman I've just entered to following settings in Zotero standalone, which are copied directly from Firefox:

network.proxy.socks = 127.0.0.1
network.proxy.socks_port = 8000
network.proxy.socks_remote_dns = true
network.proxy.type = 1

(Also, every other about:config setting that appears when filtering on "proxy" is identical in both programs.)

When I attempt to add items through the Zotero connector now, there's a much longer delay when it gets to the "Full Text PDF" notice, but it still fails.

dstillman

Did you restart Zotero after changing the settings?

YGLvGa2NtMfci

Yes. And Firefox.

dstillman

Is this an authenticated proxy? Do you get prompted for a username and password when Zotero starts? (Edit: I guess you said no login.)

Can you provide a Debug ID for Zotero startup using the "Restart with Logging Enabled" option?

YGLvGa2NtMfci

No authentication at all. No prompts for anything because none are needed.

I just did the debugging, but Zotero threw an error when trying to send it... which is logical enough, since it's not going through the proxy, and the proxy is my only way to the internet!

I looked at the output, and Zotero seems to be trying to use the wrong port (23119 rather than 8080; the address is wrong too -- it's actually rather more custom than 127.0.0.1, but for clarity's sake I stated this address above; it should actually be in the 10.0.x.x range).

Here's an example:

(5)(+0000000): POST /connector/ping HTTP/1.1 Host: 127.0.0.1:23119 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0 Accept: */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Content-Type: application/json X-Zotero-Version: 5.0.17 X-Zotero-Connector-API-Version: 2 Content-Length: 2 DNT: 1 Connection: keep-alive

I can put the whole thing up on pastebin or something if that'll be of use. It doesn't seem to contain any sensitive information, or does it?

dstillman

No, the 23119 stuff is unrelated. Search for "proxy" in the output.

YGLvGa2NtMfci

Not a single case of "proxy" there.

dstillman

Can you email it to support@zotero.org with a link to this thread?

dstillman

Pastebin or a gist is also fine. Shouldn't be anything sensitive other than your filesystem path.

YGLvGa2NtMfci

Just sent the e-mail.

dstillman

We need output just for Zotero startup, using the "Restart with Logging Enabled" option — what you sent appears to be for saving.

YGLvGa2NtMfci

Quite right! Just sent a new e-mail with the restart debug log.

dstillman

[JavaScript Error: "Error: HTTP HEAD https://s3.amazonaws.com/zotero.org/proxy-auth failed with status code 0" {file: "chrome://zotero/content/xpcom/http.js" line: 595}]
Zotero.HTTP</this.triggerProxyAuth/</</<@chrome://zotero/content/xpcom/http.js:595:8

[JavaScript Error: "Error connecting to proxy -- proxied requests may not work"]

Make sure your proxy supports HEAD requests. You should be able to test the above with curl --proxy socks5h://10.0.x.x:8000 -I https://s3.amazonaws.com/zotero.org/proxy-auth (though I don't have a proxy set up to confirm that).

dstillman

Actually, I doubt that's it — the HEAD request is necessary for triggering proxy authentication, but I don't think that should affect later use of the proxy. In any case, clearly something about the proxy settings isn't working.

YGLvGa2NtMfci

Here's the output of the curl command:

HTTP/1.1 200 OK
x-amz-id-2: PFHfv2+ho4s3ejhyrDzsWKthDLKniWJHMzOBGtiYXTKQHQHLFjPeMo2c8cWzOZEQ
x-amz-request-id: 486A7C7AC4EAA882
Date: Wed, 16 Aug 2017 15:18:28 GMT
Last-Modified: Wed, 01 May 2013 08:22:39 GMT
ETag: "e0aa021e21dddbd6d8cecec71e9cf564"
Accept-Ranges: bytes
Content-Type: text/plain
Content-Length: 2
Server: AmazonS3

dstillman

I just tested those same settings with a SOCKS5 proxy over SSH on macOS, and it works fine for me. I'd guess that you have some security or firewall software that's blocking Zotero's access to the proxy address.

YGLvGa2NtMfci

Firefox and Zotero are going through the very same proxy (or trying to, in the case of Zotero). And they're both allowed to make any connection they want in the Windows firewall.

Firefox works, Zotero doesn't.

So I have to assume there's something in Zotero's about:config that I still need to change... but what?

YGLvGa2NtMfci

Looking at my network connections, I see Zotero is connected to the following right now:

TCP: 19873 (no port)
TCP: 23116 (no port)
TCP: 23119 (no port)
TCP: 52315 127.0.0.1:52316
TCP: 52316 127.0.0.1:52315
TCP: 52319 127.0.0.1:8000
TCP: 52320 127.0.0.1:8000
TCP: 52321 127.0.0.1:8000
TCP: 52322 127.0.0.1:8000
TCP: 52323 127.0.0.1:8000

So it's actually connected to the proxy in five different instances. And yet... nothing's getting through. What could be going on?

YGLvGa2NtMfci

For what it's worth, Zotero can't sync, either.

dstillman

Not sure what to tell you. Zotero is based on a build of Firefox, so this is just the same Mozilla code, and those four settings work for me. I just tried the same on Windows 7, and that works as well. If you're using SSH for the proxy, you can pass -v for more logging. (And if you're not using SSH, try using SSH.)

YGLvGa2NtMfci

I am indeed using SSH. I just restarted the proxy connection with -vvv. It spews scores of lines per pageload in Firefox, but when I try to sync or download somehting through the Zotero connector NOTHING shows up.

Something's broken here. Don't know if it's my config (your experience indicates it's not) or Zotero or the connector, but the proxy connection is completely dead to them.

dstillman

Again, nothing to do with the connector. This is just in Zotero itself.

This is what you should see for ssh -v when Zotero starts up:

debug1: Connection to port 8080 forwarding to socks port 0 requested.
debug1: channel 3: new [dynamic-tcpip]
debug1: Connection to port 8080 forwarding to socks port 0 requested.
debug1: channel 4: new [dynamic-tcpip]
debug1: Connection to port 8080 forwarding to socks port 0 requested.
debug1: channel 5: new [dynamic-tcpip]
debug1: Connection to port 8080 forwarding to socks port 0 requested.
debug1: channel 6: new [dynamic-tcpip]
debug1: Connection to port 8080 forwarding to socks port 0 requested.
debug1: channel 7: new [dynamic-tcpip]
debug1: channel 3: free: direct-tcpip: listening port 8080 for s3.amazonaws.com port 443, connect from 127.0.0.1 port 49523 to 127.0.0.1 port 8080, nchannels 8