Standalone won't sync but the Firefox extension will

DougHunt

I had been using Zotero 4 from within Firefox 22 for some time, very successfully. However, I mostly use Google Chrome as my main browser and it was a bit of a pain having to swap browsers every time I wanted to add an item to my collection.

I decided to install standalone Zotero 4 so that I could use Zotero directly from Chrome. During the installation it asked if I wanted to use my existing Zotero library and I said "Yes". After installation I could see my existing library (correctly) in standalone.

I added a couple of new items using standalone and then went to sync and received an error message (Unable to connect). Using standalone I am able to lookup the meta data for a pdf file and so I know that Zotero standalone can connect correctly to the internet (we do use proxies locally)

I checked that Standalone had my correct account name and password and used the same data to log into my Web Based Zotero account to make sure that they were still current. No problems but I still could not sync. Searched the forums and did not find a solution.

Then decided to try exiting Zotero standalone then starting Firefox. Without Standalone running, Firefox reverted to the embedded copy of Zotero and as that pointed at the same local copy of my Zotero data I decided to try a sync from within Firefox. This worked fine.

I am not sure what the problem is with Zotero standalone but if anyone else is in a similar situation where they have both Zotero within Firefox and Zotero standalone installed and they are also having sync problems then try stopping Standalone; starting Firefox and then syncing from within Firefox.

aurimas

Please submit a Debug ID for an attempt to sync from Zotero Standalone. http://www.zotero.org/support/debug_output

adamsmith

Beyond the debug: failure to sync Standalone is very often a proxy issue - are you connecting to the internet through a system proxy?

dstillman

(Doug says he does connect using a proxy, but it's odd that Retrieve Metadata still works.)

DougHunt

@aurimas Firstly, my apologies for the delay, I have been away. I tried following the procedure for submitting a debug item from (http://www.zotero.org/support/debug_output). 388 lines of debug information collected but when I tried to submit the info I got "Invalid response from server" and no debug number.

@adamsmith and @Dan Stillman yes I do connect via a proxy (as Dan has noted) but I do not think that it is as simple as that. As I mentioned, I can use Zotero standalone to download meta data for a paper where I have a pdf and so from that I have assumed that Zotero can successfully navigate the local proxy. Perhaps the issue is slightly more complex and is related to HTTPS? I am assuming that Zotero standalone uses HTTPS when syncing whereas downloading metadata probably uses HTTP. Is there a difference in how Zotero standalone uses HTTPS? Remembering that if I close Zotero standalone and then start Firefox it uses the embedded Zotero and embedded Zotero has no problem syncing. A couple of lines from the Zotero standalone debug file are below if they help at all or I can email the whole file if anyone wants it???

++++
[JavaScript Error: "Error connecting to server. Check your Internet connection." {file: "chrome://zotero/content/xpcom/sync.js" line: 652}]

version => 4.0.16, platform => Win32, oscpu => Windows NT 6.1; WOW64, locale => en-US, appName => Zotero, appVersion => 4.0.16
+++

dstillman

Ah, good point about HTTP vs. HTTPS — that's almost certainly the issue. You can email the full debug output to support@zotero.org with a link to this thread.

DougHunt

Thanks @Dan Stillman - Debug file e-mailed, with a link to this thread.

dstillman

Well, OK, so what kind of proxy setup is this? Are you able to connect to sites from a web browser with your system proxy disabled? And does that differ between HTTP and HTTPS?

When Standalone first starts it checks to see if a proxy is required for an HTTP request to zotero.org.s3.amazonaws.com. If no proxy is required, it doesn't bother prompting for proxy authentication. It doesn't test an HTTPS request, though, so if for some reason your HTTP is direct but your HTTPS is proxied (which seems backward, if anything) and requires authentication, that wouldn't work with Standalone currently.

DougHunt

Hi @Dan. Thanks for the prompt attention. I am probably going to run out of system knowledge very quickly but here goes...

What kind of proxy set-up? Sorry, I don't know, I will enquire in the background but finding this information will take some time.

However, the characteristics of the proxy from my (user) point of view are:
When I use a computer (at my University) connected to the wired network (this is how I run Zotero standalone and Firefox)then when I log into the network my system proxy is automatically set up for me and I do not need to do anything to access any http or https website from any browser installed on that (wired) PC.

When I use a computer connected to our wireless network(Not how I am using Zotero) then once I am signed into the wireless network I also need to sign into the system proxy. I only ever do this once per wireless network session and things then work for ANY browser (i.e. I do NOT need to sign in to the proxy on individual browsers).

I am able to use non-browser products such as Drop Box, Google Drive and similar through the proxy. These products do not need to be set up or separately signed in for the Proxy (provided I am either on the wired network and signed in, or on the wireless network and I have signed in previously to the Proxy).

It seems that the Proxy settings are advertised on the network because Firefox and Chrome (and even IE) will AUTOMATICALLY detect the Proxy settings.

If I sign out of the Proxy manually (I can do this) then all Internet connectivity is lost (both HTTP and HTTPS).

That is about it for an immediate answer. I will see if IT will give me the detailed info that you have requested.

dstillman

Sorry, I didn't mean the brand of proxy server or anything like that — just how it's configured on your system (i.e., what checkboxes/fields in the system proxy settings are being used) and for what kind of requests it's required.

If I sign out of the Proxy manually (I can do this) then all Internet connectivity is lost (both HTTP and HTTPS).

You can do this even on the wired network? How?

DougHunt

As an aside, Squid lives in there somewhere too. I have once (in nine years) had a web browser problem where Squid was the cause. I am assuming that you know what Squid is, just in case, it is a caching "proxy" that takes a local copy of frequently used pages that allow caching and serves those pages locally in order to speed up network connectivity. Not sure if this helps or is a Red Herring.

dstillman

But basically you're saying that, when on the wired network, at no point do you get an authentication dialog for the proxy in any software?

DougHunt

I have a Proxy login / logout screen within my browser and on this screen there is a Logout button. Click that button and all internet access stops. Probably not much help sorry....

Another tid-bit, the System Cache is accessed via a particular "port".

DougHunt

But basically you're saying that, when on the wired network, at no point do you get an authentication dialog for the proxy in any software?

>> Yes, that is correct, but I suspect that there is a script running at login time (that I do not see) that signs me into the Proxy using my network credentials.

dstillman

OK, I'd still like to know what checkboxes and fields are enabled in the system proxy settings.

DougHunt

Unfortunately, I can't answer that question "what checkboxes and fields are enabled in the system proxy settings" because all documentation (and usage) relies on the browser automatically detecting the proxy settings.... I never get to see what checkboxes are set (or unset).

DougHunt

OK, about the best I can do is:

Sometimes on older versions of Ubuntu I have needed to set up the proxy manually and in this case I enter the following into the Ubuntu proxy settings cache.xxx.xx.xx:3128

When I next open a browser session in Ubuntu it goes to that (local) address and I am presented with a login screen. I log in then everything else is able to connect to the Internet.

Not sure that this helps much....

dstillman

You're saying that if you go into the system proxy settings in the system network preferences, nothing is selected?

DougHunt

No, I am saying that when I go into system proxy settings, what is selected is "Automatically detect settings". Nothing else is selected.

DougHunt

If I deselect "Automatically detect settings" then nothing is pre-populated into any other field and there is no documentation to cover what should be entered. I suspect that if I enter cache.xxx.xx.xx into the Proxy address field and 3128 into the port field then I will probably have an equivalent set up but I have no idea what, if any advanced settings to set or unset and there is (purposely) no documentation for this because IT do not want us mucking around with this stuff.

DougHunt

Edited output from ipconfig /all

++++++++++++++++++++++
Windows IP Configuration

Host Name . . . . . . . . . . . . : wt123-zzzzz
Primary Dns Suffix . . . . . . . : xxxxxx.xxx.xx.xx
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : xxxxxx.xxx.xx.xx
xxx.xx.xx

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . : xxx.xx.xx
Description . . . . . . . . . . . : Intel(R) 82579LM Gigabit Network Connection
Physical Address. . . . . . . . . : ZZ-51-YY-4F-B8-XX
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IPv4 Address. . . . . . . . . . . : 999.88.70.167(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Lease Obtained. . . . . . . . . . : Monday, 2 December 2013 11:50:45 a.m.
Lease Expires . . . . . . . . . . : Tuesday, 3 December 2013 2:50:46 a.m.
Default Gateway . . . . . . . . . : 999.88.70.1
DHCP Server . . . . . . . . . . . : 999.88.8.1
DNS Servers . . . . . . . . . . . : 999.88.8.1
999.88.1.12
NetBIOS over Tcpip. . . . . . . . : Enabled

dstillman

No, I am saying that when I go into system proxy settings, what is selected is "Automatically detect settings". Nothing else is selected.

OK, that's what I was asking.

And you're positive that, if you clear that box and restart your browsers, both HTTP and HTTPS cease to work?

I'm still not really clear on the mechanism your institution is using for proxy authentication, since being able to log out of a system-level proxy from a webpage seems pretty strange. My guess is that, to avoid standard proxy authentication pop-ups, the proxy server is allowing unauthenticated access by IP address for clients that authenticate via some scripted web request at startup. And then, when you log out, it's just removing access for that IP address.

I'm actually fairly surprised if HTTP access really stops working with that box cleared, because Zotero doesn't seem to be using a proxy at all — according to the Mozilla network layer, no PAC file (which is what "Automatically detect settings" should ultimately result in) is loaded and no proxy is required for zotero.org.s3.amazonaws.com — yet HTTP requests are still working. I'd be less surprised if HTTP requests still worked but HTTPS requests didn't. (That still wouldn't explain why Standalone wasn't getting the PAC file, but it would explain why (apparently direct) HTTP requests work yet HTTPS requests don't.)

DougHunt

You said: And you're positive that, if you clear that box and restart your browsers, both HTTP and HTTPS cease to work?

I say: No, that is not what I said. I said that when I click the button on my local proxy screen to log out of the proxy, all internet traffic stops.

I just tried what you asked, "close browsers, unclick automatic detection, then re-start browsers". In this case all HTTP sessions reconnected but all HTTPS session ceased to work, including The HTTPS session to Zotero forums and I needed to go back, click auto detection then reload AND sign back into Zotero.org before I could repost this reply.

DougHunt

Our Uni monitors all Internet traffic for charge-back and policy policement. I think that the system proxy operates at the network level and is tied into a particular IP or MAC address when we sign into our local proxy.

DougHunt

In addition, for reasons only known to IT, the proxy login will time out after a certain set time if you are logged in continuously. Max time out is 14 hours and so I rarely encounter an enforced log-out and logging in and out of the network also logs me in and out of the proxy so most people do not encounter this often and it just takes a simple log-in to get back to internet access so no real issue.

Perhaps, though, this indicates that Proxy logged-in sessions use resources and so the enforced log-out clears used sessions where people simply go home without logging out. This may point to why we have what seems to you to be an unusual set up.

dstillman

I just tried what you asked, "close browsers, unclick automatic detection, then re-start browsers". In this case all HTTP sessions reconnected but all HTTPS session ceased to work, including The HTTPS session to Zotero forums and I needed to go back, click auto detection then reload AND sign back into Zotero.org before I could repost this reply.

Your use of "session" here is a little confusing, but I think what you're saying is that you could no longer make HTTPS connections when proxy autodetection was disabled. The proxy (or lack thereof) shouldn't have anything to do with your cookie-based login session on zotero.org, though, so I assume that the only reason you had to sign in again was that you hadn't previously chosen to remember your session when you last logged in to zotero.org, and you had restarted your browser. Does that sound right?

Assuming it does, that at least makes sense with regard to Zotero, since it means that HTTP requests on your network actually work fine without the proxy, but HTTPS requests don't — and since Zotero isn't using the proxy at all, HTTP requests still work.

The question, then, is why Standalone isn't picking up the system proxy settings. If you go to the Advanced pane of the Zotero preferences and click "Open about:config" and type "proxy" in the Filter box, is anything in the list that shows up in bold? (Bold means non-default.) For example, is network.proxy.type bold and set to something other than 5?

dstillman

I said that when I click the button on my local proxy screen to log out of the proxy, all internet traffic stops.

And just to tie this in, this would presumably be because, with proxy autoconfiguration enabled, your system is still trying to use the proxy for all requests and the proxy is refusing them, even though HTTP requests don't actually need to go through the proxy in the first place.

(Now, I don't know why a network would be set up to allow HTTP requests without a proxy but block HTTPS requests, but that's somewhat irrelevant for our current situation.)

dstillman

If those settings are all non-bold, setting network.proxy.type to 4 (auto-detect proxy settings) and restarting Standalone might cause Standalone to detect the proxy on its own and use that. You shouldn't need to do that — Standalone should just pick up the system proxy settings (5) — but in theory that should have the same effect.

DougHunt

Nothing related to the keyword "proxy" in about:config shows up in bold.

DougHunt

After changing "network.proxy.type" to 4. Shutting down Zotero and then restarting it and adding a new paper to my database it successfully synced! Great work and THANKS!

dstillman

OK, great. That's not an ideal fix, but as long as you continue to use a proxy setup with automatic configuration, that should work. If you ever switch to something else, you'll want to switch that back to 5.