Locked out of Google Scholar

kalital
Hoping I can get some help/advice with this. I was downloading citations from a Google Scholar search, using the button in the browser address bar, to grab a whole page at a time. It was a big search and somewhere around page 35 Google gave me an "I'm sorry" page, thinking I was automatically downloading. There *should* have been a CAPTCHA box on the "I'm sorry" page, but there wasn't. I did all the stuff Google suggested, like scanning for malware, but this seems to be a Zotero/google/firefox problem. Once I was locked out, I couldn't access Google scholar at all, even after uninstalling all plugins and restarting. This is Very Bad.

I could access the Google Scholar "I'm sorry" page in Safari, and enter the CAPTCHA code, after which time I could do searches on Safari, but still could not get past the "I'm sorry" screen in Firefox.

"Fine," I thought. I'll just browse anonymously on Firefox, because there's an obvious bug here. So I did and I could reach Google Scholar just fine--no "I'm sorry" screen. BUT... the citation download doesn't work with the anonymizer. I get an error box pointing me to a "known translation issue." In fact, Zotero doesn't work with *any* anonymizer (which is a drag because for some sorts of research, one might rather stay anonymous, but that's another question).

So... now I am locked out of Google Scholar in Firefox and obviously can't use Zotero in Safari. Why doesn't Zotero warn us that we could get axed from Google Scholar if we use the tool *as it is supposed to be used*? How long does it take to get Google to let me back in? What's wrong with Firefox, that it won't show the CAPTCHA screen that could solve the problem? Can you set it up so Zotero works with anonymous.org?

At this point, Zotero is causing me far more problems than it's solving. What to do?
  • dstillman
    there's an obvious bug here.
    No, you were just making hundreds of essentially automated requests on Google Scholar, and they have every right to block you.
    Once I was locked out, I couldn't access Google scholar at all, even after uninstalling all plugins and restarting. This is Very Bad.
    It's likely just cookies from Google. Clear them or wait. (They could also block based on IP address and browser user agent string.)
    Why doesn't Zotero warn us that we could get axed from Google Scholar if we use the tool *as it is supposed to be used*?
    Nowhere does the documentation suggest using Zotero to save all results from dozens of pages of search results one after another. People do so, but many sites will block you—and any properly configured site probably should, as a protection against denial-of-service attacks.
  • kalital
    You know what? That wasn't very helpful. There *is* a bug -- at least in Firefox, because it shows the CAPTCHA on systems older than 10.5, but not on the latest Firefox version and operating systems above 5.5 on the Mac. I'm not the only person to have had this problem, and the interaction between Google Scholar, Zotero and Firefox has been noted other places, all by scholars with some panic in their complaints.

    The Zotero documentation states: "If you are looking at a group of items (e.g., a list of search results from Google Scholar or LexisNexis), a folder will appear. Clicking on the folder will produce a list of items with check boxes next to them; choose the ones you want to save and Zotero will do the rest." It would be USEFUL to indicate that there are problems with lockouts on the specific systems from which they explicitly say you can download. And it would be even *more* useful to get a warning window that says something like: "You have download 200 citations from Google Scholar. If you continue to download you may risk getting locked out." That would be great for everyone who doesn't understand that Zotero citation farming qualifies as "automatic downloading" and Google Scholar blocks people for that.

    The point, I'd think, is to make Zotero more user friendly and useful. And since it could be a terrible thing to get locked out of a major reference resource just at the moment one needs it the most, I think it's reasonable to want a warning or an explanation of the possibility.

    My current solution is to use Zotero on my older laptop, where CAPTCHA works. And I've written to Firefox to point out that Google's CAPTCHA isn't displaying properly. But the Zotero support docs should at least mention this as a possible problem.
  • dstillman
    Well, I'm sorry if you didn't find my response helpful, but I'm simply explaining the issue—and I suggested a way to temporarily restore access in Firefox.

    It's completely up to Google when to block you. There's also no particular reason to think that the CAPTCHA's not displaying is a bug—it might be, but it's more likely that they just don't display the CAPTCHA at all if you go past a certain limit, because CAPTCHAs can be circumvented either automatically or manually.

    We can add a line to the documentation, but that's about it. Sites can block based on all sorts of factors, and we have no way of knowing when a particular site will do so.
  • anders_royce
    You won't see this with other sites either. Google is just very cautious about this sort of thing. Scholar is the only site I have ever been temporarily blocked from while using Zotero, and I have had the exact same issues syncing Gcal and, separately, with syncing Contacts to too many locations. They offer free services, and at some point they will just cap your access. I should mention that when I was blocked from scholar, I got a friendly message telling me that I was blocked (and if I remember correctly, I'm afraid it was specifically that my IP was blocked), and not just a blank page. Also, do you by any chance the Zotero Scholar Citations addon installed? I was blocked another time when that sent a big batch of requests to Scholar.
  • kalital
    @Dan -- adding a line to the documentation would be very helpful. When it happened to me, I did a google search and found that I was far from the only one, and no one knew what to do. Warning us to be cautious downloading could help newbies avoid disaster. If I wasn't already familiar with how software works (I design databases and web sites, and my husband is a software developer) I would have reacted with true panic and abandoned the product. I can duplicate the mass downloading on another computer in the house and see if the CAPTCHA disappears when I get bumped.

    @anders_royce -- I didn't know there was a Scholar Citation addon, but I'll look for it. My IP was blocked too, so deleting cookies didn't help. I didn't get a blank page, but the "I'm sorry..." page with no CAPTCHA box. I'm allowed back on as of this morning. (Morning, that is, in Swiss time.) I will certainly be more cautious next time I download cites.
  • radu124
    Could zotero at least stop pushing more requests when it receives the "Sorry" page from google and warn the user?

    I had the problem with captchas not appearing in Firefox. In my case, after I deleted the cookies (not only from scholar, but also from google.com), I got the captcha and after entering it it was ok.

    As a rule, I'm not trying to push it too much, i'm only taking a couple of references at a time, I only had problems when I tried to import my existing library of PDFs and retrieve the metadata for all of them.

    I am careful now if I do that to always stop it when it fails retrieving the metadata and visit the scholar website.
  • borchers@nmt.edu
    I was locked out today (after retrieving metadata from about 12 .pdf files) and not offered the captcha to prove that I'm a human.

    A message on Google's help forum gave me the key- I started up another browser (Google Chrome) and accessed scholar.google.com. At that point I got the captcha and was able to prove I'm a human.

    My guess is that either there's a cookie being set or that Google is recognizing the web browser and refusing to offer the captcha. In any case, this seemed to get around the problem.
  • Oddible
    This is a Zotero bug. When you disable Zotero in Firefox the problem goes away. This happens to me periodically, not sure what causes it or how to make it stop. From a fresh boot, went to my university proxy, then to Google Scholar and ran a search and got the automated searches CAPTCHA page (which by the way you cannot get past). Whenever I try to access http://scholar.google.com it always redirects to my university's proxy to access Google Scholar - wasn't sure what was causing this.

    I cleared my cache, set to No Proxies, nothing worked. Read in another post elsewhere on the internet that it is Zotero sending a blast of requests when you access Google Scholar so I disabled Zotero. Problem solved - I can access scholar.google.com as well as through the unversity proxy to Google Scholar. Except now I don't have Zotero. Going to try removing my university proxy from Zotero and see if that helps. Not sure what Zotero is doing that would cause this.
  • Simon
    Oddible, Zotero only sends requests to Google Scholar if you retrieve PDF metadata. The likely problem is that your university's proxy is getting locked out of Google Scholar. If you disable transparent proxy redirection in the Proxies section of the Zotero preferences, or if you take scholar.google.com out of the list for your proxy and turn off auto-associate, that should fix things.
  • Oddible
    Yeah that seems to be the case, thanks for the clarification - it seemed far fetched for me to think that this could be a university-wide problem but sure enough.
  • Simon
    How many people are there at your university? If Google Scholar detects a number of requests in excess of what a single person would make coming from a single computer, it locks it out. (This is the reason for the issues with the Retrieve PDF Metadata functionality described above.) Since your proxy looks to Google Scholar like a single computer, and yet is probably sending out requests for dozens of users, I'm not entirely surprised that Google Scholar doesn't like it.

    We could blacklist scholar.google.com in the proxy code, but this is somewhat undesirable, since you'll hit the paywall if you click a link to any journal/site that Zotero hasn't auto-recognized. You could also ask those in power at your university to take scholar.google.com off the list of sites the proxy tries to proxy (which would prevent Zotero from auto-recognizing scholar.google.com as a proxiable site), but this is undesirable for the same reason. The best solution would be to ask Google to exempt your university proxy from its blacklist, although I'm not sure how one would go about this.
  • few
    I was locked out today, and so were the other Firefox users in my lab (we share an IP address through a router). I think that since zotero missed the captcha page, and kept trying to automatically identify PDF's, it triggered a temporary lockout (and they're timed). Google must be using the browser identifier (version and perhaps plugins) along with the ip address to set the lock. As soon as I used chrome I was able to access scholar again. It it probably worth having a warning pop up, or at least pass on the captcha to the user when zotero pulls it up during a search.
  • jhuebert
    There is definitely a problem here. I was about to blame my university when I stumbled upon this thread. I agree with another user, the problem seems to be related to something Zotero tries to do behind the scenes when Firefox starts up. I have been blocked from Scholar some days before I have even tried to connect to the site.

    It is very likely many uni users are going to connect to Google Scholar using a proxy, and want to use Zotero at the same time. In fact, this is part of my every day (many times a day) routine: perform a search, find an interesting article, hit the "add" folder icon, and select the article I want to add to my Zotero library (usually it is only one!).

    Can this be addressed soon? I love Zotero, and use it every day.
  • dstillman
    I agree with another user, the problem seems to be related to something Zotero tries to do behind the scenes when Firefox starts up.
    Zotero does not access Google Scholar by itself. It does so if you use Retrieve Metadata for PDF or if you save items from Google Scholar using the address bar icon.

    There's a Zotero Scholar Citations plugin—which we recommend avoiding for a number of reasons—that might access Google Scholar at other times, but that has nothing to do with Zotero proper.

    There's nothing in Zotero to fix for what you've described.
  • jhuebert
    Ok, fair enough, and thanks for the explanation. So, why is there all this this chatter with Scholar when I use the address bar icon to add something to my Zotero library? Is the problem the use of a proxy - or is that not related at all?

    Perhaps if I understand what's happening, I can work differently in the future. My original problem remains, and I am not trying to blame Zotero. I just need to know how to get my work done.
  • dstillman
    Zotero makes a request or two to Google to retrieve the metadata—it's basically just retrieving the same metadata file that you can download by clicking on it yourself. That won't cause you to be locked out.

    Most of the people who get locked out save pages and pages of items from Google Scholar at once.

    I can't tell why you're getting locked out, assuming you don't have ZSC installed. Maybe someone else who shares your IP address does or is saving pages and pages of references. But there's nothing we can do about that.
  • pedrobrasil
    Guys... post on this thread to ask for google to help zotero users with this problem...

    http://www.google.com/support/forum/p/Web+Search/thread?tid=0fee61a650860220&hl=en&fid=0fee61a6508602200004a9c2359e67cf

    kind regards.
  • strongbrew
    The problem is easy to resolve. Do not select Actions->Update Al Citations. It locks me out but gives me a warning that all Google Citations Could not be made do to a large request (or something like that, I'm not going to try it again.) Interesting enough. I can go to Windows IE, and get dialog that asks me to type a phrase. IE will let me in while Firefox Will Not.
  • Simon
    There's no "Update All Citations" option in Zotero, so I think this must be related to the Zotero Scholar Citations add-on, which Dan discusses above.
  • shucwu
    I was locked out of Google Scholar today, after trying to retrieve metadata for over 100 pdf files at once. While being locked out in Firefox, I could access scholar using IE after answering CAPTCHA. After that, still couldn't access scholar using Firefox until I cleared cache, cookies, etc and was able to get CAPTCHA in Firefox.

    Certainly after this experience I will not do such a big batch all at once. On the other hand, it would be easy for Zotero to act more like a "human" to Google then all this will be less likely to happen.
Sign In or Register to comment.