Or were you just giving me access to a forum that explains that this issue is being worked on and is nearing a solution?

this. You can't really install this yet, not even as a beta. But that could happen very soon.

we don't really know the answer to that - the patch doesn't circumvent google's lock-out — it makes it less likely to occur, and when it does occur it helps you to authenticate via captcha, and doesn't fail on items it can still import (via DOI e.g.).

thanks aurimas for your idea: I was blocked with retrieving Metadata, then I opened scholar.google.com, they checked if I was a robot as you guessed, once they finished checking I could retrieve Metadata again. Thank you!

The next version of Zotero introduces considerable improvements to this issue. It essentially pops up a CAPTCHA window inside Zotero and AFAICT prevents "permanent" Google Scholar lockouts. A beta version (for Firefox only atm) is available here: http://www.zotero.org/support/dev_builds#zotero_40_beta (not sure when this will be officially released)

tfjern: when I opened scholar, they asked to recognise letters and numbers (two times) and then it was unlocked. But I guess it's resolved with the new version anyway.

I'm really happy to see Zotero is working on this. I have standalone 4.0.17, but I don't see the behavior aurimas describes above. When I hit my query limit I still have to go scholar.google.com to enter the captcha. Am I doing something wrong? Also, once I've entered the captcha I case search in Scholar, but I still get the query limit reached warning in Zotero.

I also use the Firefox version (just installed today). In that version, when query limit is reached I just get a message from Scholar that just says the think my computer is sending automated responses and they can't process my requests. No captcha...the search page just doesn't show at all.

This is all in Windows 7 (most recent version of FF).

Any settings I should change or things I should try? Thanks.

Per others on the thread, FWIW, I used CC Cleaner to clear cookies, then logged on to Google Scholar through lib.umich.edu, and used Zotero for Firefox rather than Standalone after a day and a half of query limit after successfully retrieving metadata for about 150 pdfs late one night (the highest number I've ever achieved.

This seemed to work and I've retrieved about 100.

It occurred to me that since my pdfs were in alphabetical order, I kept trying with the same set, so I changed the sort order and did a search for the topics for which I most need articles, which means that it is calling different titles out of alphabetical order, and I don't try again with the same set after a query limit notice.

I have no idea which of these changes is making things work smoother, but wonder if (a) nature of types of documents (author name, discipline, etc., is something included in Google's bot detector; and/or (b) those who log in with a university account and use the Firefox version have better luck. If so, I wonder if there would be a way to integrate university accounts with Zotero.


on a more general note, I've read this thread and am thrilled about the new release. I switched from EndNote to Mendeley (which crashed, had terrible accuracy, was too expensive, etc ect) to Zotero, used EndNote for mass editing of citations for some old citations and imported pdfs that were my own field notes, etc., and have now permanently settled on Zotero. Some of the complaints people have are easily solved with add-ins like Zotfile, etc.

I just want to note that I second the point about frequently needing multiple import for huge numbers of citations (I have 4000 that still need to retrieve metadata. This is due to working with large research groups and coauthors who have massive DropBox folders of pdfs and handle everything manually. One can't click on every pdf or look at their bibliographies and search for each item. I am also working on systematic reviews and meta-analysis and using individual databases and importing/searching on work collected over many years prior to this kind of functionality is not a great solution, and dedicated systematic review software is very expensive with a steep learning curve and quite unnecessary if one uses Zotero effectively. I love so many features about Zotero, this is the only one that slows me down. (well, that and having to drag into subcollections rather than being able to right click and move to subcollections).

The beta version is working much better--I still get a query limit after entering the captcha 4 times, but I can retrieve meta-data for many more citations -- maybe 250 at a time instead of 30-50. It also seems to go faster through items that do not have OCR text, and continues on with other api calls, apparently, even if G scholars reaches its limit. Thank you!

I mean 4 times in a row WITH METADATA being retrieved in-between (50-100 items). I never made a mistake with the Captcha pop-up. I finished the retrievals over 2 days (instead of many-- THANK YOU!!). I didn't see anything strange when I visited Google Scholar. In fact, I logged in via the University of Michigan with no problem. Last night it ran for a very long time with only Captcha messages coming up and no query limit--it was about 10pm Mountain time (midnight ET), so perhaps it has something to do with time of day. In any case, it's a vast improvement and I am finally close to organizing my library completely after Mendeley and EndNote crashes back in 2009. Grateful to Zotero!

same questino as above

If you did mean 4 separate times, then could you visit Google Scholar after getting the "Query Limit Reached" message and tell us if Google Scholar displays some odd page saying that you're a robot?

I honestly cannot reproduce this. I just ran 595 PDFs through Google Scholar for metadata retrieval and everything worked as expected without any "Google Scholar Query Limit Reached" messages. I got over 10 CAPTCHAs on the way and it took me about 30 minutes (it's probably a very generous estimate of how long this should normally take, because DOI and ISBN were not considered at all for this).

For everyone who reported above and any future reporters, if you're using Zotero Standalone, I would highly recommend helping us debug this using Zotero in Firefox.

Once you're using Firefox and you run into the "Google Scholar Query Limit Reached" message, please go to http://scholar.google.com/ (in FIrefox) and if you see anything but the usual Google Scholar page, take a screenshot, post it on some image hosting website, like imgur.com and link it here. If you don't see anything unusual, retry retrieving metadata. If you continue to get the "Google Scholar Query limit reached" message, please submit a Debug ID (post it here). Then try restarting your browser. If that doesn't help, try clearing your cookies.

Let us know what (if anything) fixes this.

This is not a 'Google Mystery'. It is an historical (dating back in 2010) problem in zotoero, which is not able to circumvent the daily limit in fetching data from GScholar. Passed more than 24 hrs, I have cleared all the cookies from firefox and still I have that silly cpatcha coming out all the time even if you fill correctly.
Anyway I will post the debug id as request.

as Dan told you in the other thread, the Zotero code has changed significantly early this year and the google issue is mostly solved, so no, this is not a historical problem.
If you read the posts right above you, this has been tested with multiple hundred PDFs in a day. The mystery that janew refers to is the fact that on one day she got stuck after 100pdfs, but a couple of days later she was able to retrieve data for multiple hundreds in exactly the way that this should work and does in our tests.

When you say the captcha appears all the time - what exactly do you mean by that? For massive retrieve attempts, you'd likely have to fill out a couple of captchas, but there'd be progress in between doing so. Or are you saying it immediately pops back up? That would suggest that either you're not filling it out correctly or somehow it gets lost in translation.

This is the screenshot of google scholar (at 17.20) http://i57.tinypic.com/27y516t.png.
Other screen shots taken in order to let you know that problem is still present even if you still want to deny.

Trying to fetch a pdf article of NEJM

http://i57.tinypic.com/zt7rjt.png (1st try)
http://i58.tinypic.com/108767q.png (2nd try)
.......
And after 2 tries more we have the final result.
http://i61.tinypic.com/35a3ww7.png

RecognizePDF: No CAPTCHA entered