Sudden stop of mass download in google scholar

ejsimpso
edited January 24, 2018
Hi was happily chugging along using the mass download function in chrome on a google scholar citations page (click the folder and it gives you the option to download all citations on that page) when it just stopped working. I see the "save to zotero" box in the corner when I click the folder icon, I click 'select all', I click "ok" but then nothing happens and instead of populating, the dialog box disappears. I've made sure that connector is on, the translator is updated (sometimes I was getting a message saying translator was having a problem) I've updated Zotero, it's open, I can save citations singularly (i.e. if I'm on a page that has just one item) I've done the check to the link and "zotero is available"... help!

Report ID: 1253718443
https://scholar.google.com/scholar?start=106&hl=en&as_sdt=400005&sciodt=0,14&cites=3846434904667233521&scipsc=
  • DWL-SDCA
    This has little to do with Zotero and everything to do with Google Scholar's rules for how much it allows a computer to make requests in a given time interval.

    Google Scholar begins to reject requests from users it judges to make heavy demands on the system. GS thinks you are a robot. You will need to wait for some time interval (an hour up to a day) before GS will allow you to again use the system.

    I said "little to do with" above because the Zotero developers are working on ways to allow Zotero and GS to both play by the GS rules and at the same time reduce the frustration of Zotero users. You didn't specifically say what "mass download" means to you so I don't know if the future GS-Zotero improvements will help in your specific case.

    GS is primarily an index and _not_ a literature database. GS allows you to download metadata for the items it lists but it primarily serves to show users links to original material. The metadata downloaded from GS will almost always be less accurate and complete than the metadata you can obtain if you follow the link to the source and get the metadata from there.

  • dstillman
    the Zotero developers are working on ways to allow Zotero and GS to both play by the GS rules and at the same time reduce the frustration of Zotero users
    The upcoming changes will only affect Retrieve Metadata for PDF, so they probably wouldn't make a difference here unless @ejsimpso was also using that feature recently. There's not really anything we can do in the case of mass downloads — that's precisely what Google Scholar's limits are intended to prevent. (The good news is that this sort of throttling is mostly limited to Google Scholar.)
  • ejsimpso
    Thanks! This was really helpful. By "mass" I mean I was trying to take in about 300 citations and got cut off about 250. I'll plan to spread it out in the future.

    Also- not directly zotero-related, but in case you may know... is there another citation access method that functions this way but doesn't "throttle" extensive use? (i.e. if I'm trying to find all the publications that cite a particular document)

    In any case- I really appreciate the help!
  • dstillman
    Some tips here: https://www.zotero.org/support/getting_stuff_into_your_library#large-scale_imports_from_databases
  • DWL-SDCA
    edited January 24, 2018
    There are bibliometric studies of what is required to obtain a comprehensive listing of all items that cite a particular item. If you use only Google Scholar you will miss items that are included in Web of Science (Web of Knowledge) but GS will include items that are not in WoS. If you have access to Scopus and the original items are recent you _may_ find a few cites that the other databases omit. Scopus isn't as well studied because its high subscription cost means that fewer universities have access compared to WoS.
  • adamsmith
    Scopus, should you have access, also has a really nice API and a python library to use with it.
  • bwiernik
    For mass downloads (for systematic reviews, etc), Scoous, PubMed, and Web of Science have the best interfaces, APIs, and also metadata quality.
Sign In or Register to comment.