Zotero connector 5.0.111 failed to save snapshot of any webpage in incognito mode of Chrome.

testplayer
edited July 10, 2023
After updating Zotero connector to the latest version this morning, it simply failed to save a snapshot of any webpage. I've tried almost all in https://www.zotero.org/support/troubleshooting_translator_issues.


================
Your debug output has been submitted.

The Debug ID is D1394924421.
================

And where can I download the previous version of Zotero connector? I couldn't find it anywhere, and Google chrome extension store doesn't provide previous versions for download.
  • adomasven
    Could you disable other extensions and see if you still see this issue? You might need to refresh the page, or restart your browser between disabling extensions and trying again. If it works with other extensions disabled, could you figure out which one is causing it?
  • testplayer
    edited July 10, 2023
    I have too many other extensions installed and I can't disable them for so many reasons. I have tested on multiple different webpages and also refreshed them, restarted my browser and also restarted Zotero.

    Update: if I go to an arXiv.org page, and save it using the arXiv.org translator, it can save both the PDF and the webpage snapshot. However, if I save the same page using "Web Page with Snapshot" option, the snapshot won't be saved at all. So there seems to be something wrong with that translator or so.

    Note: the previous version (before upgrading this morning) of Zotero connector worked properly when saving snapshot of a webpage. So there must be something wrong with the latest version.
  • adomasven
    You don't have to permanently disable all extensions, we just need to figure out if interference with another extension is causing this issue, and if it is - which one, so that we can debug this. We have deployed this update to over 3 million of Zotero users and yours is the only report of this issue we received so far, which likely means that something specific to your configuration is causing it.

    Saving pages via multiple-item dialog and in some other instances trigger a different method of saving, which is why it works. There's a technical reason for it that I won't get into here. We still need to figure out why the basic use case is broken for you.
  • testplayer
    I will need much more time to test disabling the extension, because I'm currently using the web browser and can't change anything.

    Meanwhile, I have been browsing all webpages in incognito mode exclusively, with cookies enabled but all 3rd-party cookies disabled. Could this be a reason?
  • testplayer
    edited July 10, 2023
    OK, problem found and it has nothing to do with extension: saving a webpage in incognito mode won't work, whereas in regular mode it works. BUT, I simply CAN NOT use the regular mode and I MUST use the incognito mode when browsing ANY webpage. This is a bottom line and I can't make any compromise in production or non-testing environment. Could you please fix it?
  • ZoeCMa
    Most of the translators don't use cookie information, and they don't care about whether you're in incognito mode (as long as Connector is permitted to run). I've been using Connector in it with no problems. Could there be some other cause?
  • testplayer
    As I have tested, I didn't make any change to any extension, and simply switching to non-incognito mode will work and the snapshot can be saved properly. It just fails to work when I try to save it in incognito mode.
  • testplayer
    edited July 10, 2023
    Can anyone provide a link to download the previous version of Zotero connector for Chrome (.crx file)?
  • testplayer
    Can anyone let me know whether it's safe to download and install Zotero connector v5.0.110 from https://www.crx4chrome.com/crx/961/ ?
  • dstillman
    edited July 10, 2023
    Update: if I go to an arXiv.org page, and save it using the arXiv.org translator, it can save both the PDF and the webpage snapshot. However, if I save the same page using "Web Page with Snapshot" option, the snapshot won't be saved at all.
    I can reproduce this in incognito mode in 5.0.111. We'll fix — thanks.
  • dstillman
    Note that this appears to be specific to Chrome. The above works fine in Firefox private windows in 5.0.111.
  • testplayer
    edited July 11, 2023
    Thanks for your reply! You're the hero @dstillman ! However, I urgently need to save snapshots of many webpages now in Chrome's incognito mode, and I can't wait for this issue to be fixed. So could you please provide a link to download the .crx file of Zotero connector v5.0.110? Or alternatively, is it safe to download and install the .crx file from https://www.crx4chrome.com/crx/961/ (on this webpage, there is a link to download the .crx from "Google CDN", which is https://clients2.googleusercontent.com/crx/blobs/Acy1k0YFQK_CIfcF-9_P1IdIQ-pmcHOy_xeJr8UTETkwY0SMHoob-1-RFEuK7U4Z3fZZfJ6btnngzFN3QKUPhG-YM18Nf1gKzpxlJzrPneB6QhcyGkUFAMZSmuV_aRj4Wpmp5-SgQI6sWT9IcvPGqQ/extension_5_0_110_0.crx , but I'm not sure whether it's safe or trustworthy/really from Google web store)?
  • dstillman
    You can wait a day or two, use Firefox, or build it yourself for Chrome from the source code. Chrome extensions are distributed from the Chrome Web Store. We obviously can’t vouch for a download from a third-party site.
  • testplayer
    edited July 11, 2023
    I can't wait for a day or two. My Firefox profile is in a tedious migration to LibreWolf and it's inconvenient and impractical to use it for other purposes than profile migration. If I build the connector from source and load it unpacked in Chrome, will it be added separately alongside the existing v5.0.111, or will it overwrite the existing one? If overwriting, will all extension preferences/options be retained (and downgraded to the unpacked v5.0.110) or removed?

    Because a bug of a new version of Zotero Connector might be discovered only after it's released, it will be very important for a user to be able to download a last working version of the .crx file from zotero.org, especially for time-sensitive/critical situation like mine. Since a user can download the .xpi file of a previous version for Firefox easily from https://www.zotero.org/download/connector/dl?browser=firefox&version=5.0.110, is there any technical restriction that prohibits the hosting of the .crx file on zotero.org? I mean, there is no need to host all previous versions of the .crx files. It will suffice to host 1 or 2 previous versions, in case that the latest/updated version has a bug like now, and the user needs to revert back to the last working version without the needs to switch browser or build from source. I hope this will make sense.
  • dstillman
    OK, so it turns out this was already broken in Incognito mode due to limitations in Chrome — snapshots saved in Incognito mode in 5.0.110 and earlier only included the HTML page with no assets (images, CSS, etc.). That was broken behavior that should never have been enabled to begin with, so we're going to have to disable snapshot saving in Incognito windows in Chrome and Edge until they address the underlying issue (a limit on message size for the extension messaging API).

    If you want snapshot saving in a private window, you'll have to use Firefox, where this worked properly before and continues to work.
  • testplayer
    edited July 11, 2023
    That's really a terrible news. But at least for now, how can I simply revert back to v5.0.110? It will take me a month or more to finish the Firefox profile related settlement & migration, so using Firefox is not an option for me now. So could you please simply provide a link to download the .crx file for v5.0.110?
  • dstillman
    edited July 12, 2023
    OK, we were able to re-enable snapshots in Incognito mode via the Zotero app. Unlike <=5.0.110, these snapshots will correctly include images and other assets. Unlike in Firefox private windows, the snapshots will be taken within the Zotero app, so they'll take a bit longer and won't necessarily reflect what you see in the browser (e.g., changes made via JavaScript), but they won't be missing all assets like they were before.

    We've submitted 5.0.112 with this fix to the Chrome Web Store and Edge Add-ons site. The latter has already been approved, and the former will hopefully be available later today.

    When running 5.0.112 with Zotero 6.0.26, the save popup will show a red X for the snapshot, but the snapshot will be saved nevertheless. With the next Zotero 7 beta, the save popup will display proper feedback for the snapshot.

    Or you can just use the Zotero Connector in a new Firefox profile now and get correct behavior in all cases.
    So could you please simply provide a link to download the .crx file for v5.0.110?
    Again, the Zotero Connector, like all public Chrome extensions, is distributed via the Chrome Web Store. A .crx can only be installed in developer mode and isn't equivalent to an XPI. It's not something we distribute, so you can stop asking. If you want to use a browser that allows file-based installs of public extensions, use Firefox.
  • testplayer
    edited July 12, 2023
    Thanks for the updates. In most cases, the "Embedded Metadata" translator is available, and with updated connector v5.0.112, it won't show the red X when saving the snapshot. However, as you mentioned, the snapshot is saved within the Zotero app but not directly from the incognito mode of browser, so for example, when saving the this current page https://forums.zotero.org/discussion/106122/, my user name won't be displayed in the saved snapshot because there is no login/cookie stored in Zotero app.

    My temporary solution yesterday (before the updated v5.0.112) was to use SingleFile (https://github.com/gildas-lormeau/SingleFile) to download a snapshot when browsing a webpage in incognito mode of Chrome, drag it into Zotero to the item (for which saving snapshot using Zotero connector in incognito mode failed), and then delete the snapshot downloaded by SingleFile. It looks like SingleFile can faithfully scrap the webpage as exactly shown in the incognito window, and when downloading, it will trigger a regular download from blob:chrome-extension://mpiodijhokgodhhofbcjdecpffjipkle/, for which I can see the download progress and history in chrome://downloads/. Since Zotero Connector is partly based on SingleFile (or SingleFile-Lite https://github.com/gildas-lormeau/SingleFile-Lite ?), could this saving mechanism of SingleFile be integrated/implemented in Zotero Connector?

    Many thanks again.
  • adomasven
    Good investigation on your part. Before the fix we looked to see if we could make it work like SingleFile. SingleFile works because all of the SingleFile extension executes directly in the page. For the Zotero Connector we need to pass the saved snapshot to the Zotero client, which requires first sending the snapshot to the background page, which is where Chrome places limitations on us for passing blobs between an Incognito page and an extension background page.
  • testplayer
    There is a critical problem/issue with saving snapshots of webpages within the Zotero app: because the webpage is accessed and downloaded by Zotero app, any webpage that requires login credential to view won't be saved properly (it will usually save the redirected login page), and same for any webpage that has an expiration date, for example, if a webpage will expire at 12:00 noon (showing HTTP 404 or similar), and if I open that webpage at 11:59 but click the button to save to Zotero at 12:00 or later, the saved snapshot will be showing HTTP 404 or similar.

    Therefore, saving webpage's snapshots within the Zotero app is NOT a proper solution to this issue. So could you please integrate or implement the same or similar snapshot saving mechanism of SingleFile (which works properly in incognito mode of Chromium based browsers) in Zotero Connector?
  • dstillman
    Adomas already explained that the Zotero Connector can't do that.

    We're done talking about this. This is a limitation of Chrome. Use Firefox if this is important to you. If you can't use Firefox for some reason, use SingleFile and attach the snapshot manually.
Sign In or Register to comment.