Zotero connector can't saving with ScienceDirect with institutional access

kgrox

I'm using the 7.1-beta.35+03b33d995 (64-bit) version client and 5.0.165 Connector
the connector can only save using the Embedded Metadata and save some of the metadata from the snapshot of the webpage, with the Author information lost and the abstract incomplete. This only happens when the paper is access via my university's net work, if i change the network environment, the connector can successfully resolve the page and saving the correct information
https://s3.amazonaws.com/zotero.org/images/forums/u17111097/hio93btma5ayxwe5ac3y.png
debug ID: D224498233
report ID: 429911269

adomasven

What happens if you click Cite->Export Citation to RIS? What about if you visit

https://www.sciencedirect.com/sdfe/arp/cite?pii=S0378775325009486&format=application/x-research-info-systems&withabstract=true

kgrox

@adomasven click Cite->Export Citation to RIS or visiting the page jumps to the same blank extension page, the Zotero Connector shows in the right top corner and nothing happened after that, click Done doesn't change either
https://s3.amazonaws.com/zotero.org/images/forums/u17111097/taeop7tjtsjufr3fs5x8.png

dstillman

Disable the Zotero Connector and then try to export the RIS again. ScienceDirect is blocking Zotero's attempt to download metadata, so we need to see the actual webpage you're being served (possibly some sort of CAPTCHA page).

kgrox

@dstillman I disabled the Zotero Connector and Chrome instantly download the RIS file normally after clicking the Export citation to RIS

dstillman

And the above link as well?

kgrox

@distillman Zotero Connector ignore the exported txt file and "Save to Refworks", Export to BibTex is same as Export to RIS, no CAPTCHA page found when the Zotero Connector is disabled

kgrox

@distillman Sorry i just understand, yes, the above link(https://www.sciencedirect.com/sdfe/arp/cite?pii=S0378775325009486&format=application/x-research-info-systems&withabstract=true) jump to an instant download RIS file exactly same as Export to RIS did

adomasven

Are you testing this while connected to your university network?

kgrox

@adomasven yes, i can't View PDF if I'm not in my university network, and as i mentioned above, the Zotero Connector work correctly if I'm not in my university network, just can't view PDF because of the network limit
https://s3.amazonaws.com/zotero.org/images/forums/u17111097/ecj5ceyi6gvjifnvo3ld.png

kgrox

Actually using the .edu mail is also the same, basically as long as I have access to the full issue and PDF, the Zotero Connector can't save with sciencedirect

adomasven

Could you do the following troubleshooting with the network where you have issues:

1. Go to about:extensions
2. Toggle Developer Mode in the top-right
3. Under Zotero Connector -> Inspect Views click on Service Worker
4. Open the Network tab. There should be a Console section at the bottom. If there is not, press Escape. Paste and run
Zotero.HTTP.request('GET', 'https://www.sciencedirect.com/sdfe/arp/cite?pii=S0378775325009486&format=application/x-research-info-systems&withabstract=true')
in the console.
5. A web request line will appear above in the network list. Right-click -> Copy -> Copy all as HAR.

Paste the output into a text file, and send it to support@zotero.org with a link to this thread.

kgrox

@adomasven thanks a lot! The mail has been sent

adomasven

When you use the university network, and the Zotero Connector is enabled, when you export to RIS, does the import into Zotero succeed?

adomasven

@kgrox Does it work for you with Firefox or is it also broken in the same way?

kgrox

@adomasven it fixed after i reset the outside network and use email to authorize the access, although it is still broken sometimes, but it works well mostly
when i use university network, the problem still happens and Zotero client has no response to export to RIS, the extension page just jump up, show the save page, and disappear, did no change to the Zotero client.
i guess i just use the email authorize to fix the problem, and I‘ll test the Firefox later

adomasven

Let us know whether it works with Firefox on your university network and then we'll see if we can troubleshoot this further.

kgrox

@adomasven
Things are a little bit different on Firefox, the first time i clicked the Zotero Connector, it worked well, the CAPTCHA page jumped up and passed the check, the file saved successfully. BUT when i tried another paper, the Zotero Connector can resolve the page as ScienceDirect (which is different to Chrome), but after it saved the snapshot, and tried to download the pdf, a CAPTCHA page jumped up and it almost immediately disappeared before anything showed up, then the pdf saving failed, the same thing happens every time i try, and i tried clicking the "View PDF", it led directly to the pdf page. At the same time, Chrome showed the same thing as i mentioned early.
That's all i can get, if you need any log or report just let me know.

adomasven

Could you provide a Debug ID for a save from Firefox that produces the failing behaviour?

kgrox

@adomasven
i can't reproduce the "CAPTCHA page disappear part", now the Connector fails saving the pdf directly, and the debug ID of current situation is D1024318826

adomasven

You should update to the latest version of the Firefox plugin and produce another Debug ID. It's hard to say whether we can fix this since the debug log you produced is missing some lines that Zotero tries to log regarding the requests Zotero Connector makes, suggesting all Zotero Connector may be limited in some fundamental way under your university network.

kgrox

@adomasven i've uninstalled the old version and installed a new one on the official website, the bug remained and the new Debug ID is D1197481032

adomasven

Could you install the Zotero Connector Firefox Beta and see if the issue still occurs with it (or something else happens, and produce another Debug ID)?

kgrox

@adomasven the beta version reproduce the flash past of the CAPTCHA page and the Debug ID is D1136018737

adomasven

Could you do the following in Firefox:

1. Open the PDF page of the file that you are trying to save, hit F12 to open the developer panel, and select the Network tab, then Refresh the page. You will see an entry, most likely the first, for the PDF (should say "pdf" in the Type tab).
2. Right-click on the entry, Copy Value -> Copy Request Headers and paste it in a new text file.
3. Right-click on again, Copy Value -> Copy Response Headers, and paste it in that file under the previous text
4. Save the file as request_browser.txt

Then:
1. Open type about:debugging in firefox url bar. Select This Firefox on the left side. Find Zotero Connector in the list and press Inspect. A developer tools box will appear. Select the Network tab in it, and put it aside for now.
2. In a new tab navigate to the article page (https://www.sciencedirect.com/science/article/pii/S0010854525002747).
3. Press the Zotero Button to save the article.
4. Go back to the Zotero Connector developer tools box, and in the "Filter URLs" box type "pdf". Multiple entries will likely appear. Find the last one in the list that has 403 for Status (first column) and "pdf" for Type.
5. Once again copy Request Headers and Response Headers and paste them in a new text file.
6. Save the file as request_zotero.txt

Send these two files to support@zotero.org with a link to this thread.

kgrox

@adomasven
my firefox client auto updated before i tried, and during the test, Zotero Connector worked and save the pdf successfully, thus the bug seems to be saved, do you still need the headers with 200 status or should i try the same thing back on Chrome?

adomasven

Yes, try with Chrome if it's working with Firefox. Provide a Debug ID from Chrome if it doesn't work.