newspapers in Proquest

aaronbcowan

I've encountered a repeated error when trying to save newspaper articles from Proquest databases, in which Zotero seems to recognize the item as a newspaper article, but as soon I click the icon to add the item to Zotero, I get "An error occurred saving with ProQuest. Attempting to save using Embedded Metadata instead" - after which it just saves the item as a web page. Here's an example of a URL where I have this issue: https://www.proquest.com/hnppittsburghpostgazette/docview/1856896404/2C5F6A9322A64054PQ/11?accountid=13901

I should note I'm accessing the ProQuest database through my university's library, not a personal account.

Any suggestions?

adamsmith

I don't have access to the Pittsburgh Gazette -- are you seeing the same issue on Boston Globe, Washington Post, LA Times, NY Times, Chicago Trib, WSJ?

aaronbcowan

Thanks - unfortunately we only subscribe to a limited number of papers in PDF format (Pittsburgh Post-Gazette and Philadelphia Inquirer) - the problem happens with both of those papers.

However, I did find that if the newspaper articles are in the database as HTML (rather than a scanned image), it seems to recognize the newspaper format and pull the metadata with no problem. For example, this article worked fine: https://www.proquest.com/docview/371640635/abstract/35E764165C774A67PQ/1

aaronbcowan

Update: I was able to access Proquest Historical Newspapers through a public library, and the metadata pulled without any problems. So it seems to be specific to my access through my institutional library.

adamsmith

Could we get a debug ID from the Zotero connector for importing from the above Pittsburgh Gazette article?
https://www.zotero.org/support/debug_output#zotero_connectors_firefox_chrome_and_safari

Also, if you could take a screenshot of the entire window for that article (with the "Abstract/Details" tab selected on the left if that's an option) and post it to any free image hosting site (imgur.com, dropbox, google drive etc.), that'd be helpful

aaronbcowan

Debug ID is D600866363

As for the screenshot, here it is with the "Details" panel open
https://i.imgur.com/6PuOlYt.png

Thanks!

adamsmith

@dstillman anything in the debug? Does it just not get a title?

@aaronbcowan -- are there articles that are listed with their proper headline (instead of just date & page number)? If so, can you try on one of those? E.g., when I'm looking at the Boston Globe here, the first field under Details is "Title".
I'm suspecting the problem is the absence of a title, which would break import.

aaronbcowan

@adamsmith Good point - the ones that seem to work correctly have a title listed (but note that's through a different library access) https://imgur.com/w3JIV51

It seems like the problem may be that the ProQuest Pittsburgh Post-Gazette database doesn't allow you to view specific articles, only the full page on which they're located. So, for example when looking at an article from the Pittsburgh Courier, the title is listed in the heading and I can select to view only the article in PDF, or the entire page. https://imgur.com/SPjn6Kg - but with the Post-Gazette, no title is listed in the database record, and I can only view the entire page. https://imgur.com/yjLtvDX

So this really may be an issue on ProQuest's side, and not a Zotero problem at all. Puzzling that they would have article-specific metadata for some newspapers and not for others though...

qenghis

I'm getting the same error with Proquest Historical Newspapers. Examples:

New York Times:
https://www.proquest.com/docview/99391303/70936FB0204F4380PQ/3

New York Herald-Tribune
https://www.proquest.com/docview/1114067625/2F0CDDF1191B4820PQ/3

The bug first appeared with version 6.0.16. When I roll back to 6.0.15, it works fine.

adamsmith

@qenghis -- works for me (in the latest beta at least, but that should be equivalent to 6.0.18, the latest stable version, in that respect).
Which browser? Could you provide a debug ID from the Zotero connector for trying to save the NYTsarticle you link to above?

(I'm still planning on fixing the Pittsburgh Gazette error reported above, but that isn't an issue for NYT or NYHT)

qenghis

I'm unable to I'm unable to submit output. When I click the button, I get an error modal:

---------
An error occurred submitting your report.

undefined is not an object (evaluating 'Zotero.Date.dateToSQL')
---------

Here is the debug log when I attempt to save the NYT article:

---------
(3)(+0003639): HTTP POST http://127.0.0.1:23119/connector/ping: {}

(3)(+0000005): Connector: Method ping succeeded

(3)(+0003117): HTTP POST http://127.0.0.1:23119/connector/ping: {}

(3)(+0000008): Connector: Method ping succeeded

(4)(+0000003): Translate: Binding sandbox to https://www.proquest.com/docview/99391303/70936FB0204F4380PQ/3

(4)(+0000006): Translate: Parsing code for ProQuest (fce388a6-a847-4777-87fb-6595e710b7e7, 2022-08-19 18:27:20)

(3)(+0000000): Translate: Beginning translation with ProQuest

(3)(+0000001): Translate: Going to the Abstract tab

(3)(+0000000): Translate: resolving URL https://www.proquest.com/docview/99391303/abstract/E3C587D0B3F7420APQ/1?accountid=35635

(3)(+0000001): Translate: resolved to https://www.proquest.com/docview/99391303/abstract/E3C587D0B3F7420APQ/1?accountid=35635

(3)(+0000000): HTTP GET https://www.proquest.com/docview/99391303/abstract/E3C587D0B3F7420APQ/1?accountid=35635

(3)(+0000018): HTTP POST http://127.0.0.1:23119/connector/getSelectedCollection: {}

(3)(+0000044): Connector: Method getSelectedCollection succeeded

(3)(+0001158): HTTP POST http://127.0.0.1:23119/connector/ping: {}

(3)(+0000000): HTTP POST http://127.0.0.1:23119/connector/ping: {}

(3)(+0000006): Connector: Method ping succeeded

(3)(+0000005): Connector: Method ping succeeded

(3)(+0001037): Parsing cross-origin response for https://www.proquest.com/docview/99391303/abstract/E3C587D0B3F7420APQ/1?accountid=35635

(3)(+0000001): Translate: Not following link again. Attempting to scrape

(3)(+0000003): Translate: WARNING: No item type specified

(2)(+0000000): Translate: Translation using ProQuest failed:
Error: No title specified for item

_itemDone@webkit-masked-url://hidden/:609:41
@webkit-masked-url://hidden/:89:22
@
scrape@
doWeb@
@
processDoc@webkit-masked-url://hidden/:223:19
@webkit-masked-url://hidden/:246:23
@webkit-masked-url://hidden/:90:20
promiseReactionJob@[native code]
url => https://www.proquest.com/docview/99391303/70936FB0204F4380PQ/3
downloadAssociatedFiles => true
automaticSnapshots => true

(5)(+0000001): Translate: Running handler 0 for error

(4)(+0000008): Translate: Parsing code for Embedded Metadata (951c027d-74ac-47d4-a107-9c3069ab7b48, 2022-01-03 17:00:00)

(3)(+0000000): Translate: Beginning translation with Embedded Metadata

(3)(+0000002): Translate: Embedded Metadata: found 9 meta tags.

(3)(+0000000): Translate: Creating translate instance of type import in sandbox

(3)(+0000000): Translate: Translation successful

(5)(+0000000): Translate: Running handler 0 for done

(4)(+0000005): Translate: Binding sandbox to https://www.proquest.com/docview/99391303/70936FB0204F4380PQ/3

(4)(+0000009): Translate: Parsing code for RDF (5e3ad958-ac79-463d-812b-a86a9235c28f, 2022-10-01 18:10:00)

(3)(+0000001): Translate: Initializing RDF data store

(3)(+0000000): Translate: Saving item

(5)(+0000000): Translate: Running handler 0 for itemDone

(3)(+0000001): Translate: Title was not found in meta tags. Using document title as title

(3)(+0000000): Translate: Looking for authors in byline, bylines, vcard, article-byline

(3)(+0000000): Translate: Found 0 elements with 'byline' class

(3)(+0000000): Translate: Found 0 elements with 'bylines' class

(3)(+0000000): Translate: Found 0 elements with 'vcard' class

(3)(+0000000): Translate: Found 0 elements with 'article-byline' class

(3)(+0000000): Translate: No byline found.

(3)(+0000001): Translate: Saving item

(5)(+0000001): Translate: Running handler 0 for itemSaving

(3)(+0000000): Translate: Saving via queue
---------

adamsmith

and that's which browser and browser version?

qenghis

Safari 16.0

adamsmith

nothing obvious in the debug -- I'd want someone with Safari to look at this: possible it's an issue related to the newest Safari version.
Do you have another browser installed & could test in Chrome or Firefox?

qenghis

Yes, it works fine in Chrome 107, so likely a Safari issue. And again, Safari 16.0 works with Zotero 6.0.15, so it would appear to be some issue in the 6.0.16 changes.

qenghis

Hi, I'm still seeing the same error when I try to use the Safari connector with Proquest Historical Newspapers. I worked around the issue by staying with Zotero 6.0.15 for a long time. I just updated to version 7, hoping that the problem would be resolved. Unfortunately, it isn't. The Chrome connector works fine. The Safari (v17.6) connector still displays: "An error occurred saving with ProQuest. Attempting to save using Embedded Metadata instead." Is there a way to get to the bottom of this?

aaronbcowan

I also updated to 7, hoping this issue was corrected, but it STILL is not. :(
I get the same "An error occurred saving with ProQuest. Attempting to save using Embedded Metadata instead." message, regardless of browser (Firefox or Chrome)

There are many, many folks who've inquired about similar
https://forums.zotero.org/search/?Search=proquest - really wish we could get a fix for whatever's going on here...

adamsmith

There are a bunch of different issues here -- Safari-specific ones, issues specific to a subset of newspapers, etc.
I think the only actual known Proquest issue that's not limited Safari currently is with EEBO.

aaronbcowan

@adamsmith But I don't use Safari

AbeJellinek

@aaronbcowan: Please provide a Connector Debug ID. May be the same problem you were seeing before, may not be, but we always need that information to debug translator issues.

aaronbcowan

@AbeJellinek Thanks for the reply. Just tried a couple of them, with the same result.

Report ID: 47347512

adamsmith

(that's an error report ID -- we'd want debug from the connector: https://www.zotero.org/support/debug_output#zotero_connectors_firefox_chrome_and_safari )

aaronbcowan

@adamsmith apologies.
Debug ID is D523056139

AbeJellinek

Thanks, I can reproduce.

AbeJellinek

@aaronbcowan: Fixed. Thanks for reporting. Your Zotero Connector should auto-update within a few minutes, or you can update manually by clicking Update Translators in the Advanced pane of the Zotero Connector preferences.

aaronbcowan

@AbeJellinek Thank you! This has been a huge pain in the neck for so long. I appreciate your help.

qenghis

@adamsmith @AbeJellinek I'm glad to hear that @aaronbcowan's bug is fixed, but my Safari connector is still choking on Proquest. Could you please check Debug ID D14101142?

As I mentioned previously, the bug first appeared with Zotero version 6.0.16. When I rolled back to 6.0.15, it worked fine with the same browser.

AbeJellinek

@qenghis: Thanks, I can reproduce that issue. We'll look into it.

For now, click the "Abstract/Details" tab in the left sidebar before saving to Zotero, or use a different browser for ProQuest.

qenghis

Thank you, @AbeJellinek. "Abstract/Details" works. I'll use that workaround for now.