Problems Importing from Proquest

jg7138a
Hello all,

I'm a longtime Zotero user, but this is the first time I've come to the forums. This week, I ran into a problem when I attempted to import newspaper records from Proquest into Zotero. In the past, when I've selected the option "Export-->Export to ProCite, EndNote, or Reference Manager," my browser has automatically recognized Zotero and exported the files to the designated folder.

When I attempt to do this now, Firefox opens a new window and then a pop-up dialog box (Open or Save As "proquestcitations.txt). If I do save the file as an RIS document, I can import it into Zotero, but much of the metadata is lost / jumbled (much of the data becomes notes below the actual object).

I've searched the forums for a solution to this problem. One user seems to have had similar issues, and I was able to fix the problem by resetting his translators and re-installing the Firefox plugin. I tried this and wasn't successful.
http://forums.zotero.org/discussion/18397/import-problems-after-attempting-to-use-standalone-alpha/

I also attempted to follow the instructions given in another forum that deals directly with Proquest translators, including putting new translator files into my Zotero folder. This also didn't work.
http://forums.zotero.org/discussion/16034/2/proquest-translator-not-recognising-their-new-interface/

The problem left me wondering if I need to adjust certain settings in Firefox 4, but I'm really not sure what's going on. If anyone has any advice, or has overcome a similar problem, I'd love to hear from you. Thanks!

- Jordan
«1
  • adamsmith
    The way to go would likely be to get the translator to work - in general it should on proquest. Do you have a sample URL?
  • jg7138a
    Hello Adam,

    Thanks for taking an interest in my request. I created a link to at least one article I've been trying to record here:
    http://search.proquest.com.proxyau.wrlc.org/hnpbaltimoresun/docview/533659878/130D1FD09FF63FFF43D/2?accountid=8285

    I'm afraid you might run into a proxy wall, though. Like I mentioned above, exporting the file prompts the browser to open a dialog window in Firefox 4 rather than importing the file into my Zotero library. (I also don't see the usual "save to zotero" icon in my browser toolbar, if that helps).
  • adamsmith
    OK that should be an easy fix, the translator works already, it just doesn't recognize the page as a proquest document. I'll wait for ajlyon to fix this, he knows the translator better.

    Technical notes:
    Avram: The main issue here is just the regexp - the current one looks for (/pqrl|/pqdt) which this one doesn't have. The other issue is that Zotero wrongly recognizes this as a journal article because it discards the "historical newspaper" item type - that would be cool to fix, too.
  • ajlyon
    Please go to http://github.com/ajlyon/translators/raw/master/ProQuest.js and save the file to the translators directory of your Zotero data directory (http://www.zotero.org/support/zotero_data).

    It should start working again. If this works for you, please post here so that I can submit this change to be pushed to all users.
  • jg7138a
    Hello all,

    I went into my directory and inserted the new translator file; then I tried to export from Proquest and got the same error. I even went so far as to delete the other "old" Proquest translators; still the same response. Each time, Firefox opens a new tab, then a dialog box appears.

    Just to clarify - when I save the new translator, I save it as a ".js", and I put it in the Zotero "Translators" folder within my Mozilla user profile. (If my process is wrong, let me know and I'll try again).
  • ajlyon
    Oh-- don't try to do the export. That's a separate issue, but you shouldn't need to export. Just click the icon in the address bar to save, after installing the updated translator.
  • adamsmith
    What ajlyon says, but instead try this. Also, you'll have to restart Firefox after saving.
    https://github.com/adam3smith/translators/raw/9c461377db1ec1ad87638a894f031f0af83498f3/ProQuest.js

    (Avram - you forgot a pipe in the regexp).
  • jg7138a
    Well, I went back to Zotero and, unfortunately, there wasn't an icon in the address bar. I looked through every different view of the newspaper articles (Citation/Abstract, Page View, PDF, etc.) with no luck. Thoughts?
  • adamsmith
    Did you try with the version I link to or with ajlyon's? If you tried with ajlyons - retry with mine. If you tried with mine, make sure that the file is in fact proquest.js and not proquest.js.txt - the icon will look different if you open the translator folder. Also, as I said, you'll need to restart FF after replacing the proquest file.
  • ajlyon
    Sorry about that-- I don't see the hnpbaltimoresun bit in my URLs, so I had to do that by sight. You can now reinstall from the address I gave above and it should work-- I've included adamsmith's patch.

    adamsmith: Thanks for catching that.
  • ajlyon
    And this will only work from the citation/abstract views for now.
  • adamsmith
    actually, it works in pdf mode for me as well.
  • jg7138a
    Hey all,

    I went through, tried both patches (both with and without the earlier translators; always as a .js file, not a .txt file. Though the icon does appear in the address bar (yes!), when clicked, it doesn't actually export the record. It actually behaves differently on different pages.

    On the normal / PDF page of the article, Zotero small red square pops up (as it should), but it's completely empty, and the record doesn't get to my zotero library.

    On the Citation/Abstract page, the red box comes up, but it's an error message:"An error occured while saving this item. Check Known Translator Issues (http://www.zotero.org/support/known_translator_issues) for more information."

    Sorry to be such a difficult user. Also, while the browser icon would be great (and I would make it work), I am working with a few hundred newspaper articles, so the export function would be very useful.

    Two thoughts: If I installed the standalone Zotero and used Chrome, would I be able to work around this problem? And what if I went back to an earlier version of Firefox?
  • adamsmith
    No, we want to get this fixed, don't worry.
    Installing standalone and earlier versions of Firefox won't help, no.
    The idea would be that you could also use the multiple import button from the URL bar, so you don't need the export function. While it might be possible to get the export working directly into Zotero again, we won't be able to fix the jumbled import - that just means the data in the RIS is jumbled - I can't tell you why that used to work better, probably a change on their part.

    As for the errors you see - have you tried that on different pages? And none of them work? Right after triggering the error from the citation/abstract page, can you create an error report - to speed things up, don't send it, but instead copy the part that describes the translation failure and paste it here. It will somewhere refer to Proquest.js
  • jg7138a
    Hey all,

    Tried a few other articles with the same results. Here's the error report:

    [JavaScript Error: "TypeError: doc.evaluate("./div[@class=\"display_record_indexing_fieldname\"]", record_row, nsResolver, XPathResult.ANY_TYPE, null).iterateNext() is null" {file: "file:///C:/Users/Jordan/AppData/Roaming/Mozilla/Firefox/Profiles/sz9rmcny.default/zotero/translators/ProQuest.js" line: 0}]

    I'll be away from my desk for most of the day, but if you would like me to try something else, leave a message and I'll try it this evening. Thanks!
  • ajlyon
    This is quite baffling. I managed to find a way to replicate URLs of this type, and have tested with the item:
    http://search.proquest.com/hnpbaltimoreafricanamerican/docview/530980990/abstract/130D7905C86225ACC9B/1?accountid=14512

    I was hoping this would show the same issues-- but it works fine for me.

    The translator looks for the section "Indexing (details)", like in this snapshot: http://www.gimranov.com/research/zotero/proquest-hnp.png

    Can you see such a section on the abstract page?

    I've also uploaded an updated version to http://github.com/ajlyon/translators/raw/master/ProQuest.js that should be a little bit more forgiving and might fix this for you.
  • adamsmith
    That looks promising - the site you linked to didn't work for me with the old version but does with the new one.
  • jg7138a
    That did it! The new translator works ; the icon in the browser exports the file into my zotero library, and it does it on the normal (pdf) screen, as well as the citation / abstract screen. It still doesn't have the ability to do multiple files (the icon only appears when I look at individual items, not a list), but I can work around that. Thanks so much for figuring this out, and let me know if I can do anything if you're still tweaking the translators / etc.
  • ajlyon
    What URL do you see when you are looking at search results?
  • jg7138a
    Hey,

    I found something interesting ; the translator (and Zotero) works perfectly (single items, batches of items) on ProQuest when I don't log in under my username and use "My Research."

    I don't know how familiar you are with ProQuest, but it gives you the option of saving articles and organizing them in your own personal folder. In most cases, I search for articles and then put them all into this folder before I actually import them into my Zotero library. In the past, I've used this folder to, say, divide a thousand articles over a decade into annual folders. Then, I've imported each annual folder, en masse, into Zotero.

    At the moment, when I'm under "My Research," I can do single files (thanks to the new translator), but not groups of articles.

    However, if I don't save those files into My Research, but instead import them directly into Zotero, it works fine - indivdual articles or large groups.

    So perhaps it's the "My Research" ghetto that's causing all the problems. That's not a big problem at all; it just means I'll do more of my sorting on the back end, rather than in the ProQuest service.

    Whew, hope that makes sense! Thanks again for tackling this. I'm sure I speak for a lot of people when I say you guys are awesome.
  • ajlyon
    Ah yes. I don't ever create accounts on sites like that-- that's what I have Zotero for. :)

    And note that since the translator now grabs PDFs for the articles as you go, you can use Zotero (even offline) to browse everything that you might previously have browsed from within My Research.

    It'd be possible to add support for the My Research view as well, but that may just not be worth it if the other views are working smoothly.
  • jano_riha
    Dear Avram,

    I am sorry I have to report that I also have encountered a problem while working with the ProQuest translator (the latest version). When having tried to retrieve this entry, the "ProQuest Record" with fulltext in html seemed to be downloaded. However, when I clicked on the item to check it out, an error page opened with this text:

    "We seem to have encountered a problem.

    Try pressing the browser's Back button. That sometimes works!
    If you typed in the address, try double-checking the spelling.
    If you followed a link from somewhere, please let us know. Be sure to tell us where you came from and what you were looking for, and we'll do our best to fix it.
    Your session may have ended. Begin a new session

    The technical details...

    If you want to see the details and, perhaps, help us prevent it happening again, you can view it here".

    When I clicked on the link "here" to actually view what I had been advised to view, there was an error text again stating that "File not found".

    Fortunately, the translator otherwise seems to work fine, in most cases. Nevertheless, the process is still not flawless, probably due to ProQuest itself.

    Regards
    JR

    P.S.: Would it be possible to tweak the translator in such a way the link from the downloaded "Proquest Record" would be filled in the Zotero entry info and make sure it is actually in the same presentable format (that is http://search.proquest.com/docview/number?accountid=number) as the link ProQuest generates when one uses its "Cite" instrument from the menu? The reason is it is becoming a standard to include a link to a database when citing a source. At present, one has to do it manually in case of ProQuest.
  • ajlyon
    Please go to http://github.com/ajlyon/translators/raw/master/ProQuest.js and save the file to the translators directory of your Zotero data directory (http://www.zotero.org/support/zotero_data).

    The attachments should start working again.

    The choice to not put the database URL in the URL field is because we consider ProQuest to be a non-canonical source-- for justification, see https://www.zotero.org/trac/ticket/827
  • jano_riha
    Done and tested. It worked well with the article I referred to in my previous post. However, in case of this thesis, the translator dowloaded snapshot of html page with pdf named "ProQuest Record" (although I retrieved citation from the citation/abstract page) along with another pdf named "ProQuest PDF". I am sorry to report this, but it seems there still is a problem somewhere.

    As for the canonical/non-canonical source, thank you for edifying me. I am sorry I have omitted the ticket you referred to. I agree it is better to distinguish sources in this way. Nevertheless, in the prevalent understanding of ISO 690 here in the Czech Republic, it seems to me, it is considered preferential to include availability information within citation. There is a new version of this norm, ISO 690:2010, which I have not yet had the possibility to go through. Therefore I do not know how significant changes are there. However, I have looked at some excerpts from ISO 690 and ISO 690-2, and it seems to me it might be sufficient to include availability information in a format such as "Available also from ProQuest (ID number)". Would it not be possible to retrieve this ID and fill it in the "Call Number" field, so it could be then included in citation?

    Regards
    JR
  • ajlyon
    Please go to http://github.com/ajlyon/translators/raw/master/ProQuest.js and save the file to the translators directory of your Zotero data directory (http://www.zotero.org/support/zotero_data).

    I've changed the snapshot to grab the abstract instead of the PDF, and to put the ProQuest ID in the Call Number field.

    I've also noticed that the translator breaks completely if the language is changed to something other than English-- I'll have to fix that at a later date (although it'll require some serious reworking).
  • jano_riha
    edited August 3, 2011
    Dear Avram,

    Thanks again a million, I have tested the latest version of the translator on the following IDs: 227276984, 200520701, 201323086, 304408427, 02650525, 304897759. There are still problems.
    1. Not such a big deal. The "ProQuest Record" attachment includes only abstract, not fulltext as well, even there is a fulltext on the same page, and is dowloaded along with the abstract if one does it manually.

    2. A bigger deal. In case of ID 304408427, „ProQuest PDF“ attachment was not dowloaded. When I repeated the test, not even "ProQuest Record" attachment was downloaded.

    3. A big deal. When retrieving the last two database records, the Zotero asked me to restart Firefox („An error has occured. Please restart Firefox. You can report...“ and so on). The last error report in the error log before restart stated
      [JavaScript Error: "uncaught exception: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.rollbackTransaction]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: :: line 525" data: no] [ERROR: cannot rollback - no transaction is active]"]
      After the restart I realised the last item (ID 304897759) was retrieved without any attachment, while the second last (ID 02650525) with "ProQuest Record" attachment, which was however corrupted. When I repeated retrieving of the record ID 02650525, no attachments were downloaded. When I tried to retrieve the record ID 304897759 for the second time, Zotero again asked me to restart Firefox. I have got a certain feeling the „big deal“ might be related to records referring to theses. I tried to retrieve the same ID again. And again, Zotero asked me to restart Firefox, in the error log there was:
      [JavaScript Error: "[Exception... "'[Exception... "Component returned failure code: 0x80630002 (NS_ERROR_STORAGE_IOERR) [mozIStorageConnection.commitTransaction]" nsresult: "0x80630002 (NS_ERROR_STORAGE_IOERR)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: :: line 484" data: no] [ERROR: disk I/O error]' when calling method: [nsITimerCallback::notify]" nsresult: "0x8057001e (NS_ERROR_XPC_JS_THREW_STRING)" location: "" data: no]"]

      [JavaScript Error: "uncaught exception: [Exception... "Component returned failure code: 0x80630002 (NS_ERROR_STORAGE_IOERR) [mozIStorageStatement.execute]" nsresult: "0x80630002 (NS_ERROR_STORAGE_IOERR)" location: "JS frame :: chrome://zotero/content/xpcom/data/item.js :: :: line 1345" data: no] [ERROR: disk I/O error]"]

      4. I hope this report will help you to help me (of course, not only me, but many others) :)

      Regards
      JR

      ---
      Edit: Just to assure you, I use the English language settings in ProQuest. I use it everywhere possible (in EBSCOhost, unfortunately, one has to change preset Czech language to English every time).
  • dstillman
    That item works fine for me, and it's unlikely that this would be related to the translator. Follow up in the other thread.
  • ajlyon
    (1) is intentional. The only way I can easily guarantee that the snapshot won't have the embedded PDF and certainly will have the basic bibliographic data and abstract is to do it this way.

    (2) sounds bad, but I'm concerned it might be connected to (3). If (3) is resolved and (2) keeps happening, we can go back and see what's causing it.
  • jano_riha
    Dear Avram, dear Dan,

    I have already reported my recent findings in the thread on the forced FF restart. I am inclined to think the problems (2, 3) were caused by the lack of storage in the very drive where Zotero application data were stored. Zotero dealt with the impossibility to execute the command by asking for FF to be restarted. I hope there will be a solution for such cases in the future.

    Thanks again to both of you.
    JR
  • croftb
    I have a simple fix to this problem. In Proquest, click on export. Select the line with ProCite or reference manager. It will load your citation with all of the details intact. In fact, it seems better than before the switch over to the newer version of ProQuest.
Sign In or Register to comment.