WebofScience problem since update 3.0.14

andypascal

I was adding new references to my database (Zotero Standalone+Firefox plugin), when I was asked for an update to 3.0.14. After this update (to both), I twice added references from WebofScience. Each time, instead of a single reference it seems several hundred were added. This wasn't happening before the update. I hesitate to test with other libraries as the database will become unmanageable. I can create a new database to play with but could you first verify if this is just me, or there was a bug in the update? (maybe there'll be other reports..)

dstillman

How are you saving the items? Search results? Article page? Do you get the same items or different ones, and do they match your search results?

Can you provide a Debug ID for this occurring? You can easily remove all the newly created items afterward by sorting your library by Date Added.

dstillman

Also, this is using the Firefox plugin via Standalone? Does it happen if you use Zotero for Firefox alone?

andypascal

It was a single article record in WOS each time (guess this is what you mean by article page), using the plugin via standalone. It looks like the extra items were related (some same authors, and/or similar subjects) so it's possible it was getting things from a link on the page (eg to papers citing that one, or refs in the paper) - I dont think it was just all of my search results, but I didnt check that. I will look again, and try the things you suggest when i get back home. Thanks for the help...

adamsmith

If you're able to replicate this, please provide exact information on the item you're looking at - WoS doesn't have stable URLs, but you can give us database & title or the like to get us to the same page you're looking at.

andypascal

I'm not sure what details I should be giving you, but will WOS accession number do? I just made 2 searches on WOS, this time using the Firefox plugin alone. The first search I only got one item in the results, which I clicked on to go to the individual record and then saved to Zotero using the icon in the address tab. This entered just one item, the right one:
Andersson, Po, T. Gillbro, L. Ferguson, and Rj Cogdell WOS:A1991GA53900004
I then clicked on one of the authors (Gillbro) to get all their articles, ie a large number, selected one of the records and clicked to insert it into Zotero. This time 8 articles came in. The first is the one I wanted, followed by the other 7:
Zietz, Burkhard, and Tomas Gillbro WOS:000250142400021
Batchelor, C. WOS:000224749900005 (this record twice)
Granucci, G., M. Mazzoni, M. Persico, and A. Toniolo WOS:000229978300008
Mazzoni, M., G. Agati, R. Pratesi, and M. Persico WOS:000234182700006
Plavskii, V. Yu, V. A. Mostovnikov, A. I. Tret’yakova, and G. R. Mostovnikova WOS:000248834100002
Thorling, Camilla A., Yuri Dancik, Clinton W. Hupple, Gregory Medley, Xin Liu, Andrei V. Zvyagin, Tom A. Robertson, Frank J. Burczynski, and Michael S. Roberts WOS:000295441500025
Zietz, B., and F. Blomgren WOS:000236313300055
Clearly the extra articles were not just from the search, as they all would have Gillbro as a coauthor, and they're not references in the article or citing articles (I checked).

I just realised, I didn't have debugging on when I did this - should I do it again with it on to submit the output?

dstillman

Yes, a Debug ID would be helpful.

adamsmith

the accession numbers are great, though. I can get to what you're looking at (though I can't replicate your problem, hence the debug ID).
To make sure we're talking about the same thing:
You click on "Gilbro", you get the article list in WoS, you click on the yellow folder icon in the URL bar, select only the first item ("Initial photochemistry...") and then get the items you describe imported in Zotero?

aurimas

I can't reproduce this either. Could you also make sure that your translators are up to date? http://www.zotero.org/support/troubleshooting_translator_issues step 5.

Also, what do you get for step 6 and what is the URL you are looking at (exactly as it appears in your URL bar)?

andypascal

Today I'm in the lab, and the problem doesn't occur here - neither before or after update. I guess there's no point in giving this debug report... When I'm at home I'll check the translators and then run a debug there if necessary. For what it's worth - here it's on Firefox 12 under WinXP; at home (where the problem is) I have Win7 64-bit running Firefox 18.0.2 (32bit).
adamsmith - not exactly. From the list of Gillbro's articles, I click on one (on the web page) to go to the individual WOS record and then click the single article tab in the address pane

andypascal

Debug ID: D831500422
This time 22 articles were pulled in, instead of one. This was standalone+firefox plugin. Can you see the 22 articles in the debug, or should I post them here?
I just remembered something that may be important. On this computer zotero was installed from my wife's (admin) account. When I went to use it from my account, I had to reinstall the firefox plugin. There also seem to be other weird things - I just noticed that the plugin doesn't have my sync settings, whereas standalone does. Is this related? Maybe they don't talk together properly?
Thanks again for your help.

dstillman

If you can reproduce this easily using just Zotero for Firefox, with Standalone closed, a Debug ID for the save from that might be helpful. (I don't believe there's a way for Zotero for Firefox to generate a Debug ID in connector mode (like the Chrome and Safari plugins can), and the Standalone side of the debug output doesn't provide enough info to debug this. When you use Zotero for Firefox alone, the debug output includes everything.)

When I went to use it from my account, I had to reinstall the firefox plugin.

That's expected and unrelated. Zotero for Firefox needs to be installed for each user (since it's installed within the user's Firefox profile), and it also doesn't currently share settings with Standalone, just the database.

andypascal

Debug ID: D678934175
I searched for MTA Alexandre's articles, clicked on the first one and tryed to add it:
Alexandre etal. WOS:000309409000506
I got this one and 11 others with it. It looks like the original search gave these 12 results. Let me know if you want all the other article details.

andypascal

I forgot to mention - the translators were up to date, and it does identify ISI Web of Knowledge when I hover over the address pane icon.

dstillman

(3)(+0000021): Translate: Beginning translation with ISI Web of Knowledge

(3)(+0000005): HTTP POST viewType=fullRecord&product=WOS&mark_id=WOS&
colName=WOS&search_mode=OneClickSearch&locale=en&recordID=WOS%3A000309409000506&
sortBy=PY.D%3BLD.D%3BSO.A%3BVL.D%3BPG.A%3BAU.A&mode=outputService&qid=9&
SID=[...]&format=saveToRef&filters=USAGEIND+RESEARCHERID+ACCESSION_NUM+FUNDING
+SUBJECT_CATEGORY+JCR_CATEGORY+LANG+IDS+PAGEC+SABBR+CITREFC+ISSN+PUBINFO+KEYWORDS
+CITTIMES+ADDRS+CONFERENCE_SPONSORS+DOCTYPE+ABSTRACT+CONFERENCE_INFO+SOURCE+TITLE+AUTHORS
++&selectedIds=1&mark_to=&mark_from=&count_new_items_marked=0&LinksAreAllowedRightClick=full_record.do
&value(record_select_type)=selrecords&markFrom=&markTo=&fields_selection=USAGEIND+RESEARCHERID
+ACCESSION_NUM+FUNDING+SUBJECT_CATEGORY+JCR_CATEGORY+LANG+IDS+PAGEC+SABBR+CITREFC
+ISSN+PUBINFO+KEYWORDS+CITTIMES+ADDRS+CONFERENCE_SPONSORS+DOCTYPE+ABSTRACT
+CONFERENCE_INFO+SOURCE+TITLE+AUTHORS++&save_options=othersoftware
to http://apps.webofknowledge.com.[...]/OutboundService.do?action=go

(3)(+0000189): Translate: Could not find qid on page. Using 1 + previous qid: 10

(3)(+0000000): HTTP POST locale=en&colName=WOS&sortBy=PY.D%3BLD.D%3BSO.A%3BVL.D%3BPG.A%3BAU.A&
SID=[...]&filters=USAGEIND+RESEARCHERID+ACCESSION_NUM+FUNDING+SUBJECT_CATEGORY+JCR_CATEGORY
+LANG+IDS+PAGEC+SABBR+CITREFC+ISSN+PUBINFO+KEYWORDS+CITTIMES+ADDRS+CONFERENCE_SPONSORS+DOCTYPE
+ABSTRACT+CONFERENCE_INFO+SOURCE+TITLE+AUTHORS++&fileOpt=fieldtagged&
action=saveDataToRef&product=UA&numRecords=1&numRecsToRetrieve=500&qid=10
to http://ets.webofknowledge.com/ETS/saveDataToRef.do

(3)(+0000858): Translate: Creating translate instance of type import in sandbox

(4)(+0000001): Translate: Binding sandbox to
http://apps.webofknowledge.com.[...]/full_record.do?product=WOS
&search_mode=OneClickSearch&qid=9&SID=[...]&page=1&doc=1

(4)(+0000005): Translate: Parsing code for ISI Web of Knowledge

(3)(+0000025): Translate: Beginning translation with ISI Web of Knowledge

aurimas

I followed andypascal's steps exactly and I still only import 1 item. The only difference is that i do not get the following:
(3)(+0000189): Translate: Could not find qid on page. Using 1 + previous qid: 10

I added some more debug code to the translator to troubleshoot this further. @andypascal, if you could update your translators again, try the import once more (with debugging turned on), and resubmit a debug ID, we might be able to figure this out.

There might be something going on with the proxy for the first POST request. Dan, does everything look ok with the proxified URL there?

dstillman

It's actually a little weird, though I'm not sure if it's related. For the first request it's POSTing to apps.webofknowledge.com.foo.bar.tld.foo.bar.tld, whereas the sandbox is bound to just apps.webofknowledge.com.foo.bar.tld.

aurimas

That could do it. We're picking up the URL from the form on the page, so it might be incorrectly proxified by the proxy server. I added some more debug code just now that will show us if this is the case.

I could make the translator less smart and use a static URL for the post I think if this is the case.

andypascal

I did it 5 times, and each time only the entry I wanted came in. Debug ID: D697796266
Should I keep trying until it goes wrong again?

dstillman

aurimas: In those successful requests it seems to no longer being doing the double-domain thing.

aurimas

I'm guessing that it was a temporary glitch on the proxy. Let us know if the problem resurfaces. The debug code will remain in the translator, so if it happens again, we'll be able to tell for sure if it's Zotero or the proxy.

Enjoy the working translator :-)

andypascal

OK, thanks again. Zotero is great, thanks a lot for making it available