A custom translator does not work although Scafold verifies it.

onedayonenoodle

Hello,

I have made a custom web translator for Zotero using Scafold.

https://github.com/kouichi-c-nakamura/Zotero-translator-SfN-2016

Although both detectWeb and doWeb of Scafold run successfully, the same translator does not detect relevant web pages and Zotero icon on browser toolbar stays in the default data type (Web page) rather than the specified data type (Conference Article), and I'm wondering why. Restarting both Zotero and the browser didn't help.

Any suggestions?

Best,
Kouichi

dstillman

Does the target regexp pass in the Metadata pane after you load the URL in the Browser tab? The current version of Scaffold will run detectWeb even if the target doesn’t match, but the target needs to match for detectWeb to run in the Connector. (We’ve fixed that for the next version.)

adamsmith

Looking at how the page loads (e.g. https://www.abstractsonline.com/pp8/#!/4376/presentation/30154 ), I'm guessing the content loads after the page signals it's fully loaded (which is when detectWeb runs). That's the same reason the tests didn't work, but it won't show in Scaffold, since you're running detectWeb manually there after the page _has_ fully loaded.

Look at Z.monitorDOMChanges to address this (unfortunately not well documented I think, but there are a number of examples in existing translators)

onedayonenoodle

First of all, it turned out that I was using an outdated version of Scaffold.

@dstillman

You're right. "Test regex" button actually resulted in "false", and I found an obvious error and fixed it. Now the toolbar icon is "Conference Paper" as expected, even before the page is loaded properly.

@adamsmith

I noticed the slow loading of the pages, too. Even `"defer": true` did not work for tests.

Thanks for Z.monitorDOMChanges:

https://github.com/zuphilip/translators/wiki/Useful-functions#monitor-dom-changes

===

Now, the import is working, but I'm facing another issue.

In case of https://www.abstractsonline.com/pp8/#!/4376/presentation/30154, I use the poster number for Pages field; "282.18/B8" is expected to appear in the Pages field of the imported Zotero item.

When I ran

Zotero.debug(item.pages);

it showed "282.18/B8" properly in the Scaffold.

Also, `Returned item:` output shows `"pages": "282.18/B8"` in Scaffold.

However, the Pages field of imported Zotero item is empty. This does not make sense to me. Do you have any suggestions?

Because other fields seem to be fine, this is probably not due to the slow loading of the page.

Thanks in advance.

Best,
Kouichi

adamsmith

I'd recommend looking at debug from the browser connector. This sort of thing isn't any faster for us to troubleshoot than it is for you.

That said, the correct item type for presentations/posters in Zotero is "presentation" not "conferencePaper", which is intended for papers published in conference proceedings.

onedayonenoodle

Mmm, "Presentation" doesn't have a place holder for the presentation ID, which is important. It has to be Journal Article or Conference Paper at the moment.

Can you elaborate more about debugging? I want to pin down why the information about Pages is lost during import.

Did you mean Help > Debug Output Logging > Enable?

dstillman

If you're debugging something in the Connector specifically, it's the Debug Output Logging section of the Advanced pane of the Connector preferences.

(To get a new version of the translator to the Connector, you have to save to Zotero in Scaffold and then update or reset translators in the Connector preferences.)

bwiernik

What do you mean “Presentation ID”? What does that mean/what is it for?

onedayonenoodle

Thank you, guys. While messing around with Scaffold, the problem of pages has been fixed once. And then it came back although I didn't edit the .js file at all. Even for the same record item. Very weird.

For example, in the case of https://www.abstractsonline.com/pp8/#!/4376/presentation/30154, the poster has the number "282.18 / B8", that is associated.

The number "282.18" is used for citation in papers, whereas "B8" is the ID for the poster board.

So, currently, when I cite it, it appears as

Yakoubi R, Rollenhagen A, Lübke J, Lehe M von, Miller D, Walkenfort B, Hasenberg M, Sätzler K (2019) Ultrastructural heterogeneity of human layer 4 excitatory synaptic boutons in the adult temporal lobe neocortex. In: Society for Neuroscience Abstract, pp 037.01/B74

In fact, "pp" should be removed. But this has to be done by editing CSL and that can cause another issue.

Also, maybe "B74" should be removed as well. But the is the best compromise I've come up with so far.

onedayonenoodle

Here is the debug output I obtained. But I can't really see a hint.

https://gist.github.com/kouichi-c-nakamura/869cfeb8c286e8b9bb88fdff1fd0f8f1

adamsmith

well, you'd want to built relevant debug statements into the translator code you're running to
a) make sure you're running the right version of the translator and
b) understand what's not working with the current version.

onedayonenoodle

It was essentially a problem with regular expression. Although Scaffold ran without an error, when actually running on the web page it failed. Somehow Scaffold was more relaxed? This could be related to the slow loading of dynamic web page.

Anyway, I changed a regular expression, and the problem has been solved for now,