client scripting API with Zotero standalone (v5)

benjswift

This page (https://www.zotero.org/support/dev/client_coding/javascript_api) shows how to use the "zotero" js object to do scripting & batch processing of the library. However, it's written in the context of the FF add-on, where it's possible to poke around in the js internals.

Is it possible to do *client* js scripting with the new standalone Zotero? If so, how does one get a hold of the "zotero" js object?

bwiernik

There currently isn't a way to access the javascript API with the Zotero client app. It would be possible to write a simple plugin that would allow you to send arbitrary js commands to Zotero, but no one has done that yet.

benjswift

Ok, thanks. I don't know if I'll go down that path right this minute, but it's good to know the way forward.

dstillman

I've added a way to run arbitrary JS to the latest Zotero beta.

Install that, go to Config Editor in the Advanced pane of the preferences and set devtools.chrome.enabled to true, and then restart Zotero. You'll then see a Run JavaScript option in the Tools menu, which will open a window that lets you run the sort of privileged JavaScript on the linked page (though what's there is quite out of date by now, so we'll need to update that).

I hacked this together very quickly, so we can certainly improve on it, but it will hopefully do the trick for the moment. For now, you'll want to keep the Error Console (which will also appear as an option in Tools) open to see any errors in the code you run.

benjswift

Oh, that's super. Thanks much.

dstillman

I just made some additional improvements to allow the use of `await` (as now required by much of the code) and to log errors to the right-hand pane so that opening the Error Console separately isn't as necessary.

I've also updated the linked page to bring most of the examples up to date for Zotero 5, though, as explained there, the source code is still the best reference for now.

emilianoeheyns

If by "client scripting" you mean "scriping from something outside Zotero" (because if it's inside Zotero you'd usually stick it inside a plugin): if you zip up https://github.com/retorquere/zotero-better-bibtex/tree/master/test/fixtures/debug-bridge, you get an extension by which you can execute javascript inside Zotero by sending it in a POST request. While it listens only on 127.0.0.1, it's quite obviously a major security risk, so you wouldn't want to run this in an hostile environment - otoh, if you have hostiles making http POSTs on 127.0.0.1, you have more problems than just this plugin IMO.

You zip that directory up and rename it's extension to ".xpi". This xpi you install as you usually would. Set up a password as per https://github.com/retorquere/zotero-better-bibtex/tree/master/test/fixtures/debug-bridge .

Once installed (and Zotero restarted), you now have an endpoint that listens on http://127.0.0.1:23119/debug-bridge/execute . You can POST (using curl for example) a text body to that URL:

curl -X POST -H "Content-Type: application/json" --data "await Zotero.Schema.schemaUpdatePromise; return 'hello ' + query.world" http://127.0.0.1:23119/debug-bridge/execute?foo=bar&baz=quux&world=world&password=[your password]

or

curl -X POST -H "Content-Type: application/json" --data=@yourscript.js http://127.0.0.1:23119/debug-bridge/execute?foo=bar&baz=quux&world=world&password=[your password]

your code will be used as the body of an async function, and the query variables will be available as "query", so the first call would be equivalent to executing

async function test(query) {
  await Zotero.Schema.schemaUpdatePromise; return 'hello ' + query.world
}
return await test(query)

if your code errors out, you will get the error back as the response body, and the response code will be 500

if your code executes successfully, you get the JSONified return value of your script (if it doesn't return anything, "null" is returned), and the response code will be 201.

Note that a lot of of the Zotero uses coroutine/yield instead of async/await. I *think* coroutines return promises you can just await on. But in the code you post, you cannot use yield; use await (or old-school .then) instead.

Any POST to the endpoint will already automatically await Zotero.Schema.schemaUpdatePromise before running your code, so that's not necessary, that was just to show how await works in this context; technically, the curl call above is equivalent to

async function test(query) {
  await Zotero.Schema.schemaUpdatePromise;
  await Zotero.Schema.schemaUpdatePromise; return 'hello ' + query.world
}
return await test(query)

Note that if you launch Zotero and immediately after that access this endpoint, you will first get "Failed to connect" for a while during Zotero startup, and after that it will take about 5 seconds for the first script to execute as it is waiting for Zotero.Schema.schemaUpdatePromise to resolve. After that resolves it is not exactly a no-op but can be thought of as such.

In your scripts you can run whatever you would otherwise run in an extension. Zotero.debug(...) works, you can access the database, everything.

If you run long-running scripts this way that do not call await regularly, Zotero will stall and you will get popups with "this script is running for a long time, kill Zotero?" (or something to that effect).

Once done you can either remove or disable the extension but it will remain active until you restart Zotero.

dstillman

If by "client scripting" you mean "scriping from something outside Zotero" (because if it's inside Zotero you'd usually stick it inside a plugin)

Not necessarily — the linked documentation page previously recommended using the Execute JS Firefox extension for the sorts of batch editing stuff discussed there. The "Run JavaScript" feature I added above is a direct replacement for that, and should be sufficient for people who just want to quickly run some code.

Your debug-bridge certainly has lots of other uses, though.

bwiernik

This is fantastic! Thanks!

fbennett

Very late to the party here, but I just used the JS runner to refactor 1,000 case entries in an old library, and: thank you.

tamurello

emilianoeheyns, the debug-bridge tool looks quite useful. I ran into some problems executing code though.

I was able to install the add-on successfully, I think, and it looks like it loads according to the debug logs, but I am having trouble running anything (getting errors like "Endpoint does not support content-type" when I try to execute the hello world example).

In more detail:

I installed debug-bridge-5.1.105.emile.limonia.xpi

After restarting, the debug log shows:

(3)(+0000004): debug-bridge load attempt
(3)(+0000000): Installing debug-bridge

When I go to http://127.0.0.1:23119/debug-bridge in a browser, the response is:

Request not allowed

The exact hello world example I tried running with curl is:

curl -X POST -H "Content-Type: text/plain" --data "await Zotero.Schema.schemaUpdatePromise; return 'hello ' + query.world" 'http://127.0.0.1:23119/debug-bridge/execute?foo=bar&baz=quux&world=world'

Some Zotero logs from those requests:

(5)(+0107282): GET /debug-bridge HTTP/1.1 Host: 127.0.0.1:23119 Connection: keep-alive Cache-Control: max-age=0 Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3 Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.9

(5)(+0000004): HTTP/1.0 403 Forbidden X-Zotero-Version: 5.0.73 X-Zotero-Connector-API-Version: 2 Content-Type: text/plain Request not allowed

(5)(+0059274): POST /debug-bridge/execute?foo=bar&baz=quux&world=world HTTP/1.1 Host: 127.0.0.1:23119 User-Agent: curl/7.54.0 Accept: */* Content-Type: text/plain Content-Length: 70

(5)(+0000006): HTTP/1.0 400 Bad Request X-Zotero-Version: 5.0.73 X-Zotero-Connector-API-Version: 2 Content-Type: text/plain Endpoint does not support content-type

Any ideas for what might be wrong? Or other debug info that would be useful?

emilianoeheyns

The debug bridge requires a password since 5.1.105, and the content type must be "application/json".

fl0_id

For me, I had to put the url into quotation marks and use content type application/javascript, otherwise I got wrong contenttype or 'password required'.