Python with the Zotero API

hughp3

Greetings,

I'm wondering if anyone knows of a Python tool kit for interacting with the locally hosted Zotero database.

I see this one https://github.com/urschrei/pyzotero. But it seems to be for interacting with the web API...

OR is the local and web APIs the same thing, only with a destination change...?

aborel

I've been playing with it for a couple of weeks now, it seems to be only a wrapper for the web API. So you need to pay attention to syncing otherwise the results might be a bit strange when you look at or edit records in your local database.

adamsmith

There is no reasonably way to use python on the local database. Zotero doesn't have a local API that's exposed outside of the mozilla framework, so there's nothing python could hook into (it _is_ possible to write a python-based reader for the library, of course. That is part of the Qnotero tool, although likely needs updating since Qnotero is no longer maintained)

danb

@hughp3 I wrote a library that works locally. See here: https://forums.zotero.org/discussion/81195/limited-proof-of-concept-csv-to-zotero-import#latest

It uses the debug bridge to run code locally. I've used it successfully to automatically import files from disk with links, tags, notes etc, but I haven't tried it over the last month and can't guarantee that nothing broke in the meantime.

@adamsmith: Is there hope to get a local API in the future?

The debug bridge already provides most of the functionality needed for a local API. Any chance it could become an official part of Zotero?

With a local API, a lot of things would become much easier:
- Custom importers for existing libraries (often asked here)
- Text mining, similarity search, extracting references from documents. The ML ecosystem is mostly Python-based, and it wouldn't make sense to run such tasks in a browser context anyway.

dstillman

@danb: Can you clarify what you're looking for? debug-bridge isn't really an API — it just exposes the internal JavaScript environment, which is also available via Run JavaScript within Zotero. It certainly doesn't provide "much of the functionality needed for a local API". The HTTP part is trivial.

We have had a vague plan to try to expose a local HTTP API that mirrors the web API, so that tools could easily be programmed to work with either, but not sure when we'll be able to work on that.

danb

@adamsmith: I am looking for a stable way to talk to a local Zotero installation for adding items and querying the library.

The commands that I use are:

- create items with attachments, notes, tags, etc.
- used for importing documents into Zotero
- get item currently selected in Zotero.
- used to add a document with notes to existing items
- traverse Zotero library and get all files, links, notes for items.

I don't mind doing most of the processing in Python, but I fear that going through a debug interface might not be stable in the long run.

For example, I am not sure that it will still work after the switch to Electron.

The HTTP part is small by comparison, but that is where I had most of the difficulty when still using MozRepl. No issues with the debug bridge so far. It might be little code, but it is really the core component, so I would love if this approach was at least officially sanctioned including Electron.

Of course, a local API would be much better, but I understand that this is not a priority.

b.n.lawrence

Me too. I really want this. The lack of it is the one thing that keeps making me consider leaving Zotero ... (haven't yet, but the elastic gravity strap keeping me in the zotero orbit is getting pretty frayed :-).

dstillman

If this is enough to make you consider leaving Zotero (for what?), then just hook something up with debug-bridge, which already exists.

hughp3

@b.n.lawrence how deep did you go into Zotero? as in how many items do you have in a zotero Database?

b.n.lawrence

I have thousands of items in zotero (many thousands :-), all of which are duplicated in DTP. There is a useful thought on the DTP Forum here: https://discourse.devontechnologies.com/t/from-devonthink-to-zotero/61297/8

I will follow up and report.

(@dstillman, I was reluctantly looking at Bookends, I don't like it all on the desktop, but it works well enough on an ipad ... I have yet to try it in anger, I'd prefer to try and get something working on what I have using DTTG to ingest into DTP and on the desktop slurp it into Zotero and back into DTP in the right place.)

danb

@dstillman I am a bit reluctant to put more effort into my python library before the change to electron.

The library is pretty complete as far as creating items, notes and attachments etc., but there is still a lot of old code left from when I switched from MozRepl to the debug-bridge. I don't want to clean it up just to run into incompatibility issues.

Do you think it would be possible to create a minimal, stable, debug-bridge like functionality that was directly integrated in Zotero and that will remain stable when the switch to Electron happens?

This would make it very easy and less of a hack to create a local API in Python. I think there is a lot of potential for adding and processing items with notes, tags, and attachments, but also for running ML stuff on the database to find similar papers, add a citation graph etc. There are lots of libraries for analytics in Python, but currently there is no good way to connect them to Zotero.

b.n.lawrence

I'd really like to have my workflow built around something in Python, which talks to some local stable API on the standalone version, so big +1 for me for this!

dstillman

Let's continue this on zotero-dev.

yanghaozhe

you can try my lib,

see https://forums.zotero.org/discussion/87671/a-new-python-tool-kit-for-interacting-with-the-locally-hosted-zotero-database#latest

and https://github.com/sailist/pyzotero-local