A new tool to write Zotero select links into PDF files

mhucka

When using Zotero for reference management, you may on occasion want to work with the PDF files from outside of Zotero. However, when viewing or manipulating the PDF files from outside of Zotero, you may run into the following problem: when looking at a given PDF file, how do you find out which Zotero entry it belongs to?

I developed a free open-source utility, Zowie [1,2], to help deal with this problem. Zowie scans through the files in a local Zotero database, looks up the Zotero bibliographic record corresponding to each PDF file found, and writes a Zotero select link [3] into the PDF file and/or certain macOS Finder/Spotlight metadata fields (depending on the user's choice). A Zotero select link has the form "zotero://select/..." and when opened on macOS, causes the Zotero desktop application to open that item in your database. Zowie thus makes it possible to go from a PDF file opened in an application other than Zotero (e.g., DEVONthink, Adobe Acrobat), to the Zotero record corresponding to that PDF file.

As an example of use, in DEVONthink [4], I index the folder containing my local Zotero database of PDF files. Using Zowie to write the Zotero select link into the Finder comments of every PDF file means that within DEVONthink, the Zotero link appears in a DEVONthink inspector panel for the file. Thus, while viewing a PDF file in DEVONthink, I can quickly go to the corresponding entry in my Zotero database.

Zowie is a command-line program written in Python 3. At the moment, Zowie is aimed at Mac users, and needs the user to have a Python environment on their computer in order to run. Zowie is available from PyPI for installation via pip; it can also be obtained from the project home on GitHub [1,2].

[1] https://mhucka.github.io/zowie/
[2] https://pypi.org/project/zowie
[3] https://forums.zotero.org/discussion/78053/given-the-pdf-file-of-an-article-how-can-you-find-out-its-uri
[4] https://www.devontechnologies.com/apps/devonthink

rpallred

Just found this, downloaded, and tried it--great work!.

FYI: am on Catalina (10.15.7) and ran the binary just fine. One popup window asking if I wanted to allow it.

mhucka

That's great to hear! Thanks for the note.

I'm really surprised you could run the binary, though; on my test system, macOS refused to let me run it after downloading from GitHub. Have you done anything to change security settings in your macOS installation?

rpallred

I have it set to allow apps downloaded from App Store and identified developers. But other than that, nothing special that I can think of.

I downloaded from GitHub, unzipped, entered Terminal

cp ~/Downloads/zowie /usr/local/bin/

Then started Zowie. There was a popup asking me if I wanted to allow Zowie to access my files (or something to that effect), and it proceeded as expected. Entered my Zotero keys. Did a trial run, then ran it on all PDFs, verified that the Finder comments had been updated in DEVONthink.

mhucka

@rpallred Thanks for the detailed reply, and my apologies for taking so long to acknowledge it.

rpallred

No worries! Still working like a dream.

danieltomasz

Great tool! Does it works with pdfs that are moved by Zotfile?

mhucka

@danieltomasz Yes. I actually use Zotfile myself. Note, however, that Zowie does not internally implement a file-watching scheme, so you'll have to handle running Zowie after additions are made to your Zotero library. Some people use Hazel to do it (e.g., https://discourse.devontechnologies.com/t/a-new-tool-for-zotero-users/60025/6); I happen to use DEVONthink and it has something called Smart Rules that can be used to do it (c.f. https://discourse.devontechnologies.com/t/a-new-tool-for-zotero-users/60025/5).

There's one issue to worry about with Zotfile: when you add an item to Zotero, it's written to the database on disk using the original file name, then Zotfile kicks in and renames it. So, whatever file-watching scheme you use, you may want to introduce a delay between the time an addition triggers it and the time it runs Zowie on the file, to get the final file name.

(I realize it would be better to have Zowie do its own file watching, and I've wanted to implement that, but I'm limited by lack of time ...)

danieltomasz

I use Zotflie to move pdf to folder outside Zotero database. Does Zowie support pdfs stored outside Zotero database?


I tried to run Zowie both on my Zotero databse folder and my dropbox folder where I store pdfs. In case of Zotero database Zowie doesnt see pdfs that are stored outside folder, in case of Dropbox folder Zowie was unable to retrieve Zotero records for pdfs.

(Zowie is great idea, I use Python script to clean both my Zotero folder and my Zotfile storage folder from pdfs that are not linked to database or are not moved by Zotfile because the dont have parent item etcetera, I think I could learn much from your package :)

danieltomasz

A functionality that I am looking for now is a way to write specific tags from Zotero to finder/DT, this way I would have a way to recreate my collections in DT (via smart rules, file has specific tag -> replicate to folder) .

Could your package be easily extended to have such functionality?

mhucka

@danieltomasz With respect to to "in case of Dropbox folder Zowie was unable to retrieve Zotero records for pdfs" (https://forums.zotero.org/discussion/comment/373730/#Comment_373730), yeah, if you use linked attachments and store them somewhere else, there's no way to figure out the parent record where the PDF file belongs. I previously asked about this on the forums here (https://forums.zotero.org/discussion/comment/372519/#Comment_372519) and have documented this as a limitation in a couple of places in the Zowie web page (https://mhucka.github.io/zowie/).

With respect to extending Zowie to handle tags, in fact, I've already been thinking about the same, but in a more generalized way, to handle not just tags but any Zotero field. I started writing Zoinks (https://github.com/mhucka/zoinks) for that reason, but am not far along because of heavy time commitments on other things. I need Zoinks for myself, so I expect I will write it, but can't promise _when_ ...

bookhand

@mhucka - just FYI, the Alfred workflow zothero (https://github.com/deanishe/zothero ) overlaps with some of the functionality you're describing for zoinks (search by field).

mhucka

In case anyone is interested, I've released a new version of Zowie (1.2.0) that fixes a software compatibility issue, adds a new feature that will be of interest to DEVONthink users, has faster startup time, and comes with single-file self-contained runnable binaries for macOS to make it easier to install. See https://mhucka.github.io/zowie/.

mhucka

I've (finally) written up some notes about how I use Zotero in conjunction with DEVONthink. They are at https://github.com/mhucka/zowie/wiki

Ronan Le Coadic

Hello Mike,
I was trying to process my bibliographic references from Zotero in Devonthink and I discovered that you had created this program, Zowie. Unfortunately, I'm not good (at all) at computers...
I managed to get as far as "Welcome to Zowie: ZOtero link WrItEr" in the Terminal of my Mac. But then, when I saw that I was asked for an "API key", I didn't understand. I tried several passwords. Then I went back to https://mhucka.github.io/zowie/ and I eventually understood that I had to give my Zotero ID and my Zotero key. So I tried to do it but it didn't work... Now, qhen I get to "Need at least one folder path or file as argument" and give the path of my Zotero storage, I get the answer: "permission denied". And when I type my Zotero ID and API key (which, now, I'm sure are good), I get the answer: "Unable to connect to Zotero: invalid ID and/or API key".
Could you please advise me? Thanks in advance!
All the best,
Ronan

stenti1

@"Ronan Le Coadic" Did you ever find a resolution to your issue? I am running into a similar problem.

I have located my Zotero Id and I have used the command zowie -i I [my id]. I have also created a new private API key in Zotero. Then, I entered that key using the command zowie -i I [my new private key]. When I enter both, I get the response, "Unable to connect to Zotero: invalid ID and/or API key."

@mhucka Is there a correct way to uninstall zowie and reinstall it to try again? I have tried deleting zowie from /usr/local/bin. When I reinstall it, it doesn't prompt setup, as it did on the first run. Any tips you have would be greatly appreciated.

Kind regards,
Dave

mhucka

@stenti1 Can you tell me the method you used to install it in the first place (since there are different options for Zowie)? And also, what is the version of your macOS operation system?

Regarding API keys: I didn't realize it was so confusing or I would have tried to provide clearer instructions. There are two separate command line options for telling Zowie new values of the user id and api key: -i, and -a. The first (-i) is for giving the Zotero user ID, and the second (-a) is for giving the API key. To force Zowie to remember new values, just run it again with the flags followed by the new values, more or less like this:

    zowie  -i myuserid  -a myapikey  ~/Zotero

An actual value for "myuserid" will be a 7 or maybe 8 digit number like "7340123". A value for "myapikey" will be look like "1ydtqmMZnWkxuLYFoanDJpJe", so you would type something like this (don't use these values -- these are just fake examples):

    zowie  -i 7340123  -a 1ydtqmMZnWkxuLYFoanDJpJe  ~/Zotero

They are stored in the macOS keychain so that you don't have to keep supplying them on every run. On subsequent invocations of Zowie, you can just type

    zowie ~/Zotero

and it should work.

If you continue to have trouble with the permissions, you may be able to get more information about why it's happening by giving the debug flag to Zowie:

    zowie  -@ -  ~/Zotero

Note the dash alone by itself after -@. It means "send the output to the terminal". Instead of the dash, you can send the output to a file, for example like this:

    zowie  -@ ~/zowie-debug.txt  ~/Zotero

The debug output is very technical, but it may be possible for you to infer the cause of a problem by reading what Zowie logged near the end of the debug output, shortly before it quit.

stenti1

Hello @mhucka,

Thank you so much for your detailed response. The first command you suggested above appears to be working for me:

zowie -i myuserid -a myapikey ~/Zotero

I think I may have been doing several things wrong. First, I was entering my user id separately from my API key. Second, I was also adding "I" and "A" after the flags "-i" and"-a," respectively. Third, I was not adding "~/Zotero" to the end of the commands.

As for my install, I was downloading the ready-to-run program. I am on macOS Monterey, Version 12.6, running Zotero 6.0.16-beta.3+29dd0cf5d.

I greatly appreciate your support!

Kind regards,
Dave

mhucka

@stenti1 Thank you for explain what you did! What you wrote is extremely useful for me to know, because now I see another way that the instructions are confusing and this helps me know what to change.

It's great news that you were able to make it work!

stenti1

@mhucka Super. Thanks again for the support. I have been using Zowie for a few weeks now. Thank you for making a terrifically helpful tool.

markmusselman

I know that this is an old discussion but I thought I'd try. I'd like to run Zowie but I'm afraid I'm lost. I use Devonthink and Zotero independently without problems - but when it comes to script I have no idea what to do. I've read through https://github.com/mhucka/zowie/wiki/Example:-using-Zowie-with-DEVONthink and would like to try this method - but while I can manage setting up smartrules in Devonthink I don't now what to do - or where to put - the script. Can someone fill me in? Thanks.