Local LLM with Zotero as Knowledgebase

rac8f

I am following the steps in this video ( https://www.youtube.com/watch?v=b2BSZfOtD_w ) to experiment with using a local LLM to query items in Zotero. (Ollama + Docker + OpenwebUI + Zotero)
I have been a Zotero user for many years through many projects... and I don't want to use my entire library as the LLM knowledgebase. Instead, I'd like to use the content of a single collection.

The instructions in the video are to sync the Zotero storage folder as the source of the LLM knowledge. However, there doesn't seem to be a way to link to the storage for a single collection because of the way that Zotero stores items (and the fact that Zotero storage doesn't mimic the collection structure).

Any thoughts on a possible workaround? I don't believe there's a way to change the storage settings on a single collection (but correct me if I'm wrong!)

Many thanks!

matt_price

I'd be interested to hear whether you figured anything out!

wayt

I recently used Microsoft Copilot to create a custom LLM (MS calls it a "custom agent") from a 1,500+ item Zotero collection. It works well, but did require a substantial workaround. The problem is the Copilot agent creation tool, which is included in (some) MS365 subscriptions, has low limits on the number and size of files that can be uploaded as "custom knowledge." However, it can instead be aimed at a SharePoint library of any size (or at least very large size).

So the challenge I faced was to import my Zotero items into a custom SharePoint library. In tests on a small number of items, I found that simply uploading PDFs without their Zotero metadata into a SharePoint folder resulted in unsatisfactory answers to queries; Copilot had a hard time identifying relevant documents. (This may change as Copilot is continually evolving.) But I got quite satisfactory performance from the AI when I created a custom SharePoint library that has a dozen or so fields corresponding to the most relevant metadata fields in my Zotero collection (e.g. authors, item title, publication title, publication year, and tags) in addition to the PDF.

There may be a better way to do it, but the solution I used was to create an automated workflow using Microsoft PowerAutomate Desktop (for Windows) that uploads each item in the Zotero collection to SharePoint and populates each SharePoint column for the item with corresponding metadata from Zotero. Handling tags was the most difficult part, as it required first adding all the needed tag values to a custom metadata store on the SharePoint server. I was able to complete the task with only site admin privileges; I do not have server admin privileges.

So if your organization has an enterprise MS365 license or your personal MS365 account includes SharePoint, this is an option that doesn't require learning how to install, train, and run an LLM. I am hopeful and moderately confident that my custom agent will also continue to improve "automagically" as Microsoft continually updates the AI system behind the scenes. But it's already quite good, allowing me for example to quickly find specific passages in relevant papers (including very long docs, such as the IPCC AR6 reports) that discuss questions I am researching.

Bial77

You can give zotai.app a try to achieve this

wayt

@rac8f That YouTube method you highlighted is very cool. Here's a workaround to get all the PDFs from a collection into a single folder so that your local LLM can hoover them up:
In Zotero, use the pull-down menu on the magnifying glass icon to do an advanced search. In the search form, select the library and then under the search criteria choose match "all" where Collection is [desired_collection_name] and attachment file type is PDF. (Use the + button at the end of first search criterion row to add a second row.) Check the "search subcollections" box if desired to include folders within the collection. Also check "include parent and child items of matching items." The results list should show all PDFs in that collection. Highlight one and hit cmd-A (on Mac) to select all the search results. Then simply click and drag the highlighted items and drop them in an empty finder folder. You might have to move slowly to give Zotero time to process if the list of PDFs is long. The PDFs will get copied into the destination folder. This worked for me on a collection that includes over a dozen folders and more than 1,500 PDFs. I've only tried it on a Mac so far, not positive it works on Windows.