I recently used Microsoft Copilot to create a custom LLM (MS calls it a "custom agent") from a 1,500+ item Zotero collection. It works well, but did require a substantial workaround. The problem is the Copilot agent creation tool, which is included in (some) MS365 subscriptions, has low limits on the number and size of files that can be uploaded as "custom knowledge." However, it can instead be aimed at a SharePoint library of any size (or at least very large size).

So the challenge I faced was to import my Zotero items into a custom SharePoint library. In tests on a small number of items, I found that simply uploading PDFs without their Zotero metadata into a SharePoint folder resulted in unsatisfactory answers to queries; Copilot had a hard time identifying relevant documents. (This may change as Copilot is continually evolving.) But I got quite satisfactory performance from the AI when I created a custom SharePoint library that has a dozen or so fields corresponding to the most relevant metadata fields in my Zotero collection (e.g. authors, item title, publication title, publication year, and tags) in addition to the PDF.

There may be a better way to do it, but the solution I used was to create an automated workflow using Microsoft PowerAutomate Desktop (for Windows) that uploads each item in the Zotero collection to SharePoint and populates each SharePoint column for the item with corresponding metadata from Zotero. Handling tags was the most difficult part, as it required first adding all the needed tag values to a custom metadata store on the SharePoint server. I was able to complete the task with only site admin privileges; I do not have server admin privileges.

So if your organization has an enterprise MS365 license or your personal MS365 account includes SharePoint, this is an option that doesn't require learning how to install, train, and run an LLM. I am hopeful and moderately confident that my custom agent will also continue to improve "automagically" as Microsoft continually updates the AI system behind the scenes. But it's already quite good, allowing me for example to quickly find specific passages in relevant papers (including very long docs, such as the IPCC AR6 reports) that discuss questions I am researching.