Zotero Connector metadata for AI chats from different chatbots is inconsistent

edited 3 days ago
It has previously been suggested that Zotero needs a new item type for AI outputs, for example for chats with AI chatbots.

Until that happens (and I think it should), the situation as it stands now is that Zotero will save an AI chat as a 'web page' item type*. The problem with that is that each company's chats are saved differently by Zotero, often with missing data. It would be great if that inconsistency could be fixed.

* I am not talking here about saving the *contents* of AI chats, just the metadata (ie that would allow a chat output to be cited when writing). How to save complete chat contents is a whole other issue, for which it seems there is no ideal, universal solution (a web snapshot will often not show the full chat history, and not all chatbots have a Save to PDF option).

Google Gemini chat metadata are typically saved by the Connector with the chat's correct title in the Title field, and 'Google' as the author, but there is no date saved, and the chat's URL is not entered in the URL field. 'Website type' is 'Generative AI chat'.

Claude chats are instead saved with no author, but the correct title *and* the correct (private) URL are saved. There is also a date saved, which appears to be the last date on which anything was written to the chat. 'Website type' is empty.

Note that if AI chats are being cited in writing (it remains to be seen how common that will become), the chat's *public* 'share' URL would be required - so that readers can access the exact chat content (before that facility became more generally available, its lack was previously a common objection to citing AI chats at all, or instead citing them as Personal Communications).

To get at item with the public URL as well as the original one with the private URL, one would currently have to duplicate the original item and replace the URL with the public one (a problem for future web scraping is that a public/share chat URL typically has to be explicitly requested from the UI). The person who created the chat is likely to want to keep a record of *both* the private and public URLs - a 'proper' AI chat item type would have fields for both, along with other important elements like the specific model used. The way in which AI chats should be cited is still evolving under the different formats, but would be more robustly implementable in Zotero if a specific AI chat item type were available, eg
https://apastyle.apa.org/style-grammar-guidelines/references/examples/ai-references#2

An example of a very poor save is Deepseek's chats - all you get in Zotero is a generic web page item, with a title of 'Deepseek' instead of the chat's actual title, no author, no date, and the URL is simply chat.deepseek.com (ie not the actual chat's specific URL).

So with all AI chats saved to Zotero, there is a lot of manual metadata entry/editing required. eg
https://guides.qatar.georgetown.edu/c.php?g=575091&p=11211130

I use a range of AI chatbots, as I am still figuring out which work best for the type of scientific material I work with (as well as minimising my use of token quotas for any single chatbot). But beyond the above examples, I have not created an exhaustive list of how every AI vendor's chats are saved by the Connector. IIRC they *all* get saved somewhat differently, and inadequately. The chatbots I use include:
Gemini gemini.google.com
Claude claude.ai/chat
ChatGPT chatgpt.com
Deepseek chat.deepseek.com
Z.ai chat.z.ai
Kimi www.kimi.ai/chat
Meta meta.ai
Grok grok.com
Qwen chat.qwen.ai
Sign In or Register to comment.