Zotero Connector metadata for AI chats from different chatbots is inconsistent
It has previously been suggested that Zotero needs a new item type for AI outputs, for example for chats with AI chatbots.
Until that happens (and I think it should), the situation as it stands now is that Zotero will save an AI chat as a 'web page' item type*. The problem with that is that each company's chats are saved differently by Zotero, often with missing data. It would be great if that inconsistency could be fixed.
* I am not talking here about saving the *contents* of AI chats, just the metadata (ie that would allow a chat output to be cited when writing). How to save complete chat contents is a whole other issue, for which it seems there is no ideal, universal solution (a web snapshot will often not show the full chat history, and not all chatbots have a Save to PDF option).
Google Gemini chat metadata are typically saved by the Connector with the chat's correct title in the Title field, and 'Google' as the author, but there is no date saved, and the chat's URL is not entered in the URL field. 'Website type' is 'Generative AI chat'.
Claude chats are instead saved with no author, but the correct title *and* the correct (private) URL are saved. There is also a date saved, which appears to be the last date on which anything was written to the chat. 'Website type' is empty.
Note that if AI chats are being cited in writing (it remains to be seen how common that will become), the chat's *public* 'share' URL would be required - so that readers can access the exact chat content (before that facility became more generally available, its lack was previously a common objection to citing AI chats at all, or instead citing them as Personal Communications).
To get at item with the public URL as well as the original one with the private URL, one would currently have to duplicate the original item and replace the URL with the public one (a problem for future web scraping is that a public/share chat URL typically has to be explicitly requested from the UI). The person who created the chat is likely to want to keep a record of *both* the private and public URLs - a 'proper' AI chat item type would have fields for both, along with other important elements like the specific model used. The way in which AI chats should be cited is still evolving under the different formats, but would be more robustly implementable in Zotero if a specific AI chat item type were available, eg
https://apastyle.apa.org/style-grammar-guidelines/references/examples/ai-references#2
An example of a very poor save is Deepseek's chats - all you get in Zotero is a generic web page item, with a title of 'Deepseek' instead of the chat's actual title, no author, no date, and the URL is simply chat.deepseek.com (ie not the actual chat's specific URL).
So with all AI chats saved to Zotero, there is a lot of manual metadata entry/editing required. eg
https://guides.qatar.georgetown.edu/c.php?g=575091&p=11211130
I use a range of AI chatbots, as I am still figuring out which work best for the type of scientific material I work with (as well as minimising my use of token quotas for any single chatbot). But beyond the above examples, I have not created an exhaustive list of how every AI vendor's chats are saved by the Connector. IIRC they *all* get saved somewhat differently, and inadequately. The chatbots I use include:
Gemini gemini.google.com
Claude claude.ai/chat
ChatGPT chatgpt.com
Deepseek chat.deepseek.com
Z.ai chat.z.ai
Kimi www.kimi.ai/chat
Meta meta.ai
Grok grok.com
Qwen chat.qwen.ai
Until that happens (and I think it should), the situation as it stands now is that Zotero will save an AI chat as a 'web page' item type*. The problem with that is that each company's chats are saved differently by Zotero, often with missing data. It would be great if that inconsistency could be fixed.
* I am not talking here about saving the *contents* of AI chats, just the metadata (ie that would allow a chat output to be cited when writing). How to save complete chat contents is a whole other issue, for which it seems there is no ideal, universal solution (a web snapshot will often not show the full chat history, and not all chatbots have a Save to PDF option).
Google Gemini chat metadata are typically saved by the Connector with the chat's correct title in the Title field, and 'Google' as the author, but there is no date saved, and the chat's URL is not entered in the URL field. 'Website type' is 'Generative AI chat'.
Claude chats are instead saved with no author, but the correct title *and* the correct (private) URL are saved. There is also a date saved, which appears to be the last date on which anything was written to the chat. 'Website type' is empty.
Note that if AI chats are being cited in writing (it remains to be seen how common that will become), the chat's *public* 'share' URL would be required - so that readers can access the exact chat content (before that facility became more generally available, its lack was previously a common objection to citing AI chats at all, or instead citing them as Personal Communications).
To get at item with the public URL as well as the original one with the private URL, one would currently have to duplicate the original item and replace the URL with the public one (a problem for future web scraping is that a public/share chat URL typically has to be explicitly requested from the UI). The person who created the chat is likely to want to keep a record of *both* the private and public URLs - a 'proper' AI chat item type would have fields for both, along with other important elements like the specific model used. The way in which AI chats should be cited is still evolving under the different formats, but would be more robustly implementable in Zotero if a specific AI chat item type were available, eg
https://apastyle.apa.org/style-grammar-guidelines/references/examples/ai-references#2
An example of a very poor save is Deepseek's chats - all you get in Zotero is a generic web page item, with a title of 'Deepseek' instead of the chat's actual title, no author, no date, and the URL is simply chat.deepseek.com (ie not the actual chat's specific URL).
So with all AI chats saved to Zotero, there is a lot of manual metadata entry/editing required. eg
https://guides.qatar.georgetown.edu/c.php?g=575091&p=11211130
I use a range of AI chatbots, as I am still figuring out which work best for the type of scientific material I work with (as well as minimising my use of token quotas for any single chatbot). But beyond the above examples, I have not created an exhaustive list of how every AI vendor's chats are saved by the Connector. IIRC they *all* get saved somewhat differently, and inadequately. The chatbots I use include:
Gemini gemini.google.com
Claude claude.ai/chat
ChatGPT chatgpt.com
Deepseek chat.deepseek.com
Z.ai chat.z.ai
Kimi www.kimi.ai/chat
Meta meta.ai
Grok grok.com
Qwen chat.qwen.ai
Upgrade Storage
At the time we added the initial Claude/Gemini/ChatGPT translators, the consensus seemed to be that it was correct to cite AI conversations as webpages. @adamsmith / @bwiernik, is that still the best practice?
I think changes to the Claude web UI broke that translator; we'll fix it. Gemini chats do save with URLs and dates, but currently only when you share the chat and save from the share link. I'll look into whether we can improve on that.
We don't support any of those other services, but we can work on it.
I'm not really sold on importing the private URL for a reference manager.
The ChatGPT import currently produces the exact APA recommended citation out of the box (and also makes sense more generally), so that's a good template for how we should import these.
APA calls for the AI company to be listed as the Author and for the model name to be given in the position where we show Website Title. APA also calls for the full date of creation (Date), title/summary of the chat content (Title), the publicly accessible URL (URL), and “Generative AI chat” or another description (Website Type).
AI Company Name. (year, month day). Title of chat in italics [Description, such as Generative AI chat]. Tool Name/Model. URL of the chat
@adamsmith How do you feel about auto-populating the type field with something like “Generative AI chat” or “Generative AI output”?
I agree that private URLs shouldn’t be saved if we can’t retrieve or create public ones
https://s3.amazonaws.com/zotero.org/images/forums/u2433/8p65vptroyb6weyeqc63.png
@mjthoraval -- right, if there's demand for that, I think that'd make a lot more sense than a dedicated field.