The possibility of integrating AI into zotero to assist reading

dongfanghong
In the past two months, we have seen the exciting applications of GPT-3. Among them, Explainpaper(https://www.explainpaper.com/) and Scispace(http://scispace.com/) are tools to facilitate paper reading, providing quite good experience. But they cannot effectively integrate with zotero. So, I just wonder wether there is any possibility of integrating AI into zotero, e.g. through plugins or built-in functions.
«1
  • emilianoeheyns
    That would depend entirely on whether those tools have APIs Zotero could talk to. I don't readily see they do.
  • GabeTheEngineer
    I wonder if the map_reduce chain from this project: https://langchain.readthedocs.io/ (or the similar implement in this project: https://github.com/whitead/paper-qa/blob/main/paperqa/docs.py) would offer ideas.

    It would certainly be nice to search my library by embedding rather than trying to guess the right word.
  • rabitai
    LangChain has a JS implementation (https://github.com/hwchase17/langchainjs) which is perhaps easier to integrate? Paper-QA is based on LangChain too.
  • jiayipan
    Hi I am working on a Zotero extension on the exact feature ( paper-qa like experience within Zotero), will try to release it within next month:)
  • sheldon123z
    @jiayipan when you release the extension, please let us know, that's would be very cool
  • rabitai
    I created an external app (hosted in Hugging Face) that interacts with Zotero APIs to perform paper QA: https://huggingface.co/spaces/lifan0127/zotero-qa

    I thought about a Zotero plugin for the same purpose, but I believe the underlying langchainjs library is not compatible with the Zotero JavaScript environment (?).

    @jiayipan I'd be interested in your approach. It would be a very useful plugin for sure.
  • jiayipan
    Hi @rabitai, your work is impressive!

    I have made some progress with creating a minimal working example for the extension. However, before releasing it as a useful extension, there's a lot of polishing work to do. I'll try to release it in the following weeks.

    Indeed, it seems that langchain.js is not currently compatible with Zotero. Luckily, as a researcher in NLP, I am able to build a tiny infrastructure from scratch.
  • destroy314
    another extension: https://github.com/MuiseDestiny/zotero-gpt
  • abdelatif74
    Hi @jiayipan, any updates on there Zotero extension you are working on to integrate ChatGPT in Zotero?
    It would be great if this extension can combine multiple papers/sources like a folder for example.
  • jiayipan
    edited April 4, 2023
    Hi @abdelatif74, thank you for your interest in this! I've finished most of the infrastructure (vector database, LLM API, TextChunk Utils) and is mainly working on the interface. If everything went well, I will try to release an alpha version early next week.

    I imagine the first version to have the following two features:
    1. Library Chat: where the LLM is connected to your entire library, and answers questions based on them
    2. Paper Chat: where the LLM is connected to the PDF you opened, and answers questions based on it

    The feature you suggest sounds interesting and is definitely doable. I'll try to add that later.
  • atomicnature
    @jiayipan - Excited to hear about your progress! Please post here if you need any sort of assistance (technical or otherwise); I'll be willing to contribute efforts.
  • atomicnature
    Also an idea - I think Collection Level chat will be super useful. I tend to have at least some primitive organization, and it's best if the LLM can make use of existing organization.
  • Hydraginium
    @jiayipan I was just thinking about such a plugin! I am really excited about your work as well, and eager to test it!
  • ashleyriott
    @jiayipan I made an account to say this is exactly the plugin I am searching for as a research student/public speaker. I'd also love to subscribe for updates and happy to help in any way I can
  • rabitai
    Hi everyone, I am building Zotero RA, a GPT-based Chatbot Add-on. It supports search, QA and summarization based on your Zotero library. My plan is to also add other functionalities such as reference management and data visualization over time.

    Please give it a try if you are interested: https://github.com/lifan0127/zotero-research-assistant
  • EdilsonGA
    Hi @rabitai. I was pleasantly surprised by the creation of your plugin. I'm already testing it and having the same errors as other colleagues who opened an issue. I also made a note there about using other templates for tasks.
  • rabitai
    @EdilsonGA, Thank you for your feedback! I am able to reproduce some of the errors reported in GitHub. Interaction with Zotero in an open, conversational manner through a probabilistic model can lead to many different, often unexpected outcomes. I appreciate all the comments to improve/harden the system and will address the specific issues in GitHub.
  • EdilsonGA
    In time, did you see that the paper-qa update now brings the convenience of interacting with Zotero through ZoteroDB?? Tonight I will try to replicate the example given by Prof. White. He has included a module called "Contrib" for this purpose. But I repeat what I told you before, @rabitai, my knowledge in python is very limited.
  • rabitai
    @EdilsonGA Yes☺️ As I constantly refer to the paper-qa package for inspiration, I noticed that Zotero connection was added last week. However I have not tested it yet.
  • polygon
    I made this http://github.com/MuiseDestiny/zotero-gpt, maybe you can try it.
  • nectarinebridge
    +1 @jiayipan, also interested to check out your extension!
  • Marcelo Oppermann
    @jiyapan, Sign me in as a supporter.
  • smatthie
    @polygon Is there another way of opening the extension? On a German keyboard, there is no /? key and I can't find which key is supposed to be the shortcut, so I had to find a US keyboard.

    Also, is there some more documentation? I could not find out how I can use the extension. I set up the secret key and everything, but what do I actually have to do to get a summary of a paper or find some papers in my library with a question. Can you explain this in more detail?
  • smatthie
    I have tried both add-ons (ARIA by @rabitai and Zotero-GPT by @polygon) and they are both very interesting.

    I want to give some feedback and hope that the developers find it useful. I don't want to be negative, it's really meant to point out issues that you can make better. It's hard work and this is a completely new field, so it's normal that there are many issues at the beginning.

    It is hard to say how useful they are at the moment. I had some practical issues and often both didn't give useful answers or missed papers in the collection, but I don't know if this is a problem with the add-on (and how it interacts with Zotero) or with ChatGPT. At other times, they worked very well.

    In both add-ons I found it hard to move between the chat and Zotero. So if I'm looking at one paper and want to ask questions about it, it's a hard to move to the plugin; especially ARIA doesn't (at present) know which paper I'm looking at so you have to type in the reference first. The other way round, when you ask the chat for papers about a topic it returns a list (often incomplete..) but then it is hard to go from the list to the actual items in Zotero.

    I would also like some standard questions like "Summarise this article" and "What are the main conclusions?" quickly accessible from the context menu of an article, without having to type this.

    So overall they are both very promising and I'm looking forward to more development. This can become really useful.
  • smatthie
    @jiyapan Your add-on also sounds very interesting! Is it available for testing?
  • DemianDu
    Guys, I need this feature so badly but find everything proceeds too slow so I decided to develop it myself.

    I have no idea how to develop a Zotero plugin, so what I did is to develop a PDF reader integrated with ChatGPT. In this way, I can customize whatever I want in the future and I do not to compromise to any limitations or Zotero. Also, I use pdfGPT, which can cite the page that supports the bot's response, which makes it perfect for serious paper reading.

    Simply by setting this PDF reader as default reader used for Zotero, I now manage to read papers 10X faster than before!

    The app is only for MacOS (because I mainly design it for myself). You can find more information in this repo: https://github.com/Yiiipu/SimplePDF.

    It is free but you need to bring your own API Key. I plan to open source it after a few updates.
  • vladislav.ivanistsev
    pdfgear is probably what you want. One can open pdf's in pdfgear and use the build-in AI chat.
  • vladislav.ivanistsev
    Here is my solution https://twitter.com/sci_strategist/status/1709116085851594841 Thanks to https://github.com/retorquere/zotero-open-pdf
  • Marcelo Oppermann
    I loved, thanks vlsadislav.ivanistsev
  • LeighMiller2023
    thanks for doing all this
Sign In or Register to comment.