Upgrade StorageZotero PDF reader and Annotations to Markdown Workflow
I am posting this new topic as requested by @dstillman...
---
Hi, your new PDF reader interface is really good (although it does seem to have a minor bug in missing the occasional letter from the highlighted text).
I would love to see better markdown export capability.
My use case is that I want to annotate my PDFs and get the annotations into Obsidian (and really any other app that handles markdown) with links back to my Zotero reference library to open the specific annotation locations in the source PDF in my Zotero library.
---
The way I do it today is with the zotfile and mdnotes plugins. These are the steps I currently go through and it works great:
1. Annotate the PDF
2. Extract the annotations from the PDF using Zotfile
3. Export the annotations note(s) to markdown using mdnotes
4. Open the markdown file and copy and paste it into the notes app (in this case Obsidian, but this would work for Bear, Roam, etc.)
5. Have a nicely formatted note in the notes app (Obsidian) with all the links correct so that when I click on them, it opens the PDF inside Zotero at the location of the annotation
---
With the current beta of the PDF reader, this doesn't currently work. I have tried two ways getting the annotation note that I created in PDF reader into Obsidian:
1. Select all the text from within the annotations note created from Zotero PDF Reader and paste into Obsidian. The look of it is not bad, but all the links to the original PDF are lost. The loss of the links to the PDF location is the main issue here. Also, any images that have been captured using the Select Area tool copy over and render okay in Obsidian, but in the raw markdown mode the image description is hundreds of lines of gobledy-gook -- a subset example of which is:
![(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdoAAACqCAYAAAAKsY7aAAAgAElEQVR4nO3deVxN+eM/8NtmH5NiZN/NZGc0EyJrNINSI7IMY4mQQeEzYcjYZswMM8aStT
...and it goes on...
2. Use mdnotes to export the annotations note created from within PDF Reader to a markdown file. This file is a mess format-wise and also loses all the links back to the pdf.
---
Current workaround which is very undesirable:
1. Annotate in Zotero PDF Reader
2. Export the PDF so that the PDF Reader highlights get embedded into the PDF
3. Reimport the PDF with highlights into Zotero
4. Use my original workflow to use Zotfile to extract annotations and use mdnotes to get markdown and then copy and paste into Obsidian.
---
Desired workflow:
1. Annotate in Zotero PDF Reader
2. Add Item Note from Annotations
3. Right click on the item note and either copy as markdown (so I can then paste into Obsidian without creating a new file) or save as markdown and it shows up as a new item under the parent in my library (which I could subsequently copy/paste into Obsidian.
Key requirements for this desired workflow:
- The pasted/created markdown should include the reference links back into the PDF where the annotation came from. So in Obsidian, I should be able to click on the link for the highlight and it should open up the PDF file in Zotero on the right page (which is basically the "Go to Page" link in the Zotero PDF Reader Annotation note).
- Also, this should export any images that are in the Zotero Item note so they show up in Obsidian (these are the images captured by Zotero PDF Reader in the item note when using the Select Area function). But somehow these need to come over in a way where they are image files and not hundreds of lines of gibberish.
---
Even more desired: create an Obsidian plugin to do all this automatically.
---
Hi, your new PDF reader interface is really good (although it does seem to have a minor bug in missing the occasional letter from the highlighted text).
I would love to see better markdown export capability.
My use case is that I want to annotate my PDFs and get the annotations into Obsidian (and really any other app that handles markdown) with links back to my Zotero reference library to open the specific annotation locations in the source PDF in my Zotero library.
---
The way I do it today is with the zotfile and mdnotes plugins. These are the steps I currently go through and it works great:
1. Annotate the PDF
2. Extract the annotations from the PDF using Zotfile
3. Export the annotations note(s) to markdown using mdnotes
4. Open the markdown file and copy and paste it into the notes app (in this case Obsidian, but this would work for Bear, Roam, etc.)
5. Have a nicely formatted note in the notes app (Obsidian) with all the links correct so that when I click on them, it opens the PDF inside Zotero at the location of the annotation
---
With the current beta of the PDF reader, this doesn't currently work. I have tried two ways getting the annotation note that I created in PDF reader into Obsidian:
1. Select all the text from within the annotations note created from Zotero PDF Reader and paste into Obsidian. The look of it is not bad, but all the links to the original PDF are lost. The loss of the links to the PDF location is the main issue here. Also, any images that have been captured using the Select Area tool copy over and render okay in Obsidian, but in the raw markdown mode the image description is hundreds of lines of gobledy-gook -- a subset example of which is:
![(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdoAAACqCAYAAAAKsY7aAAAgAElEQVR4nO3deVxN+eM/8NtmH5NiZN/NZGc0EyJrNINSI7IMY4mQQeEzYcjYZswMM8aStT
...and it goes on...
2. Use mdnotes to export the annotations note created from within PDF Reader to a markdown file. This file is a mess format-wise and also loses all the links back to the pdf.
---
Current workaround which is very undesirable:
1. Annotate in Zotero PDF Reader
2. Export the PDF so that the PDF Reader highlights get embedded into the PDF
3. Reimport the PDF with highlights into Zotero
4. Use my original workflow to use Zotfile to extract annotations and use mdnotes to get markdown and then copy and paste into Obsidian.
---
Desired workflow:
1. Annotate in Zotero PDF Reader
2. Add Item Note from Annotations
3. Right click on the item note and either copy as markdown (so I can then paste into Obsidian without creating a new file) or save as markdown and it shows up as a new item under the parent in my library (which I could subsequently copy/paste into Obsidian.
Key requirements for this desired workflow:
- The pasted/created markdown should include the reference links back into the PDF where the annotation came from. So in Obsidian, I should be able to click on the link for the highlight and it should open up the PDF file in Zotero on the right page (which is basically the "Go to Page" link in the Zotero PDF Reader Annotation note).
- Also, this should export any images that are in the Zotero Item note so they show up in Obsidian (these are the images captured by Zotero PDF Reader in the item note when using the Select Area function). But somehow these need to come over in a way where they are image files and not hundreds of lines of gibberish.
---
Even more desired: create an Obsidian plugin to do all this automatically.
I don't think there's anything fundamental that would preclude using the same workflow as ZotFile otherwise.
The image issue may be tricky to solve though. Those are embedded encoded images, which would be the only way to have an image in html/markdown without creating a separate file.
Solving the image issue would just be a bonus.
The image issue may need to solved by using externally hosted images rather than using the new Zotero clipper. For example, adding an imagur html link to a pdf note in the new Zotero editor displays in the new note editor (after using Add Item Note from Annotations) like this:
https://i.imgur.com/p2bqS2d.png
So an updated version of mdnotes can parse the externally hosted image url and present it in markdown format. Obviously this is not a slick as the new Zotero image clipper but it is a possible solution to accessing images from outside of Zotero via markdown.
Zotero.Notes.getExportableNote()) If they have questions about how best to do this, they can post to zotero-dev.But as I've said elsewhere, if there's sufficient demand, we could consider offering a built-in way to export notes with zotero://open-pdf links. I'm not totally clear on the desired outputs here, though, so links to example documents would be helpful. Also, files with zotero://open-pdf links would inherently be local-only documents, and I'm not clear on the workflows people would be using to create final, public documents from them, so details on that would be helpful as well.
@adamsmith is right - the use case is people who are trying to use Zotero as a reference manager and want to output pdf annotations into interoperable formats such as markdown with very high efficiency (because they are dealing with many sources and notes) and into note-taking and outlining apps such as those mentioned. Currently, the workflow for most is that mentioned in some other posts on this forum: annotate pdf outside of Zotero, put annotations into a standalone note using a plug-in, export the annotations into another app for outlining, linking, drafting, etc., and then (much later) into a word processor for final revision and output.
That's perfectly fine and is exactly what we (at least me, but presumably also @jdinning and @realtime99) are looking for. The current workflow - i.e., annotating in an external PDF reader, extracting the annotations as markdown using Zotfile/mdnotes, pasting them into Obsidian - results in these local open-pdf links. The idea is that your markdown-based notes app will open the PDF at the correct location. The notes are not intended to be shared or published, so the links being local is not a problem.
The new PDF reader (which is great) has a nice way to extract annotations as Zotero notes with links, but when copied to markdown these links are lost (copied as plain text). It would be nice if there was an option to generate open-pdf links instead, so that we can use the built-in PDF reader in this workflow.
@jdinning Thank you for posting your detailed request. I spent an hour trying the find out why this function was not working for me. I searched the Internet and your post appeared in the search results. Thanks for including information on a workaround in the meantime. You are right, the current workaround is very undesirable. But it is only a workaround while the development team finds a solution. :)
Thanks for developing a wonderful PDF reader! The ability to extract both highlight and text annotations (bonus for inking annotations; an awesome new feature by the way), which can then be copied as a markdown file (with viable built-in PDF reader links) would be fantastic.
Are there options in the Advanced Config that are similar to zotfile.pdfExtraction options for styling/customizing the notes? That would be extremely useful.
In short, some way to annotate PDFs in the NPR, extract those to Markdown, and get them into Zettlr/Obsidian would be greatly appreciated!