PDF Editor Beta - Some notes on notes

First of all -- this is such good news. I had no idea you were working on this and I'm liking so many of the decisions you've made.

Some initial observations:

1. So here's what I think is a fairly common use-case: I highlight a PDF and then click + "Add item note from Annotations". This works great. But then I return to the same article and find another bit to highlight. If I run "Add item note from Annotations" again, I now have a duplicate anntations.
I see some complications, but in general I feel like it should be possible to remove the older note (perhaps unless it has been modified or so?) for a much smoother experience.

2. I think the "All Notes" view needs rethinking. I have hundreds of notes from across my library with no way of searching or filtering them and it takes up the lion's share of the right pane. This doesn't seem right.

3. For the note editor -- would it be possible to bring back source view? I know some other people have been using it for advanced features, too, and I like the additional control that gives me a lot, even if I rarely use it.

Again, though -- already working really smoothly for an early beta.
  • edited March 4, 2021
    Thanks!
    1. […] I feel like it should be possible to remove the older note
    We'll think about that. It's tricky, since there's nothing special about the note — as you say, you can create multiple, or drag one in from another item, or copy and paste. But maybe we can store a hidden property on the note that gets cleared when edited and provide a quick way to update if there's a single such unmodified note under an item.
    2. I think the "All Notes" view needs rethinking. I have hundreds of notes from across my library with no way of searching or filtering them and it takes up the lion's share of the right pane. This doesn't seem right.
    Right, so, our thinking on this is that "All Notes" basically serves two purpose: 1) to let you quickly pull up a recent note you were working on (since it's sorted reverse chronologically) and 2) to provide the logical scope for the search bar so that you can pull up any note in your library. In that way, it's much like the root view of a library — in a large library, it's probably not particularly useful on its own, but it's a place you can go to search/filter your whole library. And since we know people have thousands of notes, we only show the seven most recent ones. (There's a button to show all if you really want to scroll through them, but you don't need to press that to use the search bar.)

    An analogue here would be the left-hand pane in the Notes app on macOS. Technically you can scroll through every note you've ever taken, but really it's just there so you can click a recent one or search against the full list.

    We do plan to add many more filtering options — right now the filtering is very basic. Are there specific ways of finding notes you would find useful there?
    3. For the note editor -- would it be possible to bring back source view?
    Short answer: probably not. ProseMirror isn't an HTML editor — it's object-based, and while it gets serialized to HTML on save for compatibility with TinyMCE, the HTML is much less human-friendly (large chunks of embedded data for annotations and citations) and exposing it risks corrupting what's there. We can talk about whether there might be some way to expose some sort of sanitized view that could be combined with the internal data, but that may be tough to do safely.

    There's still a lot of note functionality we need to bring back to this version, though — colors, superscript/subscript, tables, alignment, and more. So don't be alarmed if something you use is absent.
  • edited March 4, 2021
    Thanks for this.
    1.
    maybe we can store a hidden property on the note that gets cleared when edited and provide a quick way to update if there's a single such unmodified note under an item.
    Yes, that's roughly what I was thinking. I could be wrong, but I'd guess that lots of people have "annotations only" notes and if that's the case, this will work well.

    2. Oh, so I simply overlooked the quicksearch bar because it was on top of the item notes and I didn't mentally associate it with all notes. That already makes more sense and I may have just been dense there -- not sure that's actually unclear UX.
    The only other thing I'd be looking for is to filter notes by tag

    3. OK, I was wondering if that was the case. I think the image feature alone is worth it. I don't actually use a lot of fancy things, so once tables are back I'm all good, but others likely have different views on this.
  • 4. As I was messing around, I realized that I found it difficult to find my way back to the item in the library. Would it be possible to implement a "Go to Note in Library" function, so that one can do that (and also the item itself). I imagine people will work on multiple pdfs and accumulate these tabs, and they may want to have an easier way to find their way back to the item from the pdf tab (under My Library, I assume, since the item may be under multiple collections).

    5. One reason I was trying to go back to the item is to delete the note itself. I could not find a way to delete it from the pdf tab. A Delete button there will also help get rid of a note from an incomplete set of annotations and the user may recreate a new, complete one simply in two steps without changing tabs (solves case 1 above).
  • @enozkan: Yep, both of those are high on our list.
  • Regarding All Notes, I think the appearance when there are no item notes is kind of distracting and jarring, especially in Stacked View. Could there be more empty space given to an empty Item Notes section?

    Related to 3., I would find a lot of use for a Markdown view. In RStudio for example, I switch back and forth between the Markdown and ProseMirror views really often. The biggest help for my typing habits would be being able to enter text formatting using markdown entry (eg, **a** for bold, # for headings, etc.).

    6. I frequently email PDFs by dragging the attachment item from Zotero. It would be really nice if there was a prompt asking if annotations should be included when doing that.

    7. How is the citation style for citations in notes controlled?
  • Regarding All Notes, I think the appearance when there are no item notes is kind of distracting and jarring, especially in Stacked View. Could there be more empty space given to an empty Item Notes section?
    Yes, the notes list design is temporary and will definitely be getting more attention, particularly the "No notes" part. But can you say more about what you mean by "distracting", and how increasing the empty space would make it less so?
    3., I would find a lot of use for a Markdown view
    This is complicated for some of the same reasons an HTML source view is complicated — namely, if something the note editor supports can't be represented as Markdown, it has to remain as an object (meaning that the "Markdown" view is really still a ProseMirror view with a subset of things converted between the two modes), because the alternative is a 2 KB block of encoded JSON.

    Auto-converting some additional Markdown things to rich text would be fine — we already convert Markdown syntax for blockquotes and lists. Issue created.
    6. I frequently email PDFs by dragging the attachment item from Zotero. It would be really nice if there was a prompt asking if annotations should be included when doing that.
    That we probably can't do. It varies by OS, but we're fairly limited on a technical level on drag — I haven't looked at it in a while, but if I recall the file generally needs to exist before the drag starts. It's possible we could have some additional options here in a future Firefox version. Short of that, you could have some weird thing where you chose a context-menu option and a temporary file was generated with annotations and you were given some sort of proxy icon in the UI that you could drag, but that's getting pretty esoteric.
    7. How is the citation style for citations in notes controlled?
    It's just a fake style, similar to what we show in the citation dialog. At least with citeproc-js, generating real style output is too slow for what we need. Not impossible that could change with citeproc-rs, but these are really just meant to be placeholders until you get the note into a document, whether via the word processor plugins or some other future export functionality.
  • By distracting I just mean, it’s not clear immediately that there are 2 sections and that All Notes is all of the notes in the library. A clearer distinction between these sections would make it easier to understand what they are each for. Relabeling “All Notes” to “All Notes in My Library” might help too.

    Re: 6. I’m guessing that a setting to automatically export annotations on a drag wouldn’t work then either? Maybe a permanent icon for this in the right pane for the attachment icon might be workable?

    Re: 7. Got it. I don’t think the citation style is important, but I expect this to be a mild source of confusion. We will need an FAQ page explaining that the citation style will be set when the note text is inserted/copied into a word processor document.
  • edited March 4, 2021
    Re: 6. I’m guessing that a setting to automatically export annotations on a drag wouldn’t work then either?
    No, the problem is that the file with embedded annotations doesn't exist at the time of the drag, and we can't generate it during the drag. The only other alternative would be for Zotero to maintain a mirror copy of every annotated file with embedded annotations, which I doubt anyone wants.
    Maybe a permanent icon for this in the right pane for the attachment icon might be workable?
    What I meant re: an icon is that you would have to actually request creation of the temporary file, and then once it was generated an icon would appear (hovering in a little popup, say) that served as a proxy that you could drag. But that's getting a bit out there for a non-technical audience. (Not quite the same, but it's in the vein of some utilities for macOS that give you a temporary shelf where you can drag things while you prepare a target.)
  • edited March 4, 2021
    @dstillman in the All Notes view, would it be possible in future versions to untether notes from their sources and show them separately, for example, view only notes including a particular word in the list? currently all notes for a particular item appear together.

    Edit: Never mind, they are untethered! This is amazing, it is much easier to work with notes in this interface than in the main Zotero interface.
  • Two additional things:

    - When I have a PDF open in one tab with the note editor open, if I open a new PDF from My Library, the existing note editor stays visible in the new tab with the new PDF.

    - I don't see a way to delete a note from either the notes list or the note editor view.
  • When I have a PDF open in one tab with the note editor open, if I open a new PDF from My Library, the existing note editor stays visible in the new tab with the new PDF.
    Yes, that's by design. The Notes pane persists between tabs, so you can take notes across more than one PDF. The Item pane is specific to the PDF you're viewing. (We tried variations where child notes were editable in the Item pane and remained contextual, but it ended up being too confusing to have notes in too many places.)
    I don't see a way to delete a note from either the notes list or the note editor view.
    That will be possible from the notes list soon.
  • Regarding 6: How about an option for automatically saving all annotations to the pdf? This could happen when Zotero is not working on anything else, similar to a sync. That would mean anyone regularly sharing/emailing annotated pdfs would *almost* always have that if they have turned auto-saving annotations on. I understand that this is planned for exports anyway, so implementing it here would not be much extra work?
  • @enozkan: It wouldn't work for Zotero annotations to be saved to the original file. A lot of reasons: it would result in extra file uploads/downloads, it would be much harder/slower to do on web/mobile; since Zotero still supports externally created annotations, it would result in all sorts of issues reconciling changes made to the annotations by external readers (external readers stripping identifiers from the mirrored annotations, conflicts between the two different versions of an annotation, concurrency issues from open files). It would just be a total mess.

    If disk space were no issue, someone could create a plugin that maintained a one-way mirror of PDFs with embedded annotations whenever the internal annotations changed (and, for the dragging issue specifically, overrode the drag functionality to use that version of the file).
  • Thanks for considering. I am aware how much it matters for you guys that user data and work is not locked in Zotero, so I know there will be a way to save/export pdf+annotations (one of the many reasons I like Zotero).

    How about a user-initiated File -> Save menu item in addition to "Save As" already in the menu? Because I know that I will want to save my annotations to my pdfs back in my library. This may not solve some of the issues on reconciling changes but I assume that Zotero has to deal with those now anyway.
  • @enozkan: We don't have to deal with reconciling changes now because the external annotations are read-only. In a future version we may support transferring annotations one way or another and deleting on the source side — importing annotations from a PDF and removing them from the file, or vice versa — but that's likely to be the only option (and, e.g., you'd lose Zotero tags on the annotations if saving to the file).
  • @dstillman to your question: Are there specific ways of finding notes you would find useful there? To add to what @adamsmith said: The only other thing I'd be looking for is to filter notes by tag

    It would be great to have the full tag selection interface for this, so one could drill down to a few notes by selecting two or more tags. That would give the user full control of which notes are visible
  • edited March 5, 2021
    @dstillman I am not understanding exactly what the concerns are, but I don't need to. I trust you guys can see all the issues. Thank you for another excellent improvement.

    On the confirmatory side, (1) I opened an attached pdf and annotated it in Acrobat Pro, (2) saved it in Acrobat Pro, (3) opened it in Zotero using the new pdf viewer, (4) added a new annotation, (5) saved it from the new pdf reader using "Save As", and (6) replaced the original pdf in the storage folder. This worked fine beautifully; no issues. The updated pdf opens in the new pdf reader with both annotations. So, if I choose to annotate my pdfs (including those with previous annotations) in the new pdf reader, I can do that and manually replace the file in the storage folder. Excellent!

    Edit: I do not want to give bad advice to others, though. The new file has both annotations as read-only, and you have an extra for the one created in the new pdf reader (which is saved in the library, I assume). So this workflow should not be used (and wouldn't be endorsed by the developers rightfully so). For saving your annotations as standard pdf annotations, or quickly sharing annotated pdfs with others, there may be other ways.
  • Right, to confirm, if you save the PDF with embedded annotations and replace the file in Zotero, you will indeed get duplicate annotations — the ones in the file and the originals in Zotero. That's why any such operation built into Zotero would delete the source annotations during the transfer.
  • edited March 8, 2021
    > There's still a lot of note functionality we need to bring back to this version, though — colors, superscript/subscript, tables, alignment, and more. So don't be alarmed if something you use is absent.

    That's excellent - looking forward to that!!

    A few comments

    (1)

    Also, good that the file can be exported with the annotations using save-as.
    (A) It wasn't clear to me that it would save with annotations - might be good to indicate somewhere.
    (B) I wonder whether there could be an "attach duplicate with embedded annotations" or similar, that creates a 2nd attachment with the annotations baked in?

    (2)

    Overall, I would love more keyboard shortcuts (or a way for Zutilo to generate them). It would be nice to have keyboard shortcuts for hiding panels, but above all, I would love keyboard shortcuts for the tools (maybe 1/2/3?) and colours (r/g/b/??).

    (3)

    I assume that while we cannot drag out of Zotero, we can drag annotated PDFs between libraries, right?

    As per other thread:

    [removed duplicate comments — D.S.]
  • (A) It wasn't clear to me that it would save with annotations - might be good to indicate somewhere.
    That's a placeholder menu option. It will be possible to choose whether or not to include embedded annotations.
    Overall, I would love more keyboard shortcuts
    Additional keyboard shortcuts in the note editor are planned.
    I assume that while we cannot drag out of Zotero, we can drag annotated PDFs between libraries, right?
    Yes, you'll be able to do so. There'll be a new prompt that allows you to choose whether to include various pieces of data when copying between libraries.
  • Thanks @dstillman !!

    That's excellent - looking forward to that!!

    I'd actually remove the remaining two comments from the other thread to move them here. Here they are again:

    (4) Would love an option to add turn all PDF-annotations into a note each. That note would need to start with the text of the annotation so the initial text of the note can be seen in the Zotero middle panel. That note would be tagged with the tags added to the note in the PDF. (Cudos for adding tags to individual annotations!) This would make a quick qualitative analysis of the PDF document / extracting key quotes really simple. Would love to have this.

    This could also check existing notes for exact duplicates and not recreate them, avoiding needless duplication.

    (5) ZotFile-type customisation of the extracted text. I've got my ZotFile set up so that it uses the colour of the PDF-annotation to highlight part of the Zotero-note (e.g., the reference). Will it be possible to customise the text of the Zotero-note in any way (as you can with ZotFile)?

    What I would love here is (customisation):
    (a) Show the user name who made the annotation in the Zotero-note. (User would be the Zotero user or the user embedded in the PDF, depending on the type of annotation.)
    (b) Show the highlight colour and the highlight colour name.
    (c) Show the Zotero Item key for the PDF and for the parent item.
    (d) Let the user customise character-based separators between those parts, so notes can be further processed.

    For example, one could then do this (where "⟦Note:" is arbitrary markup chosen by user):

    "Extracted text." (Author, Year, p. X) ⟦Note: ABC⟧⟦parent:123:ABC⟧⟦key:456:DEF⟧⟦Color:#FF0000:Red⟧⟦Tag:XXX⟧⟦Tag:YYY⟧⟦User:ZZZ⟧

    or even

    ⟦text⟧"Extracted text."⟦/text⟧[ref⟧(Author, Year, p. X)⟦/ref⟧ ⟦Note: ABC⟧⟦parent:123:ABC⟧⟦key:456:DEF⟧⟦Color:#FF0000:Red⟧⟦Tag:XXX⟧⟦Tag:YYY⟧⟦User:ZZZ⟧

    (The parent/key def seems to be present in the 'data-citation', but would be helpful to have a way of showing them in the plain text.)

    I know the above adds more characters etc, but we have a way of 'onward processing' of the notes (by copy/paste), and it would be really good to have this.

    Btw. I really love the way that you can drag paragraphs around in the note editor, that's very cool.
  • edited March 9, 2021
    [Moved from another thread — D.S.]

    @ @dstillman
    You wouldn't see the notes list except when you're trying to open a note, in which case this lets you choose between item notes and all notes depending on which you want to open — it wouldn't make sense to have to go somewhere else just to open a standalone note in the same editor.
    I don't have an option to choose. I click the icon and it displays both.

    I also disagree that I would only use it when looking for a note. Once you pointed out that this can be used to make notes, I have been using it this way. Having the pane open as I read and making notes as necessary works really well. I treat it as 'overall notes' as opposed to notes linked to highlights/specific pages.
    And it only shows the 7 most recent other notes by default, though you can search across all of them.
    For me, "All Notes" nearly fills up the entire pane (Macbook Air). If I have the pane open to take notes then the notes pane is full and feels messy for me. After the second note I have a scroll bar.

    I agree with the comment from the thread you linked, the notes pane takes up the lion's share of the pane. If I am in an article making notes, I'm not looking for notes from other articles. For me, it doesn't seem like the most logical place and does detract from the cleanliness of the UI.

    You noted that there will be more filtering options in the future, if this provides the means to remove/hide "All Notes" that would work well. Alternatively, a different button for "Item Notes" and "All Notes" would work.
    ...it wouldn't make sense to have to go somewhere else just to open a standalone note in the same editor.
    Again, I disagree. A different tab (I mean functioning the same way parent item does) in the pane keeps things clean. The only use case I can think of for needing "All Notes" is if when reading an article I think to look up related issues/comments/notes from other articles. Otherwise I have no need of it when viewing an article

    I understand you have a particular approach and design in mind, this is just my feedback based on the way I use Zotero, take notes when reading etc.
  • edited March 9, 2021
    @Jes.Mullins:
    I don't have an option to choose. I click the icon and it displays both.
    To be clear, I meant "choose" in that you can open either item notes or standalone notes from the same place. They're notes either way, and they both need to open in a note editor.
    A different tab (I mean functioning the same way parent item does) in the pane keeps things clean.
    We went through multiple revisions of the design where child notes were in a separate pane (generally the context-specific item pane), but it ended up being far too confusing to have multiple note lists and multiple note editors, with different tab-pinning behavior, as you switched between panes and tabs.
    The only use case I can think of for needing "All Notes" is if when reading an article I think to look up related issues/comments/notes from other articles. Otherwise I have no need of it when viewing an article
    That's understandable. I think it will depend on people's workflows. If while viewing a PDF you would only ever work on child notes, I can see having other notes show up there being distracting.

    Much of the functionality of the new note editor is based around the idea of using standalone notes further into a draft as you combine material from various sources, and the All Notes section — and the pinning of notes between tabs — is meant to address that use case. But I understand that not everyone will want to work that way, or may want to do that part outside the PDF reader.

    We'll see what we can do here. Thanks for your feedback.
  • I really like the new PDF editor and am looking forward to many of the proposed features. Here are some of my thoughts on notes in the editor.

    1. Would it be possible to view standalone notes in tabs? I often work with multiple standalone notes while reading a single PDF. Switching between notes through the notes-pane is not ideal. It would be easier if all the notes I need access to were open in tabs.

    2. Is there going to be an option to hide or shorten the citation that is shown after an annotation in the notes? I want to keep the citation, but having the whole (author+year+page) after every annotation is visually distracting and seems unnecessary. For me the best option would be to show just the page number, and that to work as a link to the specific page in the PDF. And as a side note: the "hide author" feature in the citation dialogue doesn't seem to work in notes.

    3. Would it be possible to have the quotation marks as optional when copying annotations into notes? I prefer not to have them.
  • Would it be possible to view standalone notes in tabs?
    Yes, this is planned. In the meantime, you can open notes in separate windows from the library view.
    Is there going to be an option to hide or shorten the citation that is shown after an annotation in the notes? […] For me the best option would be to show just the page number, and that to work as a link to the specific page in the PDF.
    Notes are designed to be combined into a larger standalone notes, so if we did this, it would have to be conditional based on the view. We could consider showing only the page number 1) when viewing a child note, 2) when viewing it in the library view or the PDF reader with that specific PDF open, and 3) when there are no citations for other items in the note.
    Would it be possible to have the quotation marks as optional when copying annotations into notes? I prefer not to have them.
    I don't see us doing that. This is a situation where we know the specific context of the text, and we want to help people avoid accidental plagiarism or misattribution.

    (Quotation marks are currently added when copying text from the PDF to anywhere, though, including fields, and that's a bug.)
  • A thought regarding the size of Item vs All Notes:
    1) have a draggable resize thingy so you can make one or the other bigger or smaller
    2) have a button to collapse/expand each of the Notes sections.
  • I strongly agree with @adamsmith 's comment about updating annotations rather than creating a duplicate. Though I have nothing to contribute to the discussion about whatever wizardry you'd need in order to implement it...
  • edited January 5, 2022
    I like to take item notes that reflect the structure of the document using expandable/collapsible headings for sections/chapters. Is there any interest in this?

    (edit: I currently use workflowy for this).
  • If disk space were no issue, someone could create a plugin that maintained a one-way mirror of PDFs with embedded annotations whenever the internal annotations changed (and, for the dragging issue specifically, overrode the drag functionality to use that version of the file).
    Now that some time has passed, has anybody created a plugin like this? It would be a very useful feature in my workflow if i could take advantage of both the native annotation tagging in Zotero’s PDF reader and the nested folder organization of Zotfile.
  • @aaaaaaaaaaaaaaa: It's just not a very good solution — it would duplicate every file and require a lot of processing every time you made a change to an annotation.

    You can of course use nested collections in Zotero for organization, so I'm not sure what you mean there about ZotFile. But if you're saying you want to regularly open PDFs outside of Zotero, you should probably use an external PDF reader.
Sign In or Register to comment.