The new Docear? Zotero + VUE (or another map app)?

I have been impressed with the Open Source app Docear, which integrates reference management (JabRef) with mindmapping (Freeplane) and adds things like searchable pdf annotations inside the pdf - still difficult to come by in most ref managers including Zotero - and then it maps these annotations and pdfs. However, while I love the Docear concept, I still find it buggy, at least on a Mac. Part of this has to do with JabRef and Freeplane apps themselves. I have contacted Docear but they are unable to consider other open source alternatives at the moment. So, I wonder if Zotero has considered such collaboration. I love Zotero and still choose it over other ref mgrs. (I just wish it would add in-app pdf viewing and annotation.) I found the VUE app today and very impressed. It looks like one of the more powerful mapping apps I've seen. Is Zotero considering a Docear-like integration? Apparently the VUE plugin doesn't work any longer. However, I eagerly cast my vote for such a move! Perhaps there are other integration projects already in the pipeline?
  • I concur. Docear has great potential. I find it working alright on Windoze. If Zotero played nicely with Docear - I think the pairing would yield a VERY powerful research tool.
  • As far as I know, there is currently no mind-mapping tool add-on under development or planned, though obviously, especially since it'd be an add-on, 3rd party devs are more than welcome to do so using VUE or any other open tool.

    As for PDF annotations - do have a look at Zotfile (www.zotfile.com), which extracts and links to PDF annotations made with any standard PDF reader. That seems to me quite similar to what Docear is doing.

    Re: "playing nice" with Docear - I don't think there is much Zotero (or Docear) can do in that respect. The BibLaTeX export from Zotero to Docear should work quite nicely (as should the reverse), but the database structure of the two tools is too dissimilar to allow anything like a sync between them.
  • edited April 24, 2014
    The Zotfile plugin certainly helps and does what it promises. However, I would like a way to map and comment further on the pdf annotations separately, say in a mind map for writing purposes. I have not yet found a way to easily export the notes for use in another app (ideally this would be in rtf and opml formats). Also the context hyperlinks back to the pdf page seem to break outside Zotero (Docear for one could not read the pdf notes generated by Zotfile and they broke the bibtex import process) or if I copy and paste a report with the listed annotations into a word processor. Nor do I believe the notes are indexed and searchable with Mac's system search (Spotlight). Thus it seems there remains plenty of opportunity to improve on the handling of pdf annotations stored inside Zotero. (Citavi comes close to this functionality but it only runs in Windozz.)
  • to be quite honest - unlikely to happen any time soon (i.e. the next year, probably longer).

    The context links to PDFs are, indeed, dependent on Zotero - I believe that's the only way to make them work cross-platform (and across computers).

    A translator that exports notes into OPML would almost certainly be possible, but it's a good amount of work and best I can tell you've been the only person to even ask about that so far, so I'm not going to put in the hours, sorry.
  • edited April 24, 2014
    Well, it's tough to be a visionary sometimes. ;)

    I know these things take time. On the other hand the topic of import/export searchable pdf annotations from other ref managers (e.g. papers, bookends, sente) to repository databases (e.g. devonthink) or other applications (mindmapping, word processing) currently fills several workflow blogs. Various scripts have also been written for extracting pdf annotations from pdf readers into repositories like DT. As people increasingly use the iPad and other tablets for pdf reading and annotation I suspect this concern will only increase. No one wants their data locked inside one piece of software. To my knowledge, Docear is one of the few if only apps that can search and display editable pdf annotations made in another app (e.g. Acrobat - although in my experience it finds only comments and not highlights). Thus I am a little amazed that you suggest I'm the only one thinking about this issue. Perhaps I'm not describing the problem correctly, or people are not expecting this kind of functionality from Zotero. Either way, I know others have related concerns and for me it is one of the make or break features that would draw me to another ref manager (incl inline pdf viewing/ annotation). In the meantime I'm sticking with Zotero because it continues to perform brilliantly for most of what I use it for.

    I really appreciate your efforts and attention.
  • I mean - Zotero can extract and display PDF annotations made in Adobe (and any other PDF software using standards) via Zotfile. I agree that's important. Just like Docear, Zotero expects users to rely on 3rd party tools for annotating PDFs, and thankfully those annotations are mostly standardized. Thousands of people are annotating PDFs on their tablet using one of the several available workflows from the simple ZotFile tablet function to ZotPad with goodreader or PaperShip with built in annotations. Again, all of these annotations are standard and you'll keep them if you switch to a different ref manager.

    Zotero also does export its notes in multiple formats (i.e. in pretty much every bibliographic export format it supports), so it's not like they're locked in any reasonable meaning of the term. Admittedly, the hyperlink to the PDF page gets lost on export/import, but I'm not sure to what degree that can be solved.

    The one thing Zotero doesn't do is to put notes in to separate files, but stores them in its database. That has pros and cons, but I don't share your sentiment that many people consider that a major issue. If they did - Joscha has offered for people to include it in ZotFile, a translator that exports notes - if desired with a zotero://select link to the original items or so - would also not be terribly hard to do.
  • Thank you again for you clarifications.
  • Hi there,

    My question goes into a slighly similar direction. I really like Docgear's mind map feature. What is the best approach to have a mind mapping feature next to Zotero's great note feature?

    Thanks ahead!
  • edited September 6, 2014
    Dropping in mid-thread, this may be orthogonal to the discussion. Regarding ...
    ideally this would be in rtf and opml formats
    I did a little work on other tools recently to get OPML transfers working. In the process, I came to appreciate Charles Miller's evaluation of OPML as an exchange format.

    To wit:
    Occasionally, someone will come up with a problem that looks vaguely outline-like: "I need to store data nested inside other data", and suggest OPML as a possible solution. Predictably, the next thing you will hear is the pained cry of a large number of developers shouting "Please, God no."

    The reason for this is that OPML, as specified, is a non-format. It's the alluring vapor of a specification that isn't there.
    This is, unfortunately, an accurate statement. There are a few tools that use OPML for narrowly constrained purposes, and it pretty much works in those contexts; but as a format for general-purpose content such as notes, we really shouldn't go there.
  • What is the best approach to have a mind mapping feature next to Zotero's great note feature?
    There's currently no reasonable way to add mind-mapping functionality to Zotero.
    So really the only way would be to write (or get someone to write) an add-on for an existing open mind-mapping software the way VUE worked (I don't believe it does anymore).
  • @adamsmith

    I will contact the VUE developer for more information on this.

    Thanks!
  • Rather than start another thread, I'm posting this here, although it has broader scope than just mind mapping.

    If you're old enough, think back to the old days when academic work used index cards for bibliographic information and notes, you kept photocopies of journal articles in files, and books and journals on shelves. Each note card had a separate idea and, usually, information about the source of the idea. To use these notes, one would typically lay them out, putting similar notes in piles, related topics near each other, and so on. Often they would be moved around, either to rethink a given organization for a single paper or to organize the notes for another paper.

    Now fast-forward to today. We have great reference managers like Zotero, which also can be used to organize digital libraries (with plugins like Zotfile). These reference managers typically come with add-ons for popular document preparation software like Word, LibreOffice, LyX (with the LyZ add-on), LaTeX (BibTeX), etc.

    But what's missing are good interfaces to intermediate software that's either used to organize literature before writing, as with note cards, or that's used to prepare presentation documents of major ideas. For a time, the VUE add-on partly fulfilled this need, but it no longer works. Docear has the potential to fill this particular gap, but it has problems. And in both cases, neither one is sufficient.

    As the note-card example illustrates, academic writers need multiple ways to take and to organize notes. Mind maps, concept maps, outlines, word clouds, cork boards (think Scrivener), QDA (qualitative data analysis) are examples of different ways of organizing references and notes. Call this category of app, "research organizer" (RO).

    Then there are different categories of information: bibliographic, tags, related references, attached documents, annotations in attached documents, etc. Ideally, good software for academic writers can handle all of these, and automatically extract what we might call "implicit terms," i.e., commonly used substantive terms in notes, annotations, and documents themselves, as well as "explicit terms," such as tags that the user deliberately assigns to individual references, notes, or in-document annotations. Call this category of app, "research document processor" (RDP) and the category combining this with organizing, "automatic research organizer" (ARO). Call the entire category of RO, RDP, and ARO software, "authoring" software.

    I know of no existing app in any of these categories, although tools like Scrivener and Docear go partway. This implies that there is whole category of software yet to be developed. In the meantime, users probably will rely on individual tools with limited functionality, like Docear for mind maps, or on multiple tools for different purposes, like Scrivener or Trello for corkboards, etc.

    The Unix/software tools philosophy of making focused tools that do one thing well and making them easy to integrate with other tools, seems appropriate here. Unless Zotero's developers want to start a whole new project, what makes sense for them to focus on is making sure Zotero's architecture, export formats, and API lend themselves to these kind of applications if and when they become available. Hopefully some standards already exist, or Zotero's programmers can propose new ones.

    I'd hate to see Zotero become a bloated, do-everything tool that quickly backs itself into a corner because it can't do some things as well as software dedicated to a single purpose. I'd also hate to see Zotero incorporate interfaces for individual authoring apps that effectively locks Zotero users into a small subset of authoring apps, as would be the case for example with document preparation software if Zotero only worked with Microsoft Word. We've seen what happened when VUE and Zotero became incompatible. Don't bet it won't happen with Docear or other software.
  • Any news on this front?
  • I've taken a quick look at some of the tools mentioned by @marsh; Scrivener is in no way interested in connecting with other tools (many people have tried to get L&L to move on this over the years with nothing to show for it); VUE looks like it hasn't been updated in a *long* time.

    I don't know the full history of the Zotero-VUE connection, but I would venture to guess that it broke mainly because of the move from Zotero 4 to 5, when pretty much all extensions broke. It was certainly possible (if painful) to port extensions to 5, but in my case it took the better part of 6 months to get it done; no new functionality, just re-creating what already worked for Z4. I can readily imagine the authors of the VUE connector not being up for the port. And there's a Zotero architecture change planned for the not too distant future that could well be equally disruptive.

    But the same kind of extensibility that allowed for the VUE connector is conceptually still there in Zotero. Zotero users are only "backed into a corner" to the extent that a) vendors like Scrivener don't offer any kind of facility to integrate, so the field is limited to vendors (such as MS) that allow hooking into their products, and b) the largely volunteer-driven integration plugins need maintenance and porting when Zotero changes internally.

    In comparison, you are backed into a corner *much more* with any other reference manager. When I got a sense of just how massive the task would be to port from 4 to 5, I seriously looked at other reference managers. I looked at (I think) all of the Linux-compatible (I'm on Linux) reg managers on https://en.wikipedia.org/wiki/Comparison_of_reference_management_software and not a single one was as easy to extend as Zotero from my POV. None of the proprietary ref managers (Mendeley, Citavi, Colwiz, etc) allow for any extensibility at all, and Mendeley takes "being backed into a corner" a few steps further by encrypting your database and not allowing you to decrypt it making integration by reading their database (previously possible) or even migration to other tools such as Zotero either impossible or a *major* pain in the ass.
  • edited 11 days ago
    In reference to my earlier post from Feb. 2016, and Emilianoe's comment, I'd strongly advise using Zotero to export a collection as a *.bib file and then importing the file into one or more of the FOSS tools, typically associated with LaTeX or even just Linux. See https://askubuntu.com/questions/175894/is-there-a-mind-mapping-software-that-can-handle-latex for discussion of such tools.
  • edited 11 days ago
    But what do these tools do with the BibTeX so generated? Mindmapping is all about making connections, and the generated BibTeX will have nothing to connect the items among themselves.

    Perhaps the question I should have asked is: what did the VUE connector previously do?

    (my name is Emiliano BTW; the 'e' at the end in my username is my middle initial which I added because at some point my development work I really screwed up my original account Zotero account "emilianoheyns" by syncing in out-of-spec items)
  • Ah, thanks for clearing up the mystery of your name. I was wondering.

    As for what they'll do with the BibTeX output, I'm not exactly sure. I suspect it will depend on the tools themselves. Some would most likely just import the bibliographic data, and then the user would have to make connections manually. Others might automatically make the connections, but this would depend on whether the BibTeX file can contain such linking information and whether Zotero would export this information properly. It's been a long time since I've read the BibTeX specifications, so I don't recall is there are fields for such links.

    Besides, these days I only use BibLaTeX, which is likely to have a more extensive set of fields. Since it's newer than BibTeX, some mind-mapping software might not support it. OTOH, mind maps are likely to be implemented better in software supporting BibLaTeX data.

    The only softwares of this genre that I've personally used are Tikz and Freeplane. Tikz is a very powerful drawing program that uses structured data. It has a steep learning curve, but it's very powerful once you learn it. I've used it to construct macroeconomic flow diagrams, not mind maps. But this tutorial discusses mind-mapping using Tikz.

    OTOH, Freeplane is a GUI-based research tool built around mind- and concept-mapping. It can use bibliographic metadata and can be directly integrated with Zotero (no need to export & import a *.bib file). See this tutorial.
  • You can add pretty much anything you like to bib(la)tex, that's not really the problem. The issue is that there's not really a source for the linking information. Zotero does have the concept of reference connection but they're unlabeled and directionless, so you can indicate that two items are related, but not through what concept they are related, and no parent-child relations. Those directionless relations are pretty easy to export to bib(la)tex fields should that be helpful, AAMOF a BBT postscript would make that possible without any changes to Zotero/BBT, or a Zotero extension could create something like an HTTP API that other tools could use to inspect those relations. But I'd need to know more about what those tools do/expect.

    Tikz isn't really a diagramming toolkit, it's more a general drawing toolkit. You could generate mindmaps using it but it's not the first tool I would reach for for the job.

    The freeplane video isn't really an integration with Zotero, it just manually adds links into Zotero in the mindmap. That will work with pretty much any tool that allows clickable links.
  • @Peter100 and @marsh - I'm using Zotero with FreePlane (on Windows) and the integration works pretty well for me, although it requires a workaround.

    Basically, I highlight pdfs, use Zotfile to extract annotations, and the extracted annotations in Zotero (which are in a note) all include zotero links like this: zotero://open-pdf/library/items/RS4NMVSS?page=1

    The link text looks like (Smith 2015:55) but the URL is like above.

    I can copy/paste the extracted annotations directly into FreePlane, and clicking the link works properly to open the pdf to the correct page. BUT, there is a trick to keep the link clickable rather than turning into plaintext. You must either 1) highlight a node and paste, creating a new child node, or you must paste into the note panel in FreePlane. If you try to paste into a node in the map that is in 'edit' mode--i.e., there is a blinking cursor--the link will paste as plaintext.

    I suppose that one improvement would be a way to paste a link in FreePlane that would highlight a Zotero item. Can anyone tell me if that's possible with a different URL scheme?
  • Yes, install the Zutilo plugin. Then, an option to copy “Zotero select link” will be added to the right-click menu. These links will select the item in Zotero when clicked.
  • Thanks. FYI for others, 'Copy select item links' selects items in Zotero standalone, and 'Copy Zotero URIs' gives you a URL that takes you to the item on the Zotero web site.
Sign In or Register to comment.