Can anyone confirm a working instance of Voyant export extension after the latest round of upates?

kcorwin
Everything was working fine until the latest round of Zotero updates, now the extension has disappeared from the UI menu. Running Windows 10; Zotero 5.0.34 .
  • hamblha
    Did you get any updates on this? I've never been able to get the Voyant extension to work, but I did just install it last week.
  • kcorwin
    edited July 24, 2018
    No updates provided, but the GitHub page seems to indicate that an update is unlikely and that a Voyant extension is probably dead for now
  • sdspieg
    I contacted the developer, but am not sure she will fix this. Has anybody tried using an older version of Zotero with the plugin? Also - can anybody confirm that up-to-date Zotero collections/libraries are backwards compatible within version Zotero 5.x? I don't want to mess up my database by installing an earlier Zotero 5.0 version and opening it from there...
  • sdspieg
    edited August 22, 2018

    Here's what I am trying:


    • I exported the library I want to visualize and analyze in Voyant from my default (latest up-to-date) version of Zotero to an rdf-file with all notes and files included

    • I created a new firefox profile (in the current version of Zotero - I always have the 'user profile selection' open up first when I start ZOtero so I can select the profile) which I called 'Voyant - Zotero 5.0);

    • I downloaded the first 5.0 version of Zotero

    • I installed it in a separate folder which I called Zotero 5.0 and also created a separate data folder for that version (not to corrupt my main 'production' library)

    • I installed the Voyant export plugin - which is now visible when I right-click a collection in the program

    • I am trying to re-import the rdf-file into 5.0, but it went very slow until it hit about 3000 (out of 35k) items, and it now seems stuck - to be continued...

  • sdspieg
    Update - The import went very slow (about 100 per minute - and this library has 35k items) and then stopped after importing about 3100 items. I'll retry. But so in the meanwhile I stopped the process, and restarted Zotero. At that point the Voyant menu was once again no longer visible. But removing and re-installing made it appear again. I selected 'Export collection to Voyant', it suggested a zip-filename, which I approved. But the resulting zip file is only 1kb big and only conatins an empty bagit.txt file.
    I wonder whether we would be able to find the exact (presumably still beta) version of this on which Cora created this...
  • kcorwin
    edited August 22, 2018
    No joy on updates -- see the github page. My workaround has been to use Voyant Tools setup as a local server (see the documentation for that approach, where one can speed up the process by setting params for memory allocated to the needed processes); I simply export what I need from Zotero. It's more steps, but more stable and efficient than dealing with the plug-in. I would happily contribute to dev time to get this re-connected to updated zotero. Having natural language processing is a huge benefit leveraging a library as the corpus for text analysis.

    Also direct message or another thread on how a full import is made useful would be of tremendous help. Bringing in an entire library as a corpus has generated absurd amounts of noise on my end. It is simpler/quicker to pick something like abstracts, or notes only, and analyze those for me -- but I could be missing a lot of technique and use case scenarios.
  • sdspieg
    Yes but we really need the metadata etc. It was so easy with paper machines - I may even go back to 4.0 just for this...
  • bwiernik
    Do you have any comfort programming in Python or R?
  • sdspieg
    Not enough to make everything work the way we want to make it work. That's why we are still trying to get Zotero collections into ITMS (or Voyant).
  • emilianoeheyns
    What kind of data do these tools expect?
  • sdspieg
    All kinds of data - but the 'problem' here is how to get the Zotero metadata (which you see in Zotero) connected to the underlying pdfs (or the raw text that's extracted from them), snapshots, etc.
  • emilianoeheyns
    So a dump of the reference info from zotero and the full-text content of the pdf? In what format?
  • sdspieg
    edited August 23, 2018

    So a dump of the reference info from zotero and the full-text content of the pdf?

    and also the text content of snapshots, yes.

    In what format?

    Well, in ITMS, Olga Scrivner, the developer, is still tryiing to figure out the best way of getting Zotero collections in there. It so far (kinda) works with an exported rdf file with the pdfs and html files in the corresponding files in the underlying 'files'-folder (the standard way the Zotero built-in rdf translator exports rdf files). For Voyant, Cora did apparently manage to get everything exported in the required format through the export plugin she created. But that's the one that doesn't work now for many of us. You can look at the github code, of course, if you know these things (which I myself don't).

  • emilianoeheyns
    I have looked at the github code but couldn't quite figure out what it was aiming at; in general terms I'd rather work towards what the consuming program expects than to revive code that's not my own.
  • emilianoeheyns
    But if it kinda works with a default rdf export, what is the problem? Rdf export with file attachments is stock zotero functionality.
  • bwiernik
    Zotero has web and JavaScript APIs for retrieving item data and full text cache. I’d say that a proper solution would use those APIs.
  • sdspieg

     I’d say that a proper solution would use those APIs.

    That's what Olga is trying to achieve now - thanks for the tip!

    But if it kinda works with a default rdf export, what is the problem?

    The problem is that it doesn't seem to have the date of publication. Since Voyant doesn't (yet) have diachronical functions, that wasn't necessary; but most of us (even if 99% of people don't know it yet ;) WILL want the ability to explore big full-text corpora over time - to see which topics become more or less important (streamgraphs), which n-grams start co-occurring more or less, etc. .
  • pancoast
    I have had great success with the system compiled from https://github.com/corajr/zotero-voyant-export/

    *** Make sure all PDF/document items have a parent metadata object, or else the export will abruptly fail without warning.
  • jmkembro
    It is great to know that it can work, however I have no idea how to compile the system from the github files. Could you explain how to do so or unload your compiled version that works?
  • rokalinin
    Tried the add-on from https://github.com/corajr/zotero-voyant-export/ but with no success. Could anyone explain how to make it work?
Sign In or Register to comment.