Bug when pasting in a note: href="zotero://open-pdf/..." removed

edited December 21, 2020
I have written a python script that copies a selected text from a .pdf and add, to this text, the direct link of the source (ei: zotero://open-pdf/0_1A2B3C4D/10). I can then paste everything directly in a Zotero note as my clipboard is HTML formatted (so I don't need to open the source code window). I have tested my script on several editors and I don't have any issue except on Zotero Addons and Zotero Standalone.

The trouble is that Zotero removes the href attribute of my link if it doesn't start with http://. So after pasting, if I look into the source code I don't have < a href="zotero://open-pdf/0_1A2B3C4D" target="_blank">Source< /a> as expected but < a target="_blank">Source< /a>.

Some debugging steps:
○ I modified my python script to send to my clipboard this < a href="http://zotero://open-pdf/0_1A2B3C4D" target="_blank"> and Zotero didn't remove that href.
○ If I manually create the link by adding zotero://open-pdf/0_1A2B3C4D via the Insert Link icon, then Zotero accepts the link.

So when I paste, how can I make Zotero accept a link whose href doesn't start with http:// ?


  • (I don't think there's an easy solution here that doesn't involve patching TinyMCE in Zotero, sorry).
  • Thanks for your answer! Are you sure it's related to TinyMce because if I create a link with this url zotero://open-pdf/0_1A2B3C4D via the Inser Link icon then the href isn't removed.

    If it is, do you know which file should I edit? I have looked into the zotero-master\resource\tinymce\themes\advanced\js\link.js file and found that function:

    checkPrefix : function(n) {
    if (n.value && Validator.isEmail(n) && !/^\s*mailto:/i.test(n.value) && confirm(tinyMCEPopup.getLang('advanced_dlg.link_is_email')))
    n.value = 'mailto:' + n.value;

    if (/^\s*www\./i.test(n.value) && confirm(tinyMCEPopup.getLang('advanced_dlg.link_is_external')))
    n.value = 'http://' + n.value;
    },

    Is this the one I should edit?
  • edited November 27, 2016
    I posted a question about editing zotero@chnm.gmu.edu.xpi here: https://forums.zotero.org/discussion/63316/modifying-tinymce-files-where-is-zotero-jar-and-how-to-edit-zotero-chnm-gmu-edu-xpi, but bwiernik adviced me to don't split the thread so I will continue here.


    @bwiernik : The edits I would like to make will really save me a lot of time so I don't mind having to reapply them after each update. Do you know if zotero@chnm.gmu.edu.xpi is the same as zotero.jar?


  • I did some more testing, and apparently links like zoter://open-pdf... (notice the missing last o), zoterH://open-pdf... or zoteroA://open-pdf... are accepted, but not zotero://open-pdf.... Any random string before the :// is also accepted (except file://).

    So it seems that zotero:// is intentionally forbidden (only when pasting, not when added via the Add Url icon). Any idea why? Also I still couldn't find the file causing this, so if you have any clue, that would be very welcome!

  • edited November 27, 2016
    @EnoraNedelec

    No, zotero@chnm.gmu.edu.xpi is not zotero.jar.

    Here is a quick development hack I use.

    Close firefox and zotero.

    Look for ~/.mozilla/firefox/profile_name/extensions/zotero@chnm.gmu.edu.xpi

    Change the file extension from .xpi to .zip

    Unzip to see the internal hierarchy.
    - chrome
    - components
    - defaults
    - META-INF
    - resource

    - etc.


    Edit /resource/tinymce/ .. the note pane is rendered from resource/tinymce/note.html

    Re-zip contents of unzipped and edited archive.
    Change .zip extension to .xpi

    I documented my own experiments (with Zotero standalone and tinymce) here ...

    https://groups.google.com/forum/#!topic/zotero-dev/nmtCLEb9zLA

    Regarding saving zotero:// I suspect that this is because zotero:// is a reserved protocol handler (as are file:// and http://).

    Thinking more about this you could create your own custom protocol handler to paste into your notes. But this might not work if you are sharing notes since other users would need to have the custom handler installed.
    ..

    P.S. when I am editing Zotero standalone (not the extension) I have to change zotero.jar to zotero.zip and go through the steps above.
  • edited November 27, 2016
    @dragonfly Thanks a lot for all this details! Finally I found the zotero.jar (it is in C:\Program Files (x86)\Zotero Standalone - I don't understand why windows didn't find it this morning...).

    If I got you correctly, zotero.jar is for Zotero standalone and zotero@chnm.gmu.edu.xpi is for Zotero addons. Is that correct?

    I saw your thread this morning when looking around for so direction about modifiyng TinyMce. Thanks a lot for sharing with us what you did!

    Concerning the custom protocol handler, I do not often share my notes so this won't be an issue. I looked into how I should create one, apparently it's via the registry*. But I am not sure how I should implement it to my own situation, so that the custom protocol handler I am creating can redirect to the correct path zotero://open-pdf/... Let say, my protocol handler will start a python that will open my pdf. So far here is what I have:

    REGEDIT4

    [HKEY_CLASSES_ROOT\foo]
    @="URL:Python Start Zotero Pdf Protocol"
    "URL Protocol"=""

    [HKEY_CLASSES_ROOT\Python Start Zotero Pdf Protocol\shell]

    [HKEY_CLASSES_ROOT\Python Start Zotero Pdf Protocol\shell\open]

    [HKEY_CLASSES_ROOT\Python Start Zotero Pdf Protocol\shell\open\command]
    @="\"C:\\Python35\\Lib\\site-packages\\Python Start Zotero Pdf Protocol.py\" \"%1\""

    Is good so far?

    Also how can I use the argument %1 to be the key of the pdf (the number after the zotero://open-pdf/0_ in the actual link) so that the .py knows precisely which pdf to open?

    *http://stackoverflow.com/questions/389204/how-do-i-create-my-own-url-protocol-e-g-so"

  • edited November 27, 2016
    You are a Windows user. I am using Ubuntu and it is some time since I have used Windows (although I still have Windows in dual boot partition for rare usage)

    But off the top of my head if you follow the instructions here ...

    https://msdn.microsoft.com/en-us/library/aa767914.aspx

    (as you have posted above) you could paste a URL with (say) invented custom protocol .. zopdf://

    pdf_direct_link= ("zopdf://open-pdf/0_" + foldername + "/" + Pdf_Page_number)

    the zopdf:// URL handler would then launch a wrapper (proxy) .exe file to actually implement ..

    zotero://pdf etc.

    Here in Ubuntu land the URL protocol handler in fact launches a *.desktop.

    ...

    However, on reflection, there might be an easier approach than using a custom URL protocol handler.

    Once you get into editing note.html you could add a custom button to run your introduced Python script which would paste your zotero://open-pdf/.... link into tinymce.

    ...

    Incidentally, looking into your Python script ..

    Zotero_Pdf_Highlight_and_url.py

    at a cursory reading I see that you are accessing zotero.sqlite. My understanding from reading other threads is that it is safer to use the Javascript API. i.e. the database might become corrupted.
    But this is just a hunch.
  • edited November 28, 2016
    When I read about your post I didn't think it could apply to my situation... On a second thought, I think it would be difficult. Here is the whole set up. I often read pdf on my computer and as I read them I copy the interesting part and paste them in different notes. Because I am copying while I am reading and I spread the info in many different notes, the Zotfile annotation extraction tool doesn't help me much (in particular because it extracts all the highlights at once, while I just need the last one). But I also need to keep track of the source, so I made this .py to retrieve the pdf link and the page (Zotfile inspired me on that one!). So I don' really see here how adding a button to TinyMce could fit into my workflow.

    Also the whole problem is about pasting the zotero://open-pdf/ in the note, so I guess even if the .py is launch from TinyMce, I will run into the same problem... well except if I can make the .py edit the source code instead of pasting in the html mode.

    --

    For accessing zotero.sqlite, I am doing it on a copy with shutil.copy2 of the zotero.sqlite : I tried to access the .py directly but I get an error as the file is locked because Zotero is open when I run .py. So do you thing simply copying the .sqlite can corrupt my db?


    Well that's a lot of new info, just when I was running out of ideas. Thanks so much @dragonfly for your help, for writing such detailed answers and for the time you spent writing them. I really appreciate it!
  • edited November 27, 2016
    I will just add (if you read my zotero-dev thread) that I found it difficult to understand the js amendments made in tinymce (by Dan S.). So I added a fresh tinymce editor to allow me to experiment with a later version tinymce (including adding plugins)

    If I have time (probably now tomorrow) I'll try adding zotero://open-pdf as links in the later version of tinymce which does not have all the zotero amendments.

    I do have similar interests to build a corpus of notes extracted from html and pdf for semantic analysis. I would like to see a smoother workflow. I use Foxit to annotate PDF files.
  • edited November 27, 2016
    If I got you correctly, zotero.jar is for Zotero standalone and zotero@chnm.gmu.edu.xpi is for Zotero addons. Is that correct?
    no, that's not quite right. Both versions run essentially the same code and the Firefox add-on uses a zotero.jar , too. As bwiernik indicates in the other thread, packing and unpacking zotero.jar is a rather tedious method of trying to tweak Zotero code and the standard/recommended way would be to run the Firefox add-on straight from the github checkout, i.e. https://www.zotero.org/support/dev/source_code#working_with_git_repositories but obviously hacking straight in the .jar works. It's just a waste of time, especially if you're trying things out.
  • @EnoraEdelec
    Reflecting .. in your opening post you wrote ...
    I have written a python script that copies a selected text from a .pdf and add, to this text, the direct link of the source (ei: zotero://open-pdf/0_1A2B3C4D/10).
    However, Zotfile does this exactly (including adding the zotero://open-pdf protocol) and when clicking on the link added to the annotation the source PDF is launched in default PDF Viewer (I use Foxit Reader).

    So have we been chasing a red herring with your script and discussing URL protocol handlers? I'm not sure now what additional functionality you really need or your script brings.

    ...

    @adamsmith
    I still believe that hacking zotero.jar - although clearly not the Zotero approved way and frowned upon - is easy and quick for prototyping new features. Two clicks and I am into editing the source code using a two pane editor like Total Commander (in Windows) or Krusader (in Ubuntu).
    The approved github way can follow later after first rough hacking/testing.
  • @dragonfly -- Zotfile writes this into the note directly in html using the javascript API. EnoraEdelect is trying to paste from an html clipboard to the rich text (i.e. default) view of the note, so that's a bit different.

    As for editing zotero.jar -- I think I have said this before: this has nothing to do with "approved" or not. There's nothing harmful in editing the zotero.jar. I just think working with a github checkout takes 3mins to set up and then saves you a lot of time when testing code and I don't want to encourage people to use what most people would consider a far inferior setup. I'm not trying to talk you out of it, but I do discourage you from talking other people into it ;)
  • edited November 28, 2016
    @dragonfly Yes, adamsmith is right, the procedure is different. But if I got it right, Zotfile also has to use a work around (cf Zotfile\chrome\content\zotfile\zotfile.js: att.setNote(note.innerHTML.replace(/http:\/\/zotfile.com\//g, 'zotero://'));)

    By the way, did you have the time to try to paste zotero://open-pdf/0_1A2B3C4D/10 in your tiny 4.4.3: I am really curious to know if it works. I tried to implement your set up, but I am afraid my programming knowledge isn't enough. Apparently I have to edit tinymce\note.html, tinymce\themes\advanced\editor_template.js and tinymce\tiny_mce.js to created a new button. But there I am quite a bit lost. I don'T really knwo what to do to open a new window with an editor and also to pass back the content of that editor to "the zotero ecosystem" as you put it! Would you mind sending me your whole zotero.jar, so I can have a look at it?

    @adamsmith I am not yet familiar with git repositories, but I am looking into it! Also, you keep mentionning the zotero.jar so I am confused. If I want to had the button dragonfly is talking about in Zotero addons, what file should I edit : zotero@chnm.gmu.edu.xpi (in firefox/profile/...), zotero.jar (the only one I found is in C:\Program Files (x86)\Zotero Standalone) or both?

    Thanks a lot to both of you for your help and your in depth comments. I am starting to see the light at the end of the tunnel!


  • edited November 28, 2016
    Just to remove some of the confusion which I helped to create.
    If you are using the Firefox add-on you need to look inside zotero@chnm.gmu.edu.xpi.
    If you are using Zotero standalone (as I am with Chromium connector) you need to look inside zotero.jar. I am not using Firefox at the moment.

    I prefer not to post my zotero.jar as you request since it has several rough tests underway.
    Instead I will post information on the steps I've taken.

    Let's make a start by suggesting that you simply paste zotero://open-pdf/0_1A2B3C4D/10
    into your browser URL (the example below comes from Chromium Web Browser in Ubuntu).

    When you launch this URL you should see a popup window

    External Protocol Request

    Chromium needs to launch an external application to handle zotero: links. the link requested is zotero://open-pdf/0_1A2B3C4D/10

    The following application will be launched if you accept this request.

    xdg-open

    Click on [ Launch Application ]
    We can discuss where to go from here. So far we are not touching Zotero. Just experimenting with URL protocol handlers.

    I can confirm that I can post your link zotero://open-pdf/0_1A2B3C4D/10
    into tinymce 4.5.0 (it has moved on from 4.4.3).

    More steps will follow.
  • @Dragonfly, I can't believe how kind you are! Thanks so much for taking the time to walk me through all this. I completely understand your choice about sharing your zotero.jar. That's a good news that the zotero://open-pdf/0_1A2B3C4D/10 can be pasted in tinymce when there is no Zotero edit.

    Here is what I get:
    "C:\Program Files (x86)\Zotero Standalone\zotero.exe" -url "zotero://open-
    pdf/0_1A2B3C4D/10"

    When I click on "Launch Application" Zotero Standalone is launched by Zotero Standalone doesn't open the pdf. (I am not sure it can as the argument is an url)

  • Thanks so much for taking the time to walk me through all this.
    In fact this exercise is refreshing my memory to resume some tests I've been running to capture and analyse text from collections.

    Next step, I suggest that you download tinymce 4.5.0 (Community) into a folder completely divorced from zotero and begin to understand how to use it outside zotero environment. You will need to create a note.html (or index.html) file from the tinymce examples.

    https://www.tinymce.com/download/

    e.g. read about links here ... https://www.tinymce.com/docs/plugins/link/



  • I posted a message yesterday but apparently it didn't go through. I am making progress with note.html. I was able to personalize the toolbar and add plugins like the link and to make the tiny editor run in chroma as well as firefox. I try to paste the clipboard that my python script creates in the editor: when it is open in chrome everything is good (the href="zotero://open-
    pdf/0_1A2B3C4D/10" isn't removed) but if I open it in Firefox, then the href is removed. I have to disable zotero addons to be able to paste it. (I did as you said, the note.html is in a folder divorced from zotero).

    This is kind of a funny situation, I need to paste my clipboard in zotero but to be able to paste it I need to disable zotero!



  • edited December 1, 2016
    Please clarify if you are testing against the default version (3.5.7) of tinymce as embedded in Zotero or if you are testing a later version of tinymce. Note that I found significant differences between tinymce 3.5.7 embedded in Zotero and tinymce 4.4.3 (or later version) fresh download.

    It will help you to understand if you read default tiny_mce.js in /resource/tinymce/
    You might need to "prettify" the script to make it readable.


    // Contains modifications by Dan S./Zotero

    (function(win) {

    var whiteSpaceRe = /^\s*|\s*$/g,
    undef, isRegExpBroken = 'B'.replace(/A(.)|B/, '$1') === '$1';

    var tinymce = {
    majorVersion : '3',
    minorVersion : '5.7',
    releaseDate : '2012-09-20',


    If you search "zotero" through that js file there are 7 hits.
    I do not know why the default tiny_mce.js was modified in this way instead of making a separate js file for the zotero modifications. Perhaps a developed zotero plugin for tinymce might make it easier to integrate. Currently v 3.5.7 makes integration of later versions of tinymce difficult to fathom out.

    That is why I decided to leave this "well alone" and (for testing purposes) from note.html launch a separate instance of tinymce 4.4.3 (or now 4.5.0) just for my local experiments.

    My launch window button is placed in note.html body and not in the tinymce 3.5.7 toolbar.


    <body>
    <button onclick="openNewTinyMCE();">launch window</button>
    .... ... ...


    I added a new folder "extapps" in /resources/ and in there I dropped tinymce_4.4.3 which I had downloaded.

    My test script which launches a second window with second instance of tinymce is here ... placed in script tags in head of note.html


    function openNewTinyMCE() {

    // this test script expects /extapps/tinymce_4.4.3 folder
    // to be located in /resources/ in parent zotero.jar

    OpenWindow=window.open('../../extapps/tinymce_4.4.3/index.html', 'newwindow', config='height=400, width=800, toolbar=no, menubar=no, scrollbars=no, resizable=no, location=no, directories=no, status=no');

    }


    And an index.html has to be created in /resources/tinymce_4.4.3.
    Clearly the end goal is to have only one instance of tinymce running but for my purposes I'm just testing the default note editor alongside a later version.


    This may not answer your question but gives the steps I took to experiment with different versions of tinymce in an open instance of Zotero standalone. I also added svg capability but that is another story.
  • edited December 1, 2016

    This is kind of a funny situation, I need to paste my clipboard in zotero but to be able to paste it I need to disable zotero!
    There seems to be some conflict when pasting zotero://...
    Can you paste other protocol handler strings.

    mailto://me@example.com

    or href="$zotero://open-pdf/0_1A2B3C4D/10"

    I'm musing that you might be able to parse the content of tinymce active editor after pasting a non conflicting protocol string, as an alias. e.g. changing "$zotero" to "zotero" after pasting. But this is just a work around, not the answer.

    This below grabs the content of the note editor as a string to be processed. See tinyMCE API.

    tinyMCE.activeEditor.getContent();
  • The version I use for the external TinyMce is 4.5.0.

    I was able to follow your instructions. I now have a button in my regular Zotero note. When I click on it, a new window opens with the external 4.5.0 TinyMce. I wasn't yet able to work with the tinyMce.activeEditor.getContent. I will in more details tomorrow

    There are several things that are not yet clear to me with this approach. I will only use this external editor to paste the link with href="zotero://open-pdf/0_1A2B3C4D/10" and so my workflow will generally be like: I write a couple of sentences in an already existing regular Zotero note, and then I need to add a link with this href="zotero://open-pdf/0_1A2B3C4D/10", so I will need to open the external note with tinyMce 4.5.0. I can then click on a button to close the note but at this point I would like to get back to my note (with the updated content) in the regular Zotero. I generally work on many different notes at once so the external editor can't be my by default editor (maybeit could be if I get a way to pass to it the ZoteroID of the note).

    Even in this external TinyMce 4.50 note, I can't paste the href="zotero://open-pdf/0_1A2B3C4D/10". $zotero:// doesn't work (as it doesn't break the match zotero://) but something like zZotero:// or zoteroA:// work.

    +++
    I was also looking around for the other solution you talked about with the Protocole Handler. Does something like this will work?

    REGEDIT4

    [HKEY_CLASSES_ROOT\foo]
    @="URL:Python Start Zotero Pdf Protocol"
    "URL Protocol"=""

    [HKEY_CLASSES_ROOT\Python Start Zotero Pdf Protocol\shell]

    [HKEY_CLASSES_ROOT\Python Start Zotero Pdf Protocol\shell\open]

    [HKEY_CLASSES_ROOT\Python Start Zotero Pdf Protocol\shell\open\command]
    @="\"C:\\Program Files (x86)\\Zotero Standalone\\zotero.exe\" \"zotero://open-
    pdf/0_1A2B3C4D/10\""

    The trouble is to pass the argument zotero://open-
    pdf/0_1A2B3C4D/10\ not to zotero but to open-pdf because zotero standalone doesn't seems to be able to open.

    Also I don't really understand where I should add my custom protocol handler like if I choose "zoteroA://" where should I add it?


    Thanks a lot for your help and your time!

  • I do hope that you are regarding this exercise as just a walk through the various options in using tinymce, to gain experience by trial and error.

    So now, instead of the described test above which uses separate files (note.html in tinymce 3.5.7 and index.html in tinymce 4.5.0) let us now use just one note editor file .. the existing note.html .. amended to hold two instances of tinymce editor. But we still need to include the tinymce 4.5.0 library into /extapps/ folder to create the second instance.

    Then with two buttons we can toggle between the two tinymce instances.

    textarea1 = tinymce 3.5.7 active textarea
    textarea2 = tinymce 4.5.0 active textarea

    So, clicking note pane button1 - launch new editor, toggles to display tinymce 4.5.0 and also clones content of textarea1 into textarea2.

    And, clicking note pane button2 - close new editor, toggles back to display default tinymce 3.5.7 and also clones content of textarea2 into textarea1.

    Hopefully this answers your question ...

    I can then click on a button to close the note but at this point I would like to get back to my note (with the updated content) in the regular Zotero. 
    If (as you write) you only wish to use the second editor for pasting zotero:// links then the window can be greatly simplified cutting out menubar etc.

    I suggest that we forget about adding another custom URL protocol handler. This is adding more complexity. But at least you understand what these handlers do.

    I'll try to create a single note.html which does the above.

    The puzzle about zotero://open-pdf remains unsolved for now.
Sign In or Register to comment.