Support for citations in Apple Pages.app

cems
Yes I realize that Word is the major word processor. However, there are also many even better endnote generating citation manager for Word. Other word processors like Pages.app entirely lack automated citation insertion methods.

In the case of pages.app I've already scoped out how hard this would be. Answer, it looks pretty easy in principle. Namely, pages docs are written in plaintext XML with a fairly guessable set of Tag names. one can insert new fake "font formats" containing the citation data then tag words inthe text body with these to mark the citation insertion points. All that has to happen is to save the document, inject the tags, then revert to saved and the program will load the tagged document back in.
  • bdarcus
    I wouldn't even consider doing anything with Pages until Apple shows where it's going with the application. I'd expect that to happen sometime in the next few months.

    I say this because I think there's a decent chance for significant file format changes in the application (the current file format is a mess, and only used by a single application).

    Finally, I've been telling Apple for the past two or three years they need to do something even more fundamental: they need to add citation support to the OS and application framework. One reason they haven't done it yet is because they claim not to hear enough demand.

    So I'd suggest people who care about this send in a request to Apple. Why, after all, should developers have to invent spoecific code for every application that might to use citations?
  • cems
    Hi,
    Well apple has a new version of Pages out. An office suite. Maybe you could update your comment about when Pages support for Zotero might be considered?
  • bdarcus
    First, I don't speak for Zotero.

    Second, I think my earlier comments stand. I don't see any evidence that Pages is any more open or extensible than previous versions (though I could be wrong; might be that Apple just isn't promoting it). For OS X applications, one should look at a generic solution that works across Cocoa applications. E.g. if possible (and I think it is) a citation solution for Pages ought to work for Mellel, Nisus Writer, etc.
  • MacZoteroAdvocate
    Apple's Pages now boasts MathType and Endnote support.

    My clients would love to be able to use Zotero with Pages.
  • lizrodwell
    Yes, Zotero for Pages '09 would be lovely....
  • bdarcus
    As I said on another thread, then write Apple. They were the ones that (stupidly) chose to limit the citation and bibliography support to Endnote, so they're the ones that need to fix that.

    Note, though: I don't speak for Zotero, and so don't know if they would support Pages even with changes from Apple.
  • Simon
    edited January 19, 2009
    As of 1.5, we have a new modular Python infrastructure that makes writing word processor plug-ins much simpler. We currently use this for the Word 2008 plug-in and the 2.0 version of the OpenOffice plug-in. It is relatively easy to write a new module for this infrastructure to support another word processor, provided this word processor exposes a field API via Python (or via AppleScript). Nisus Writer and Mellel may expose the requisite API, although given the number of other things they have on their plate at the moment, the Zotero team is unlikely to release a plug-in for these word processors in the future. (This does not, however, preclude an enterprising individual from writing such a plug-in him/herself, as the API is fairly well-documented in the source files.)

    Unfortunately, the Pages/EndNote® integration appears to use a field API to which we do not have access. (In my testing, I was actually unable to get the integration to work at all, but I noticed that insertion of an EndNote bibliography is associated with a special <sf:bibliography> tag in saved documents.) We might be able to use hyperlinks or bookmarks (both of which are accessible from the Pages UI) in lieu of fields, but there is no way to manipulate these through AppleScript either. Thus, for now, it looks like Apple has granted Thomson Reuters a monopoly on citation management using Pages.

    If you are interested in Zotero integration with Pages, you are encouraged to contact the Pages team through the appropriate feedback form.
  • wisam
    I sent pages 09 team a message already. I would like to have page 09 work with Zotero too.
  • xaxa
    edited May 3, 2009
    Dear Simon, dear all
    There should be a way to integrate Zotero in Apple Pages 09. Sente, starting with version 5.7.6, uses a new Pages xml parser and injects sf:merge-field tags. This might be the way to go.
    Have a quick look at it - first it defines the merge-fields in the header; one for each reference plus one for the bibliography:

    <sf:fields><sf:merge-field sf:property="SenteCitation" sfa:ID="SFWPMergeField-0"><sf:table-field sfa:string="Aegerter 2004"/></sf:merge-field><sf:merge-field sf:property="SenteBibliography" sfa:ID="SFWPMergeField-1"><sf:table-field sfa:string="Bibliography"/></sf:merge-field></sf:fields>
    </sf:merge-field><sf:merge-field sf:property="SenteBibliography" sfa:ID="SFWPMergeField-1"><sf:table-field sfa:string="Bibliography"/></sf:merge-field>


    ... and then, in the body (sf:text-body), these are referenced:
    <sf:merge-field-ref sfa:IDREF="SFWPMergeField-0">(Aegerter, 2004)</sf:merge-field-ref>

    ... and for the bibliography:
    <sf:merge-field-ref sfa:IDREF="SFWPMergeField-1">My Bibliography, etc, etc</sf:merge-field-ref>

    Looks quite simple, doesn't it? I am good with applescript and xml, but unfortunately I have not much experience with python, otherwise I'd try to write the plug-in myself.
    Anybody out there who could do it?

    I'd love to move to Zotero. Its just that I would never go back to MS Word and Pages is the only option that handles my South-Asian Script needs properly.
  • dstillman
    xaxa: That's not a plugin, though. Sente just copies temporary plaintext codes via drag and drop and then post-processes the file separately. That's a fine approach, but it's not, strictly speaking, "integration", which is what Simon was referring to.

    The next beta of Zotero will offer post-processing of RTF files, and that feature could probably be extended to work with any text-based file format (zipped or otherwise), including the Pages format.
  • xaxa
    Dan, good to see that this forum is active :)

    Three things:
    - There is no need to do the same as Sente does. One could directly work on the xml. Save, inject xml-node, reload - this is fast. At least fast enough. (I use it for a quick and dirty implementation of simple cross-references in Pages 09)
    - on the other hand: post-processing has its good sides. It makes collaboration with MS-Word users easier, because fields hidden in proprietary pages xml and used in an "unproper" way will probably not be retained when exporting to Word and back to Pages. Therefore the possibility of "unscanned" documents has a certain advantage over direct "integration". (Although I haven't tested yet what happens to merge-fields on import/export).
    - RTF post-processing is very good news and makes Zotero usable in many more situations. Any ETA yet?
  • dstillman
    There is no need to do the same as Sente does. One could directly work on the xml. Save, inject xml-node, reload - this is fast. At least fast enough. (I use it for a quick and dirty implementation of simple cross-references in Pages 09)
    Well, if you're talking about editing the file directly, it's still a form of post-processing and not an integrated plugin, no? You'd just be inserting Pages fields rather than plaintext citations, and you presumably still have to close Pages first. I don't believe the current RTF scan implementation on the trunk inserts RTF fields, but it could, and a Pages mode could do the same.

    Also, if we're editing the document through anything other than an official API, I think we'd generate a copy of the file rather than edit the original, which would be too risky.
    RTF post-processing is very good news and makes Zotero usable in many more situations. Any ETA yet?
    Next beta is in final testing now.
  • xaxa
    Dan, of course, you are right - it would be ugly, a hack, temporary, etc. etc. I am not expecting anybody of you folks to waste time on this. But maybe this leads to a next idea that suddenly makes things seem possible and worth considering...

    I just did some testing: merge-fields are _partly_ retained during export to Word and reimport to Pages (and there are some warnings that pop up). Partly meaning, the sf:property attribute is retained (and one could store an UUID there), most of the rest including the sfa:ID is lost.

    URLs/Hyperlinks seem to be no problem when moved between Pages and Word.

    I'll have a look whether it is possible to inject/modify xml-nodes and reload a pages document without closing it.

    looking forward to the next beta.

    xaxa
  • bdarcus
    Dan, of course, you are right - it would be ugly, a hack, temporary, etc. etc. I am not expecting anybody of you folks to waste time on this.
    And just to underline the point some of us have been making here: Apple chose to integrate a closed citation API into Pages. Even if it were open, I don't think it should be any kind of priority for Zotero developers to work on integration with a marginal word-processor, that is available only on a single platform.

    Don't get me wrong; I'm typing this from a Mac. But if people care about these sorts of things, I strongly recommend not using Pages. Use Word or OOo instead. If OOo has some problems with language handling that precludes this for some reason, file a bug report.

    If that's not attractive to you and you're attached to Pages, then I think you need to accept the limitations of your choice (like needing to hack your own, far-from-ideal, solution). It's quite enough work for the developers just to get decent support for Word and OOo.
  • dstillman
    Bruce: I certainly wasn't volunteering our time. I imagine the file scanning architecture in the trunk, which currently only handles RTF, could fairly easily be extended to take additional modules (detect, scan, insert) for other formats, and those could be written by others.
  • xaxa
    Dan: I will look into this once the beta is out.

    In the mean time another, different approach: it might actually still be possible to work on an integration (in Dan's sense) with Applescript and bookmarks, just as one of the modes in the MS Word PlugIn for Mac.
    I only spent about half an hour on this, but I see a way to a) enter a citation with AppleScript (with text and a Zotero Ref as the bookmark's name) and b) to edit an existing citation, retrieving the bookmarks name and text based on the current cursor position. This would then have to be processed by Zotero and the changes added to the document again, just as above). See scripts below. A similar thing should be possible with Add and Edit Bibliography. "Set Document Preferences" should not be too difficult to handle. Only "Refresh" would be done writing to the pages XML (and yes, there is a way to change the xml of an open Pages document and reload it).

    The Scripts:

    -- Insert Citation
    -- assuming the following values were retrieved from Zotero
    set bref to "ZOTERO_BREF_3NMxIRfD3Num"
    set btext to "(Salick et al. 2006)"
    insertCitation(bref, btext)

    on insertCitation(bref, btext)
    tell application "Pages"
    activate
    set selection to bref
    tell application "System Events" to click menu item "Bookmark" of menu "Insert" of menu bar item "Insert" of menu bar 1 of application process "Pages"
    set sel to selection
    set startoffset to (character offset of sel)
    set endoffset to startoffset + (length of sel) - 1
    select (characters startoffset thru endoffset) of document 1
    set selection to btext
    end tell
    end insertCitation



    -- Edit Citation: returns {bref, btext} if the cursor is inside a bookmark, otherwise false
    getCurrentCitation()

    on getCurrentCitation()
    tell application "Pages"
    set mydoc to path of document 1
    save document 1
    end tell

    -- this part relies on the Satimage XMLLib Osax. For a final solution independent of any osax we would use xmllib on the command line.
    set myfile to mydoc & "/index.xml.gz"
    set xml to XMLOpen myfile
    set ns to {{nsprefix:"sfa", nsurl:"http://developer.apple.com/namespaces/sfa"}, {nsprefix:"sf", nsurl:"http://developer.apple.com/namespaces/sf"}, {nsprefix:"xsi", nsurl:"http://www.w3.org/2001/XMLSchema-instance"}, {nsprefix:"sl", nsurl:"http://developer.apple.com/namespaces/sl"}}
    XMLSetContext xml namespace ns

    set insertionPoint to XMLXPath xml with "//sf:insertion-point"
    -- is the insertion point inside a bookmark?
    set mybookmark to (XMLXPath insertionPoint with "ancestor::sf:bookmark") as item
    if mybookmark ≠ {} then
    -- ok, we are inside a bookmark, so lets find the bookmarks name, etc.
    set bref to XMLGetText (XMLXPath mybookmark with "@sf:name") as item
    set btext to XMLGetText mybookmark
    return {bref, btext}
    else
    return false
    end if
    end getCurrentCitation
  • bdarcus
    edited May 3, 2009
    @Dan: yes, I know. The comment was aimed at people requesting this.

    Of course, if we collectively manage to solve the interop issues I've been harping on, users have to worry less about any given tool (like Zotero) supporting their workflow. In theory, for example, there's no reason why Andrea's CSL implementation couldn't also do this sort of thing (and he's intending that it do just that).
  • vanamee
    Hi,

    Is there any hope for using Zotero in Pages? Please let me know if there are changes since May when the last entry was posted.

    Many thanks,

    - Eva
  • dstillman
    It's not equivalent to the Word/OO plugins, but Zotero 2.0 supports post-processing of RTF documents, which you can create from Pages.
  • jposner
    please create a plugin for pages. Using the RTF is a work around and wont work for scientific documents with lots of figures, etc.

    If you need the interested parties to bug APPLE, then let us know specifically what we should ask for.
  • adamsmith
    edited November 2, 2009
    read the thread above. Apple has created a closed source API for pages - Zotero - even there were resources for writing and maintaining a third/fourth plugin - can't create a plugin for pages. Even beyond that, though, I doubt that the Zotero core team could or should put up the necessary time.
    Zotero is open source - if there is demand, I'd suggest finding a third party to code a plugin.

    P.S. good luck bugging apple: In spite of its bay-area hippie image, they're notorious for keeping things closed and tightly controlled.
  • Manusnake
    The RTF Scan works, but it's far from convenient when working with Pages. I agree Apple isn't the most open partner to work with, but their products are pretty good (I ditched MS Office 2008 for iWork 09, which is far lighter on my MB and has all the features I need).

    According to these forums and some pictures of Zotero "classes", Zotero has a very big Mac community (in term of percentage), so yes, doing something that works with Pages is relevant. Personally, I've made alot of my fellow graduate students switch to Zotero, except for the majority of iWork users, who represent the majority of Mac users in my university (have to admit, the local computer shop did a lot of advertising for Macs and iWork).

    I know the Sente example bugs people, but it is a good one, since it works. However, I do prefer Zotero over Sente (and not just for the price tag), so it would really be nice (I filled a suggestion on Apple's site btw). Maybe they'll open up their APIs with the rumored iWork X, but we cannot count on it. I'm no programmer, but it doesn't seem impossible to do according to older posts.

    Dan, I hope you and Zotero's team can manage this. I know we're demanding alot, especially for a free product, but it's this kind of thing that do make a difference, and I know you know it (I don't doubt of your effort nor sincerity, just wanted to make a point).

    Thanks!

    Manu
  • adamsmith
    I think you're misreading the above.
    There is actually no way of doing this right without an open API.
    The Sente example doesn't bug people, but as Dan pointed out it's essentially just a fancier version of an RTF scan, so it's not actually a solution.

    If you feel so strongly about this, find a couple of Pages users, join together and contract an independent programmer to write a pages plugin/hack. It's probably going to be less than buying 10 copies of Endnote. But you can't really expect open-source programmers to spend a lot of their very, very scarce time because people insist on using (in terms of total users quite marginal) software that does everything to prevent interoperability with open source.
  • bdarcus
    Another thread that really needs to be closed.
  • Manusnake
    edited November 15, 2009
    Ok, got the point, thanks for the clarification.

    I don't feel "so strongly" about it, I just thought it would be good for Zotero. I can use both Zotero and Pages without too much trouble (I copy paste from my library into my document, "the old fashioned way", a bit slower of course but not so bad when you're used to it).

    You're right, I misread, I thought it was easier to do than it apparently is. Sorry for that.

    And you know, people (at least those I know) don't insist on using Pages for the sake of it (even if fanboys probably use it because it's made by Apple, which is a very bad reason).

    Maybe they just thought MS Office for Mac was slow and a bloatware and that both NeoO and OOo were lacking some ease of use and integration (I don't blame them, I know what it is to do stuff for free aside from your work and all the rest, I actually manage my department's review in history).

    They choose Pages because it fitted their needs, that's all. No need to judge them. Even if I agree Apple could improve its support from open source (by making iWork compatible with ODF for example). I actually hesitated a lot before switching.

    Now, I don't want to start anything here, just to make my thoughts clear (even if I'm the only one to care), no need to flame or anything.

    bdarcus> thanks for the constructive comment, wasn't necessary.
  • bdarcus
    @Manusnake: the comment was not aimed it you. It was aimed at the forum moderators, who have previously closed similar long-running "please support X app" threads.
  • Manusnake
    Ok, my bad.
  • dstillman
    Manusnake: Based on my reading of his post, adamsmith was neither flaming you nor denigrating Pages as a product, but merely clarifying the situation and offering a possible path forward. He also said pretty much the exact same thing in his post immediately before yours.

    So I'm going to close this thread to save everyone time. The situation is as it stands above. If there's third-party interest in figuring out some hack around the Apple-imposed restrictions (or in extending RTF Scan to support the Pages XML format), we'll reopen.
This discussion has been closed.