Upgrade StorageSupport for citations in Apple Pages.app
Yes I realize that Word is the major word processor. However, there are also many even better endnote generating citation manager for Word. Other word processors like Pages.app entirely lack automated citation insertion methods.
In the case of pages.app I've already scoped out how hard this would be. Answer, it looks pretty easy in principle. Namely, pages docs are written in plaintext XML with a fairly guessable set of Tag names. one can insert new fake "font formats" containing the citation data then tag words inthe text body with these to mark the citation insertion points. All that has to happen is to save the document, inject the tags, then revert to saved and the program will load the tagged document back in.
In the case of pages.app I've already scoped out how hard this would be. Answer, it looks pretty easy in principle. Namely, pages docs are written in plaintext XML with a fairly guessable set of Tag names. one can insert new fake "font formats" containing the citation data then tag words inthe text body with these to mark the citation insertion points. All that has to happen is to save the document, inject the tags, then revert to saved and the program will load the tagged document back in.
This discussion has been closed.
I say this because I think there's a decent chance for significant file format changes in the application (the current file format is a mess, and only used by a single application).
Finally, I've been telling Apple for the past two or three years they need to do something even more fundamental: they need to add citation support to the OS and application framework. One reason they haven't done it yet is because they claim not to hear enough demand.
So I'd suggest people who care about this send in a request to Apple. Why, after all, should developers have to invent spoecific code for every application that might to use citations?
Well apple has a new version of Pages out. An office suite. Maybe you could update your comment about when Pages support for Zotero might be considered?
Second, I think my earlier comments stand. I don't see any evidence that Pages is any more open or extensible than previous versions (though I could be wrong; might be that Apple just isn't promoting it). For OS X applications, one should look at a generic solution that works across Cocoa applications. E.g. if possible (and I think it is) a citation solution for Pages ought to work for Mellel, Nisus Writer, etc.
My clients would love to be able to use Zotero with Pages.
Note, though: I don't speak for Zotero, and so don't know if they would support Pages even with changes from Apple.
Unfortunately, the Pages/EndNote® integration appears to use a field API to which we do not have access. (In my testing, I was actually unable to get the integration to work at all, but I noticed that insertion of an EndNote bibliography is associated with a special <sf:bibliography> tag in saved documents.) We might be able to use hyperlinks or bookmarks (both of which are accessible from the Pages UI) in lieu of fields, but there is no way to manipulate these through AppleScript either. Thus, for now, it looks like Apple has granted Thomson Reuters a monopoly on citation management using Pages.
If you are interested in Zotero integration with Pages, you are encouraged to contact the Pages team through the appropriate feedback form.
There should be a way to integrate Zotero in Apple Pages 09. Sente, starting with version 5.7.6, uses a new Pages xml parser and injects sf:merge-field tags. This might be the way to go.
Have a quick look at it - first it defines the merge-fields in the header; one for each reference plus one for the bibliography:
<sf:fields><sf:merge-field sf:property="SenteCitation" sfa:ID="SFWPMergeField-0"><sf:table-field sfa:string="Aegerter 2004"/></sf:merge-field><sf:merge-field sf:property="SenteBibliography" sfa:ID="SFWPMergeField-1"><sf:table-field sfa:string="Bibliography"/></sf:merge-field></sf:fields>
</sf:merge-field><sf:merge-field sf:property="SenteBibliography" sfa:ID="SFWPMergeField-1"><sf:table-field sfa:string="Bibliography"/></sf:merge-field>
... and then, in the body (sf:text-body), these are referenced:
<sf:merge-field-ref sfa:IDREF="SFWPMergeField-0">(Aegerter, 2004)</sf:merge-field-ref>
... and for the bibliography:
<sf:merge-field-ref sfa:IDREF="SFWPMergeField-1">My Bibliography, etc, etc</sf:merge-field-ref>
Looks quite simple, doesn't it? I am good with applescript and xml, but unfortunately I have not much experience with python, otherwise I'd try to write the plug-in myself.
Anybody out there who could do it?
I'd love to move to Zotero. Its just that I would never go back to MS Word and Pages is the only option that handles my South-Asian Script needs properly.
The next beta of Zotero will offer post-processing of RTF files, and that feature could probably be extended to work with any text-based file format (zipped or otherwise), including the Pages format.
Three things:
- There is no need to do the same as Sente does. One could directly work on the xml. Save, inject xml-node, reload - this is fast. At least fast enough. (I use it for a quick and dirty implementation of simple cross-references in Pages 09)
- on the other hand: post-processing has its good sides. It makes collaboration with MS-Word users easier, because fields hidden in proprietary pages xml and used in an "unproper" way will probably not be retained when exporting to Word and back to Pages. Therefore the possibility of "unscanned" documents has a certain advantage over direct "integration". (Although I haven't tested yet what happens to merge-fields on import/export).
- RTF post-processing is very good news and makes Zotero usable in many more situations. Any ETA yet?
Also, if we're editing the document through anything other than an official API, I think we'd generate a copy of the file rather than edit the original, which would be too risky. Next beta is in final testing now.
I just did some testing: merge-fields are _partly_ retained during export to Word and reimport to Pages (and there are some warnings that pop up). Partly meaning, the sf:property attribute is retained (and one could store an UUID there), most of the rest including the sfa:ID is lost.
URLs/Hyperlinks seem to be no problem when moved between Pages and Word.
I'll have a look whether it is possible to inject/modify xml-nodes and reload a pages document without closing it.
looking forward to the next beta.
xaxa
Don't get me wrong; I'm typing this from a Mac. But if people care about these sorts of things, I strongly recommend not using Pages. Use Word or OOo instead. If OOo has some problems with language handling that precludes this for some reason, file a bug report.
If that's not attractive to you and you're attached to Pages, then I think you need to accept the limitations of your choice (like needing to hack your own, far-from-ideal, solution). It's quite enough work for the developers just to get decent support for Word and OOo.
In the mean time another, different approach: it might actually still be possible to work on an integration (in Dan's sense) with Applescript and bookmarks, just as one of the modes in the MS Word PlugIn for Mac.
I only spent about half an hour on this, but I see a way to a) enter a citation with AppleScript (with text and a Zotero Ref as the bookmark's name) and b) to edit an existing citation, retrieving the bookmarks name and text based on the current cursor position. This would then have to be processed by Zotero and the changes added to the document again, just as above). See scripts below. A similar thing should be possible with Add and Edit Bibliography. "Set Document Preferences" should not be too difficult to handle. Only "Refresh" would be done writing to the pages XML (and yes, there is a way to change the xml of an open Pages document and reload it).
The Scripts:
-- Insert Citation
-- assuming the following values were retrieved from Zotero
set bref to "ZOTERO_BREF_3NMxIRfD3Num"
set btext to "(Salick et al. 2006)"
insertCitation(bref, btext)
on insertCitation(bref, btext)
tell application "Pages"
activate
set selection to bref
tell application "System Events" to click menu item "Bookmark" of menu "Insert" of menu bar item "Insert" of menu bar 1 of application process "Pages"
set sel to selection
set startoffset to (character offset of sel)
set endoffset to startoffset + (length of sel) - 1
select (characters startoffset thru endoffset) of document 1
set selection to btext
end tell
end insertCitation
-- Edit Citation: returns {bref, btext} if the cursor is inside a bookmark, otherwise false
getCurrentCitation()
on getCurrentCitation()
tell application "Pages"
set mydoc to path of document 1
save document 1
end tell
-- this part relies on the Satimage XMLLib Osax. For a final solution independent of any osax we would use xmllib on the command line.
set myfile to mydoc & "/index.xml.gz"
set xml to XMLOpen myfile
set ns to {{nsprefix:"sfa", nsurl:"http://developer.apple.com/namespaces/sfa"}, {nsprefix:"sf", nsurl:"http://developer.apple.com/namespaces/sf"}, {nsprefix:"xsi", nsurl:"http://www.w3.org/2001/XMLSchema-instance"}, {nsprefix:"sl", nsurl:"http://developer.apple.com/namespaces/sl"}}
XMLSetContext xml namespace ns
set insertionPoint to XMLXPath xml with "//sf:insertion-point"
-- is the insertion point inside a bookmark?
set mybookmark to (XMLXPath insertionPoint with "ancestor::sf:bookmark") as item
if mybookmark ≠ {} then
-- ok, we are inside a bookmark, so lets find the bookmarks name, etc.
set bref to XMLGetText (XMLXPath mybookmark with "@sf:name") as item
set btext to XMLGetText mybookmark
return {bref, btext}
else
return false
end if
end getCurrentCitation
Of course, if we collectively manage to solve the interop issues I've been harping on, users have to worry less about any given tool (like Zotero) supporting their workflow. In theory, for example, there's no reason why Andrea's CSL implementation couldn't also do this sort of thing (and he's intending that it do just that).
Is there any hope for using Zotero in Pages? Please let me know if there are changes since May when the last entry was posted.
Many thanks,
- Eva
If you need the interested parties to bug APPLE, then let us know specifically what we should ask for.
Zotero is open source - if there is demand, I'd suggest finding a third party to code a plugin.
P.S. good luck bugging apple: In spite of its bay-area hippie image, they're notorious for keeping things closed and tightly controlled.
According to these forums and some pictures of Zotero "classes", Zotero has a very big Mac community (in term of percentage), so yes, doing something that works with Pages is relevant. Personally, I've made alot of my fellow graduate students switch to Zotero, except for the majority of iWork users, who represent the majority of Mac users in my university (have to admit, the local computer shop did a lot of advertising for Macs and iWork).
I know the Sente example bugs people, but it is a good one, since it works. However, I do prefer Zotero over Sente (and not just for the price tag), so it would really be nice (I filled a suggestion on Apple's site btw). Maybe they'll open up their APIs with the rumored iWork X, but we cannot count on it. I'm no programmer, but it doesn't seem impossible to do according to older posts.
Dan, I hope you and Zotero's team can manage this. I know we're demanding alot, especially for a free product, but it's this kind of thing that do make a difference, and I know you know it (I don't doubt of your effort nor sincerity, just wanted to make a point).
Thanks!
Manu
There is actually no way of doing this right without an open API.
The Sente example doesn't bug people, but as Dan pointed out it's essentially just a fancier version of an RTF scan, so it's not actually a solution.
If you feel so strongly about this, find a couple of Pages users, join together and contract an independent programmer to write a pages plugin/hack. It's probably going to be less than buying 10 copies of Endnote. But you can't really expect open-source programmers to spend a lot of their very, very scarce time because people insist on using (in terms of total users quite marginal) software that does everything to prevent interoperability with open source.
I don't feel "so strongly" about it, I just thought it would be good for Zotero. I can use both Zotero and Pages without too much trouble (I copy paste from my library into my document, "the old fashioned way", a bit slower of course but not so bad when you're used to it).
You're right, I misread, I thought it was easier to do than it apparently is. Sorry for that.
And you know, people (at least those I know) don't insist on using Pages for the sake of it (even if fanboys probably use it because it's made by Apple, which is a very bad reason).
Maybe they just thought MS Office for Mac was slow and a bloatware and that both NeoO and OOo were lacking some ease of use and integration (I don't blame them, I know what it is to do stuff for free aside from your work and all the rest, I actually manage my department's review in history).
They choose Pages because it fitted their needs, that's all. No need to judge them. Even if I agree Apple could improve its support from open source (by making iWork compatible with ODF for example). I actually hesitated a lot before switching.
Now, I don't want to start anything here, just to make my thoughts clear (even if I'm the only one to care), no need to flame or anything.
bdarcus> thanks for the constructive comment, wasn't necessary.
So I'm going to close this thread to save everyone time. The situation is as it stands above. If there's third-party interest in figuring out some hack around the Apple-imposed restrictions (or in extending RTF Scan to support the Pages XML format), we'll reopen.