Upgrade StorageWord plugin status update
A lot of people have been posting questions about the MS Word plugin. Here's the current status:
1) We're aware of the problems people are having. (It's "alpha" for a reason.) Sometimes copying the contents to a new document will fix things, but generally speaking, there aren't individual fixes for these problems. The large majority of them should be resolved in the next version of the plugin.
2) John McCaskey contributed some major changes to the plugin that fix many of the existing problems (though there are still a few on the Zotero side that we'll need to fix). However, his changes are not upwards-compatible, so existing documents will break if loaded into the new version. We're currently working to restore upwards-compatibility to John's version and add a couple other features, and we hope to have a new release out very shortly.
3) Ian Laurenson has been working on making the current plugin compatible with OpenOffice.org. He's created a version that uses a different method of storing data. We haven't reviewed this version and don't yet know how feasible it will be to merge Ian's changes with John's, but we're hoping that it's possible.
We appreciate everyone's patience as we work to get the next version out, and a special thanks to John and Ian for their contributions.
1) We're aware of the problems people are having. (It's "alpha" for a reason.) Sometimes copying the contents to a new document will fix things, but generally speaking, there aren't individual fixes for these problems. The large majority of them should be resolved in the next version of the plugin.
2) John McCaskey contributed some major changes to the plugin that fix many of the existing problems (though there are still a few on the Zotero side that we'll need to fix). However, his changes are not upwards-compatible, so existing documents will break if loaded into the new version. We're currently working to restore upwards-compatibility to John's version and add a couple other features, and we hope to have a new release out very shortly.
3) Ian Laurenson has been working on making the current plugin compatible with OpenOffice.org. He's created a version that uses a different method of storing data. We haven't reviewed this version and don't yet know how feasible it will be to merge Ian's changes with John's, but we're hoping that it's possible.
We appreciate everyone's patience as we work to get the next version out, and a special thanks to John and Ian for their contributions.
I am from the ICE content management project, which hired Ian to do his work. We're very pleased with his approach of combining the two versions into one macro with compiler directives for the platform specific code. I'm assuming this will be a good way to go forward.
We've had a bit of discussion with John off-list about interoperability issues. He's got concerns with using bookmarks, but we really need to find an interoperable solution, and if the 'official' Zotero solution is not interoperable then we'll be really stuck as our application MUST work with both Word and Writer.
I'd also like to echo Bruce's concerns about unique citation IDs. It's essential that teams can share reference data along with documents. I'm working with tags - add a tag for each paper then make a saved search for the tag. I'm hoping then to be able to automatically export the refs for a paper into a defined location and sync them with colleagues using subversion, but any other solution that achieved the same end would be very welcome.
Could you comment on the issues raised here?
Peter
In terms of the thoughts of Peter, Bruce, and others interested in the development side: obviously this is a particularly thorny issue, dealing with multiple platforms, multiple word processors, multiple versions of the same word processor (e.g., Word changed formats this year, and is going to lose VB in 2008), and, in Peter's example, multiple users sharing references.
I have a somewhat pragmatic bent, so my initial thought is that we have to tackle word processor integration on several levels, from basic to advanced, at once. For instance, I think Dan Stillman's addition of drag-and-drop citations in the last beta is a stroke of genius that will satisfy a lot of (very casual) use cases (including providing unsophisticated but functional support for random word processors that don't have active dev communities). There will undoubtedly be subprojects for specific word processors and different versions of word processors given the peculiarities of those software packages and interface methods. I wish there was a single solution, though at the very least it seems (though perhaps I get out of my depth here a little) like there are pieces that can be reused across specific word processors.
For the most sophisticated use cases, I do think that the idea of IDs has great merit. (And IDs raise other interesting possibilities such as collaborative annotation.) I just remain a little unclear about how it would work and which pieces would be reusable for any particular word processor integration. Bruce and Peter--could you explain your thinking in a little more detail? What would we need to add to the main Zotero client code? Or does your vision expand beyond the client--e.g., would you want to see URIs on our server for references?
Leaving aside the ID issue for a bit I'd like to talk about the Word / Writer plugin.
There' s fundamental question to be answered here. Are you at Zotero concerned with interoperability between word and Writer via the .doc format? To me it seems obvious to seek it, and Ian's work on an integrated solution is a good idea.
I'm a pragmatist too, so that's why I'm arguing that even if it compromises the Word version a little (and I think the jury is still out anyway) the bookmark approach is a better bet because we can have a single code base and allow users to interchange documents. We can do some further investigation to see if some other mechanism will work, but Ian's pretty experienced in these matters so I think bookmarks may be the best we can do. In any case, we can make sure that the plugin can maintain backwards compatibility if we find a better mechanism - Ian has already looked at that for the Word plugin.
While you say there will be sub-projects for different word processors and different versions I don't think this is necessary for Word 97 - 2007 and OpenOffice.org 2.x - which at this point must account for 98% of the installed base around the world (that's a guess I admit). It definately covers the whole of the word processing we see at our university. The work Ian's done is well on the way to dealing with all those platforms in a single version - Word's APIs may be more stable than you think as MS have had to deal with a large installed base for a long time now. AFAIK they have not broken anything major since around version 97. Even Word 2007 can run macros written on earlier versions, although it is impossible to add toolbars to it.
I would not like to see separate Word and Writer version of this plugin when there's already an interoperable on on the table.
My project <http://ice.isq.edu.au/>; is prepared to fund any merge that needs to be done between John's and Ian's code to get a version we can all evaluate. Would you like us to do that?
Peter
First, I'd like to underline Dan C's note about VBA going away in future Mac versions of Office. Need a plan for that, as well for one which encompasses OOXML and ODF 1.2 (which both have more specialized/standardized ways to encode citations). I'd admit this shouldn't be the first priority, but one any design needs to keep in mind.
Second, to present big picture/long-term goals here, it is that citations are encoded in ways that are round-trippable between different word-processors and bibliographic databases. Peter's concerns absolutely address the first: he's got a university of Word and OOo users, and he knows they get pissed if their citations break when moving the documents around. Nothing more pragmatic than that ;-).
I'd be content to start with OOo and Word, BTW, but in the end it needs to be even more open-ended. I'd love to see apps like Google Docs support this stuff too, and the KOffice guys (who I know through the OASIS ODF work) also have expressed interest.
So I'd like to see moving in a design direction which can evolve this into this fairly gracefully.
Third, and related, on Dan's questions: It would just need to be able to take a URI and match it to a local record. That too. Look at it this way: if citations use good global IDs (URIs), they can be processed even if the user doesn't happen to have the metadata in their own database. And they can also more easily work in collaborative contexts in general.
Admittedly this is kind of a big issue with some tricky details (I outlined it here), but it is an important one to solve, particularly in a distributed environment.
I am hoping the biblio ontology work might also include best practice suggestions on the URI question, BTW.
Given the number of unique identifiers (URL, DOI, PMID, arXiv number, ISBN, etc.), a good server should probably have a list of equivalent identifiers which refer to the same resource. Few apps (including the Zotero client) really allow this & there are issues of how some IDs refer to different manifestations of the same resource.
One is to offer NeoOffice / Writer to the Mac users - once we get a bibliographic solution this is viable. I think MS are really shooting themselves in the foot with this one.
The other is that I believe we have a site license for Parallels and we definitely have a blanket license for Word so we could let them use the Windows version, provided they are using a recent Intel Mac.
The core code is in core.vb. There are platform-specific versions of the ZoteroRPC function in mac.vb and windows.vb that need to be combined with core.vb to create a working platform-specific plugin. The .dot and .dmg contain the latest merged versions (again, not currently upwards-compatible—John has suggested a migration approach, but I imagine that would change after a switch to bookmarks). Since this isn't the dev list, I'll mention that we obviously don't recommend that end users install an unreleased version.
We're happy to defer to others on the fields vs. bookmarks issue, especially since Simon's time is very limited at the moment. (Simon's the Zotero dev who created the plugin initially.) Interoperability seems like the goal, but are there any clear limitations with bookmarks that you'd like feedback from us on?
Also, good to see the collaboration here. I'd just urge that you maintain (or extend) the current functionality (drag-and-drop citations, autoomatic footnoting and switching between note and in-text, etc.).
There is a real problem with the current model. I have tried exporting references, getting someone else to import them and trying to format a bibliography with the plugin, but it fails because of the arbitrary IDs assigned by Zotero.
Looks like this will also be a problem even for a single user. What if I move computers and import my references? Unless you happen to be working with a portable Zotero installation from the start then looks like the bibliography plugin is not going to work long term.
Bruce has posted here before about using a better identifier. I'll quote what he wrote to me, with minor changes here.
The details need to be worked out, but I think we need rules; e.g.:
- if an online resource, use the perma-URL
- else if item has a DOI, use it in info URI form (info:doi:xxxxxxxx) [To which I add that handles in general should be treated this way, not just DOIs which are a special case of handles]
- if book, use worldcat.org accession number URI; else use urn:isbn
Now almost all Zotero resources will have a URL, so why can't we use that as the ID, with the other strategies if that fails, and in the worst case at least try to use something like a version of the title. This is not quite the utopian situation Bruce describes as people can still use dodgy URLs but it would at least solve the short term problem that the plugin is severely limited in its utility at present by the use of the current IDs.
It would also allow a new workflow where the plugin could find links, and ask Zotero to make new records for them if they don't exist.
There may be some issues with multiple citations pointing to the same URL, but the user should be warned about that situation anyway.
Does Zotero have the APIs to do this at the moment? Where would one look to change the code?
Once the next release is out (soon?!?), maybe support for Word 2007 won't take too long after all.
[caveat: notwithstanding the file format issue, there is how MS is handling interoperability with previous formats. In short, they are not. If you open an OOXML file with citations created in Word 2007 in Word 2003 or 2004, the citations will become plain text. This is an absolutely brain-dead solution on MS's part which seveerely complicates things not just for users, but for developers (like Zotero). That might suggest, then, not using the new citation field, and instead just using the same approach as in earlier versions.]
There's a bigger problem with MS, though: they are dropping support for VBA from the forthcoming Mac version. So it will no doubt have the same bibliographic and citation support, but no VBA.
<http://www.zotero.org/blog/feature-spotlight-zotero-microsoft-word-integration-alpha/>;
Namely (for mac):
<http://www.zotero.org/download/Zotero-Word-Plugin-1.0a1-Mac.dmg>;
are incorrect. I found the correct links at:
<http://www.zotero.org/documentation/microsoft_word_integration>;
namely:
<http://www.zotero.org/download/Zotero-Word-Plugin-1.0a2-Mac.dmg>;
Since the previous page is the one that comes up first in google searches, maybe you could update those links.
No one seems to have mentioned this, but it would be a great convenience. It may be that this is something that needs to be done on the Zotero side...
1. [Name], [Firstname]. [Title] ([Year]).
then one could manually reorder or add information into the footnotes:
1. cf. In [Year] [Firstname] [Name] suggested that a equals b ([Title]).
Thus it would still be possible to update the citation, without screwing up additional content.
We appreciate everyone's patience as we work to produce a functional Word plugin. As mentioned above, there are a couple development lines of the plugin contributed by outside developers (John and Ian). One of our developers is reviewing both versions, and we hope to release a new, supported, beta-quality version soon along with the first Release Candidate of Zotero 1.0.
In the meantime, if you want to use Zotero for a long paper, we'd suggest dragging items into Word to create static references (see the Quick Copy settings in the Zotero prefs) rather than using the released plugin, which is really an experimental alpha version not intended for daily use.
But again, we hope to have a working beta plugin out very soon.
Meanwhile, it would be really neat if there was a simple way to install earlier versions of Zotero other than the latest one. This would help some of us keep a stable system working despite Zotero moving ahead, and we can catch up with things when we have time to iron out the challenges that are sometimes part of the process.
The version I am using, with the small customisations I have made, is doing the job just fine for me. I will monitor the situation and submit some suggested code snippets when the add-in has become more stable.
2) When you do upgrade, the database schema is often irrevocably changed, but a pre-upgrade copy of the database is stored in your data directory. If you've saved a copy of the previous XPI, you can always reinstall it and put the previous database in place of the upgraded one.
Not sure if this is a place for feature requests but here's one:
It would be nice to have an option for only using the year (no parens) for in text citation. That's what I use in EndNote (and what will probably keep me stuck with it for a while). It give me much more control over the way I cite in text. Sometimes, I have an author with multiple publications, etc. The formats I use a lot are author (year), (author year), (author1 year, author2 year, etc.), author (year1, year2). Pretty much any text I write needs this level of flexibility. Would this be a nice feature or am I missing something?