Great! I'll put some time in some of the case archive translators in the next week or two, to try to improve the quality of the data Zotero gets from them.
I'm a law student in the UK with some experience with dynamic languages. I'd like to help!
I'm afraid I've got a very newbie question to start off with though. I've seen these pages http://code.google.com/p/zotero-for-lawyers/wiki/BAILII , but as an end-user I'm not sure what I'm meant to do with those .sql snippets to bring those translators into Zotero. Is this method of translator distribution out of date now in 2.x? Must I install Scaffold to import them? (if so how are non-dev's meant to use them?)
yes, that's long out of date - translators are simple .js files that are placed in the translator folder of Zotero and automatically loaded on FF restart.
See here for a current guide.
http://www.zotero.org/support/dev/how_to_write_a_zotero_translator_plusplus
I'm not sure if the .sql there is at all helpful anymore.
The documentation on creating and modifying translators is a bit of a mess right now, and the guide adamsmith linked to is incomplete and not yet suitable for reading.
In short, translators are stored as individual files in your Zotero profile, and can be edited with any tool, but many authors of Zotero translators (me included) prefer to use the Zotero 2.0-compatible Scaffold 2.0: http://www.zotero.org/support/dev/scaffold (the info on this page is up-to-date).
Welcome aboard! Development for law has been languishing, but not for want of good intentions; Zotero 2.1 will have much-improved infrastructure for legal styles, and this is certainly a good time to dust off the xLII translators.
If you do any work on legal translators, please consider signing up for BitBucket and adding folders for them in the following development repository -- it's in need of a little housecleaning at the moment, but that can be done if the archive becomes more active:
Sounds like the best thing to do would be to just look through some of the standard translators (http://www.zotero.org/support/dev/creating_translators_for_sites sure didn't inspire confidence in the current docs).
@fbennett, good stuff, will get in on that.
Some more basic questions:
1) A cursory search doesn't show up any translators for UK statute law sites (statutelaw.gov.uk, legislation.gov.uk). Is there an official central resource for translators, something like addons.mozilla.org, that I should be checking for work already done?
2) @fbennett, I only see CornellLII in the bitbucket repo. Further up this thread I see @grieth called for the initial plugin to work across all LII sites. Is this in any way feasible? What is the level of consistency / coordination across LII sites?
3) Following on from 2), what's the most up to date LII translator I should be basing my work on? I'll be starting with BAILII, since that's what I'm most familiar with. Looks like the CornellLII translator is probably the most up to date, but BAILII provides no META headers or any other metadata as far as I know, so not sure if it's worth working off the Cornell code. Thoughts?
1) no there isn't. Most stuff has been discussed on zotero-dev in the past, but there isn't much that exists and has not already been included in Zotero.
(and for what it's worth, while Rintze is right that the guide I link to is very deficient, it's currently the best there is - if you, on the way, want to help improve it, it's in Wiki form, so that'd be much apprecidated).
1) A cursory search doesn't show up any translators for UK statute law sites (statutelaw.gov.uk, legislation.gov.uk). Is there an official central resource for translators, something like addons.mozilla.org, that I should be checking for work already done?
The up-to-date translators are all shipped with Zotero. Anything else lying around can be assumed to be a work in progress, or abandoned code.
2) @fbennett, I only see CornellLII in the bitbucket repo. Further up this thread I see @grieth called for the initial plugin to work across all LII sites. Is this in any way feasible? What is the level of consistency / coordination across LII sites?
There is no coordination to speak of. LII at Cornell was the first to use the LII moniker. AustLII leveraged the name when they started up, and the same is true of the rest. The people who run these sites attend the same conferences and know one another, but for the most part they don't know one another's code (except when borrowing and hacking it).
3) Following on from 2), what's the most up to date LII translator I should be basing my work on? I'll be starting with BAILII, since that's what I'm most familiar with. Looks like the CornellLII translator is probably the most up to date, but BAILII provides no META headers or any other metadata as far as I know, so not sure if it's worth working off the Cornell code. Thoughts?
The Cornell LII translator is completely separate. Your best bet would be to start from scratch, or to use the existing BAILII draft translator as a starting point.
Re working from old translators, it's worth noting that there are two new page retrieval functions that make things much easier. Older translators needed to do async calls with a registered callback to process separately retrieved pages. This can now be coded synchronously with the functions retrieveSource() (text only, but works across domains), or retrieveDocument() (DOM, but works only in the current domain). The best reference for using the new functions is the source code, available for viewing here:
Also, re statutes, we need to work up conventions for what metadata goes in which fields for this item type, as well as requests for additional fields, as required. This needs a community discussion to settle the issues, but our population has until now been very small ... if you have input, feel free to file tickets on zotero-legal-utils.
I see that I was overtaken by events, and the existing BAILII translator was included in the most recent update (2.1b2). There are a couple of improvements I could think of:
All judges are usually listed between , and the whole list of them are put into one Author entry (e.g. Author is "LORD JUSTICE DENNING<br>LORD JUSTICE MORRIS<br>and<br>LORD JUSTICE PARKER"). Seems like the first step of splitting on <br> would be easy enough, although there are edge cases with titles (like Master of Rolls) and names split over multiple lines. Can't think of an elegant solution for that, I envisage something like if( line is like /^Lord|Lord Justice|Sir|Sir Justice|MR|blah$/ ) { line = line . nextLine; }
The "BAILII Citation Number" is inserted as a note attachment. I think it's taken from the , maybe . However, later in the page a more complete list of citations appears after the text "Cite as:". This usually includes Weekly Law Reports citations and the like which would be more useful to law students if no-one else. Haven't looked at the DOM for that section but it seems it would be just as easy to have all the citations pulled into the note.
No snapshot is taken by default, but that would just be my personal preference. Don't know what the general policy on that is for translators.
Any other BAILII users have suggestions for improvements I could look at?
Somewhat relatedly, the only reason I like to keep snapshots is that I like to annotate things quite thoroughly. What are the technological barriers to simply allowing annotations on live web pages, without requiring a snapshot? I've used Diigo and other web annotation plugins in the past, so it's clearly doable. I'd be interested to know whether Zotero just hasn't had the tuits for it or if there's some technological reason it's not implemented.
Upgrade Storage
(Note: um, it's the other way around ...)
I'm a law student in the UK with some experience with dynamic languages. I'd like to help!
I'm afraid I've got a very newbie question to start off with though. I've seen these pages http://code.google.com/p/zotero-for-lawyers/wiki/BAILII , but as an end-user I'm not sure what I'm meant to do with those .sql snippets to bring those translators into Zotero. Is this method of translator distribution out of date now in 2.x? Must I install Scaffold to import them? (if so how are non-dev's meant to use them?)
Thanks!
See here for a current guide.
http://www.zotero.org/support/dev/how_to_write_a_zotero_translator_plusplus
I'm not sure if the .sql there is at all helpful anymore.
In short, translators are stored as individual files in your Zotero profile, and can be edited with any tool, but many authors of Zotero translators (me included) prefer to use the Zotero 2.0-compatible Scaffold 2.0: http://www.zotero.org/support/dev/scaffold (the info on this page is up-to-date).
If you do any work on legal translators, please consider signing up for BitBucket and adding folders for them in the following development repository -- it's in need of a little housecleaning at the moment, but that can be done if the archive becomes more active:
http://bitbucket.org/fbennett/zotero-legal-utils/
Sounds like the best thing to do would be to just look through some of the standard translators (http://www.zotero.org/support/dev/creating_translators_for_sites sure didn't inspire confidence in the current docs).
@fbennett, good stuff, will get in on that.
Some more basic questions:
1) A cursory search doesn't show up any translators for UK statute law sites (statutelaw.gov.uk, legislation.gov.uk). Is there an official central resource for translators, something like addons.mozilla.org, that I should be checking for work already done?
2) @fbennett, I only see CornellLII in the bitbucket repo. Further up this thread I see @grieth called for the initial plugin to work across all LII sites. Is this in any way feasible? What is the level of consistency / coordination across LII sites?
3) Following on from 2), what's the most up to date LII translator I should be basing my work on? I'll be starting with BAILII, since that's what I'm most familiar with. Looks like the CornellLII translator is probably the most up to date, but BAILII provides no META headers or any other metadata as far as I know, so not sure if it's worth working off the Cornell code. Thoughts?
(and for what it's worth, while Rintze is right that the guide I link to is very deficient, it's currently the best there is - if you, on the way, want to help improve it, it's in Wiki form, so that'd be much apprecidated).
Re working from old translators, it's worth noting that there are two new page retrieval functions that make things much easier. Older translators needed to do async calls with a registered callback to process separately retrieved pages. This can now be coded synchronously with the functions retrieveSource() (text only, but works across domains), or retrieveDocument() (DOM, but works only in the current domain). The best reference for using the new functions is the source code, available for viewing here:
https://www.zotero.org/trac/browser/extension/trunk/chrome/content/zotero/xpcom/utilities.js
I see that I was overtaken by events, and the existing BAILII translator was included in the most recent update (2.1b2). There are a couple of improvements I could think of:
All judges are usually listed between , and the whole list of them are put into one Author entry (e.g. Author is "LORD JUSTICE DENNING<br>LORD JUSTICE MORRIS<br>and<br>LORD JUSTICE PARKER"). Seems like the first step of splitting on <br> would be easy enough, although there are edge cases with titles (like Master of Rolls) and names split over multiple lines. Can't think of an elegant solution for that, I envisage something like
if( line is like /^Lord|Lord Justice|Sir|Sir Justice|MR|blah$/ )
{
line = line . nextLine;
}
The "BAILII Citation Number" is inserted as a note attachment. I think it's taken from the , maybe . However, later in the page a more complete list of citations appears after the text "Cite as:". This usually includes Weekly Law Reports citations and the like which would be more useful to law students if no-one else. Haven't looked at the DOM for that section but it seems it would be just as easy to have all the citations pulled into the note.
No snapshot is taken by default, but that would just be my personal preference. Don't know what the general policy on that is for translators.
Any other BAILII users have suggestions for improvements I could look at?
Somewhat relatedly, the only reason I like to keep snapshots is that I like to annotate things quite thoroughly. What are the technological barriers to simply allowing annotations on live web pages, without requiring a snapshot? I've used Diigo and other web annotation plugins in the past, so it's clearly doable. I'd be interested to know whether Zotero just hasn't had the tuits for it or if there's some technological reason it's not implemented.
Thanks!