It looks like the PubMed people are on our side. Here's the copyright notice on the NCBI site, and here is an FTP download link for their lists (PubMed Journals and NCBI Molecular Biology Database Journals). The lists contain two abbreviation versions for each entry (MedAbbr and IsoAbbr). The ISO abbreviations are probably offered for sale somewhere, but you have some cover from the important work of Public.Resource.org on government publication and incorporation by reference of privately published standards.

I think this is a marvelous project. Abbreviation list always has had me look back at Endnote occasionally.

I have a couple of questions and a question/suggestion.

1. Could you elaborate on the importing process for those who are less familiar with JSON or CSL files? For testing, I downloaded one of your MLZ styles, but it's a CSL file, not a JSON file that the plugin looks for when importing.

2. I've downloaded the PubMed abbreviation list. How do I convert this into a format that the plugin can parse out? I'm assuming it will require some find & replace operations to the file.

3. I think a repository of abbreviation lists for this plugin on the web where anyone can add to the lists in a wiki-like collaborative manner would be wonderful. Would there be a legal hurdle to such implementation?

Thanks!

The abbreviation lists are separate from the styles. Nothing comprehensive is yet available; I've just been building small lists for testing in connection with the style proofsheets, to get the structure right. When the styles stabilise (probably in the next month or so) I'll look at filling out their content a little, but the focus for those will be on basic legal resources.

Here is a simple example of the JSON structure. Building a list of journal abbreviations manually would just be a matter of filling in the blanks, as it were. To convert data from a public list to this format would require some modest scripting, but the abbrevs are just one-to-one mappings, so apart from data cleanup etc it shouldn't be a huge task. Journal abbreviations go in the container-title segment; the others can be left empty:
{
"default": {
"container-title": {
"European Human Rights Reports": "EHRR",
"All England Reports": "All E.R."
},
"collection-title": {},
"institution-entire": {},
"institution-part": {},
"nickname": {},
"number": {},
"title": {},
"place": {},
"hereinafter": {},
"classic": {},
"container-phrase": {},
"title-phrase": {}
}
}


batch editing is one of the top priorities for the 3.5 version of Zotero - if I understand you correctly, that would solve this, or at least help to solve it.

Thank you for your answers. Yes, you understand me right. And I agree with that that both things should be seen as separate things. So I’m looking forward for the batch processing, which will indeed make the work with abbreviation lists much easier.

The file isn't valid JSON. The plugin should complain rather than failing silently, but that's why the import is failing. (On a quick check the only problem I spotted were unescaped double-quotes inside quoted string content, but there may be other issues.)

it seems to me that with ~25000 entries, zotero and firefox just stopped responding.

There is no progress indicator (yet), but it's working, and you just need to wait until the import clears. I didn't do a full import, but in 15 minutes on a slow machine it processed about half of the file. It is recommended that we perform database operations asynchronously (which I haven't done, for simplicity), to avoid locking up the UI. Async coding is higher math for me, as it were; I'll keep things as they are in the plugin for the present, but this will likely improve at some point.

The speed of import itself can be improved. It's running in a single transaction (which is faster than not), but we can speed things up a bit further by using precompiled storage operations. I'll look into setting that up as time permits.

Is it possible to make the plugin ignore the capitalization?

This could be done, but it would mean data loss, and so increase ambiguity. I'm not sure it would be a good idea.

The PubMed list itself is pretty messy, with some journals registered in abbreviated form. Forcing everything to lowercase would increase the possibility that the abbreviation of one journal (registered as its "full" name) overlaps with the proper name of another. If matches are case sensitive there will be more misses, but the user can still register a missed journal name form when it is encountered, which seems adequate.

I updated "Abbreviations for Zotero" to the version 1.0.120.
The speed is improved, however the browser kept popping up the warning:
"Warning: Unresponsive script.........Script:chrome://abbreviations-for-zotero/content/xpcom/import.js:80", even when the importing was finished. For example, if I chose to import the whole 26,000 entries to replace the entire local list, the browser was busy for ~10sec and the warning popped up, if I click cancel at this first warning, and then export the list, only about 8000 entries were imported.
As a result, I have to click continue upon this unresponsive script warning for 3 times and click cancel at the 4th time when the warning popped up to have the entire list imported. If I didn't click cancel at all, this unresponsive script just kept popping up forever (at least >10 times).
It looks like the script just didn't tell the browser it has finished the importing task.

On the other hand, it would be really helpful if you can add an option for the plugin to ignore the case, because almost 90% of my journals are not matched because of this capitalization issue.

Thanks a lot for your help,

thank you, fbennett for the effort of managing abbreviation lists! I've been playing around with it for a while and now the abbrev. window just shows up really small. Could you please add window resize possibilility as reinstall apparently does not fix this problem :(

Screenshot:

https://dl.dropbox.com/u/5277753/AddEdit%20Citation_2012-06-04_15-56-02.png

@Rintze, thanks, did not know that!

Some feedback about the script - we need a an option to create versions with and without full stops, depending on citation format. Currently the Index Medicus list compiled by qztseng has some abbrevation with full stops and other without.

The other issue is slightly different versions of journal names in index medicus and Zotero. But this has already been mentioned, I won't elaborate on that.



I think the best solution rather than a list would be kind of a dictionary - there are quite a few words that tend to repeat that could be easily replaced by a script.

thanks. it is just that the current Index Medicus list has full stops on and off. Someone with patience needs to fix that then :)

The first thing to do would be to run the file through a syntax checker. It has to be valid json. Then try taking it in smaller pieces first, to be sure the plugin is functioning correctly on your system. After you have confirmed that your json file is valid and that valid json imports correctly, it's worth trying the full-sized file.

An unresponsive script warning isn't an error; it just means that the import hasn't finished yet. If simple small imports are confirmed to work and you have a valid json file, you would want to click "Continue".

Yo, that's great news. Thanks for the effort you've put into this.

There is no community repository yet, but we can open one. Shall I open a space for lists on GitHub, and reflect it on CitationStylist as a solution for the time being?

Yes, I thinks that's a good idea. I'll upload my list.
Now I'm looking for a good list for statistics and mathematics...

The json link returns "file not found". If you can put it online, I'll be happy to take a look.