Automatically classifying journal articles

sdspieg

Does anybody know a way to automatically classify academic journal articles by their acedemic discipline(s)? Such lists do appear to exist (e.g. journalseek) - so "African Journal of Political Science and International Relations" OR Afr J Polit Sci Int Relat would automatically get the tags
Social Sciences - Africa
Social Sciences - International Relations
Social Sciences - Political Science
If anybody could point us in the right direction on how to do this, we'd be most grateful (and of course put the results of our efforts in the public domain).

-Stephan

DWL-SDCA

Although I cannot imagine how the current Zotero database structure and user interface could allow this; I would be extraordinarily pleased if we could have an extended journal table with publisher date and place ranges, print and electronic ISSNs, etc.

mronkko

Zotero does not store this information, so it is not possible. Your best option is to export the data and then merge that data with a list of journal classifications that you have obtained from somewhere else.

sdspieg

I understand it is currently not possible of course. But I was hoping somebody might be able to suggest a way to do it. The Journalseek list seems to be publicly available, so what we'd have to find is a way to match a journal title (which IS in a Zotero field) with the appropriate discipline as indicated in the JournalSeek list, and then to enter that discipline as a tag in that record. Unfortunately I don't have the programming skills to do this myself, and I also don't know how much work this would be, but it seems to me this is a lot easier than some of the other things that Zotero is already capable of. Am I wrong on that?

adamsmith

I think it's absolutely feasible. It's also the type of thing that applies to a relatively small group of people and can easily be handled by hooking into Zotero with a third party plugin - either via the server api or via the local javascript API - that Zotero core devs are unlikely to get into.
Your best bet would be to find someone interested in the feature with the requisite coding skills.

http://www.zotero.org/support/dev/server_api
http://www.zotero.org/support/dev/client_coding/javascript_api

sdspieg

I understand. So I hope then that somebody with those skills reads this post and gets in touch with us! Fingers crossed...

sdspieg

I now have an Excel-file, based on JournalSeek's publicly available webpages, of about 50'000 journals mapped to disciplines at two levels (if anybody wants it, just let me know and I'll gladly share it):

• first-level: Arts and Literature; Astronomy/Astrophysics/Space Science; etc.; and


• second level: (under Arts and Literature:) Architecture, Art History, Children's Literature, Culture Heritage, etc.; (under Astronomy/Astrophysics/Space Science:) Celestial Mechanics, Cosmology, Extragalactic Astronomy, Interstellar Matter, Meteors, Miscellaneous, etc.); etc.

If anybody has any ideas about how to map this information to either a Zotero-tag or even a Zotero-field (e.g. the 'extra' field), please do let me know. Now granted, it would be a lot better if we could use a 'live' feed of journalseek (through the api), but for the time being (based on currently available publications), this should already help...

-Stephan

sdspieg

A colleague of ours has now produced a first fully functional version of a Zotero plugin that classifies journal articles by academic discipline. It scans the 'Publication' field of any Zotero library against the 64000 journals that are classified by discipline/subdiscipline in (an embedded copy of) the Journalseek database and then adds a new tag with that discipline/subdiscipline into the Zotero record. If anybody wants to try it out and help us debug it, please let me know.

fbennett

Is the code online? It sounds like something that could be implemented as a plugin, and it would probably reach a wider audience that way. If the code is available, someone might be induced to plugify the changes, even if you don't want to go that far yourself.

sdspieg

I sent it to you by email Frank...

sdspieg

Frank - have you had a chance to look at this?

fbennett

Oops, thanks for the ping. I was busy with some infrastructure work for the multilingual/law book, but that's now out of the way. I'll do a diff of the source and post again either today or tomorrow.

sdspieg

Any thoughts? This one was useful for our current purposes, but I am sure there must be more 'elegant' solutions for this. It seems to me that one of the main implications of the increased available of full-text academic work will be more cross-disciplinary work. That is exactly what we did in the project for which we made this plugin. We searched on EBSCO for all articles that contained the word 'definition' within 5 words of the word 'security', downloaded those (1000s of them) into Zotero, and then sorted them by academic discipline with our plugin. Using both automated (textmining) and human analysis, this allowed use to identify some systematic differences in how various disciplines define security. And the results were just astounding - I have been in the 'national security' field for my whole life, but I was totally unaware of rich veins of thinking about security in psychology (e.g. attachment security); economics (e.g. tenure security), law (security regulation) etc. I bet many of us will be increasingly interested in systematically comparing concepts and insights across disciplines. And that is where this type of plugin might come in very handy!

fbennett

Stephan,

Thanks for your patience. I've been distracted by the same purpose -- enabling cross-disciplinary, cross-jurisdiction collaboration by getting the integrated suite of MLZ styles into shape. I'm working through a round of proofing today, and when that's done I will take a look.

fbennett

Stephan: a response at last! I've installed the plugin. It seems to go in (Firefox 11.0, Linux, starting Zotero in a tab) without error, but the plugin item in the gear menu doesn't show up. Possibly it has to do with loading sequence -- maybe the patch into the menu needs to be postponed on some platforms or in some configurations. I think running it from JS would do that, but I'm not sure. More news tomorrow as I play with it a bit more.

fbennett

Okay, that was it. When Zotero is started as a browser pane (sliding up from the bottom of the screen) the plugin menu item shows up. When started as a app tab or as a browser tab, it fails as described above. I think recasting the menu attach in JS will be enough; if not, any solution would need to be in JS anyway, so the effort recoding it would not be wasted.

fbennett

The plugin runs, and does classify things in the library (great!).

The result menu comes up transparent on my system, which is a bit distracting. Normal white background would be better.

The ability to select items to classify seems like it might be wanted, but if performance isn't affected much by library size, maybe that isn't so important.

fbennett

Oh, I see. Tagging is limited to the currently selected collection. Never mind the comment about selecting items, this is the right interface.

This is really cool, by the way. Any chance you can place the code on GitHub? I think you'd attract code contributions; this would be useful to a lot of people.

fbennett

Remembered that I had to add to the Z toolbar for one of my own projects, which is a very similar thing to adding to the gear menu. The overlay setup used for that is here.

sdspieg

Hey Frank. Not it was I who was distracted for a while! Thanks for your comments. I see no reason not to place the code there. Let me look into it.

sdspieg

The code has now been made available at https://github.com/swashbuckler/zotero-classify-articles. We hope some people get some use out of this.

fbennett

Great stuff. I'll try to look at the panel-versus-pinned-app activation issue sometime in the next week or two.

dstillman

Due to Zotero's handling of tags, executing the plugin's classification while having an open tag window will likely cause your system to run out of memory. Keep the tag window closed and nobody gets hurt. This really should be fixed, but it involves tampering with Zotero's internals.

Huh?

Adding a tag will trigger a tag selector refresh. If you're adding lots of them at a time, wrap the loop in a single DB transaction—Zotero.DB.beginTransaction() and Zotero.DB.commitTransaction()—to prevent a refresh until the end (and make the DB writes much faster in general).

nesredna

I obviously should have been reading the documentation/code better. Thanks Dan, I've made the change.

sdspieg

Guys - is there any way we could have this plugin added to the plugin list? I have heard from some people who have found it useful... See here - https://github.com/sdspieg/Zotero-classify-by-academic-discipline

adamsmith

I'm happy to add it, but that github doesn't have any code/plugin in it.

sdspieg

My bad - it's actually here: https://github.com/ndrsn/zotero-classify-articles. Could somebody also take care of the Firefox/signing issue?

ajb59

Currently getting a 404 on https://github.com/ndrsn/zotero-classify-articles.

adamsmith

Remove the period at the end and it works

adamsmith

signing isn't something that other people can "take care off" for you easily. It should be done by the person maintaining/offering the add-on.

sdspieg

Ok - I'll ask him.