Support for DOAJ Link

Oddible

This is a request for support for the Direct Open Access Journals (DAOJ) site. Not sure how this works in Zotero but in most major publishing sites (Springer, ACM, MITPress, etc. Google books, Amazon, my university library), I get the little link in the address bar to be able to add an item from the website to my Zotero database. This is currently not appearing on the DAOJ site. Is there something they need to publish in their web content or is it something that Zotero has to add to support? Is there documentation I can send them on what they need to do somewhere?

http://www.doaj.org/doaj?func=loadTempl&templ=faq#metadata

adamsmith

here is the documentation,
http://www.zotero.org/support/make_your_site_zotero_ready
although it would probably make more sense to write a dedicated translator:
http://www.zotero.org/support/dev/creating_translators_for_sites

DWL-SDCA

Is there any progress on getting DOAJ article-level metadata in a form that is readable by Zotero? I have written to them and their reply was to point me to the link mentioned above in the Oddible post. I pitched making the DOAJ site Zotero-ready. I even offered the (free) help of my web programming crew given that my own SafetyLit site uses UnAPI. (They said that it was trivial to implement -- something like 4 to 5 hours time. Most of that time was taken by reading the documantation and looking at sites where UnAPI was implemented.)

DOAJ has a wealth of information and it would be convenient to bring it into Zotero. Has anyone found a way to get data from the site? Is there an alternative journal portal that supplies reference matadata?

ajlyon

Can you provide a sample URL for article-level metadata in the database? I seem to be finding only journal-level data.

fbennett

I figured out how to get a DC metadata page for an individual item, but you need the OAI Identifier to fetch it. I took a look at a content page, and all I could find is an "id" from the page URL. There doesn't seem to be an obvious way to get the OAI Identifier from a page/document "id". Maybe the site maintainers can help.

ajlyon

Yes-- I've had a similar situation with OAI-PMH systems in the past. Sites don't usually take into account the possibility of OAI-PMH clients using the websites as a way of browsing the data. I was working with people at USC and said:

[..] Is the fully-qualified item identifier available from item pages? On a
page like http://digitallibrary.usc.edu/impa/controller/view/impa-m42162.html,
a client like Zotero (http://www.zotero.org) might be able to discern
the identifier "impa-m42162", but I had to dig around a bit in the
OAI-PMH output to learn that the full identifier for OAI-PMH access is
"oai:usc.edu:impa-m42162".

A related issue is that there is seems to be no meta tag or other
indication of what the base URI is for OAI-PMH to the repository, or
any indication in the page code that the system acts as a repository.

I should have a working prototype of Zotero access to repositories via
OAI-PMH in the coming week or so, and while I can make site-specific
translators to support USC's collections, autodetection of things like
this is certainly preferable to making ad-hoc solutions for each
resource.

They quickly added links to the OAI-DP records from the page header. I still haven't done my part and written code to read from OAI-PMH.

ajlyon

Please feel free to crib from my message in requesting similar actions of other database maintainers.

DWL-SDCA

I'm uncertain, given ajlyon's comments, that you still need this but here is an article level url:
http://www.doaj.org/doaj?func=abstract&id=724732&q1=violence&f1=all&b1=and&q2=&f2=all&recNo=356&uiLanguage=en

and here is a url for the results of a search -- it lists multiple articles:

http://www.doaj.org/doaj?func=searchArticles&q1=violence&f1=all&b1=and&q2=&f2=all&p=36&uiLanguage=en

It seems to me that the consistency of the records isn't perfect but acceptable as a starting point. Also, to get the doi number (for articles that have one) it is necessary to click a full-text button; go to the journal publisher's website, and find the pdf version of the article.

The accuracy of the article-level information is quite good -- much better than is obtained if I get the article meta-data from the pdf by bringing it into Zotero. It seems that articles from obscure journals don't do well when found via Google Scholar. (Author first/last names reversed, page numbers wrong, misspelling of words in title, etc.)

fbennett

Scraping the information from the page is possible, but it's cumbersome and sets up a snowball of work down the road, since a screen-scraping translator needs to be updated every time the underlying site changes.

What ajlyon is looking at is getting the metadata in a structured form (which is what the OAI standard is for). The information is available, via links that look like this:

http://www.doaj.org/oai.article?verb=GetRecord&identifier=oai:doaj-articles:09deef8e70fa208b06c1ef7cb7d399f8&metadataPrefix=oai_dc

That gives standard OAI-encoded Dublin Core metadata, that is assured not to change even if the visual appearance of the site changes. The problem is that the user-friendly page does not contain the "identifier" key; if the site maintainers include that string in the page, a solid Zotero translator can be quickly built for the site.

DWL-SDCA

A moment ago I wrote to DOAJ again with the information ajlyon and fbennett provided. I also included a link to this discussion. If i get a reply I will share the news.

ajlyon

I just had a very productive discussion with staff at the Newberry Library in Chicago today about support for ContentDM, which will certainly be built off of the OAI-PMH system.

We clearly need a new import translator for the format, possibly with a web component and some changes to the RDF translator to account for the mappings used by OAI-PMH.

The Newberry folks are interested in writing a translator for ContentDM-- if we can put together something for OAI-PMH in general, we'll be able to work with a very large range of collections with minimal effort.

kslarsena

Is there any update to this issue? I use ContentDM sites regularly, and a translator would be VERY useful.

ajlyon

No, no update.

This still would be a pretty simple addition-- if you have any JavaScript hackers around, you may want to talk to them about trying to write this. I don't have the time now to do much new translator development, but the docs have progressed significantly in the past year, so it's now much easier to get started making a translator like this one. Please direct any interested people to this thread and to the development mailing list, http://groups.google.com/group/zotero-dev