CERN CDS and Inspires

roberto.franceschini · September 16, 2021

Hi there,
as far as I know CERN CDS https://cds.cern.ch/ and Inspires https://inspirehep.net/ have machine readable interface to their content via an API.
Are these available somehow in Zotero to add paper via their Identifiers on these databases?
Maybe I have missed some plugin or something like that ...

adamsmith · September 16, 2021

The principle way to get references int Zotero is via the browser add-on. See https://www.zotero.org/support/adding_items_to_zotero#via_your_web_browser

That works great for Inspire; CDS is wonky, but so is their metadata. Might be possible to improve, but I'm not super optimistic given what I have seen.

Add by identifier is only available for a small set of well defined and easily identifiable IDs (such as DOI, PMID, arxiv ID). I don't think either CDS or Inspire use an ID system that would qualify as such. (Also, adding by ID duplicates work on web import, so often hard to justify)

roberto.franceschini · September 17, 2021

I think I disagree on Inspires not having unique IDs. As a matter of fact I have python scripts (for BibDesk) that import from Inspires and CDS.
Here is the doc https://github.com/inspirehep/rest-api-doc
and one example of document retrieval in JSON
https://inspirehep.net/api/literature/451647

CDS as well has an API
https://cds.cern.ch/help/hacking/search-engine-api
and one example of XML output
https://cds.cern.ch/record/2743741/export/xm?ln=en

adamsmith · September 17, 2021

I'm not saying they don't have unique IDs. Almost every database does. I'm saying "well defined and easily identifiable IDs" and both INSPIRE and CERN CDS just use numeric IDs. So if you were to search for "451647" -- no way to tell it's INSPIRE (that's also true for PMIDs, yes, but they were here first ;). Compare that with an ISBN (which has a standard start and a checksum), a DOI (which has an easily regexable structure), or an arxivID, which includes a prefix.

The CDS data you link to is MarcXML, which is a mess for articles and papers -- Zotero does import that (and imports from CDS) but item types are off.

As I said, Zotero's principle mode of import is via the browser. You don't have to use that, obviously, but that's what capabilities are going to be optimized for.

roberto.franceschini · September 18, 2021

I now understand better your point. CDS entries several serchable IDs such arXiv number, DOI, and other IDs. For most of these there are better sources, but when it comes to CERN internal notes and stuff like that CDS is the only source.

An internal note has an ID that is pretty human readable, it's search-able, and unique. For instance you may want to search for CMS-PAS-HIG-17-005

https://cds.cern.ch/search?ln=en&p=reportnumber:CMS-PAS-HIG-17-005&action_search=Search&op1=a&m1=a&p1=&f1=&c=CERN+Document+Server&sf=&so=d&rm=&rg=10&sc=1&of=xm

and you will get an XMLish reply. I cannot locate the help page explaining the fields of this XML (it's horrible in my opinion). I have some python that extracts the most important fields if someone has interests.

Anyhow, CDS can in principle return bibtex straight away from a reportnumber,

https://cds.cern.ch/search?ln=en&p=reportnumber:CMS-PAS-HIG-17-005&action_search=Search&op1=a&m1=a&p1=&f1=&c=CERN+Document+Server&sf=&so=d&rm=&rg=10&sc=1&of=hx

wouldn't this suffice to get it into Zotero as unique ID?

Xuefeng Ding · April 6, 2023

For the particle physics community, the CDS and inspirenet are important inputs.

The request is simple:

When I add a CDS / inspirenet entry to the zotero, it's not saved as a webpage snap, or simple pdf item. Instead, you create a paper or report or document entry with year / author / title properly added.

adamsmith · April 6, 2023

I don't understand the request -- as I say above, that's already the case when you add from either database via the browser connector

Xuefeng Ding · April 6, 2023

Yes, if I add from the browser connector on the item page, say, http://cds.cern.ch/record/2721370, an item is properly inserted with correct title/author/year etc., yet the pdf is not downloaded.

If I add it on the "fulltext" webpage, say, http://cds.cern.ch/record/2721370/files/CERN-ESU-015-2020 Update European Strategy.pdf, a webpage item is inserted, without title/author/year.

If I download the pdf, drag it into zotero, and "create parent entry", then it's asking for "DOI / ISBN / PMID / arxiv id / ADS Bibcode". It does not understand the cds url.

So, there is only one solution: add the record from "http://cds.cern.ch/record/2721370", download the pdf, then drag the pdf to the item.

I am requesting you to add "URL" as one option after "DOI / ISBN / PMID / arxiv id / ADS Bibcode" for "create parent item". I believe this should be easy.

adamsmith · April 6, 2023

OK, so it'd seem like importing the PDF from CDS is the actual request here. Going through the URL in add parent item is rather convoluted (and there are good UX reasons not to add the URL there).

(INSPIRE PDFs actually used to work, that's just broken, but looks like a simple fix)

adamsmith · April 17, 2023

@Xuefeng%20Ding Both CDS and INSPIRE now import PDFs directly with the connector

Xuefeng Ding · April 18, 2023

Thanks. I just tested one inspirehep record the pdf file is downloaded automatically. This is very helpful.

The CDS is still not working. There are several types of CDS document. Part of them are without DOI.

I tested under "articles & preprint", pdfs of thesis and journal articles are not downloaded.
say,
- https://cds.cern.ch/record/2851382?ln=en
- https://cds.cern.ch/record/2849085?ln=en

(But this is fine for me personally. I have put most CDS records I want to add in my zotero last week.)

Thank you for your work.

adamsmith · April 18, 2023

This works -- make sure it says "CERN Document Server" when you hover over the save icon. If it doesn't, your translator just hasn't updated -- should happen within 24hs or you can force it from the Advanced Tab of the Zotero preference and the connector preferences.

Xuefeng Ding · April 18, 2023

Looks good. Now it works on CDS! Thanks.

roberto.franceschini · May 22, 2023

Checked also inspirehep.net Thanks a lot!