PubMed import partially broken

beogl

The one right above my answer, i.e. enable PDF attachments.

aurimas

@adamsmith, Dan, Simon, etc.
Can someone else confirm that this is not working?

beogl

@aurimas: I did not see the image until now. I looked at the old place, but there is nothing there. And there is nothing in the new place.

But why not place it in the old place? I see no reason to just leave it empty. That just makes it harder for those looking at it. Please make the DOI clickable instead.

aurimas

I'm not sure what else to tell you, since the link is properly attached (as shown in the image) when importing from PubMed using Chrome and Zotero Standalone when attaching PDFs is enabled. You can post a link for a page that does not work, but I don't think that will be the problem. Make sure your Zotero (both Standalone and Chrome connector) is up to date. Make sure your translators are up to date. This page may help: http://www.zotero.org/support/troubleshooting_translator_issues

But why not place it in the old place? I see no reason to just leave it empty.

The reason for this is explained several times throughout this post.

Please make the DOI clickable instead.

The DOI is already clickable. Try clicking the part that says "DOI".

aurimas

Actually, one thought does come to mind. If you scroll over the Zotero icon in the URL bar, what does the tooltip that pops up say? It should say something like "Save to Zotero (NCBI PubMed)"

Also, when you import the item from PubMed, the Library Catalog field should say "NCBI PubMed".

beogl

aurimas: It says exactly that. And it does not work.

I still think the change is a very, very bad idea. And I see other translators that does save the URL too.

Going to the DOI is just a bad idea if you just want to have a look at the abstract. Pubmed is just more readable. It is two very different kind of links.

Please revert this change. I have read the thread but I can see no clear logic for the change. If there is one then perhaps it can be written down in a sentence or two so we can discuss the logic? ;-)

beogl

One more thing: If you go to the Pubmed page there is always a link to the DOI page. The other way round is never possible.

beogl

So I actually search for the Pubmed page before adding the reference. And now you have just made that work a waste.

adamsmith

The logic is:
"URLs should only be included for items that include the full text of a work (such as journal publishers, google books, but also full-text databases like Proquest and JSTOR), not for catalogs like PubMed or Library Catalogs, that only contain bibliographic information."

There are good bibliographic reasons for that: A URL which is in the URL field may be cited (either if there is no page range, or if the "include URL" box is checked). That means you accessed the article on Pubmed, which isn't true --> your citation is wrong.

(This will solve itself in the medium run - we'll get a PMID field in Zotero which then can be made clickable just like the DOI field is now).

adamsmith

And just to re-iterate: While Aurimas technically made the change, this is in line with long-standing policy in Zotero and was signed off on by Simon, the lead developer for translators.

beogl

Thanks adamsmith.

With this logic you can of course not include the URL for a book unless it is an electronic book, or? ;-)

More troublesome is that if you include the link to the full text of an article with restricted access many people can not see even the abstract without trouble. All you do is actually giving the publisher a link for free. From the readers point of view it is a bad decision.

The bibliographic reason seems useless. You create bibliographic references from within Zotero and that reference does not depend on if the Pubmed URL is in the entry in Zotero or not.

So again: Please revert this change! When the PMID field is clickable both in the web interface and in the Zotero standalone interface it could be introduced as an option - perhaps. (Though I still can see no good usage of that option. Unless people really, really want to got directly to the DOI page instead of to Pubmed first.)

beogl

adamsmith: Yes, I understand it is in line with a long-standing policy. And I think that policy unfortunately is not very good (for the reasons I have given).

adamsmith

With this logic you can of course not include the URL for a book unless it is an electronic book, or? ;-)

correct. And we don't. No library catalog saves a URL, nor does the Amazon translator.

The bibliographic reason seems useless. You create bibliographic references from within Zotero and that reference does not depend on if the Pubmed URL is in the entry in Zotero or not.

of course it does. The URL field is bibliographic information and whether it's there or not does change citations (and it should!). Trust me, I wrote a large share of the citation styles.

More troublesome is that if you include the link to the full text of an article with restricted access many people can not see even the abstract without trouble. All you do is actually giving the publisher a link for free. From the readers point of view it is a bad decision.

I'm not sure who "the reader" is here. If you retrieved the article from a paywalled source, that's what you should link to. That's just correct bibliographic practice. If the article is paywalled, the reader needs to have access to read it.

It just seems like you're misunderstanding the distinction here: The URL field _only_ matters for bibliographic purposes. For everything else, the attached link has the exact same use. I don't know why that's not working for you, but that's an entirely separate issue that should probably be moved to a new thread.

beogl

adamsmith:

Yes, I know there is no link attached for example when I save from a Google Books page. And that is quite troublesome. It means the users of the library where I save this will have a lot of trouble finding the books. They are not librarians. They are professionals who needs the book. So I think it is a bad decision too.

If the URL field is used for bibliographic information then the URL has to be stored somewhere else because users often need the URL from where the information was saved to Zotero. So please add the URL somewhere else before removing it from Pubmed! (And it to to Google Books for example the same way.)

Most references I save are to paywalled sources. Most professional users reading the references do not have access to the sources. So again that is why the URL used for saving is important.

A bit of history: When Pubmed was created it was a public secret among those interested in the access to the abstract that Pubmed actually was not allowed to save even the abstracts. I know because we did not dare to use the abstracts in internal databases. Now this is history (because of Google and other search engines) so anyone have access to the abstract in an indexed form.

adamsmith

If the URL field is used for bibliographic information then the URL has to be stored somewhere else because users often need the URL from where the information was saved to Zotero. So please add the URL somewhere else before removing it from Pubmed! (And it to to Google Books for example the same way.)

again - this is the case and we did just that. The link/URL is attached - on google books, on pubmed, on amazon - for everyone here except you. That was part of the change to Pubmed. I don't know why links don't attach for you, but if it's also missing on google books, that doesn't seem pubmed specific, so it should probably be addressed in a separate thread.

beogl

Thanks adamsmith. Hm, that is strange. I just tested the recipe from aurimas:

"Or you can enable PDF attachments in Preferences -> General -> Automatically attach associated PDFs and other files when saving items"

Now it works for both Pubmed and Google Books, i.e. a link with the URL is attached. (It did not work last time I tested. Or did I do something wrong then? Hm, there is not very much I could do wrong...) That is nice. ;-)

However I am unable to test this in the web interface because the new entries does not show up there even after clearing the cache in Google Chrome (and syncing Zotero). I will check again later.

Thanks for the help. It seems like this could be a good solution then. (If the web interface works too, since most users will use that.)

beogl

I mean that the users will use the web interface for looking at the database. BTW there are several problems with the web interface, but that is another question.

DWL-SDCA

Even from the beta stage of PubMed, late 1994-95, abstracts were included if they were available from the publisher. Early PubMed was a free version of / or interface to Medline. At that time, only about half of the Medline article records had abstracts. Also, at that time many publishers didn't provide abstracts to any literature service. At that time many publishers required payment from databases that wanted to license and index articles from their journals. By the end of 1998, much of this changed. With only a handful of exceptions, most publishers were pleased to license metadata with abstracts without charge. For most publishers, this also included free access to full text, either print or electronic versions. The licenses prohibited all redistribution or sharing of the full text material. Full text was offered to databases to facilitate minor modifications or amendments to the authors' abstracts.

jose.eizayaga

I got mad until I found this large discussion about the matter, thinking there was a problem with my Zotero or Firefox. Though I understand the points against, I agree with the critical importance of the URLs in citations and would really love to get them back. In the meanwhile I found a not very happy solution: immediately after adding the PubMed item to Zotero, I copy-paste the URL in the corresponding field in the right column.

aurimas

Though I understand the points against, I agree with the critical importance of the URLs in citations and would really love to get them back. In the meanwhile I found a not very happy solution: immediately after adding the PubMed item to Zotero, I copy-paste the URL in the corresponding field in the right column.

If you read the discussion above, then you understand that including PubMed URLs in citations is incorrect behavior.

As far as "the critical importance of the URLs in citations" goes, I believe DOIs provide a much better alternative.

I wouldn't hold my breath waiting for PubMed URLs to make it back into the URL field. It's unlikely to happen.

jose.eizayaga

I understand. But one thing are citations in a paper and another thing is to send citations to colleagues.

adamsmith

also as per above, we will have a dedicated PMID field in the not super-distant future though, which would make it trivially easy to create a citation style that includes the pubmed URL. Already, I believe, there is a NLM grant style that prints the PMID on the repository, which would make it quite simple for colleagues to look up an article.

alex.gray

I suggest a more positive approach to this issue.

Rather than stating that in some "not super-distant" future there will be a good work-around for the users affected by this change, why not back out the change until the work-around is actually available.

I understand the importance for many Zotero users of being able to include url's pointed to PubMed (we want to do this), and why they would be upset at the removal of this long time feature for what are apparently theoretical reasons (I did not see any mention of large numbers of user complaints about the way it was working before).

One of the advantages of open source environments is the ability to quickly respond to user concerns.

When you make a change that upsets many users and disrupts their use of the product, why not be open to backing it out until you have a good solution for the affected users ?

And it is clear that a good solution means an easy way to include a clickable url for PubMed.

I believe that this is a win-win approach, and will build community rather than divide it.

adamsmith

see above. We're not going to revert back to a status that creates wrong data & citations. While those of us developing Zotero and its translators have tried to explain the rationale for this at length and have made specific suggestions on how to address various use cases, those of you who want us to revert this haven't as much as acknowledged the issue at hand (cf. "theoretical reasons") nor proposed any viable solution to it. That strategy is not going to change anyone's mind.

The advantage of open source is not really that we can respond more quickly to users than proprietary software (why would that be the case?), but that you (or anyone else interested with the necessary (in this case not very significant) skills) can change/adjust things in the code that you don't like (and then share her/his solution.
If you're interested we can give you some pointers on what to do.

I'd also like to point out that the "many users" you're referring to have been four so far (you included), of which one (beogl) was satisfied once the new solution was working for him correctly.

Simon

We could add a (hidden) pref that tells Zotero to use the first link/snapshot attached to an article as the URL. This doesn't screw up the data model and will offer the behavior that others have requested here. There are more pressing issues on my plate at the moment, but one of the advantages of open source environments is that the core developers aren't the only ones who can write code :)

alex.gray

AdamSmith:

Actually, I did propose a viable solution.

I proposed deferring the PubMed import change until the dedicated PMID field was added to Zotero. That seems pretty viable to me. You can still make the change since you feel so strongly about it, but we are asking you to wait until a work around is available.

You are changing an aspect of Zotero that has been present for six years, so it is not clear what the rush is.

I am proposing a compromise where the import change still gets made, but is deferred until a good work around is available.

I also think that calling clickable links to PubMed "wrong data" is a bit extreme. PubMed abstracts are a key and reputable tool for medical science. It is clear to me (and many others) that PubMed links in citations are very useful for practicing physicians and busy researchers.

We are aware of other users who have been adversely affected by the change so I believe that this forum is a "tip of the iceberg" as far as the impact.

Our group have been considering contributing code to Zotero development, but if your tone is representative of the community attitude, it does not feel very welcoming.

adamsmith

under your proposal, users will continue to import wrong data. "Wrong" in the sense of: a) no corresponding to the standardized Zotero data model and b) leading to wrong citations in the sense of "not corresponding to style guides, includinpg NLM".

As we've said repeatedly here, that's not something we consider a viable solution, not even intermittently.

I've offered to help you with a workaround, so I'm not sure why you consider my tone unwelcoming to people interested in contributing to Zotero, that's certainly not my intention.

alex.gray

Simon:

Thank you for your note on another possible solution.

I am not familiar with hidden Zotero preferences.

Is there some documentation that you can point us to which discusses hidden preferences and how to create code for them ?

dstillman

A hidden pref is just a pref that's not in the UI.

You can look for instances of Zotero.Prefs.get() in the code. Default prefs are stored in defaults/preferences/zotero.js. (Prefs don't need to be defined there, but if put there they'll show up in about:config, which makes changing them easier.)

dstillman

And the actual change would probably go in retrieveItem() in cite.js, which is what generates JSON from Zotero items to pass to the citation processor.