Simplify PubMed Translator

sjimon

Hi,
It seems that the behavior of the PubMed translator has changed recently so that when a citation is imported, it

1) links to the journal's website (rather than the current PubMed Link), and
2) imports a child item/attachment with the PubMed Link.

Is it possible to opt out of this more complex behavior, in favor of the simpler behavior of just importing the reference and linking to the current PubMed page (with no attachments)?

ie clicking a zotero reference that came Pubmed takes you back to that Pubmed page, rather than anywhere else.

My rationale is as follows:
1) If I wanted the zotero reference link to be the journal page, I would have imported from the journal page. Since I am importing from PubMed, I would prefer the default link to be PubMed.
2) I use the presence of an child item/attachment to indicate that I have a local copy of the full pdf. If the PubMed link is attached, it looks like I have the full pdf, when I do not. I always keep the 'attach PDF' and 'take snapshot' behaviors off by default, but this does not seem to prevent the PubMed link attachment any longer.

My preference is the simpler behavior, or at least to have ability to choose the default behavior.
Thanks for your efforts.

sjimon

Sorry, I forgot to mention this issue is related to these threads:

http://forums.zotero.org/discussion/25635/pubmed-import-partially-broken/

and

http://forums.zotero.org/discussion/25697/automatic-link-to-pubmed/

Where the discussion on the matter is somewhat complicated.
IMO, I would keep the translator behavior very simple, or at least provide the ability to choose....
Thanks again.

adamsmith

you can't opt out of the behavior, at least not easily, no.

The rational for the change is that the URL field is meant for links to full-text versions of articles - it is never populated for catalogs, be it pubmed, amazon, or library catalogs. The main reason for that is that the URL field is bibliographic data that may be cited and while pubmed IDs may be cited, pubmed URLs should not (one example were that would happen is a pre-publication article retrieved from pubmed).

There are two things that we could change, Zotero:
1. The default behavior on double-clicking the item in Zotero: IMHO that should default to the attached link, not the URL in the URL field. I'd be curious what Dan's and Aurimas's thoughts are on that.

2. Not import the URL of the journal page. I understand that might be problematic, especially since there may be other ways to get to the full text PDF than the journal publisher. I have no opinion on that one.

Both are independent choices, but either would solve your issue 1) and neither your issue 2).

sjimon

The previous default behavior on double-click was perfect. It would open the attachment if one is present. If there was no attachment, it would open the URL (which would take me to the PubMed page where I could review the abstract in a standardized format, and make a decision as to how I want to obtain the pdf from the various options, some of them requiring payment).

Regarding the rationale for the URL, I would have imported the reference from the journal page if I want it to point to the journal URL, and import from PubMed if I want it to point to PubMed.
That seems like the most straight-forward approach - I would let the user choose and control this URL behavior by choosing where to import the reference from.

My preference is to always import from Pubmed because in my experience it is more consistent. I always leave the journal page and go back to PubMed before importing for this reason. However, I can see how others might prefer to always use the publishers URL. So, it seems the simplest to let the user control what URL is stored by simply storing the URL that the reference came from.

This would also solve the extra PubMed Link attachment that now appears, as it would not be necessary to attach this additional information.

adamsmith

that's not going to happen. As I say above: The URL field in Zotero should only every contain URLs of full-text versions of items. We're not going to change that.
We're happy to think about other ways to make this work for you, but please take that as a given.

aurimas

1) links to the journal's website (rather than the current PubMed Link), and

I'm fairly sure there is no link being imported into the URL field. the behavior you are seeing is Zotero following a DOI, which is as stable as it gets as far as links go.

1. The default behavior on double-clicking the item in Zotero: IMHO that should default to the attached link, not the URL in the URL field

I think that would make sense. One downside is that it's more intuitive to click on the attachment than on the URL label if you don't want the default behavior. We could make the URL label look more like a hyperlink though. Perhaps same with DOI.

2) I use the presence of an child item/attachment to indicate that I have a local copy of the full pdf. If the PubMed link is attached, it looks like I have the full pdf, when I do not. I always keep the 'attach PDF' and 'take snapshot' behaviors off by default, but this does not seem to prevent the PubMed link attachment any longer.

The indicators for file attachments will change in the next major release of Zotero. It will be much more obvious to see which items are missing PDF attachments.

Rintze

While I understand the rationale behind avoiding the URL field, I think the link-attachments are rather clunky.

Since each item already has some non-citable metadata in the Info column (Date Added and Modified), maybe it would be more elegant to introduce a new field, e.g. "URL Added"?

Rintze

(alternatively, the PMID field could be made clickable once it is introduced)

dstillman

Since each item already has some non-citable metadata in the Info column (Date Added and Modified), maybe it would be more elegant to introduce a new field, e.g. "URL Added"?

I don't think that field name works, but at least from a performance standpoint this would be much better.

sjimon

I think you are correct aurimas - there is nothing in the URL field currently.
It makes complete sense that the DOI field points to the publisher's fulltext.

So therefore if the DOI and URL are essentially redundant, I'm not understanding why the URL field can't point to the user-chosen source (such as PubMed). This is the default behavior in other biblio/organizing software.

Rintze's suggestion of a new field is also fine (although would call the field 'Source URL', 'PMID' is even better).

dstillman

"Source URL" was my first thought as well.

sjimon

I agree that this is a good idea, as this new 'Source URL' field eliminates the need for the PubMed Link attachment.

I would be great to be able to choose the default double-click behavior (in the absence of attachment) between the DOI and the new field 'Source URL', as it is clear that people have strong and differing opinions on this. Choice would be good.

aurimas

I don't think that field name works, but at least from a performance standpoint this would be much better.

What kind of performance hit are we talking about? Iterating through attachments when double-clicking an item? IIRC we already do that to figure out if there are any file attachments.

Also, while idk if this is a common case, I can imagine multiple catalog URLs being attached to a single item. I guess you're not suggesting that we remove the catalog link though.

I would call the field "Imported From". IMO "Source" is a bit ambiguous. On the other hand, more often than not, this field would be either the same as the URL field, or would be left blank.

aurimas

I would be great to be able to choose the default double-click behavior (in the absence of attachment) between the DOI and the new field 'Source URL', as it is clear that people have strong and differing opinions on this. Choice would be good.

Choices are good, but at best, this would be one of the "hidden" about:config options, which 99% of the users are not aware of and would never use. So we still need to figure out what is the best behavior for a default double-click.

IMO, navigating to a catalog entry (or "source URL") makes more sense by default, because typically catalogs provide a link to the publisher's website.

sjimon

I agree, the catalog (PubMed in this case) provides the link to the fulltext, but the reverse is not true.

Also, another reason I prefer the reference-doubleclick to go directly to Pubmed, is that I like to make use of the 'Related citations in PubMed' feature to expand my searches.

adamsmith

(sorry about the confusion above - I remembered we talked once about adding the publisher URL, but thinking more about this I'm now pretty sure that's a bad idea).

Choices are good, but at best, this would be one of the "hidden" about:config options, which 99% of the users are not aware of and would never use. So we still need to figure out what is the best behavior for a default double-click.

IMO, navigating to a catalog entry (or "source URL") makes more sense by default, because typically catalogs provide a link to the publisher's website.

+1 to both.

aurimas

I remembered we talked once about adding the publisher URL, but thinking more about this I'm now pretty sure that's a bad idea

Let's say in the next Zotero release we are finally able to fetch PDFs from publisher websites when importing from PubMed. If you do get a pre-print PDF so that a citation would have to include a URL, is it appropriate to use the publisher's URL?

Honestly, I think dx.doi.org urls (which we can construct) should be used in citations, but I'm not a very big expert on what is actually correct behavior.

adamsmith

(Digressing a little).
right - if we do get PDFs we should get URLs.

As for which URL - most styles require the use of DOIs instead of URLs where those exist for prepublication. These are then live-linked for publication (and we could consider adding an option to live-link DOIs to citeproc, it'd be neat I think). So in most cases we'd see the URL in citations only when there is no DOI. Otherwise we should use the original URL, though.

(digression end)

@sjimon - none of these changes will happen immediately. If this needs to work for your right now amd you're interested and somewhat computer literate (i.e. able to follow somewhat more complicated instructions) we can provide you with instructions to install a custom version of the PubMed translator that uses the old behavior. You should then delete that version once you see the Source URL field appear - probably in one of the next major Zotero updates.

sjimon

Yes, thank-you, that would be very helpful.
If you provide or point me to some instructions, I think I can fumble my way through it. I could use that as a patch in the meantime.

dstillman

What kind of performance hit are we talking about? Iterating through attachments when double-clicking an item? IIRC we already do that to figure out if there are any file attachments.

I'm referring to syncing and the server, mainly, where having additional items is much more expensive than having additional fields. Each additional item has to be uploaded, stored, cached, indexed, searched through, and downloaded separately.

But additional items make the client slower, too. Most of that is avoided by loading child items on demand, but more items still means more rows in the database, which means more data to search through for various queries. (It also means that something like pressing "+" to show all child items takes longer than it would otherwise.)

Also, while idk if this is a common case, I can imagine multiple catalog URLs being attached to a single item. I guess you're not suggesting that we remove the catalog link though.

Remove existing catalog links, you mean? No, though we would have to decide whether we would try to migrate existing catalog links to the new field, at least when there was only one.

Rintze

If you do get a pre-print PDF so that a citation would have to include a URL, is it appropriate to use the publisher's URL?

Honestly, I think dx.doi.org urls (which we can construct) should be used in citations, but I'm not a very big expert on what is actually correct behavior.

In the life sciences it seems to be most common to include a (non-hyperlinked) doi. Some examples:

Uptake of α-ketoglutarate by the citrate transporter CitP drives transamination in Lactococcus lactis
Agata M. Pudlik and Juke S. Lolkema
Appl. Environ. Microbiol. AEM.02254-12; published ahead of print 30 November 2012, doi:10.1128/AEM.02254-12

(see http://aem.asm.org/citmgr?gca=aem;AEM.02254-12v1 and http://www.pnas.org/citmgr?gca=pnas;1214753109v1 )

sjimon

@adamsmith - I think I have a temporary solution to simplify the translator behavior. I modified the NCBI Pubmed.js file, which I will update to the latest version when it is fixed in the next release.

In the meantime, to prevent the storage of the NCBI url link as an attachment, I have commented out the commands in the "newItem.attachments.push(( ))" statement. This appears twice in the js file and I commented them both out.

To store the PubMed URL in the URL field, I have added the statement:
newItem.url = "http://www.ncbi.nlm.nih.gov/pubmed/" + PMID
I have added this line twice, once inside the 'articles' for-loop and once inside the 'books' for-loop.

This seems to work. The translator behaves similar to before. Please let me know if I have missed something that could cause a problem.

Actually, this works really well because when I import a citation, there is no attachment and if I click on the DOI, it takes me to the fulltext; if I click on the URL, it takes me to pubmed. Exactly the behavior I was hoping for.

adamsmith

this will be overwritten automatically every 24hs unfortunately - you'll want to change the translator ID, too, to prevent that (I believe you should be fine just changing any one letter or number, but I'm not 100% sure).
Otherwise what you did sounds fine.

soaringbear

I just stumbled onto this conversation while looking up about preferences, and have to say that I object to this sudden and hidden change from going to the pubmed page to going to the publisher site. Neither is more right than the other and choice ought to be in the users hands.

adamsmith

Zotero does not link to the publisher site from PubMed entries.
The only reason double-click takes you to the publisher site is that it follows the DOI, that behavior also hasn't changed.
You can still get to the Pubmed entry by clicking on the attached link.

soaringbear

clicking where? I'm not clear where you mean the attached link is?

adamsmith

Let's stay in your other thread - I don't really think this one is relevant for your issue and having essentially the same conversation in two places is just going to confuse things.