Upgrade StoragePubMed import partially broken
Since today or so the import from PubMed has problems: Zotero misses the URLs of references from PubMed and the date, when the reference was added (Firefox 15.0.1, Win XP, Zotero 3.0.8). Can anybody confirm?
Thanks, Michael
Thanks, Michael
Since PubMed is a catalog of articles and doesn't actually store any articles, the URL and accessed date shouldn't be part of the citation. The URL is still attached to the item as a link, though.
From a software usability point of view I think it is also almost always a bad idea to take away functionality which has been years in place and many users are accustomed to.
I hope this change is reversed, Zotero otherwise will have lost much of its value for me.
The person who made the change is, in fact, a published biomedical researcher, I assume he uses PubMed regularly.
To clear up some things: that's why Zotero has a "Date Added" field, which is and should be distinct from a date accessed field. as I say - it's "attached as a link" - not in the URL field but as an attachment that says "PubMed Link".
If you export a PubMed imported item - say to RIS or to Zotero RDF to pass it on to a colleague the link will be included. You yourself can get to the pubmed site directly by double-clicking on the attachment.
Given that, I still don't really see what you're losing.
So you confirm that one user had enough influence to change the perhaps most used import filter of Zotero for hundreds or thousands of users? Sorry, but that`s bad change management. I can understand the original argument of your user if he only looks at reference lists as an output for papers.
But I use reference lists from Zotero much more often for quick email communications with colleagues (individually or hundreds of them on dental mailing lists), and then I need more information as I get now. That means: Choosing some references, push the right mouse button, choose citation style, copy the reference list to the clipboard, from there to email and I`m done.
I still can`t see why the URL should not remain part of the reference as it did till yesterday.
For me "date added" doesn`t make much difference from "date accessed". But I understand the difference. Is there a still a citation style which gives me URL and "date added" or "date accessed" for references imported from PubMed from today on?
"Given that, I still don't really see what you're losing." You don`t have to see that. You might simply accept my view, because you would loose nothing doing so.
Is there a way for to get the old import filter back? Is somewhere a repository of old import filters? It seems it has been a mistake to trust automatic updates by Zotero. I will switch off these automatisms if possible and go back to my automatic backup of my Firefox profile from last night.
If your writing requires that those URLs be included in the citations, then we can help you make changes to the translator to populate the URL field-- I don't recommend you turn off translator updates, since many sites change frequently and you will no longer seamlessly maintain access to the thousands of sites parsed by the translator collection.
> Post by: *ajlyon*
> There isn't a way for citation styles to access the
> attached link, whereas they could access the contents of the URL
> field. This is correct citation practice
No, it depends. If you are doing EBM/EBD or statistics about medical literature on different topics, you most often will never touch the original article. You will wade through hundreds of references and their abstracts, delivered by PubMed, and PubMed will be the only source. PubMed sometimes doesn`t deliver correct references, especially foreign author names may be wrong, pages missing, so PubMed is a source of it`s own and should be handled as that by Zotero. Based on this original source (PubMed) a preselection of references will be done, which will be documented in EBM/EBD (URLs remain handy for this), and only a small percentage of original papers will be ordered.
The same is true of course of amazon.com (or any other library catalog). If I get a reference from Amazon, I want to have the URL from Amazon with my reference. Amazon frequently makes mistakes (shortened titles, wrong author names).
Your mental frame, how you see library catalogs, is wrong. If you only use a library catalog or anothers author`s work for citing the reference, that has to be shown in the reference. If you get the original article, you may correct the import from PubMed, remove the URL or replace it by a better, direct URL, if you have one.
But you don`t have to remove the URL manually in Zotero for a usual reference list, if you don`t want, because there is already an option for this: If you go to Zotero/Preferences/Styles you find "Include URLs of paper articles. When this option is disabled, Zotero includes URLs when citing journal, magazine, and newspaper articles only if the article does not have a page range specified".
If you have a paper article, you will have a page range, problem solved. If no page range, URL will show. By not importing URLs from library catalogs you have destroyed the reminder function of URLs on reference list ("get a page range!"). You have taken away much value.
> and ultimately the main reason the link had to move to a link
> attachment-- it would be incorrect for citations to articles
> retrieved from PubMub to include the PubMed URL, as they are not
> published there, but merely indexed.
Everybody knows that original articles are not in PubMed directly accessible, where is the beef? And as I wrote above, there is no such thing as "merely indexing". It`s a librarian`s work, who adds value (MESHes, translating of foreign titles), and people and systems make errors. Zotero already ignores an important part of this original, valuable work, done by librarians, by no longer importing MESHes form PubMed into tags.
> The switch, which was signed off on by the development lead (and
> neuroscientist) who handles site translation,
You should better have asked modern librarians for confirmation, e. g.:
Rethlefsen M, Rothman D, Mojon D. Internet Cool Tools for Physicians. 1st ed. Springer; 2008.
(That I got this reference from Amazon.com is missing now, and that`s bad.)
> is to provide correct data and citation for the larger user
> base in accordance with data curation standards.
I don`t buy that. I`m using Medline/PubMed for 25 years and scientific libraries for 30 years. I think I know what I`m talking about.
> If your writing requires that those URLs be included in the
> citations, then we can help you make changes to the translator to
> populate the URL field
That would be a nice, first step. I still would need a citation style to get this URLs out automatically.
> adamsmith 2 hours ago
> (for quick access, note also the option of using the "National
> Library of Medicine (NLM) - Grant with PMID" which will include PMIDs
> for articles retrieved from PubMed. Another alternative would be
> using a citation style like APA that includes DOIs, which also allow
> for quick access)
PMID or DOIs are not part of an URL in these styles. That means extra work for readers of electronic output (ebooks, presentations etc.). My mentioned presentation will be offered for download as a PDF. I want that the readers can open my PDF, read it, click the URL in the reference and arrive in my source. My whole workflow from PubMed to my PDF worked perfectly automatically this way and now it is broken for no good reason at all.
https://github.com/zotero/translators
if you want any more detailed help I at least will either need a hefty consulting fee or an even heftier change in attitude.
> if you want any more detailed help I at least will either need a
> hefty consulting fee or an even heftier change in attitude.
You want a fee for bringing back a function which was taken away after being in Zotero for years, dragging users into this software? That`s a nice business model. I would never have started using Zotero without the functions I`m mssing now. Eliminating the import of Meshes already slowed me down sometimes, but now Zotero has become much less useful for serious work with PubMed, at least for me.
Whether Zotero is free or not doesn`t matter (at least not for me, I chose Zotero because of features, not of price). Zotero ist part of the service industry (like me), anyhow. I have written already, how software developers should act. And the companies I`m dealing with (and paying for) are acting this way.
I wanted to recommend Zotero in my presentation. I cannot do this without reservations which I had not a few days ago.
But thanks for the hint to \zotero\translators (no repository with old binaries? weak). I went back to my backup of 2012-09-29, grabbed "NCBI PubMed.js" (inside it reads: "lastUpdated": "2012-03-12 01:14:39") and replaced the broken one in my active Zotero (anybody interested can get it here: http://www.logies.de/NCBI%20PubMed.js or http://www.logies.de/NCBI PubMed.js). It took about 3 hours for me (us) to solve this problem, sorry, that`s more than I can stand. Now at least PubMed is working again as it should, other library catalogs like Amazon are still broken.
It seems, that the integration of Zotero with Scrivener does not become better. After finishing this project I will test Endnote and perhaps Bookends. Thanks for the years I could use Zotero, but it seems that time has come to move on:
1. Anon. Literature and Latte • View topic - Zotero integration. Available at: http://www.literatureandlatte.com/forum/viewtopic.php?f=4&t=18847&p=133722&hilit=bibliographic+software#p133722. Accessed October 6, 2012.
2. Anon. Literature and Latte • View topic - Best bibliography tool to use with Scrivener? Available at: http://www.literatureandlatte.com/forum/viewtopic.php?f=2&t=18942&hilit=bibliography+software. Accessed October 6, 2012.
I'm also not part of the core Zotero development team, so what I'm about to say is my personal opinion and not that of Zotero developers.
Zotero, first and foremost, is a bibliographic management software to help you (A) organize your research and (B) produce correct citations. Everything else that you can do with Zotero is a huge bonus. Having said that, I think everyone here is excited to see Zotero being used for more than its intended purpose. In your case this means generating citations that are, while useful, not technically correct.
I think typically the developers here are quite accepting of new ideas and suggestions for improvements to help make Zotero more useful. Not all of these requests can be accommodated for one reason or another though (e.g. incompatibility with other features or lack of developers working on Zotero). It is unfortunate that your "my way or the highway" tone and language has turned this potentially fruitful discussion into a somewhat sour situation. Nevertheless, I do use PubMed myself and I see how including a catalog link in the citation for personal communications could be useful, so I would like to turn this discussion around and try to come up with some sort of solution.
Here are some ways that I can see this being resolved:
1. Additional, possibly hidden, option to substitute catalog links for URLs when the URL is missing (either upon import or when generating citations. I prefer the latter). [technical note: In order for this to work correctly, I think we would also want to properly distinguish catalog links from other links (e.g. file links) through an additional attachmentType property (this could replace snapshot:true with attachmentType:snapshot).]
2. Same as above but implemented as a Zotero addon. I can see this option having much fewer objections, but I believe some changes would still need to be made to the Zotero client.
3. If someone really wanted this right away, I think this can be accomplished via a custom export translator. Of course you wouldn't be able to flip through different citation styles without hard-coding them, so it's not a solution for the general public.
4. Not exactly a complete solution (since sometimes links are not available), and I'm not sure this would even be a correct way to handle it, but when (if?) we do turn PubMed (and other catalog translators like Google Scholar) into a recursive translator (i.e. have it navigate to the linked site and use that sites translator to retrieve additional metadata/attachments), we could store the URL of the site hosting the resource. This doesn't even have to wait for recursive translators. We could find the direct URLs right now. My concern is whether it would be correct to say that the article was retrieved from the linked website when the user never visited it. I feel like the answer is no, but I'm interested in other opinions.
It's not orthodox but what about adding a dedicated field for these URLs? Under "Library Catalog" and "Call Number".
I think that would be my preferred solution, though I understand the concern about not actually visiting the site. too hacky for my taste. Also, there would still need to be a CSL variable to map that too and I doubt that's going to happen.
http://dx.doi.org/ + DOI to the end of the citation style - that would likely be more stable than the URL link to the original resource.
@aurimas: Thanks for trying to find a solution. I think if a software problem needs 6 years or more to be tackled, it is no problem worth being tackled because all users annoyed by this problem will already have left the software. Then a change can only annoy more users, here it has been me.
"Zotero, first and foremost, is a bibliographic management software" No, I can`t agree. For years it had been advertised as a tool to store all kind of information. And usually it is much better to have some information at all instead of having it not because it doesn`t fit into a predefined "bibliographic style". Somehow I feel that this is a cultural problem. Here in germany, at least in the medical field, I can`t imagine that bibliographic style is taken too seriously, and of course it is not a part of the medical curriculum (as it is in the humanities). In medicine, the reference should allow to identify the used source and, at first look, should look more or less, as the journal wants, and if a URL is added as a bonus, that`s fine, regardless whether there is a style for that or not.
I have frozen my Zotero now and will see how long it will work till some important catalog change will break it. PubMed seems to be quite stable, though. I think, that they know their customers well... Any busy clinician will hate any change in the software he/she uses if it is not absolutely neccessary or brings a definite benefit. We heal patients, not reference notes. The change to "NCBI PubMed.js" was superfluous and damaging from this point of view.
So nice that you try to solve the problem. If this should happen, I hope that I notice it somehow and may try.
1. Maserejian NN, Trachtenberg FL, Hauser R, u. a. Dental composite restorations and neuropsychological development in children: Treatment level analysis from a randomized clinical trial. NeuroToxicology. (0). Available at: http://www.sciencedirect.com/science/article/pii/S0161813X12001970. Zugegriffen Oktober 6, 2012.
2. Maserejian NN, Trachtenberg FL, Hauser R, u. a. Dental composite restorations and neuropsychological development in children: Treatment level analysis from a randomized clinical trial. Neurotoxicology. 2012. Available at: http://www.ncbi.nlm.nih.gov/pubmed/22906860. Zugegriffen Oktober 6, 2012.
Both should be the same, but they are not. ISSN and Date are different, the name of the journal is written different. That`s a bad joke.
The PubMed record will have the original add date and a last-revised date. For the vast majority of early PubMed records,much of the information is incomplete or not yet in standard PubMed format. I'm not yet writing about the records with missing MeSH terms and those listed as ePub. -- author names, article titles, and other things will change as edits move the article to maturity. The number and nature of MeSH terms will also change from none, to several, to more, or to fewer. In other words, the typical PubMed record undergoes several changes throughout the first 4 to 6 months after being added. Much later, when ePub articles are updated with volume, issue, and page range info, there are often other changes made.
That suggest to me that the best medium-run way to make this work is the PMID - unfortunately it's currently not being saved in a way that can be turned into a URL (it has PMID: in front of it in the extra field) - I think with the current Zotero fields that's the only reasonable way to do this, but once we (finally) get a PMID field in Zotero, a citation style that does with the PMID what I suggest for the DOI above would be easy enough.
As DWL notes above, they're not present for all items in PubMed, especially newer ones, but where they are they're imported (I don't know if PubMed updates these at different times - seems unlikely, but just in case, what matters is the XML, not the display on the page.)
When was this change introduced, i.e. when was the Pubmed URL removed? (I have to go through old entries and fix them.)
If you have another use case for this, please let us know.
1) The tags they add there. It is more uniform.
2) The abstract is often more readable there. You can see it from a mobile too.
3) You have a link to the original article. And if the whole article is available for free in Pubmed Central there is a link to it in Pubmed, but I have never seen a link from the original article to Pubmed Central. (Maybe I just have not looked for it in all cases... ;-)
4) In some cases not even an abstract is available in the original place without a login.
So I really want the link to Pubmed. Everyone involved in reading the references should know what it means, I guess.
This will be fixed in the next Zotero version that is due by the end of the week. If you want to have a sneak peek you can install the 3.0.9 "beta" from http://www.zotero.org/support/dev_builds
Or you can enable PDF attachments in Preferences -> General -> Automatically attach associated PDFs and other files when saving items"
Are you sure you are looking in the right place? http://imgur.com/qLNLW