Upgrade StorageFeature Request (for v7 or add-on developer): Publish or Perish integration
Hi all,
I'm currently undertaking a systematic literature review, I'm very grateful for where Zotero is at with version 6 so thank you all for the hard work.
I've come across Harzing's Publish or Perish, which is a very small-footprint desktop app which allows me to search the more "open" databases such as google scholar, PubMed etc without having to log in to sites, and then lets me access full text.
I am not a coder but the feel of this programme is not a million miles away from Zotero, and would love to see this database search functionality built in, or added on for a workflow that starts even earlier in the research process!
I'm currently undertaking a systematic literature review, I'm very grateful for where Zotero is at with version 6 so thank you all for the hard work.
I've come across Harzing's Publish or Perish, which is a very small-footprint desktop app which allows me to search the more "open" databases such as google scholar, PubMed etc without having to log in to sites, and then lets me access full text.
I am not a coder but the feel of this programme is not a million miles away from Zotero, and would love to see this database search functionality built in, or added on for a workflow that starts even earlier in the research process!
The request sounds a bit strange to me, because I feel that Zotero and Publish or Perish are competing products: they aim for rather similar goals (collecting and managing references) through different paths. Can you explain in more details what you would like to do?
I'm not suggesting that Zotero does the same strange integrations with universities that EndNote does. Still, I feel that the "PoP" functionality, which would speed up searches of the more esoteric engines, would add a lot - because those are the most laborious sites to navigate as an student.
Publish or Perish tries to use its own "machine intelligence " to help you to construct simultaneous searches of Google Scholar and some other online databases. The results of PoP queries (in my opinion) are adequate but not approaching excellent. With little effort you can on-your-own search databases with "better" results. By that I mean you can get results that include items that an automated search will miss while also omitting irrelevant items that the automatic search term and query construction will find.
A "systematic literature review" requires time, effort, knowledge of the topic, and (alas) some knowledge and experience with the query systems of the databases you will use. Also required is knowing which databases to use and how much overlap there is between / among those databases.
If you use a cross-platform automated query you are almost certain to have results that omit entire blocks of areas of relevant knowledge. When you do the queries yourself you are likely to find articles on pertinent things that are directly relevant but that the "smart" search will dismiss (never show you) as chaff.
None of us has unlimited time to accomplish our tasks but quality is related to the time and informed effort you apply. A few minutes of conversation with a librarian can help you to do a better database search than relying on automation.
With the current state of technology, if Zotero were to add an automated query capacity I wouldn't use it.
I feel that this is misrepresenting the intent of the request, however.
I use PoP as a way of constructing and running searches of INDIVIDUAL databases from a desktop interface rather than having to keep links to multiple database hosts on my desktop or tabs in the browser.
I'm interested in the idea of being able to obtain statistics from searches, store said searches, replicate searches precisely (or in similar ways), and select then transfer results straight to a collection without the awkwardness in between - ResearchRabbit seems to want to be this missing link using AI but I agree a manual (but convenient) step is required.
At the moment, you could use the following workflow:
1. set up your searches in PoP
2. Execute them as needed
3. Export to a Bibtex or RIS file
4. Import the file into a Zotero collection
If the PoP export can be performed incrementally (so that you don't need to worry about duplicate records in Zotero), I think it would essentially meet your need without any development. What do you think?
I hate EndNote with a passion but it does seem to manage searches in-line well, so I was thinking that with PoP codebase being so light, it might be a good scaffold for building the functionality straight into a Zotero tab.
However, you should be aware that importing RIS as new collections or sub-collections is optional. It doesn't need to be as messy as you have observed, and even if you create unnecessary collections, you can simply delete them without any impact on the actual records: in Zotero, collections behave like tags more than as folders.
Are you primarily interested in the bibliometric (number of citations) information that PoP collects? Have you spent any time visiting the database websites and using the Zotero import button to capture the record(s) you want? Granted, you will get duplicates if you pull from multiple different databases or publisher websites but you will also get duplicate records when importing from within PoP. Zotero allows you to merge records and keep the metadata items that you want from either duplicate record.
Is the main benefit of PoP that you can enter search terms once and automatically query several databases? Sorry, I still don't understand what you want to achieve. I don't mean to be dense.
There is a device, once common until banned by the US Consumer Product Safety Commission, called a baby walker. It is also called infant walker, babywalker, or any of more than 25 other terms used by authors. Entering "baby walker" into a PubMed or MEDLINE query until recently would miss more than half of the records in the database because there was no automatic term-mapping to the other terms. Now, baby walker is mapped with several of the other terms authors used to describe these devices.
Another example, lets say that you wish to find journal articles about disaster-related drowning. You enter those two terms into the PoP query field. You will miss many, many relevant articles. This is because it is common for authors in disaster and civil engineering fields to never use the word "drowning" when they write about disaster deaths from things such as storm surges, dam failures, levee collapses, lost ships, etc. They write about "resulting fatalities", "loss of life", "missing and presumed dead", "lost at sea", etc. Drowning as a result of these failures is understood as a "given". Even MEDLINE records with assigned MeSH terms by indexers will not have the MeSH term "drowning" and these articles will not be found because of term mapping. (The articles have been assigned MeSH terms "Built environment", "Disasters", "Floods" but not the MeSH term "Drowning" even though the full text of the article describes flood-related deaths.) In my opinion, these articles are of primarily interest in disaster-related drowning. In my correspondence with NLM indexers they say that the primarily focus of those articles is the structural failure that led to loss of life and not the resulting deaths themselves. (I know this because I was recently asked to help with a drowning literature project.) To summarize, you are requesting the Boolean Drowning AND Disaster of the database and because the word or term "drowning" is not in the article you will not find the article among your results.
This is with a database that is heavily curated and has "keywords" (MeSH terms) assigned by professional indexers. Other databases such as the Web of Knowledge family, Scopus, etc. use text-word based searching assisted by some form of synonym mapping. PoP depends upon these maps as well as some of its own to find the articles it "thinks" you want. Without even basic knowledge of the rules for each database you are likely to miss many relevant records and possible even omit whole but key areas of research because you didn't use the proper jargon in your search. PoP's mapping may help with this jargon problem but it is grossly insufficient for real university-level scholarly reviews. If you look closely at published reviews, one of the authors is likely to be a reference librarian or a reference librarian will be mentioned prominently in the acknowledgements. Know that, if your manuscript is a review, most journals are likely to assign a reference librarian as one of your peer-reviewers. It may even be assigned to me.