Publishing a large bibliography on the web, with a faceted search interface?

2»
  • @bjohas It certainly would be a good opportunity for me to learn what people would like to do with Kerko, and perhaps to explain more what one can currently do with it!

    @parkjinn Really happy that you got it up and running!
  • @parkjinn , how did you manage to deploy the kerkoapp to heroku? I generated a requirements.txt file using `pip freeze > requirements.txt`, but I am struggling with the Procfile contents. Any pointers on that?
  • Hello @dlesieur and @floriand - shall we set a date for a session on kerko, with some trouble shooting tips?
  • @bjohas I also want to know how did you handle deploying your app with Heroku. A code example or repository could be great!
  • I've just discovered this thread David and am very excited and interested. I have a bibliography that I would love to make available on the web. It now has about 10K entries and is currently in ProCite, but could easily be migrated to Zotero. The bibliography at http://quescren.concordia.ca/en/search would seem to be perfect for my application. But I'm a little concerned that the last comment is from 2021. Are you still working on this?
  • Very exciting, thank you. In your documentation you say it's best to "limit the number of distinct facet values under a few hundreds." I'm not sure I understand your use of the word "facet" here. Do you mean keywords, assigned "subject headings" or what?
  • @Karrowr This is about the number of distinct filters on the left-hand side of the Kerko search interface. These can come from different sources in Zotero: collections, tags, fields. That will depend on how you choose to configure Kerko. Most projects I have worked on use collections to define facets (as per https://whiskyechobravo.github.io/kerko/1.0/config-guides/#defining-custom-facets-based-on-zotero-collections) and prefer not to expose their tags as filters.

    Technical limitations aside, it is usually preferable to not overwhelm the end-user with a huge number of filters.
  • edited September 11, 2023
    Edit The above post was delivered while I was typing my own post here. I guess that I wasn't the one you were referring to...

    I'm not sure if I'm the "David" you were referring to or the Kerto contact David. My database software is currently managing more than 3/4 million records and has a thesaurus that allows for jargon and lay-term synonym queries. Records newly-added to the database are optionally distributed to subscribers to any of 30 RSS feeds based on topics of interest.

    The offer to freely share this system still holds. The only cost would be for my web developers to strip the SafetyLit-specific "branding" from the scripts. Zotero records (Articles, books, book sections, conf proceedings, reports, theses, etc.) via MODS are the input source for the database MySQL content.

    The Kerto public display UI is very very nice.
  • Thanks to both Davids. I feel I need to point out that I am a complete newcomer to Zotero and don't really understand most of the technical details about Kerto. I'm a retired map curator and cut my teeth typing 3x5 catalog cards. I do love ProCite and think I'm probably the only human still using it happily. But I have successfully migrated some records to Zotero, so am confident I could get that far. I'm a hale and hearty 78, but not sure I really want to devote a couple of years to learning to use Kerto (unless you can convince me that it's not that difficult).

    DWL-SDCA: What would it mean to "freely share" your database software? Are you saying that you could somehow get my database on the web, at some cost, of course?
  • @Karrowr Kerko (not Kerto) is a web application that has to be installed on a web server. Thus, some technical work is required to get it up and running, more so if you wish to also change the default looks of the user interface.

    Since you are new to Zotero, just in case you didn't know I should mention that you can always choose to put your records in a Zotero group library and make that library public. No third-party software needed, no web server to manage! Kerko is useful only if you need some of its features and customizability, and/or if you need to establish a strong web presence for your library.
  • edited September 12, 2023
    @Karrowr If you expect to have a database of several hundred records (or maybe a few thousand) then Zotero Groups may be sufficient. But that also depends upon (among other things) how many users will access the database and how many people will have the authority to add, edit, and delete records.

    My web database software was designed by me about 25 years ago and improved and updated through the years by a team I trust. I've offered it to others at no cost other than what it would cost to remove my brand and server addresses. It is designed for ever increasing size (number of records) and quick search response to queries. (That is, except for the author-name search for temporary technical reasons.)

    There are automated administrative utilities that fetch metadata for ahead-of-print journal articles after predetermined time intervals from initial publication; identify potential duplicate authors but with differing levels of name completeness ( based on coauthors, year of publication, and subject/topic) [we manually standardize author name completeness through this utility and with the aid of authorities such as ORCID and VIAF]; duplicate article detection with the ability to mark items as non-duplicate; a system that tracks how recently a journal's website has been visited/searched to identify relevant items (with 'knowledge' of the journal's publication frequency), and other housekeeping tasks.

    I offer the software because much of my time and the time of my wonderful developers was supported by government contracts and grants from NGOs. The software without much tweaking can handle a database of a bit over three times its current record count. The cost of removing SafetyLit-specific stuff wouldn't likely be very dear if you are a not-for-profit. My developers offer a very significant discount to non-profit educational organizations. After that work, you could use any web developer to manage your site.

    The system now runs on AWS and that is better and less costly than our previous co-located servers with additional CloudFlare assistance.

    Metadata from Zotero can be exported in a format that can be effortlessly imported into my system. Our current system is wholly dependent on the power of Zotero -- its ability to capture records from publisher websites, the ease of editing those records within Zotero, and the ability to export each item type in a way that can be imported into my web database system. Essentially, if you know how to capture records into Zotero, edit the records to add missing metadata such as language tags, and how to export from Zotero you know enough to maintain my fully automated website.

    It may well be far more powerful than you need now. There are about 80,000 users each month who query the database. At busy times we have 20 or so simultaneous users/searchers without any noticeable slowdown. There different levels of administrative authority. SafetyLit is free to use and is presented without advertising. You can contact me directly from the website. (I'm not sure of the rules about that sort of contact info thing here.)
  • @DWL-SDCA Thanks for this explanation. I've now spent some time with SafetyLit and it looks like it would work very well for my application. But obviously, if I am serious about this, my first goal should be getting my 13K ProCite records into Zotero,
Sign In or Register to comment.