Does anyone know of software which can create a customized report from RDF data? Or from any of the other formats to which Zotero data can be exported?
Zotero reports are nice but not very customizable.
More importantly though I am thinking through options for sharing a set of citations in a format that others can both view and edit.
Of course any of the Zotero export/import formats will work. I am trying to figure out what format is the most generic and likely to be usable/readable by the largest number of people, whether they use Zotero or not.
Groups have their place but I do not think they work well for viewing a larger number of citations. Plus if someone wants to download and "own" the list and then edit it to a personal version, it is not easy with Groups.
Zotero RDF is probably best - that's pretty easy for someone else to download and then import into their own local Zotero installation.
What size collection are we talking about and in what way do you intend to share them?
Groups do have the added benefit of being very flexible by allowing you to join and sync, import to Zotero via the browser button, but also export to a wide range of formats and citation styles. The downside, of course, is that the online options are limited to 25 items at a time.
On a customized website you can use the Zotero API to provide, with no scripting, access to collections with up to 100 items in a variety of formats. For anything larger, you'll need php.
As for your initial question -- I don't think you'll find good reporting tools that make sense of Zotero RDF in a very usable way. I certainly am not aware of any.
For very basic customization, there's the report customizer; I can't peek into your head for your needs of course, so perhaps I'm missing something, but for all else, seems to me, nothing beats groups here.
As for my typical use case or number of items, the situation I am most interested would be the ability to curate a list of "best" or "classic" articles on a given topic. Typically such a group might contain 50 to 250 citations.
Zotero has many features which help in such a project. But particularly in reference to Groups or Reports, I am pondering how to do two things.
First, there is no easy way to look at a group and quickly read all of the abstracts. In fact, it is hard to even include a Group on a blog or web page open to the public. The reader needs to have or set up a Zotero account, login, sign up for a group, then create a report on the local installation of Zotero. It would be much more helpful if the Group could simply be published online for all to see. I suppose ZotPress is the closest to what I am seeking - I do not think ZotPress can display abstracts but maybe I should look into that more. Perhaps it can do that with a custom citation style? Even still, I think Zotpress has a limit of 99 citations to be displayed - can that be changed?
Second, is there a way to annotate/comment on individual citations within a Group and show those when the Group is viewed? Perhaps by using the Extra field in Zotero and then a custom citation style in ZotPress to show that field?
Maybe "report" is not a good word for what I am trying to do. I think "web publishing of a Zotero Collection" is more clear.
If I use the Zotero report feature, then I need to manually do that process every time I make a change to the collection, correct?
Groups and Zotpress are the only ways to keep a Zotero Collection continuously updated/published on the web? Zotpress is the only way to do this in a way that is integrated into my own web page?
Does anyone know of software which can create a customized report from RDF data? Or from any of the other formats to which Zotero data can be exported?
...
It would be much more helpful if the Group could simply be published online for all to see.
since you previously discussed export formats, a static HTML file seemed to fit the bill. But for continuous updates, yes, you'd either need Zotpress, or manually update. Technically, the manual part can be ameliorated a little by services like https://www.pancake.io/, but you'd still need to remember to export.
The third option is to make your group public. I think public groups are viewable on zotero.org for everyone. Zotpress is the only one that directly integrates to my knowledge, but this isn't an area I've looked into in any depth.
Example exporting a CSV file and then importing that to Seektable (hover over top right of report to open the Media Explorer and see report full size):
(1) Zotpress is nice because it syncs automatically
(2) The Zotpress report looks really nice graphically.
(3) The Zotpress report is formatted as a bibliography, which allows for a nice appearance but is harder to read for detail than the table format of Seektable. I have not added all the fields yet to Zotpress, but even when the information is in an identical format it will be harder to quickly compare details using Zotpress.
(4) Thus the main feature apparently missing from Zotero reports is the ability to output as a table (except CSV, which then requires reformatting by other software); Zotero (and especially Zotpress) assume that a "report" should be in bibliography form.
(5) Given these two options, I would probably go with the Seektable report since it is much more usable from a content perspective even though not quite as pretty and even though I need to manually export the CSV and update Seektable rather than having Zotero sync the data as it does with Zotpress.
Unless seektable can pull from REST services. Maybe in that case it can be made to pull from the Zotero REST API. But you’d have to use one of the existing Zotero translators.
ZotPress assumes that your desired output can be rendered using a CSL style. That precludes tables I think, although I see something indicating it could do block display (but this would likely be stacked, not side-by-side blocks).
(It’s Emiliano BTW. I am OK with people abbreviating it to Emile, but I’m not fond of being addressed as Emilio, which unfortunately happens quite a lot)
Interesting ideas Emiliano. Seektable has an API for analyzing data but not for uploading files. However, Vitaliy (their founder and lead developer) just let me know that they now have a beta version of Zapier integration which includes among other features the ability to auto-update a CSV file from Google Drive or DropBox. So that will make the update much nicer; I will be able to use the CSV Export feature that you wrote and save it to a consistent location and then the rest of the report update will proceed automatically.
Any chance your CSV Export could run on a regular schedule for a specific Collection and to a user-specified Dropbox location? That is interesting because Seektable can also now auto-email reports regularly. If the export could be automated, then it would be possible to set up an RSS Feed in Zotero and get a nicely formatted report regularly sent.
Automatic regular CSV export would also require Zapier or a local chron job. That's not a job for an API itself (I think Zotero/Zapier integration would be cool; IIRC, the big upside of Zapier over ITTT is that you can actually script integration yourself and don't have to rely on them to provide it).
Emiliano - interested? How hard do you think that would be to set up? If we extended it to other things like adding citations or alerting of new citations aside from saving to CSV, there are so many ways it could be integrated with other web apps.
@adamsmith What's nice about Zapier is that they host it for you, but IFTTT maker + heroku (or AWS lambda I suppose) gets the same done. Also it'd be nice for such things if webhook delivery would be supported, which means there'd be no need for cron, but cron/zapier will do the job.
In any case, I think @"rkaplan@umrpc.com" meant scheduled export from the Zotero desktop client. I'd recommend against that; I think it should be possible to use the Zotero public API to pull content from a Zotero library, push it through an instance of translation-server, and put the results on dropbox or google drive. That's a lot of moving parts, non of which I'm very familiar with. I have no indication how hard it would be to set up. But this pipeline wouldn't require having the desktop client running, and would just talk directly to the API.
(the reason I'm inclined to think he meant desktop export is because he's using a custom csv translator I put together which wouldn't help him with the public server)
Is it possible through the Desktop client? In my particular use case, I may occasionally use images in the Notes field. Those do not sync to the server.
The entire note doesn't sync? Or the image doesn't sync? If the latter it wouldn't make much difference because a translator wouldn't be able to export the images either.
OK - I can work around that if needed. I can understand why especially for use by others it would be better to work from the server. I think Zapier for Zotero would be of interest to many many people.
Interestingly as well - though I get errors saying the images to not Sync, I just checked online and it turns out the image is in fact there. Anyway, if that particular feature does not work with Zapier I can deal with it somehow- getting Zapier/Zotero to work otherwise and on a schedule would be fabulous.
I'd have been surprised if that translator exported images that didn't sync. I didn't do anything special to the notes, the images are inlined into the HTML, so beyond note size I don't see why Zotero would care what's in them. The editor has no image button so perhaps it is not expected to be there; I'd just report the error as an anomaly and see how the Zotero devs respond. In any case, since images in notes do in fact sync as you say, error or no, I'd very strongly advice to go the server route. Automating the desktop client is going to be more work, more fragile, and will require a running desktop which is just an inherently unstable environment for this kind of thing. I've actually automated exports in a plugin and I'd replace that in a heartbeat if I saw an alternative.
A general Zapier integration would only be of partial interest to you I think, because I don't see Zotero running a server where any custom translator can be dropped in; AAMOF, the translation server supports only a small number of formats from the stock Zotero formats out of the box AFAICT; I see 17 hard-coded ones, and no avenue to add new ones without changing that code. A Zapier integration would help you in the sense that it could be copied for your private case.
(Agree on no custom translator support, but if there's a compelling use case, Zotero devs have absolutely been willing to add existing export translators, e.g. biblatex most recently. Dan doesn't love .csv export, so that might require some persuasion, though)
I suggest that the use case for CSV export is very strong. It is the most universal means of export of all. Having a CSV option allows for export of data to software not specialized for bibliographic use but still very useful for Zotero data - Trello , Coda, Seektable, and Airtable are but a few of a large number of web apps which are open to Zotero users with a CSV export option.
Upgrade Storage
More importantly though I am thinking through options for sharing a set of citations in a format that others can both view and edit.
Of course any of the Zotero export/import formats will work. I am trying to figure out what format is the most generic and likely to be usable/readable by the largest number of people, whether they use Zotero or not.
But if you want to share a set of citations in a format that others can both view and edit... that's what's zotero groups are for.
If you're looking for something comprehensive, that'd be zotero rdf. Editable, CSV.
Thanks
Groups have their place but I do not think they work well for viewing a larger number of citations. Plus if someone wants to download and "own" the list and then edit it to a personal version, it is not easy with Groups.
Zotero RDF is probably best - that's pretty easy for someone else to download and then import into their own local Zotero installation.
If you do want a private copy, drag and drop would do it.
But for import /export, zotero - native, your best bet is zotero rdf for cleanest round tripping.
Groups do have the added benefit of being very flexible by allowing you to join and sync, import to Zotero via the browser button, but also export to a wide range of formats and citation styles. The downside, of course, is that the online options are limited to 25 items at a time.
On a customized website you can use the Zotero API to provide, with no scripting, access to collections with up to 100 items in a variety of formats. For anything larger, you'll need php.
As for your initial question -- I don't think you'll find good reporting tools that make sense of Zotero RDF in a very usable way. I certainly am not aware of any.
As for my typical use case or number of items, the situation I am most interested would be the ability to curate a list of "best" or "classic" articles on a given topic. Typically such a group might contain 50 to 250 citations.
Zotero has many features which help in such a project. But particularly in reference to Groups or Reports, I am pondering how to do two things.
First, there is no easy way to look at a group and quickly read all of the abstracts. In fact, it is hard to even include a Group on a blog or web page open to the public. The reader needs to have or set up a Zotero account, login, sign up for a group, then create a report on the local installation of Zotero. It would be much more helpful if the Group could simply be published online for all to see. I suppose ZotPress is the closest to what I am seeking - I do not think ZotPress can display abstracts but maybe I should look into that more. Perhaps it can do that with a custom citation style? Even still, I think Zotpress has a limit of 99 citations to be displayed - can that be changed?
Second, is there a way to annotate/comment on individual citations within a Group and show those when the Group is viewed? Perhaps by using the Extra field in Zotero and then a custom citation style in ZotPress to show that field?
A report is HTML, so if you save that, you tackle two issues at once:
I'm not familiar with ZotPress (nor am I keen to read PHP code), so I can't comment on that.
Maybe "report" is not a good word for what I am trying to do. I think "web publishing of a Zotero Collection" is more clear.
If I use the Zotero report feature, then I need to manually do that process every time I make a change to the collection, correct?
Groups and Zotpress are the only ways to keep a Zotero Collection continuously updated/published on the web? Zotpress is the only way to do this in a way that is integrated into my own web page?
I was referring to
since you previously discussed export formats, a static HTML file seemed to fit the bill. But for continuous updates, yes, you'd either need Zotpress, or manually update. Technically, the manual part can be ameliorated a little by services like https://www.pancake.io/, but you'd still need to remember to export.
The third option is to make your group public. I think public groups are viewable on zotero.org for everyone. Zotpress is the only one that directly integrates to my knowledge, but this isn't an area I've looked into in any depth.
Thanks for your thoughts
Here are some specific examples to show what I am trying to work out - this is just test data as I work out the formats
I will be posting my reports in Discourse as a basis for discussion topics.
Example report using Zotpress:
https://discourse.reference.solutions/t/zotpress-iframe-demo-to-read/77
Example exporting a CSV file and then importing that to Seektable (hover over top right of report to open the Media Explorer and see report full size):
https://discourse.reference.solutions/t/seektable-pivot-table-test/78
Comments:
(1) Zotpress is nice because it syncs automatically
(2) The Zotpress report looks really nice graphically.
(3) The Zotpress report is formatted as a bibliography, which allows for a nice appearance but is harder to read for detail than the table format of Seektable. I have not added all the fields yet to Zotpress, but even when the information is in an identical format it will be harder to quickly compare details using Zotpress.
(4) Thus the main feature apparently missing from Zotero reports is the ability to output as a table (except CSV, which then requires reformatting by other software); Zotero (and especially Zotpress) assume that a "report" should be in bibliography form.
(5) Given these two options, I would probably go with the Seektable report since it is much more usable from a content perspective even though not quite as pretty and even though I need to manually export the CSV and update Seektable rather than having Zotero sync the data as it does with Zotpress.
Unless seektable can pull from REST services. Maybe in that case it can be made to pull from the Zotero REST API. But you’d have to use one of the existing Zotero translators.
ZotPress assumes that your desired output can be rendered using a CSL style. That precludes tables I think, although I see something indicating it could do block display (but this would likely be stacked, not side-by-side blocks).
(It’s Emiliano BTW. I am OK with people abbreviating it to Emile, but I’m not fond of being addressed as Emilio, which unfortunately happens quite a lot)
Any chance your CSV Export could run on a regular schedule for a specific Collection and to a user-specified Dropbox location? That is interesting because Seektable can also now auto-email reports regularly. If the export could be automated, then it would be possible to set up an RSS Feed in Zotero and get a nicely formatted report regularly sent.
Emiliano - interested? How hard do you think that would be to set up? If we extended it to other things like adding citations or alerting of new citations aside from saving to CSV, there are so many ways it could be integrated with other web apps.
In any case, I think @"rkaplan@umrpc.com" meant scheduled export from the Zotero desktop client. I'd recommend against that; I think it should be possible to use the Zotero public API to pull content from a Zotero library, push it through an instance of translation-server, and put the results on dropbox or google drive. That's a lot of moving parts, non of which I'm very familiar with. I have no indication how hard it would be to set up. But this pipeline wouldn't require having the desktop client running, and would just talk directly to the API.
- Richard
(2) The translator you wrote for me does a great job exporting the images to the CSV file. Try it.
OK - I can work around that if needed. I can understand why especially for use by others it would be better to work from the server. I think Zapier for Zotero would be of interest to many many people.
Interestingly as well - though I get errors saying the images to not Sync, I just checked online and it turns out the image is in fact there. Anyway, if that particular feature does not work with Zapier I can deal with it somehow- getting Zapier/Zotero to work otherwise and on a schedule would be fabulous.
https://imagebin.ca/v/4ZV3yZ4O1a3z
A general Zapier integration would only be of partial interest to you I think, because I don't see Zotero running a server where any custom translator can be dropped in; AAMOF, the translation server supports only a small number of formats from the stock Zotero formats out of the box AFAICT; I see 17 hard-coded ones, and no avenue to add new ones without changing that code. A Zapier integration would help you in the sense that it could be copied for your private case.