I can't imagine any sensible use for a rating system at all. I'm interested in what people have to say about items, but not remotely in how someone (even me) 'rates' items along some fake measurement scale (if we had a list of Wittgenstein's top 10 philosophy books, how much good woudl it do us?).

Anyway, I'll no doubt be in a minority on this one ;) So I'll request a feature instead: will I be able to turn it off?

Hi Dan,

I think the proper way to add rating support to zotero is to move the tag system to a full taxonomy system just as drupal and wordpress did. Then It will allow users and developers to add rating features, tags, categories, locations, school of thought, etc.

You can take a look how wordpress handled it: http://codex.wordpress.org/WordPress_Taxonomy

I should probably provide a bit more context. One question is the potential use of a rating system as a single-user organizational tool. The other is usefulness in a multi-user environment, particularly in a recommendations system. Focusing on the latter for the moment, CB, you're right that a single user's ratings might provide little utility (though perhaps a bit of novelty). However, in aggregate, ratings data potentially becomes much more interesting.

The fundamental problem with not having a rating system is that there's not really another good metric for identifying the references that users have in their Zotero libraries but don't actually like or find useful. External citation databases could be a start, but users will have materials in Zotero that aren't in such databases, and the number of times an item is "cited" in Zotero isn't a clear measurement. Citation frequency could also be misleading in general—The Bell Curve might be cited frequently but in vastly different contexts. It seems a recommendations system should have some way to assess that difference. Even having just like/dislike buttons would provide the system with an added dimension to the data (and it would to some degree avoid the problem of different users having different criteria for what goes where on a point scale).

The fundamental problem with not having a rating system is that there's not really another good metric for identifying the references that users have in their Zotero libraries but don't actually like or find useful.

I'm a tad dubious about dealing with the lack of a good metric by using instead a meaningless one. If I rate some things high because I like them, and others because I find them useful, I can't remotely be considered to have said the same thing about both (except to an economist, who'd expunge 'like' and 'find useful', and call them both utilities). I can't see how aggregating these kinds of ratings amounts to anything other than noise (which is just what aggregated ratings do become wherever they're used).

I won't flog this as it just seems to be a fact that ratings systems are popular (even my uni library is using them in our opac, heaven help us), so I guess your users will want them.

But as an aside, it's probably not *too* hard to come up with more useful ideas for aggregating something approximating people's judgements, though implementing them would be a scarier matter. Here are two off the top of my head:

Idea 1: adjectives with a rating scale. Have a kind of 'tag' that is, roughly speaking, an adjective (would this be a unary relation, or a binary relation with an adjectival concept, Bruce?). Tagging with an adjective would require placing it on a scale. The scale for each adjective could be averaged across users (or, ultimately and better, across chosen sets of trusted users or groups). Thus you could see at a glance see how people think of an item in terms of specific descriptions (eg. 'well-written', 'good experimental design', 'good fact checking' etc), rather than contentless preferences.

Idea 2: offer a tag-like 'cloud' for items, that scales substantive terms by their frequency of appearance in notes which users have opted to share (or at least to make available for aggregating purposes). I've no idea how informative this would be in practise, but coming up with algorithms to extract contextually substantive words would be a fun project for someone thus inclined ;)

Sorry, not meaning to be pointlessly negative here. This topic just happens to have set one of my itches a-tingling.

Bruce: All good questions.

what if I absolutely hate an article or a book because I disagree with the ideas, but I consider it essential reading. How do I rate that. and to do so in such a way that it distinguishes it from, say, an article I find simply weak (uninspiring, thinly-supported, whatever)?

It's an important distinction, though it's possible it would come through simply by volume. If something is essential reading, it seems likely that it will be in a large proportion of the Zotero libraries of people for whom it is essential, even if it is widely disliked. Something that is both bad and unimportant might not be.

Or, as an alternative, if this is the fundamental distinction: have both like/dislike and relevant/irrelevant. ("Relevant"/"irrelevant" may not be the right phrasing, but, basically, the question of "Is this something I need to read?".)

The issue of different audiences might come out in the wash. It'd be a bit more subtle than explicitly saying, "This item is unimportant for undergrads but essential for grad students," but the items appropriate for each group should be sufficiently clustered already by other metrics (libraries they appear in, collections they appear in, research fields of users whose libraries they appear in, age/grade/degree of users whose libraries they appear in) for the recommendation system to figure out the places they belong.

CB: There might be a distinction between blunt yet fundamental questions of the worth and relevance of an item and the more descriptive sort of adjectives you list. The former might be more useful in a recommendations system, whereas you'd certainly want the latter to be available for filtering, searching, or visualizing the recommended items.

Dan: from my pov a more ramified system would be better for generating recommendations (if only because a digg-style setup sets such a low bar, being of no worth at all).

The only kind of recommendations I'd be interested in would come from a *weighted* set of adjectives or similar, with the weighting decided by me, and taking input only from a trusted group or network. There 'ain't no wisdom of the crowd, and star-ratings are for tabloid movie reviews ;)

It's high time for a simple 5-star, one-click rating option. To me, it's not about how others interpret my ranking criteria, it's simply having a way of quickly separating the wheat from the chaff in my collections. When I collect material for research, I just add relevant material first and sift through it later. As my collections grow, have a handy way to visually flag important stuff is invaluable.

I was going to suggest this as well.

I actually think the * rating tags are better than a workaround--properly deployed, it means you can very easily see all 5-star papers with a given subject tag or combination of tags using the tag selector pane, or create saved searches for same.

Though now that I'm thinking through some use cases, I suppose one wouldn't be able to sort by ratings with the tag system, since each rating would be its own tag.

I also would love to see a rating feature.

It should be
- quick and easy to use.
- usable as sotring criterium
- directly visible in the item window

It would be nice if it had an averaging feature for groups.

Of course I can add tags like "*, **, ******", or "bad, boring, ... good, excellent", but they don't show up in the overview, and when I change my mind, they are not the fastest to change.

A 5-star ranking is something everyone is used to, so it would be intuitive to use.
Starting with a single user ranking would be good, but I see the strenght of such a primitive ranking when it comes to groups. If you have an average voting and the number of votes you can quickly see, if something is: good/bad, common/special knowledge.

And if you just call that field "Ranking" then everybody can use it for their own purposes, so user A can use it for personal liking, while B uses it for relevance to his topic and C for something completely different.

I would favour a rating system that is minimal and visual.

library item 1
library item 2
library item 3
library item 4
library item 5

In this way, I could highlight only the best publication in a folder with many items for easier retrieval later on. Also, it's flexible and could be used for something entirely different by another user and it does not use up precious space in the already crowded zotero window.

There's a decent chance that color-based marking will happen, and there's a good chance that, if it does, it will use tags.

+1
for a 1-5 star rating system as a simple single user organizational tool. And putting off the social/collective aspects of it for future deliberation. I think these are two separate issues and the implementation of complex collective rankings should not indefinitely defer (the first post in this thread is Mar 2008) this relatively simple request.

Personally for me, a simple 1-5 ranking for every citation is a very basic and necessary requirement to organize publications and its absence leads to a lot of pointless toil. For instance, I did an extensive literature search yesterday for a paper that I am writing and now have at least 55 references in that particular folder. Some of these need immediate in depth reading while some were added as references for minor points (and the rest fall in the middle in their importance). And I knew which was which when I added them but as I sit this morning to start reading I find myself at a loss. I switched to Zotero from Papers(which has a similar rating system); while Zotero offers much more than Papers and I don't intend returning yet I do miss this very simple feature.

I have tried using tags for this particular purpose and they don't compare well with the ease and simplicity of a simple 1-5 star rating system.

I vote for keeping it simple with a five star system. each user can decide if they want to use for how much they like it vs how useful/essential they consider the article. but I would love to have this option NOW NOW! : )
wildly grateful for zotero,
Sara

I would also like very much for a rating system, 5-star is fine.

For someone who asked why aren't Tags enough, here's a quick reply: I use tags to categorize my references, and I search in these categories by having a Saved Search with each category. If I used tags to rate, then I would have to create many more saved searches to view only the "well rated ones", for instance. With a real rating system, I could add the ratings to the view columns in zotero, and so when I'm browsing my saved searches, I could just sort by rating and have quick access to the papers that I selected as the best in that category.

Btw, I would really love this feature as soon as possible... I saw the ticket with it, and saw that it was even planned for version 1.0, then 1.5, then 2.0, but never implemented... Why???