zotero votebox?

I think it would be very useful if the Zotero site could implement something like Dropbox's "votebox," where people could vote for the features, fixes, etc. that they would most like to see implemented.

I am not saying that the developers should be obligated to prioritize whatever gets the most votes. And I do think there needs to be a hierarchy of decision-making for a project like Zotero to be successful. Usually the developers give good reasons for their conclusions about how the program should develop.

I also think that there is currently not a good way for the development team (or anyone) to know what is most important to the most users. Everyone has a different workflow. There is not always a solution or conclusion that is objectively preferable.

The forum is not a great way to get feedback. It is time-consuming to participate and I think people have to have a somewhat thick skin for these discussions.

I have great respect for Dan and Adam and other developers. You all are very responsive and open to input. However, sometimes a person will comment that they wish there could be change X, and the response will be along the lines of, "It's that way because of reason Y," or "no, I don't agree, it makes more sense to me the current way," and then it seems that there is no point in pursuing the issue. Or, 10 people might all post about some issue that they want addressed but no one on the development team responds, presumably because they don't think it's as important as other issues they're dealing with.

But if in fact hundreds of people want change X, there should be a way for the developers to know so that they can make decisions with some knowledge of user preferences. I assume that if 250 people voted that they want change X, the developers would want to know about it and would be open to reconsidering it, even if it didn't seem completely consistent with their opinion about UI design or whatever.

I agree with the development team's opinions almost all of the time. But I don't think that any person can really know what will work best for the most users without some feedback system.

Thanks!
  • I think that there are a couple of threads about this already. You should search for 'voting' and 'vote' to see those.

    If I remember correctly, the developers have nothing against this if there is someone who volunteers to maintain the voting system.
  • It's been tried, and it didn't work. For intance, an attempt to start a UserVoice forum died a silent death. From the side of the developers' side it was quickly made clear that development is not in the first place driven by user requests. Yet as I argued back then (and it is still true right now) a lot of energy is wasted on the forums because of duplicate threads and the sheer difficulty of locating devs' statements on even the most commonly commented on features or feature requests.

    As I said then:
    I don't think Zotero is doing as good a job in communicating to the userbase as it could. Although the devs are confident they got the main priorities covered, the opacity-of-mind problem remains and this leads to a lot of time wasted in the forums. My intent was to suggest an improved interface that would help take away some of these problems. If this is not a priority right now, so be it. I look forward to the improvements that a better alternative for Trac will bring. I remain skeptical that the forums will become a much more enlightening place through it though.
  • You can check what is planned here

    https://github.com/zotero/zotero/issues
  • It is not a software issue. It is about finding a person that is committed to maintaining such tracker.

    Once there is a well working and maintained tracker, I have no doubts that Zotero will link to the tracker from the forums.

    Before suggesting something like this, you should have a response for two questions:
    1) Who will maintain it
    2) What will be done differently so that it will not end up dying like this attempt
    http://forums.zotero.org/discussion/12945/give-this-a-try-for-user-feedback-zoterouservoicecom/
  • edited July 3, 2012
    What made that UserVoice attempt die though was not lack of maintenance or a fundamental flaw in the execution (it is the same kind of forum that is used successfully by many other projects). It was active disalignment from some vocal users (one of whom downvoted every single thread just to prove the point they didn't like it), a misunderstanding on the part of some devs that this was being proposed as the primary forum for driving Zotero development, and disinterest on the part of other devs because "it's not lack of awareness of user priorities that prevents things from getting done—it's lack of developer time, and our having other priorities for a variety of reasons" (Dan).

    It is still the case that the developers profess to having a very clear idea of what users need and want; and that users have difficulty finding out what is planned, what other users have requested before, what proposals are being considered, and what developers have said in response to requests or proposals. In other words, there is still a disconnect between what devs know and can provide for, and what users know and should (or need not) ask for. If someone would come up with a solution to address this it would be great, but from past experience it is not going to be straightforward.
  • edited July 3, 2012
    But if in fact hundreds of people want change X, there should be a way for the developers to know so that they can make decisions with some knowledge of user preferences.
    If hundreds of people want change X, then some fraction of those users post here. As I said in the other thread, "It's usually clear when there's demand for a feature because it leads to actual thoughtful discussion."

    Beyond that, as the person ultimately implementing a lot of these requests, I don't really care how many people would vote for something. I implement something if I think it's a good idea, there's a good technical solution, and I have time to work on it. And I'm quite happy to engage in discussions about those things here.

    For what it's worth, while I'm sure quickfold11 wanted to make the more general point, the post that (I assume at least partly) prompted this thread was in fact answerable by a basic search ('search delete clear'), which would have led to a succinctly named thread ("Quick filter/search box reset when item added or deleted") that included links to the relevant issues on GitHub and an update from me when a patch was checked in. I don't think there's all that much to improve upon there, process-wise. But that search would've worked only on the built-in forums search. I've been asking Faolan for a while to drop the fairly useless Google-powered search in the navbar and revert to always using the Vanilla search, which is the only one I ever use. I think that change is coming with an upcoming site update, and that may go a long way towards helping people find relevant threads.

    I don't think there's much more to say on this that wasn't covered in the other thread. The general consensus seems to be that the things that would be most helpful would be 1) the ability to highlight important posts within forum threads at the top of the threads, the way some other forums work and 2) a wiki page that outlined some of the more popular feature requests and provided a status summary with links to relevant forum threads. The former requires Faolan or someone else writing a Vanilla plugin to do it, if one still doesn't exist. The latter could be done by anyone—but, then, that was also the case a couple few years ago, when it was first suggested.
  • edited July 3, 2012
    Please drop the "fairly useless" nav bar Google search. Even though I have learned from experience that I should go directly to the Search link in the left-hand column, I nonetheless sometimes absentmindedly try the nav bar search. I've never had much success with the Google search.
  • I think that the most important problem with searching is the large number of false drops. This is especially an issue because of the feature requests by inexperienced users that are in reality a help requst. Their use of words that mean one thing to them and another thing to more experienced users (library / collection) helps the textword search to return a large portion of useless results. Also, as I am doing with this thread, a discussion on one topic can develop into one on a different topic quite easily.

    I've been thinking about the possibility of assigning index terms to each post. I am mainly thinking of this for my own site but I believe that it could work here. Zotero already asks that messages to the forum be assigned to categories. If there could be a requirement that each poster also add terms from an index list (sub- categories?) some of the jumble of search results might be cleared. Instead of only the first poster having control of the topic category, the poster of each reply would assign terms to her or his comment.
  • (FWIW if there are sufficient people using it/knowing about it, one place where UserVoice would be very useful is translator development. I don't think I - or anyone else writing translators - has a good idea which translators would be useful to many people, especially now that the existing translators are in pretty good order and we have two (community) people (Aurimas and me) working on translators regularly. The old site translator threads were/are a mess. Translators are distinct from other dev decisions in that implementing a translator has no impact on larger design decisions, so "democracy" would actually be pretty useful for me to pick a translator to work on).

    Having spent some more time looking at Mendeley's implementation of UserVoice over the last year, I've come to agree with Dan in his main critique of it: It's a bad place to communicate with users - if you look at the actual comments, many of them could actually be addressed relatively easily in a forum thread, or they'd require some back and forth to hash out details etc. As a venue for feature requests, it does more harm than good.

    I still very much agree with mark's main reason for suggesting it - I don't think Zotero does a good job communicating to users what the current status of some of the most frequent requests/concerns is (FWIW I don't even think it does a very good job communicating _existing_ features - my bet would be that 50%+ of all users have no idea about the pretty awesome "Store References in Document" feature, for example).

    If someone wants to start a wiki page for that - the top 20 most frequent requests and their status or so - I'm happy to help maintain it.

    That's a different concern from the one expressed by quickfold - i.e. that Zotero devs don't have a way to get good feedback from users - that's hard to know, but reading almost the same number of threads that Dan does, I think I can safely say that you get a pretty good sense of what comes up frequently. Sure, Dan doesn't know whether something is requested by 50 or 500 people - but as he says, that's not the most relevant piece of information anyway.
  • IMO what would be sufficient is a sticky thread for feature requests and another sticky for translator requests.

    I know there's a translator request thread, but it was never stickied so it's essentially lost to the user that drops by the forum.

    A wiki page would be almost the same, but I would argue that the forums are much more visited and a sticky at the top of the category would be more visible and more frequently updated. Plus, I would suggest keeping the thread clean (maybe even locked) and providing very succinct information on each feature request (description, status, etc.), including declined features, and linking to the main thread for further discussion. One problem I can see is that taking over the thread (i.e. the first post) after the initial maintainer decides to abondon it could only be done by a moderator/admin, which is where a wiki page would fit much better.

    Perhaps we can create a fake sticky thread that links directly to a wiki page??

    Either way, I would really like to see what features need to worked on.

    Finally, I think a similar wiki page/thread would be useful to list for goals for the next major Zotero release. Not as important, but I think it might attract more community developers.
  • Perhaps we can create a fake sticky thread that links directly to a wiki page??
    That sounds like a good idea to me. Agree that forum is more visible, but I think having the list editable by multiple people is important. I don't think Dan et al will have any objection to create a sticky thread with a link once the wiki page is at least decent.
    Finally, I think a similar wiki page/thread would be useful to list for goals for the next major Zotero release. Not as important, but I think it might attract more community developers.
    not sure whether forum or issue tracker + help wanted tag is better for that?
  • edited July 3, 2012
    not sure whether forum or issue tracker + help wanted tag is better for that?
    We already have a Help Welcome tag on GitHub (and an old helpwanted tag on Trac). If we just populate it and link to it from somewhere, I think that'd be fine.
  • I like it!
    (BTW - I thought custom fields were out completely, but Dan has mentioned that it might still be something that could happen in the long run if the various issues are solved)

    @Dan - yes, I know. That's why I mentioned that as an option.
  • Yeah, custom types/fields are not completely off the table. There might need to be a third category for "Possible long-term features", with the warning that people shouldn't hold their breath for those.
  • We already have a Help Welcome tag on GitHub (and an old helpwanted tag on Trac). If we just populate it and link to it from somewhere, I think that'd be fine.
    I was also trying to say that a list of goals for the next major release would be somewhat motivational. It's fun to tick off things from a list :-) Plus things that are planned for the next release indicate higher priority. Though I've read somewhere on the site that Zotero doesn't want to make promisses in case it cannot keep them. Anyway, just a thought.
    BTW - I thought custom fields were out completely, but Dan has mentioned that it might still be something that could happen in the long run if the various issues are solved
    That was just the first thing that came to mind. I must admit I did not thoroughly research it.
    There might need to be a third category for "Possible long-term features", with the warning that people shouldn't hold their breath for those.
    Perhaps. On the other hand, are there features that are absolutely never going to be implemented? I think almost any rejected feature could end up on the "Possible long-term features" list. No?
  • It's fun to tick off things from a list :-)
    Good point. The translator counter e.g. is a great motivator.
    Perhaps. On the other hand, are there features that are absolutely never going to be implemented? I think almost any rejected feature could end up on the "Possible long-term features" list. No?
    There are things like "make collections behave like folders" or "let people create a webpage item type from the green plus (create new item) menu," that are design decisions.
  • It's fun to tick off things from a list :-)
    It is. I would just like to have fewer lists in general. Closing a GitHub issue with a commit is pretty satisfying, too.
    Perhaps. On the other hand, are there features that are absolutely never going to be implemented? I think almost any rejected feature could end up on the "Possible long-term features" list. No?
    Sure, but I think it's more about current plans. There's a difference between a current maybe and a current absolutely not.
  • I did a little rearrangement on that page and put things into categories based on what area of Zotero they affect. I think that will be more useful to the user looking for a feature that he/she has in mind.

    Also added a bunch of feature requests. I went through the 100 most recent posts and picked out some feature requests that have been discussed. I threw in a couple feature requests that I remember reading about previously, but I'm sure I've forgotten a lot of them. I'm not sure if all of them are worthy of being on the list, so feel free to remove them.

    This is clearly not meant to be a final copy. I got a little tired of browsing through the forum, so if someone else wants to chip in and fill in some blanks, that would be wonderful. Otherwise, I will get back to it tomorrow.

    Thoughts on the format, usefulness, etc. of the list??

    http://www.zotero.org/support/requested_features
  • looks great. I'll help fill some of these out tomorrow.
    My biggest concern currently is how to highlight that "future version" means "don't bet on this happening in the next couple of years". Raising false hopes annoys people.

    Two ideas
    1. Instead of "Planned for future versions" we write: "Not planned for medium term. Possible implementation in future versions (>>24months)" or so.
    2. We go back to your first idea and put those items in a separate part of the page & write a brief note along the lines above.
  • I'm not sure if all of them are worthy of being on the list, so feel free to remove them.
    Yeah, I would say it'd be better to restrict the list to fairly major features (batch editing, custom item types/fields, per-library sync, on-demand download, bookmarklet) and just make sure most other "planned" things are in the issue tracker with links to relevant threads. It doesn't make sense to manually replicate the issue tracker on a wiki. I'm fairly concerned about maintenance of this page, particularly since the core devs probably won't be doing much to maintain it—we'll just close tickets and post to the relevant forum threads.
    I did a little rearrangement on that page and put things into categories based on what area of Zotero they affect. I think that will be more useful to the user looking for a feature that he/she has in mind.
    I'm not sure that's really the use case for this page, though. In my view, this page makes sense as a way to help average users see big upcoming features in a less rigid way than a roadmap. People just searching for individual issues are going to need to search, one way or another, and hopefully we'll be able to improve search results and figure out a way to summarize forum threads on the threads themselves.

    Alternatively, if the goal is really to present a much wider set of planned features in a more user-friendly way (and also not have average users posting to GitHub), we could use the GitHub API to present the full issues list in some searchable, read-only fashion, with links to forum threads extracted from the descriptions.
  • I have comments on many of the points brought up here, but before starting a discussion on whether there is an effective way to collect user feedback, Dan's comment makes clear that the central issue regarding user feedback has nothing to do with implementation. I apologize that this post focuses to some degree on Dan personally, but I can't see a way to avoid it. Dan says:
    Beyond that, as the person ultimately implementing a lot of these requests, I don't really care how many people would vote for something. I implement something if I think it's a good idea, there's a good technical solution, and I have time to work on it. And I'm quite happy to engage in discussions about those things here.
    I take it the last thought is more fully expressed as "I'm quite happy to hear what you think, but once you have made your case, I'll tell you my decision and everyone will accept it."

    My first post assumed that the developers did care what was most important to their user base. In fact, I assumed that a grant-funded open-source project like Zotero was driven by an aim to produce something that is the most useful for the most people. If hundreds of people think it is important to implement feature X, but Dan doesn't think the feature is elegant or intuitive or whatever, then that's the end of it? Dan certainly knows more that the users about technical possibilities, the project's resources, etc., but solely in terms of whether a feature of the program is desirable or should be given more attention, why is Dan's individual judgment more important than the collective opinion of the user base who actually use Zotero every day as a central part of their workflow? It is not just a numbers issue. I assume that the developers work on Zotero frequently or daily, but also that they do not regularly use it for hours each day as a reference tool for writing research. My apologies if this is not the case. If it is, however, then I'd propose that what seems like a good idea conceptually may not seem like a good idea to a person who is actually using the tool to perform hundreds of operations a day/week/year.

    As I said in my original post, there are a lot of things that do not have objective best answers, and I would hope that the developers would agree. Perhaps the response to this post will be the common open-source response of "if you want it, do it yourself." I have offered to help with the project as a non-programmer, but I'm not a programmer. I'm a humanities researcher. If there is a way for me to maintain a votebox without doing programming, I'm volunteering right now to do it (or at least to help, depending on the magnitude of the project). I don't know whether the Zotero staff are volunteer or full-time employees on a grant or a mix. If Dan is volunteer, then I can understand his comment.

    But, if he is employed by a grant, then this comment floors me. What was written in the grant proposals to get funding for this project? We propose to create a useful software tool based on what the lead developer personally thinks the tool should do, and secondarily, it should ideally but not necessarily address the wishes and workflow of its user base?

    One reason I suggested a votebox (which is very different conceptually and in practice than any type of forum) is because there are issues with Zotero put forth in threads that have not been engaged with by the developers in any serious way. That seems against the spirit of this type of project.

    I'll stop here to see any responses before I comment on the other issues with UserVoice, etc.
  • I take it the last thought is more fully expressed as "I'm quite happy to hear what you think, but once you have made your case, I'll tell you my decision and everyone will accept it."
    Well, not really. First, I did mean engage in discussions, not just hear what people think. That's part of the reason I favor the forums over other venues—because the forums allow for real, in-depth discussions, which are worth far more than vote tallies.

    But after such a discussion, if I (and the other paid developers) decide not to implement something, then no, everyone doesn't need to just accept that. Someone can write a plugin to implement the feature and distribute that. Someone can create a patch that demonstrates how it actually works and ask us to take a look. Someone can fork Zotero (temporarily or permanently) and distribute the alternative version. If it's mainly an issue of time/resources, someone can offer to find funding for its implementation by us or others.
    why is Dan's individual judgment more important than the collective opinion of the user base who actually use Zotero every day as a central part of their workflow?
    Because Zotero isn't designed by committee. (Or, if it is, it's a very small committee.) Our goal is to produce the best software possible, and that requires our having opinions about how it should work. If someone disagrees with our opinions, they can try to change our minds, do one of the things I listed above, or use something else.
    But, if he is employed by a grant, then this comment floors me.
    I'm not sure why. We don't get grants to then go take votes about what to do with the money. We get grants to implement specific things that we think are worth doing (where the "we" here means, in large part, the humanities scholars that conceived and direct the project). And then we do plenty of other non-grant-funded things that we think are worth doing. Discussions here play a large role in determining exactly what those things are, but ultimately, our job is to make those decisions.
  • edited July 11, 2012
    Two other things :
    1. Pretty much everyone except Dan and fcheslack* that you'll meet here is a full time researcher and does use Zotero for that. Sean, the project director is a Historian, Simon, the other senior developer is a grad student and MIT. Fbennett, who wrote the CSL processor and maintains the multilingual fork of Zotero is a lawyer. Bdarcus is a geographer. Aurimas works in the life sciences. I'm a political scientist and have my dissertation and multiple articles/chapter written with Zotero. etc. So yes, you're wrong to think that people here don't actively use Zotero for scholarly work.

    2. Being a political scientist, I do feel I have a professional obligation to point out that not just isn't votebox/userVoice democracy (I think that's obvious), it's also a poor mechanism to aggregate user demand. a) because by the nature of it you almost entirely get votes _for_ something, but people who dislike e.g. a feature change will pop up only when it's implemented (in other words - how do you know that 500 votes among 500,000 users are a majority?) and b) there is a large literature in political theory and sociology on the advantages of deliberation and deliberative democracy over decision-making based on (majority) voting. It's just not the case that the aggregation of low-cost operations such as casting an online vote are very useful to decide complex questions.

    To turn towards something more specific - I think the forum works much better than you seem to think it does. So I'd be interested to know of a couple of issues where you think using votebox would add value. Those issues would have to be feature requests that devs aren't aware have high priority among users or feature changes without significant opposition (since as per above, votebox isn't suitable to distinguish between alternatives).


    * I don't actually know what Dan does when he doesn't do Zotero
This discussion has been closed.