Give this a try for user feedback: zotero.uservoice.com

Having seen multiple threads and messages over the last few years related to improving the way Zotero development and user feedback influence each other, and having seen Mendeley's fabulous implementation of UserVoice, I decided to go ahead and set up a proof of concept.

Find it here: https://zotero.uservoice.com/

I set it up as a free UserVoice account. This is just a proof of concept and may be deleted. I hope Zotero can spare a few bucks a month to set up something like this and to have additional features like user moderation and different categories, if needed. UserVoice is incredibly slick at providing domain name aliasing and JSON single site login, so perfect integration would be possible. However, even this free account allows a fair amount of possibilities.

Needless to say, as a loyal Zotero user, I would be more than willing to hand over this particular subdomain to staff if needed. On the other hand, I can easily imagine something like this being maintained by committed users — by way of a service to the developers which may help them gauge the priorities in the usersbase more effectively. That is to say it needn't cost valuable developer time, it might actually improve the development cycle by being a more efficient alternative for the current feature request system.

Small selection of earlier threads/comments by users related to the feedback process: Migugg; Rintze, Adam Smith and me; and Scot.
«1
  • Rationale for this kind of system: If properly tended — and this can easily by done by committed users— a setup like this is a great tool for developers to get a clear idea of what the userbase considers to be high priority issues. The rating system gives voice to many users that might not feel like placing a comment otherwise. Here on the forums there are pile-on threads too, but they get lost amidst other threads and often become dormant after a while without it being clear what happened to it.

    Yes, trac tickets are created sometimes and devs reference these tickets sometimes in the original thread, but in the current setup there is often a great disconnect between the user feedback and what is done with it — witness the threads referenced above.
  • I like this idea myself.
  • I'm skeptical, but would be happy to be proven wrong.

    Before we throw yet-another-tech (that happens to be used by a few other projects that have larger development teams), we should answer these questions:

    What are the top communication issues that need to be addressed and is this tool actually help to address them? Who will be the uservoice wrangler when there is not really a trac wrangler?

    Currently, users have a single point of communication with zotero (these forums) to address bug reports, feature requests, and support requests. These "brainstorm" webapps only really address feature requests. And the single thing they are able to do best is to see what popular opinion says are the most important issues. Developer priority is (and both should and must be) different than user priorities. Can one not already accomplish a similar thing by glancing at popular forum threads? Is the problem that this softwar solves even a real need for Zotero?

    Migugg seems to most want a system to keep track of the next steps of program development. I don't know whether I agree to the importance of this (it seems more important that the scarce resource of developer time is spent improving Zotero). But I also don't think brainstorm software actually addresses issue tracking.

    Mendeley's system actually demonstrates some of this disconnect: the second most popular request, over a year old, is for them to release their code under an open source license. The status is still listed as 'under review,' and the Mendeley developers only responded to it in the past few months to say "no, we can't do this."
  • edited June 4, 2010
    I'm not sure whether the the suggestion form will help -- I've never worked significantly with a project where I found it useful. That said, I'm not a typical user.

    As I believe I've said before, I think a more productive shift would be to have more issue tracking and bug fixing take place via Trac -- the more problematic disconnect, I think, is between zotero-dev and Trac, and between patch submission and patch landing.

    If there's a tech-of-the-month that I'd like to see applied to Zotero, it's a bug bounty site, which could also be non-official. A good number of the long-time feature requests are things that could probably be done by a sufficiently motivated outside programmer in only a day or two (or an hour or two), and money can sometimes be a good motivator.
  • Mark - thanks for this.
    I haven't fully made up my mind, but I think I tend towards noksagt and ajlyon. Gathering and organizing feature requests doesn't seem a priority to me. And Dan has suggested that developers often don't feel they can make realistic predictions on issues, so respsonses might not be particularly useful.

    Like miggug (and somewhat contra noksagt) I would place more value on issue tracking - I guess the tracker could work for this in theory, but does not in practice.

    Personally, I'd be happy with something very simple - like a "to do" list of the core developers for the larger issues by order of priority - it wouldn't even have to include ETAs, but just knowing the next couple of things they are planning to do - and updating that maybe once every 3 months - wouldn't seem like a big use of developer resources and would already help - so basically a little more detailed roadmap than the one we currently have and I for my part would be content.

    But I'm also perfectly happy to be convinced that a uservoice type forum can be useful.
  • As I believe I've said before, I think a more productive shift would be to have more issue tracking and bug fixing take place via Trac
    Agreed. We're currently evaluating an alternative ticketing system, Redmine, which appears to be pretty close to Trac but is possibly a lot faster. Trac's slowness (at least with our configuration) has been the main issue that's preventing us from using it more (and instead just dumping things quickly onto our own lists)—and I imagine the same goes for other contributors.

    As for the above, it's not lack of awareness of user priorities that prevents things from getting done—it's lack of developer time, and our having other priorities for a variety of reasons (including, most notably, grant requirements) that have nothing to do with how many users vote for a feature.
  • Thanks mark for this. I do understand that programmers time is scarce and that there are good reasons why some issues are not adressed or delayed. BUT: A lot of time is also wasted by having discussions in the forums on issues that are not adressed, and as far as I can judge, could easily be adressed. Probably even more time is wasted by explaining why some issues are not adressed. This also applies to users. Countless hours are spent with asking for features and nobody knows whether this is even heard. I often wonder whether I should add a "+1" to a feature request, or whether this is not simply wasted time because I have no idea whether anybody listens.
    If there were a system that would indicate to me if my request is a) heard and b) shared by others and c) taken care of (or, for good reasons not) that would help a lot.
    noksagt's example of mendeley's feature of going open source is unfair. It is very obvious why they do not adress it, and noksagt just play's the obvious "bad commercial mendeley" card here. Apart from this issue, as far as I can judge, mendeley's system works perfectly fine and makes sense.
  • Exclusive reliance on a voting forum as the sole user input/feedback mechanism would be inappropriate for Zotero. There is a lot of fruitful discussion of use cases on the forums, ranging from the universal to the obscure, and we wouldn't want to lose that; the years (!) of accumulated feedback on the forum threads have been an invaluable source of information for CSL and processor development.

    That said, I think there is a place in the feedback ecosystem for a voting forum. Sometimes we as users just want to chip in our two bits' worth and move on, and it serves that purpose well. A voting forum shouldn't drive development, for sure, and the peripheral role of such a tool in forward planning could and should be made clear and explicit. But don't think the value of such a forum derives from any sense that users somehow control the development process. People who participate in digg-like systems are communicating with one another, as much as with dev central. To that extent, it can release some of the pressure to be recognized, and that can save time for everyone in the long run.

    One possible use for a voting forum might be to float ideas for third-party plugins. It's easy to come up with armchair visions for such things, but the necessary attention (for development, or for writing the grant applications necessary for development) can be galvanized by evidence of widespread demand.

    Anyway, mark has set up the account, let's see how it goes. Perhaps there is a positive stance to be found toward such an enterprise that lies somewhere between benevolent dictatorship and the democracy of the Borg. :)
  • edited June 5, 2010
    Dan, on:
    As for the above, it's not lack of awareness of user priorities that prevents things from getting done—it's lack of developer time, and our having other priorities for a variety of reasons (including, most notably, grant requirements) that have nothing to do with how many users vote for a feature.
    What do you think about ajlyon's point that ...
    ... the more problematic disconnect, I think, is between zotero-dev and Trac, and between patch submission and patch landing.
    ...? Is there something that can be done to address this?
  • I think this is a great idea - thanks mark!

    Agree with fbennett that we should cultivate a more diverse set of information tools around the zotero project. Forums are good for some things but here far too broadly used, for purposes where other tools would work better.

    Also agree that there are big issues with trac. That's a key part of the problkem here. (Dan, see follow-up post on Redmine.)

    Finally, I don't think you can talk about these questions without talking about the definition of zotero's core mission, and medium-to-long-term project planning. To what degree are users involved, and how? Project planning and management is integral because, having a mission, that's how you "make it so."

    The "open" part of "open source" is flexibly vague. Open source ... information ... planning ... process? Every project is different, and the range runs from individual fiefdoms to large-scale participatory chaos. Every project needs to work out its definition.

    Some background about me - now I'm working on a PhD in architectural history but I worked for years in software (no formal education though). I've defined and set up development infrastructures numerous times. I'm a big partisan of open-source software, and that has to do with how I think about human interaction, at all scales, and even the nature of knowledge, as much as more pragmatic aspects.
  • Dan, about Redmine.

    I'm currently using Redmine for my (bill-paying) job. I love it. It's the perfect balance of features you want without bloat. Running about 9 months with zero problems. A few admin and UI quirks but nothing major. I set it up primarily for my own use but colleagues liked it so much they've defined their own projects and are using it - actually more for project planning and task management, not specifically software development.

    Contact me offline (see my profile) if you have questions or would like to hear more (I can give you an account on my system if you like). I myself installed and configured it and the database back-end. I'd be happy to help you guys out if I can.
  • to follow on bdarcus and ajlyon and the "disconnect ... between zotero-dev and Trac" - two specific comments ... I might not be making any friends here :-0

    * I remember the first time I looked at zotero-dev. I said, "this is a joke, obviously not where the real communication is taking place."

    * The norm for open-source projects is that anybody can sign up and use the bug database.
  • edited June 5, 2010
    Thanks all for your comments. To be clear, I agree with noksagt that "Developer priority is (and both should and must be) different than user priorities." The problem lies in the followup question: "Can one not already accomplish a similar thing by glancing at popular forum threads?" Looking at how this forum is presently organized, the answer is a qualified no. There is no straightforward way to find out what popular forum threads are. Even if you find one, there is no way to find out what the developer's stance is and whether it has been noticed (except by knowing the user accounts of the devs and spotting rare links to Trac). There is no feeling that it means anything to pile on with a +1 on such a thread.

    I do agree with noksagt and others, including Dan Stillman, that perhaps this is not the greatest hurdle in Zotero's development cycle. Yet it is a palpable problem that quite a number of users have called attention to over the years. If Dan says, "As for the above, it's not lack of awareness of user priorities that prevents things from getting done", we are ready to believe him. Dan is taking the developers' point of view; fair enough. But from the users' point of view, there is an opacity-of-mind problem that can be quite annoying — a disconnect between what the devs are aware of and how the users can get to know what the devs are aware of. An ideas forum (where the status of ideas can be simply and clearly marked as "planned", "under review", "declined", with a brief explanation) helps solve this problem and in the end saves lots of valuable time.

    I appreciate fbennet's insightful comments on the value of an ideas forum.
    "...don't think the value of such a forum derives from any sense that users somehow control the development process. People who participate in digg-like systems are communicating with one another, as much as with dev central. To that extent, it can release some of the pressure to be recognized, and that can save time for everyone in the long run."
    Also, his point about the use of such a voting forum to developers of third party plugins is an important one:
    It's easy to come up with armchair visions for such things, but the necessary attention (for development, or for writing the grant applications necessary for development) can be galvanized by evidence of widespread demand.
    In short, I'm glad some of you agree with the possible usefulness of an ideas forum like this. We'll have to see whether it flies or not. I'm waiting a bit to see if there's more input.
  • edited June 5, 2010
    noksagt's example of mendeley's feature of going open source is unfair. It is very obvious why they do not adress it, and noksagt just play's the obvious "bad commercial mendeley" card here.
    So why'd it take them a year to comment in that thread, wasting the users' time? Why is it still not marked "can't/won't fixed?" I stand by this example to highlight the fact that tools do not solve communication issues. Developers can fail to comment in a brainstorm voting platform just as much as they can fail to comment in a forum just as much as they can fail to fully use an issue tracker.

    But I'd say ANY other issue request there shows my point too. Their number one feature request is, similar to a popular Zotero request, duplicate detection and handling. That thread is also over a year old & has been marked "started." But what does that mean? When will it be implemented? What works in their (private) development builds right now? That is one of the threads that is most commented on by the Mendeley developers, but it still doesn't provide anymore information than we have about Zotero's duplicate detection (which is: there is code that can be enabled by a hidden preference to find dups already, there isn't currently heavy development on that feature, but there are plans to improve this).

    A well-used issue tracker seems far superior to handle the kinds of problems you say you want solved. If there are other issues (of the kind that Frank Bennett notes) that this would address, then fine. But let's not fool ourselves!
  • I guess it's worth asking what, concretely, an example like this offers that a traditional issue tracker does not. I guess I'm tending to lean towards noksagt's point on further reflection that it's "not much." Yes, it can give a way to gauge relative user interest in particular features. But that's a fairly small part of the issue here. The more fundamental questions are about transparent priorities going forward, about how work is organized, about how and how quickly patches get reviewed and committed (which is another way of saying how control of the code base is organized), etc.
  • I remember the first time I looked at zotero-dev. I said, "this is a joke, obviously not where the real communication is taking place."
    That's because most communication is taking place in the forums, which are accessible to a much wider audience. In my view that's absolutely a good thing. zotero-dev is meant for technical discussions and, frankly, there aren't that many of those outside of translator development. If there were more people contributing actual code, zotero-dev would be more active.
    The norm for open-source projects is that anybody can sign up and use the bug database.
    We've addressed this many times. 1) We don't have a triage team and 2) the forums are a far more accessible place. That doesn't mean we shouldn't use the ticketing system more so that it's clearer where actual issues stand—which is what we're hoping to address with Redmine—but, for Zotero, I don't think using a ticketing system for bug reports (which are almost always actually support requests) or feature requests (which are almost always duplicates) would help developers or users.

    Most people in this thread have ticket-creation privileges in Trac already. I suspect they don't create tickets currently for the same reason we don't—i.e., it takes forever.
  • What do you think about ajlyon's point that ...

    "... the more problematic disconnect, I think, is between zotero-dev and Trac, and between patch submission and patch landing."


    ...? Is there something that can be done to address this?
    The majority of the patches that have been waiting to land are translator patches. I try to get to them as I can (and I try to get to the high-priority ones immediately), but this isn't my department. We're simply without a dedicated developer to deal with translators at the moment. But, as I've noted, we're working on a better system for managing this process. (Trac isn't ideal for this.) I don't know the ETA for the new system.

    In the meantime, though, I'm happy to give translator commit access to a few more of the usual contributors. We still need to push them to clients ourselves, but that's a bit easier once they're already committed.

    For other patches, I would say it's not really any different from how you get a patch to land in most projects: try to get other people to test it, be persistent, and be patient.
  • In the meantime, though, I'm happy to give translator commit access to a few more of the usual contributors. We still need to push them to clients ourselves, but that's a bit easier once they're already committed.
    I would be willing to do some of this in the meantime, although I am certainly looking forward to a new system for translator submission and testing.
  • I think giving ajlyon commit access for translators would be an obvious step at this point - he's atm the main person fixing them.
  • Developers can fail to comment in a brainstorm voting platform just as much as they can fail to comment in a forum just as much as they can fail to fully use an issue tracker.
    Sure. As I mentioned, though, it's not necessarily the devs that should comment. If there was a source of up-to-date information (like a well-tended Trac), users could enlighten painful pile-on threads too. Some of you say "so improve Trac and the problem's gone". Well in fact users have been trying to disseminate whatever sparse information there is on Trac in various threads — we can just look at the loads of duplicate threads and the uncertainty in even the most popular pile-on threads to see how well that has worked so far.

    This is not just about Trac. The problem is that these forums are a particularly bad place, interface-wise, to solve the disconnect I've observed above, because (1) there are no clear ways for users to quickly find what the devs have said about some topic; and as an immediate consequence, (2) many users just start a new thread if they can't find this information soon enough, wasting valuable time of many people including devs. The resulting duplicate threads make problem 1 only worse, reinforcing problem 2.

    My perception of this is obviously different than the developers' perspective. I don't think Zotero is doing as good a job in communicating to the userbase as it could. Although the devs are confident they got the main priorities covered, the opacity-of-mind problem remains and this leads to a lot of time wasted in the forums. My intent was to suggest an improved interface that would help take away some of these problems. If this is not a priority right now, so be it. I look forward to the improvements that a better alternative for Trac will bring. I remain skeptical that the forums will become a much more enlightening place through it though.
  • hmm - actually if we use that type of forum to keep track and help users to keep track of what devs say or have said on feature requests that could make sense -

    would that still work if the core developers really don't go there? - I'm happy to check out pretty frequently and together with the dozen or so other power users here we probably know everything that a developer has ever written on a topic.
    But how would requests feed back to developers?

    Also, I think there is still the problem of forking - many feature requests are, as Dan points out, not feature requests but help requests - how should those be dealt with?

    I really like the fact that it essentially forces you to search before you can put in a feature request, I've often wished the forum did that.
    Another advantage is that this could (and probably should) be moderated much more aggressively, especially in terms of merging threads/requests.
    The voting would, in that case, be more useful in pushing common requests to the tops and less important as a signal to developers - I think Dan has made it clear that their future priorities are not going to be determined mostly by user votes.
  • edited June 7, 2010
    I really like the fact that it essentially forces you to search before you can put in a feature request, I've often wished the forum did that.
    Faolan actually implemented that for these forums months ago—we just never turned it on. We can, easily.

    The main thing that's not clear to me is how another system (which, no, I probably wouldn't participate in) wouldn't become just as messy as the forums. It might seem cleaner initially when there's simply less content, but that would change if it were actually used. We're pretty aggressive about merging and closing threads here.

    And the inability to properly deal with the inevitable support requests seems like a pretty serious problem to me, since one of the main points of these forums is that the developers can step in if need be.
    there are no clear ways for users to quickly find what the devs have said about some topic
    Given that we wouldn't be saying anything at all on a separate system, I'm not sure how well it would help with that, unless other moderators were willing to do a whole lot of quoting or linking. But then you're mostly just fragmenting discussion into a forum where developers participate and one where they don't.
  • ok, I agree that there is a danger of getting lost in too many solutions, but it does not solve the problem. One solution then could be a better organised forum. More categories, better search results. Far more aggressive closing of discussions. Also, would it be possible to sort the discussions and search results according to "most viewed" and "most recent" and "most contributions to discussion"? That would already help. Because now, if I do not have/know the exact term I do not find the relevant discussions. And as can be seen for the endless list of repeating requests, people DO NOT find existing discussions.
    Also it would help, if somewhere on the entrance to the forums a note says: "if you want to push the development of a feature, simply add an entry with "+1"". Then, sorting according to "most contributions" would return the most wanted features. That would be a low tech alternative to Marks initiative.
    Further, but probably this is technically not possible, a summary of the discussion on top of a discussion would help. It is quite annoying to find a discussion and then to discover somewhere in contribution 22 that a it is fix or a fix is planned or it can't be fixed.
  • And as can be seen for the endless list of repeating requests, people DO NOT find existing discussions.
    But people do the opposite—post in irrelevant threads—just as a frequently as they fail to find relevant threads, and frankly posting in irrelevant threads is far more disruptive. Enabling auto-search will help with duplicates but will probably increase the frequency of irrelevant posts (which is one reason we haven't yet done it). I don't think we should overestimate the extent to which these can both be solved.
    Also it would help, if somewhere on the entrance to the forums a note says: "if you want to push the development of a feature, simply add an entry with "+1"".
    I don't think this is really a practice we need to encourage. It's mostly just distracting. We don't develop based on vote counts—we develop based on what we think will improve the software and what we have time to implement. It's usually clear when there's demand for a feature because it leads to actual thoughtful discussion.
    Further, but probably this is technically not possible, a summary of the discussion on top of a discussion would help. It is quite annoying to find a discussion and then to discover somewhere in contribution 22 that a it is fix or a fix is planned or it can't be fixed.
    I think this would be great and go a long way towards addressing some of the concerns here. This could be custom built, but maybe there's a Vanilla plugin to highlight particular posts in threads or add summaries to threads...

    The other thing we've considered adding is a feature that gives some indication of the status of people posting, since being able to quickly tell who's a developer or power-contributor would probably be helpful to newcomers.
  • edited June 8, 2010
    I like migugg's proposals too. Marking of dev/power-user posts would not be enough as long as these posts are buried deep inside threads — what really would help is a summary of the status of the issue (including links to trac tickets) upfront. I'm not aware of any Vanilla plugin that provides for 'thread summaries', but that would definitely go a long way towards making the forums a more helpful place.

    Dan, pile-on threads may be 'distracting' from a developer's point of view, but I do think that you can see why from a user's point of view it can be important to have a way to say "this is my issue too!" Basically, you're never going to avoid it as long as Zotero has users and forums, so why not try and streamline the process? As migugg and various others have pointed out, users would not just vote to drive development; it's also, and perhaps more importantly, to 'release some of the pressure to be recognized' (fbennett) and to get common requests pushed to the top where users that are likely to be looking for them can find them easily.
  • Quoting noksagt: "information than we have about Zotero's duplicate detection (which is: there is code that can be enabled by a hidden preference to find dups already, there isn't currently heavy development on that feature, but there are plans to improve this)."

    Sorry to warp the thread, but do I read you correctly that there is a hidden preference to enable (some alpha version of) duplicate detection? Could you please give me a link to more information on that?
  • Jon - see my comment here:
    https://zotero.uservoice.com/forums/60719-general/suggestions/807201-duplicate-detection?ref=comments
  • First: Zotero is wonderful. Thank you !
    I am fully committed, which also means fully dependent on the functioning of Zotero.
    If it went away, I'd be dead... So please keep up the great work :) !

    That said, from my personal user experience, the forums leave a lot to be desired.
    It can take hours to read the multiple and highly repetitive tracks that usually exist for popular topics. The forums are far too fractured - do not know where to go. If there is some useful information at all, it is often buried somewhere in the middle of a thread. Way too time consuming, and this time is to be multiplied with the number of users reading... No wonder many users avoid the search and open yet another thread.
    Something like a summary, and more guidance/pruning would be highly welcome.

    Feature requests are awkward. A search will lead to several threads, which may be on topic or not, have to be read completely, may have been answered or not, and the status and prospects often remain unclear.
    And it is completely non-obvious what requests are popular to what extend.

    Final aspect: Of course the developer point of view has to be different than that of the user. But somehow I seem to read here that the developers will go in the direction that they think the greatest and/or that is best for satisfying grant requirements, largely ignoring user opinion. Huh ??? Zotero is great, because it is so useful - to the user, of course... - and wouldn't that also the ultimate goal of a grant ?
    One point in case is the lack of duplicate detection. Four years old in the forums, a "show stopper" to several people who commented, and by far the most popular request on Mendeley. A very simple detection on import would be enough for many people, as has been commented often - and still there is nothing. Why ?
  • edited October 17, 2010
    But somehow I seem to read here that the developers will go in the direction that they think the greatest and/or that is best for satisfying grant requirements, largely ignoring user opinion.
    The fact is, grants do currently primarily fund Zotero development. But the applications that secured those grants were written by historians who are also Zotero users, and the purpose of those grants are pretty practical (not oriented towards "research").

    Almost inevitably, my sense is that these issues come down limited labor, and a long list of competing priorities. Priorities attached to funding don't necessarily conflict with "user" priorities.

    If you have a comment on the duplicates issue, post it on the relevant thread.
Sign In or Register to comment.