Zotero ever going to offer a server version/Group Sync on OTHER SERVERS?

Hello everybody!

To make it quick:
- We = Research Institute, 1200 people, 600+ of them scientists.
- We love Zotero but can't properly use it
- We must not and actually we don't want to upload many of our files to outsourced servers, not gonna happen
- we DESPERATELY want to be able to use the group-functionality in Zotero INCLUDING FILES
- I know many more institutions like mine, whose scientists feel the same, around the world
- We'd pay big bucks to have this, licensed or one time purchase. 10.000€+? no problem, give it to us, we have the money.
- There is NO alternative software and the IT departments burn away a ton of money on Endnote, that is less elegant, less flexible and hence most importantly: that most of us simply don't want to use!

@members
What are your thoughts?
I know there is a fiddle-solution out there but that is not what we are interested in. Wouldn't at least some of you also be interested in a server edition, even for significant money?


@admins,
please don't close this, it's 100% serious and I think we should have an open discussion on this issue.
  • edited January 18, 2012
    The Zotero server code is open source. There is no need to license anything.
    There are first reports on successful local implementations. If you have 10k, I'd just find someone who gets this running for you for 5k and then spend the next 5k on maintenance. If you write to Sean Takats, the Zotero project director, he may even know people willing and able to provide that type of service.

    There is someone (i.e. a third party - I don't remember the site, if someone does feel free to post the link) who is working on a Zotero fork with easier server implementation, but the basic problem is that setting up a sync server and maintaining it is a lot of work. There is no one-time package that would just "work" - the Zotero server doesn't just "work" either.
  • edited January 18, 2012
    thx adamsmith.

    If it all is open source, why can't "a server version" be downloaded?
    There must be a reason that the zotero-team doesn't work on this and compile and offer anything of the sort then or not?
    Am I missing something obvious here?

    edit:
    yes, I am aware of this not being a simply click and install thing. I'm no IT-professional but I'm no noob either.
    At least the solution I found seemed pretty knocked up and our IT won't even look at something like that... something posted on some forum by a random user, that just won't fly (too bad, but it is how it is)
    No server setup has to be enduser-friendly, we both no most aren't.
  • here are some instructions including a link to the downloadable server version.
    http://www.zotero.org/support/dev/dataserver_setup
    note the warning at the top. Also, I don't believe people have gotten file syncing to work in that set-up (see the "not working" notes at the bottom).

    As you can see, installing a complex server infrastructure is just not as trivial as installing a program on a computer, and again, you need someone to care for a server like that - e.g. the Zotero team has committed a dozen small commits, mostly specific issue fixes, this year alone.
    The Zotero team simply wouldn't be able to do any developing if it started providing full tech support for that type of installation.
  • Exactly, that was what I remembered. Thanks.
    Leaving the obvious (intalling & maintaining) aside. Something that immediately caught my eye:

    Step3: Host Entries

    Why is this even necessary? I see no reason than the design-decision of the client-version not being considered at all for local server support.
    Why not just a local IP + port in the client-version sync-preference-pane and be done with it?

    This is an example of what I mean by "support". I know one could fiddle with the host table at each client computer, yes, but that is not a clean solution then, is it?
  • @andybrandy: The server source is online and up to date:

    https://github.com/zotero/dataserver

    The code is clearly stable (the GitHub code is running on zotero.org), but someone would need to dig into it and learn how it works before deploying it locally. If your IT staff are reluctant to take a serious look at the server code with a view to possible deployment, that would probably be the first hurdle you'd need to clear: As it would be core infrastructure for your organization, that would surely be a threshold requirement anyway, even if a plug-and-play package of the code were available.
  • Step3: Host Entries
    Well, yes. For an institution-wide deployment, you would presumably customize the client itself to access your local dataserver at its proper address, and then have your staff install the modded client. Sync would then just work for them, assuming the dataserver is running happily.
  • edited January 18, 2012
    thx fbennet

    I was just looking at that one, adamsmith's link obviously also points there. At first glimpse I can tell that quite likely I'd be able to set it up rather quickly myself BUT, as I already wrote prior to your post.

    A good example of a no-go for many IT-departments is the necessity for something simple like adding a host to all clients. Sure, can be done and applied network wide with ease but.... they'll just say no, I'm 99% certain but will check with them. I'll simply go and try to get server and some clients running on VMs on my own machine and show it to them.
    ... but again... adding a host like this... I see the IT-head happily dismissing the proposal with a wide grin and great satisfaction.... just the way it is.

    ==================================================
    edit in response to your latest post:
    That's the thing. That's exactly what I meant, maybe I just phrased it badly. Using Zotero with a local server is simply not "provided" for. Modifying the client version would cause update-headaches and overhead all the time. Zotero releases a new version... somebody would have to get back in there and modify the new version AGAIN. If the official version could just feature a setting for local server support, the need to modify it would disappear for ANYONE.
    .... and obviously that was just ONE example on the issue.

    Did I get my point across better this? ;)
  • I see the IT-head happily dismissing the proposal with a wide grin and great satisfaction
    I certainly understand your frustration. Zotero hasn't been taken by my own institution in a serious way (yet), although I've built both the citation processor and a variant of the client with local requirements in mind. I think it's just risk aversion at work. Admins like having an upstream party to whom they can pass along issues when they arise. Running a data container locally puts them on the spot with ultimate responsibility for keeping the show afloat, and that's a position to be avoided in corporate environments. The market for systems that work will eventually bring them around.
  • edited January 18, 2012
    The technical aspect of this discussion should move to zotero-dev.

    The general issue is this: There doesn't seem to be much interest from Zotero in spending a lot of time and resources on this. This may be partly because of funding considerations, it's also because of the brand - they want to be able to guarantee the same user experience to everyone using Zotero, and finally my sense is that they and their funders also place value on the potential worldwide public-ness of data that's shared on a central server.

    But, the beauty of open source is that, if there really are multiple institutions willing and able to shell out the type of money you suggest, it really shouldn't be that hard to fix this up locally.

    edit: If you search zotero-dev, Dan has been quite explicit in saying that Zotero will not include a GUI option for local servers.
  • edited January 18, 2012
    then why would zotero not tackle these easy to implement things and "sweeten" the whole thing... preparing the client would be the most obvious one. Simpl on the dev side, but a huge appeal towards the ones having to implement it. I'm not getting it!

    Could I or some colleagues familiarize myself with the project and then be THE GUY enabling Zotero at our institute? Quite possibly, yes. But that is not my job, I'm a physicist and IT is my hobby. I just won't spend that amount of time on a task like it.

    Don't get me wrong, but that should be considered by the team developing Zotero. They're doing a great job, yes, but why not sweeten the whole thing a bit?
  • edited January 18, 2012
    Zotero releases a new version... somebody would have to get back in there and modify the new version AGAIN.
    I do this regularly for the multilingual client version that I maintain, which has close to 60,000 lines of my own code in the patch. It costs me maybe 20-30 minutes per week to keep the versions aligned.

    To divert the server, all you would need to do is fork the client code on GitHub and modify a couple of lines of code. The burden would be trivial, so long as someone was willing to spend a couple of minutes on it (literally) from time to time. The dataserver itself would be a different story, of course.
  • edited January 18, 2012
    @adamsmith
    Sad to hear that, but ok, gotta live with it then. If Zotero officially wants everything on public servers... that says something about the state of mind behind the project (no positive or negative pun intended). In this case, Zotero will die a certain death in the working community then, because sooner or later somebody will fill this position commercially. Admins won't care, scientists will have to suffer for it though for not having anything useful for several more years. Can't change that then. :(

    ..wrong... anything can be changed... if the interest is large enough and that is partly why I started this thread. Technicalities aside (yes, we can engage in those somewhere else) - If people want it, why not provide it?


    @fbennett
    yes, I'm getting you, BUT. I already administer and maintain some bits and pieces, that in my view should be taken care of other people being employed for the job. Everybody has his limits and mine is reached. I want to do my PhD, save the planet (literally related with projects like these) and not fiddle with code needed for a "tool" at work now. You do it in 30 mins/week, anybody new to it would definitely need quite a bit more time to penetrade the code. I'd do the client, yes ok, but client, then server, then keeping all in sync, testing... no, don't see me or any capable colleage deal with all of that or the overhead generated by coordinating all of it if shared between many.


    other question:
    Why don't you provide your modified version openly to whomever it might concern? :P

    PS:
    If I was back in my student-days... I would totally do this, server + client. A little polish in this direction, that's all it would need to take off - imho. I'm not the only one thinking this. The software industry offers nothing, here's your chance ;)

    PPS:
    I understand and support the idea of having things openly available. But the power of Zotero is it's usability too, and that gets destroyed if people have to have two workflows, one for documents they can sync outwards and one for the ones they can't. We have tons of especially theses that we will NEVER distribute around, nobody will do that, it's just the nature of the task.
  • If you search zotero-dev, Dan has been quite explicit in saying that Zotero will not include a GUI option for local servers.
    http://groups.google.com/group/zotero-dev/msg/7c550be3589d47f0
  • edited January 18, 2012
    Thanks Dan,

    I understand and agree with many of your points made in that link, but this I don't get at all:

    ============================================================
    On 11/16/11 6:54 AM, Robin Paulson wrote:
    > to be honest, i don't see any difference between allowing the user to
    > change the sync server, and change the storage server, the latter
    > already being exposed for anyone to change.

    They're not comparable. Zotero syncs files with our own storage service
    or with a WebDAV server. We run the former, and the latter uses a very
    basic subset of a fairly simple spec that mostly hasn't changed since
    1999. If a WebDAV server is broken, file syncing just won't work. If
    there's a problem with a data server, it has the potential to destroy
    someone's database or break their client in subtle or major ways.
    ============================================================
    Again, maybe I've just asked in a very well encrypted manner:

    What counts for me and most research-users is the fileserver, not the sync-server! What matters is the actual DATA, the files! And there I just don't understand Zotero atm.
    Why can I have my personal files be synced on any WebDAV share but NOT the group-FILES. It is not about the Database. If an item is called "secret document XYZ, plan to take over the world by andybrandy" and that metadata is on the zotero.org sync-server so be it! If the zotero sync-server get's hacked all they'll see is the metadata....that I have a great plan (title), who is involved (author) but no details :D
    Honestly, what I and most others in the field can't accept is that the attached PDF/*.* is uploaded elsewhere, not sharing files introduces a huge overhead in sending them around via mail internally though. All other things aside, Zotero is great!

    That's the whole point of this topic. If you guys go and separate the sync server from the file-storage-location for groups as well - all my objections rest. You said it yourself:

    "If a WebDAV server is broken, file syncing just won't work"

    that's exactly what I would have said. So, where's the problem? Then why not offer an alternate storage location for groups just as it's done for the personal library now? I'm no IT newbie but technically I don't get where the problem could lie.
  • edited January 18, 2012
    Everybody has his limits and mine is reached.
    Absolutely. If I were in your position, I wouldn't consider building core infrastructure to be my responsibility. Lobbying your IT staff is about all you can do.
    Why don't you provide your modified version openly to whomever it might concern?
    I do just that.
  • Ah... nice... I'll have a close look, first glimplse looks great, thanks.

    ....and what's your notion on why it wouldn't be possible to separate group storage from the sync DB just like it's done with the personal library? As I said, that'd enable "official" Zotero sync utilization permit for many researchers.
  • I haven't looked at or touched that part of the code, so I wouldn't know.
  • What counts for me and most research-users is the fileserver, not the sync-server!
    while you're obviously the authority on the former, you're likely wrong on the latter. Most people raising this topic in the past have voiced concern more about data than about files.
  • What counts for me and most research-users is the fileserver, not the sync-server!
    I suspect that wasn't clear to anyone responding to you so far. (For many institutions the data server is the issue, for things like notes.) If you really are just asking for WebDAV support for groups, most of the above doesn't apply—you're essentially just asking for a new Zotero client feature (and maybe a few server changes).

    In any case, WebDAV support for groups has been requested before. I'll just quote what I said in the past:
    The features that Zotero File Storage provides that WebDAV doesn't (web-based access to files, group syncing with built-in permissions) are due to significant technical challenges with implementing those over third-party WebDAV servers, and any solutions would be awkward at best. That doesn't mean they won't happen, but it does mean they won't happen first. And as Mark notes, they could certainly be implemented by an institution that was willing to contribute developer time.
    If you're willing to pay for this feature—which it sounds like you might be—you could probably make it happen, whether the work was done by Zotero or an outside developer.

    As I say, though, there are significant technical issues that would have to be resolved, and even in the best case the experience wouldn't match ZFS. But that may be acceptable for institutions interested in this.
  • edited January 18, 2012
    yes, granted, didn't think about the notes features as none of us has ever used them.
    It is certainly not ideal to have the metadata out there, but it would do the trick. No internal issues over puclication-licensing i.e., the head-organizations library department is very strict on not distributing papers outside the intranet, i.e.

    So this is not a "planned feature"?
    The reason to why the user "experience" would take a hit though eludes me. It doesn't when syncing multiple computers with the same personal account and having the same WebDAV storage location selected for all of them, why would it for a group?

    BTW:
    Aren't notes just files as well? Ah... guess not.. they're editable in the client... ok.
  • edited January 18, 2012
    it sounds like it's a planned feature, with high costs and low-ish priority. There are still plenty of open tickets from 2006... (some of them for important stuff, like hierarchical item types, too).

    If this is going to happen soon, it'll happen because someone else codes or funds it.
    The reason to why the user "experience" would take a hit though eludes me.
    As I understand Dan they're related to technical limitations of WebDav and the need to work around them. If someone is serious about attacking this I'm sure she'd find devs willing to go into more detail on those limitations over at zotero-dev.
  • The reason to why the user "experience" would take a hit though eludes me.
    With ZFS, access is taken care of automatically. Once you join the group, Zotero syncs the files, and you can also access the attachments (currently single-file, soon all) via the web.

    With WebDAV, every user invited to the group would need to enter WebDAV info separately after joining each group, which would be much more confusing, and there would be no web-based access to attachment files, since WebDAV attachments (and all multi-file attachments) are compressed before uploading. (The first issue could be ameliorated somewhat by having the WebDAV URL be configured by the group owner and synced to all users, but credentials obviously still wouldn't be synced.) These aren't dealbreakers, obviously, but they'd make for a worse experience than with ZFS. There might be other issues that I'm not thinking of.
Sign In or Register to comment.