phpZoteroWebDAV 2.0 - php based WebDAV server and library and attachment viewer

krueschan

Hi all,

I am very pleased to announce phpZoteroWebDAV 2.0 which is based on the previous version of phpZoteroWebDAV, discussed in this thread. In a nutshell, both the old and the new phpZoteroWebDAV versions allow users to use any php enabled webspace as their target for syncing of the attachments of their zotero library (for details see the other thread).

In addition, the new version uses the zotero server API to display the user's library on their own website (as opposed to on zotero.org but with a very similar library view - in fact it uses the same stylesheets that are used on zotero.org to format the library view) and allows them to view attachments that are synced to the webspace.

This is mainly intended to folk who, for whatever reason (e.g. cost considerations or the implications of zotero.org's servers being hosted in the USA, or simply preferring to have maximal control over their own data) cannot use zotero's own sync storage for their attachment syncing.

Note that the library items' metadata is still synced to zotero.org - only the attachments are synced to the webspace on which you install phpZoteroWebDAV. Note also that the script does not (yet?) support collections or subcollections - it displays the user's library in its entirety (the reason is simply that I don't personally need this feature, but please indicate in the comments if you do). Also, there is no support for Group Collections; as group collections can only be synced to the zotero file storage.

Download and install and configuration instructions can be found on my blog. As always, please share experiences and problems you have installing and using the program. Answering to requests I received, there is now also a live demonstration installation of the library and attachment viewer on my site.

krueschan

just to add that I have put the code on github as well: https://github.com/krueschan/phpZoteroWebDAV

mark

Sounds beautiful. Look forward to telling our IT dept about this.

Can it handle multiple user accounts? And if so, what do the user management tools look like, and could they be scripted/API-d/be made database-accessible?

krueschan

Yes and no. (no, in that this is not a feature that is coded into the script, and YES, in there are definitely easy ways to set this up). If I were to set up multiple user accounts I would do this by way of subdirectories, e.g.

/public_html/zotero/user1/
/public_html/zotero/user2/
/public_html/zotero/user3/

with each of the sub directories containing a complete copy of the script's source code (which is tiny) and the user's individual settings.php file (which must contain the user's API key and zotero user ID). This would certainly be scriptable and database-able (via scripting) as this would be simple file management tasks, including the generation of .htaccess/.htpasswd files and settings.php. Users would then access their individual libraries via http://example.com/zotero/user1 etc
[Note also, as explained elsewhere, that the use of the viewer component of my program does NOT require the use of the WebDAV server component to be used as well - your IT people might prefer using a different WebDAV server for syncing]

Alternatively, it would also be possible with only small changes to the script to have all users in the same directory (e.g. by passing a username to the script on each call). However since the script does not have any security features of its own and relies on users (or admins) to protect their data themselves (e.g. using .htaccess/.htpasswd files) this would mean that users in a multi-user setup would have access to each others data if they knew each other's username (I realise this might not be a problem or even desired in certain cases).

There should be no problem in the actual sync storage with multiple users' attachment synced to the same folder (as file names are names using zotero's unique itemkeys) but in case a user leaves the organisation/reasearch group/etc it would be not straightforward (albeit totally possible) to remove that user's attachments from the shared storage.

Thus, in conclusion and in my own view, the first method is definitely the one to be preferred.

marksample

Thanks for this! I'm one of the users who praised your original script, and I've been using it since 2008. Version 2.0 was a seamless update for me, and in a matter of minutes I had it up and running. It's working great so far, and I'm thrilled to be able to access my attachments from any browser.

evosens

Hi,

First, I would like to thank you for sharing this script, very useful.
I've installed it, and sync works nice.
The only problem I've encountered is with index.php. It does not locally display the library , whereas it is visible on zotero.org account.
Any advices?

frederic

My settings.php:
<?php
// Zotero API settings. Obtain from zotero.org after logging in by going to https://www.zotero.org/settings/keys
// the script only needs read access to your personal library, so you can disable everything else.
$API_key = 'U8zLqHKldbnN98im7wDu0uKO'; // obtain from https://www.zotero.org/settings/keys [no default]
$user_ID = '821xxx'; // obtain from https://www.zotero.org/settings/keys [no default]
$fetchlimit = 50; // [default=20]; how many items to fetch with each API call (maximum allowed by zotero is 99). use your value for items per pag
e below ($def_ipp) unless there are issues, e.g. server timeouts


// I know, those are not API settings, but I need to know anyway...
$data_dir = 'data/zotero'; // [default="data/zotero"]; directory with sync data, either relative to the script's location or absolute path (on
the same server though!). (NB: due to the way zotero sync works, this path ends most definetely in "/zotero)
$cache_dir = 'cache'; // [default="cache"]; cache directory, relative or absolute. must be writable by the script. will attempt to create
if not found
$cache_age = 0; // [default=0]; in minutes. files will not be re-read from the sync storage to the file cache if they have already
been grabbed within this time period. Also, cache folders that are older than this will be purged from the cache. set to 0 if you want to empty cac
he after each use (NB: if the attachment in question is a web snapshot and can be accessed by web the attachment will not be purged until the next
use of the script)
$cache_base_URL = ""; // [default=""]; if the cache directory is NOT a subdirectory of the directory which contains the script AND it (th
e cache) is web accessible, provide its URL here for use to display web snapshots. if the cache directory is not a subdirectory of the script's own
directory and nothing it provided here, display of web snapshots will not work (all other attachment types are unaffected)


// library view defaults
$def_ipp = 20; // [default=20]; items per page
$def_sort = "dateAdded"; // [default="dateAdded"]; The name of the field by which entries are ordered
// valid values= "dateAdded", "title", "creator", "type", "date", "publisher", "publication", "journalAbbreviation", "l
anguage", "dateModified", "accessDate", "libraryCatalog", "callNumber", "rights", "addedBy", "numItems"
$def_sortorder = "desc"; // [default="desc"]; The sorting direction of the field specified in the order parameter, valid values: "desc", "asc"

?>

krueschan

Hi Frederic,
I had a look at your settings.php and all seems in order. However, using your API key I get a “Forbidden” error message back from zotero.org so it seems like you are using the wrong permissions. Have a look at this screenshot and make sure your settings match.
Christian

P.S. Since you posted this API key in the forum, you should consider it compromised and I would very strongly recommend to change it once you have sorted out your problem.

evosens

Christian,

You're right , I forgot to check the "allow library access" in the key settings.

Thanks
Frederic

mark

krueschan, I was thinking -- could perhaps you set up a demo library that allows people to get an impression of what the library view looks like?

krueschan

following mark's suggestion, there is now a small 24 item library with attachments that can be viewed at

http://blog.holz.ca/phpZoteroWebDAV-demo
I tried to have a variety of item and attachment types (there are pdf, xls and web snapshot attachments).

Since phpZoteroWebDAV was primarily written to meet my own needs, there might be things that you would like differently. In this case, feel free to fork the project on git or let me know and I can see if I want to implement your suggestions.

Also, there are some built-in options for changes, for example, the file inc/header.php can be adjusted to include whatever logos/headings/information you want to go above the main library view.

mark

Thank you!

krueschan

you are welcome. Please note, that the demo installation linked to above only serves as a demonstration of the library view part of the script (I removed the WebDAV server part of the program).
This is simply because I didn't feel like inviting the whole world to have full read and write access to (a directory on) my web server. Also, the WebDAV server component of the script is well established and widely tested so it really doesn't need a demo installation (plus, there is really nothing to see there anyway).

krueschan

Another note about the demo installation: attachment viewing for attachments other than web snapshots was broken on the demo installation until yesterday. It is fixed now, so if you had a look at this during the last 5 days and were confused about the dysfunctional attachment view, .... well have another look.

re4med

Strange problem (and maybe it is my ignorance). I want to be able to SYNC only with my server that I added this script to. I have followed the instructions and the script works (tested and new entries were showing on my site). However, it is also synching with the Zotero server as well (which is what I am trying to avoid).

I see that the META data is still synced to Zotero. I do not know what that means. Do I need to turn off anything in the sync preferences after adding the new path to my server?

krueschan

What you are describing sounds like expected behaviour.

"Meta data" refers to the bibliographical information of your items, e.g. the title, authors, abstract, URLs etc. The term "meta data" is used to differentiate from "attachments" which are the pdf files, web snapshots, xls files, Word documents, movies, etc that you might attach to your items.

phpZoteroWebDAV only deals with attachments. Meta data syncing only works with the Zotero server.

However, if you set it all up correctly, you are not syncing attachments to the zotero server, so the data contained in the attachment files is only sent to your own server. Depending on the reasons you have for not wanting to send data to the zotero servers, this (in combination with zotero's privacy settings) might be sufficient to solve the concerns you might have in sending data to zotero.

Alternatively, the zotero server code has been released and I think there are reports of users successfully running own zotero server instances. Doing this is non-trivial but seems to me the only way of having sync without sending any data whatsoever to zotero.org.

Hope this helps.

re4med

Thanks for your most recent answer. I have another question (great script, btw).

If I wanted to set up a SECOND directory on my server for my wife or whoever to use and called that second directory "zotero1" I get an error (The link is not a valid WebDAV url"). Any thoughts or solutions to this?

Thanks!

ajlyon

Yes-- Zotero requires that the files be stored in a directory called "zotero". So you can have /files/zotero and /files-wife/zotero, but not /files/zotero and /files/zotero-wife.

krueschan

what ajlyon said.

also, the fourth post in this very thread deals with this question in more detail.

jdigerness

krueschan, fabulous piece of work. kudos. this is exactly what i've been looking for. it's working great for me outside of my office, but i'm having trouble behind my company's firewall, etc.

i'm serving your scripts and webdav with apache on my ubuntu box at home, and while i'm there, everything works great (using zotero standalone on MacOSX). when i try to do a storage sync server verification at work (on zotero standalone on win7), i keep getting the "You don't have permission to access /zotero/webdav_server.php/zotero/ on the WebDAV server" popup; however, all credentials are entered correctly. the apache error log file reports "Request body read timeout".

i've adjusted apache's mod_reqtimeout module to what i think are sufficiently long timeouts for header and body requests. it takes a little while for the verification to bail (~30 seconds).

i think the hangup is when zotero tries to PUT the "zotero-test-file" . here's an excerpt of the zotero debug output (with the username and server address obfuscated with plus symbols...):

----------------begin debug:------------
(3)(+0004801): Verifying storage

(3)(+0000000): Getting WebDAV password

(3)(+0000001): HTTP OPTIONS for http://++++++:********@++.+++.+.+++/zotero/webdav_server.php/zotero/

(3)(+0001308): Date: Wed, 14 Dec 2011 18:15:02 GMT
Server: Ubuntu
X-Powered-By: PHP/5.3.6-13ubuntu3.2
X-Dav-Powered-By: PHP class: HTTP_WebDAV_Server_Filesystem
MS-Author-Via: DAV
X-WebDAV-Status: 200 OK
DAV: 1
Allow: OPTIONS, PROPFIND, PROPPATCH, MKCOL, GET, PUT, DELETE, COPY, MOVE, HEAD
Vary: Accept-Encoding
Content-Type: text/html
Cache-Control: proxy-revalidate
Content-Length: 20
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Content-Encoding: gzip


(3)(+0000000):

(3)(+0000000): ===>200<===(number)

(3)(+0000000): HTTP PROPFIND <?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
<D:prop>
<D:getcontentlength/>
</D:prop>
</D:propfind> to http://++++++:********@++.+++.+.+++/zotero/webdav_server.php/zotero/

(3)(+0060397): <?xml version="1.0" encoding="utf-8"?>
<D:multistatus xmlns:D="DAV:">
<D:response xmlns:ns0="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/">
<D:href>/zotero/webdav_server.php/zotero/</D:href>
<D:propstat>
<D:prop>
<D:displayname>/ZOTERO/</D:displayname>
<D:creationdate ns0:dt="dateTime.tz">2011-12-12T19:30:04Z</D:creationdate>
<D:getlastmodified ns0:dt="dateTime.rfc1123">Mon, 12 Dec 2011 19:30:04 GMT</D:getlastmodified>
<D:resourcetype><D:collection/></D:resourcetype>
<D:getcontenttype>httpd/unix-directory</D:getcontenttype>
</D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
</D:response>
</D:multistatus>


(3)(+0000000): ===>207<===(number)

(3)(+0000000): HTTP PUT ' ' to http://++++++:********@++.+++.+.+++/zotero/webdav_server.php/zotero/zotero-test-file

(3)(+0000205):

(3)(+0000000): ===>403<===(number)
----------------end debug------------

any thoughts or ideas about this? any help would be greatly appreciated.

many thanks.

krueschan

i am just guessing here, but I have compared your debug output with mine and noticed that your webdav server respond incudes a <D:displayname>/ZOTERO/</D:displayname&gt field which is absent from my (Mac OS) debug output (maybe do a comparison of your own outputs to see if that really is down to using different clients or if your web server would send that information to a Mac OS client as well)

Maybe that has to do with Windows (I don't do Windows myself) somehow, which I believe is known for being a bit funny what lower/upper case of file and directory names is concerned, or with the combination of a Windows and *nix server.

I could imagine that Windows tries to access the .../ZOTERO directory thinking that this is the same as .../zotero and since that directory does not exist on your web server (which is aware of the difference between a 'Z' and a 'z') the server returns a 403 - Forbidden code.

Maybe worth trying is to create a ZOTERO symlink to the zotero directory on your web server...? It's a bit brute-force-ish but could in this case your Linux web server could actually serve the .../ZOTERO directory Windows is asking for and this would still point to the same files as the .../zotero directory that your Mac and Linux installations access.

Keep us posted on your progress.

jdigerness

Hi Christian,

After reading Dan Stillman's last post in this thread, which suggests forcing Zotero to believe that the webdav server verification has taken place, and implementing the suggested workaround, everything is working as it should.

I realize it doesn't address what may be my real issue (still TBD), but hey, it works!

I suspect (with my very neophyte's knowledge of all this), that it still has something to do with Zotero writing a zero-length file in the form of "zotero-test-file" . . . but I'm sure I could be wrong about this.

Thanks, btw, for the rapid response!

krueschan

jdigerness wrote:

I suspect (with my very neophyte's knowledge of all this), that it still has something to do with Zotero writing a zero-length file in the form of "zotero-test-file" . . . but I'm sure I could be wrong about this.

Well, I am not so sure about that - this seems to be normal behaviour (both zotero installations that I have easy access to do this while testing WebDAV server setup) and a Ubuntu server should be perfectly fine with writing a zero length file. I would rather think that it is a Windows issue (especially given that the same server works fine with a non-Windows zotero installation).
Either way, glad that you fixed the issue. Happy syncing.
C

b.c.hamans

Very nice work! I already have setup my own WebDav share, pretty much as soon as the sync option was introduced in Zotero. I have used it happily since. Is there an easy way of setting your scripts up without modifying my WebDav implementation? I guess that would be more or less as in the demo? What components do I have to disable/modify to get to this state?

re4med

Strange problem -- when my sync gets to the EXACT same place each time it hangs and then fails. I have verified that my server is up and running.

Thoughts?

Wm. Hill

re4med

Here is the error I am also getting
Unexpected Status Code 500 in Zotero.Sync.Storage.WebDav_get storageModificationTime()

re4med

When I sync with my server using WebDAV --- it always gets to a certain point in the process and then stops -- the sync arrow spins endlessly. However, all the data syncs correctly between my two machines. Not sure if this is a bug of some nature but it is a strange issue.

Wm. Hill

re4med

Any ideas to my above question?

re4med

I am getting this error when attempting to access an attachment in a browser:

Fatal error: Call to undefined function zip_open() in /home/re4med/bhill.dreamhosters.com/zotero/bhill/inc/include.php on line 24

krueschan

Hi,
@b.c.hamans: I am pretty sure that you would only have to copy all the files from the new version over the old ones (take a backup of your old install to be on the safe side!). Version 2 only expands on the feature set of the first version and I am pretty sure that I didn't touch anything of the original WebDAV implementation. You then only need to edit the settings.php as described on the installation and setup page linked to above to enjoy the complete new feature set.

@ re4med: thanks for your feedback. zip_open is a standard php function that is included in PHP 4 since version 4.1.0 and PHP 5 since 5.2.0. If your hoster uses older PHP versions (which clearly seems to be the case) there is nothing that I can do about this. Did the other question get resolved?

wilko.hardenberg

Zotero is not able to verify my server and states:
http://www.wilkohardenberg.net/zotero/webdav_server.php/zotero/ is not a valid WebDAV URL

If I try to access webdav_server.php directly over my browser I get:

Warning: require_once(Console/Getopt.php) [function.require-once]: failed to open stream: No such file or directory in /home/mhd-01/www.wilkohardenberg.net/htdocs/zotero/inc/System.php on line 23

krueschan

Hi Wilko,
This is interesting. This error should have occurred on every single installation of phpZoteroWebDAV (including my own) and never has. I guess your php server is more strict than most others.
The solution is to edit the file ../inc/System.php and simply delete line 23.
That should do the trick.
Or simply re-download the files from github where I have made the change.
Christian