Zotpress for Wordpress

Congratulations to kseaborn on releasing a new plugin to feed Zotero collections into WordPress!

Details at http://wordpress.org/extend/plugins/zotpress/

[And apologies for pre-empting any official announcement, but this just popped up on my radar.]
«13
  • Hi! You beat me to it. Zotpress is still in its early stages, so I'm very open to comments, suggestions, feedback -- and bug reports. :)
  • Would it be possible to cache individual citation data? I've added a couple of citations using [zotpress item_key="..."], but they take a very long time to load. When the user sets a specific item ID, or perhaps in all cases, it would be nice if Zotpress cached the item data to display immediately on each pageview, then looked for updated item data in the background.
  • Hmm, interesting. I'll see what I can do!
  • edited February 10, 2011
    We're working to speed the API up, but ideally Zotpress would use an If-Modified-Since header in its API requests and use cached data if it gets a 304. Unfortunately the API doesn't truly support that yet—in some modes it does return a 304, but it's a very inefficient 304 that doesn't save time, only bandwidth. But it's probably still a good idea to start using that header now so that the plugin benefits when the API speeds those up.

    To prevent hammering the API, the plugin should probably also cache the data for a short while even before making an If-Modified-Since request.
  • I should also mention that the plugin is very slick. I definitely did not expect such an elegant little tool!
  • @ajlyon Thanks!

    Next up: Caching and 304 checks. Really appreciate the feedback and ideas.
  • My citations won't load: http://www.gimranov.com/avram/w/archives/60
    There are two Zotpress tags, for two different individual items.

    I looked at the code for the first time, and I wonder if it wouldn't be better to initiate the Zotero API call from PHP, rather than in client-side JavsScript. This would improve performance and not leak API keys. The key leakage is a big deal, especially since this has the potential to allow selective library display without creating extra groups or making your main library public.
  • They're loading for me, but, yeah, I assumed this was making the API calls from PHP, which would definitely be the better way to go. Everything I said above regarding caching pertained to PHP-based requests. If 1,000 people load that page in a minute, that should trigger one API call, not 1,000. We haven't yet implemented any mechanism for throttling API requests, but client-based requests from a medium-traffic site would quickly be throttled when we do.
  • edited February 12, 2011
    @ ajlyon They're loading for me too.

    I'll see about making the switch to a purely PHP-based approach. (Caching is currently driven by PHP.) Entering into unfamiliar territory ... stay tuned.
  • I can help with PHP if you get stuck, and the Zotero sync server is all PHP, so the core devs may be able to give some hints if we can't make this work. A high-quality WordPress plugin that doesn't hammer the API would be a great thing.
  • And apparently the problem I'm seeing is that jQuery isn't being included when I'm logged. It works for me too-- so long as I'm logged out. I'm using the WordPress 3.1 release candidate.
  • Thanks, and I agree! I appreciate the support. I haven't tried WordPress 3.1, but I'll set up an implementation and see what I can do to make sure jQuery is included when needed.
  • Alright! Zotpress 2.1 is released. Here's what's new:

    I switched most of the request code to PHP and used a combination of cURL and PHP DOM XML. It's still jQuery on the front-end so that the rest of the page can load instead of waiting for the request to go through. Regardless, all private keys are now hidden and/or encrypted. I've also implemented a new way of caching: citation data is stored in a local table in the Wordpress database instead of in user-specific PHP sessions. The data is per-request, too, so Zotpress won't try to update every piece of data in the table at once—only the request needed. Also, Zotpress will send out a request to the Zotero server ONLY IF a request is made AND it's been 10 minutes since the last request. The 10 minutes is arbitrary. Also, this is temporary until the If-Modified-Since header is properly implemented on the Zotero server, and I can check for the 304 status. Finally, I fancied things up a bit ... you can poke around and see, it's a bunch of little aesthetic updates.

    @ajlyon I also installed a local copy of Wordpress 3.1 and tested out a Zotpress after adding a one-liner check for jQuery. It's working on my end, but let me know if it's not for you.

    I hope this solves the above issues in the best way currently possible. I'm certainly open to further suggestions.

    Also, next on the to-do list is implementing OAuth. As I understand it, this would obviate the need for Zotero users to register private keys.
  • Thanks for the update, but I'm afraid to report that version 2.1 gives me several errors:
    Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when in safe_mode or an open_basedir is set in /www/htdocs/w00814ec/wordpress/wp-content/plugins/zotpress/zotpress.php on line 179

    Warning: DOMDocument::loadXML() [domdocument.loadxml]: Empty string supplied as input in /www/htdocs/w00814ec/wordpress/wp-content/plugins/zotpress/zotpress.default.php on line 169

    Warning: curl_setopt() [function.curl-setopt]: CURLOPT_FOLLOWLOCATION cannot be activated when in safe_mode or an open_basedir is set in /www/htdocs/w00814ec/wordpress/wp-content/plugins/zotpress/zotpress.php on line 179
    In my case, the error does not seem to come from PHP safe mode which is disabled by my provider. So the problem seems to be open_basedir. I don't really know what this error means, but according to my research this is neither something I can change nor something my provider will change since it affects security.
  • @simifilm I've released Zotpress 2.2, which should address this issue. Let me know if you're still coming across this error.
  • @simifilm I've released Zotpress 2.2, which should address this issue. Let me know if you're still coming across this error.
    It works now.Thanks a lot for fixing this so quickly.
  • My turn to report an issue. Now in Zotpress 2.2, in the Citations tab of the Zotpress preferences, I get:Warning: DOMDocument::loadXML() [domdocument.loadxml]: Empty string supplied as input in /[snip]/wp-content/plugins/zotpress/zotpress.default.php on line 169
    And the citations show up as "null" on the blog post (see link above). The null issue was in 2.1 as well, perhaps the preferences error was as well.
  • edited February 14, 2011
    Zotpress was more or less working for me in an earlier incarnation (1.6?), but now I can't do much of anything. After adding an account, when I return to the Citations tab I cannot select a collection or tag. Both of these dropdowns are empty. I verified at zotero.org that my key hasn't been used in days, so it looks like Zotpress currently isn't trying to connect to the API at all.

    One small note regarding the tooltip text: it states that user and group IDs "should be a series of five digits." In fact they currently range from one to six digits and will hit seven soon enough.

    Many thanks for this cool plugin!
  • Sean has user ID 3.
  • Sorry about the delay -- I've stopped receiving forum notifications for some mysterious reason. I'll look into these issues ASAP. Are they still occurring in Zotpress 2.4?
  • You only receive notifications once after your last visit to the forums.
  • Sean's issue was due to the cache tables not existing in the database after he upgraded to the version with caching. Once he erased the original tables, all the tables were created properly and it worked.

    One thing I noticed while debugging this for him: is there a reason you're making multiple external requests back to the local web server? It's not much of a problem for small sites, but it's not a great practice unless you have a really compelling architectural reason to do so, since it adds extra load to the server and slows down the original request unnecessarily. It also litters the access logs with the Zotero API key, which is OK if the logs are under the control of the original user but bad otherwise. (Using POST would fix that, but it'd be better just to eliminate those subrequests, which shouldn't be necessary—you're already in PHP when you make those calls.)
  • edited February 19, 2011
    @ Dan Thanks for the information on the issue Sean was encountering—strange, because I'm using Wordpress's update mechanism for tables, which should have just added the missing cache table. (I'm certain I checked this on one of my other servers and had no issues.) Just to make sure we're on the same page: what version of Zotpress is Sean (and yourself, if different) running?

    I'm not certain what you mean by multiple requests. Zotpress should only send a request if a user makes one and it's been 10 minutes since the last request (from version 2.1 onward). It sounds like I've misunderstood something. Also: once I've implemented OAuth, will this issue still exist?

    Off topic, but is there a way to continue to receive notifications from a forum? I've subscribed to "Forum", "Category" and "Discussion" for this post.
  • He was running 2.3.
    I'm not certain what you mean by multiple requests.
    I'm referring to requests made to the web server running WordPress from itself, not requests to the Zotero API server. Sean's access logs showed requests to zotpress.rss.php originating from the blog web server itself. It's possible these were triggered only by the admin section, but there were definitely a number of those requests and they definitely put the API key in the access logs:

    ...zotpress.rss.php?account_type=users&api_user_id=3&public_key=*****&data_type=collections HTTP/1.1" 200 - "-" "Mozilla/5.0"

    ...zotpress.rss.php?account_type=users&api_user_id=3&public_key=*****&data_type=tags HTTP/1.1" 200 - "-" "Mozilla/4.0"
    Also, is there a way to continue to receive notifications from a forum?
    There's no way to receive continuous notifications, but you can ensure you'll get notified of the next post by just clicking the link in the e-mail and then closing the browser window (as long as you're logged in). The notifications aren't meant to be a replacement for the thread itself.
  • Ah! I see. So, until I implement OAuth (or even in that case?) I should instead have Zotpress send a request with the user's private key to the server once, say when the user first adds their account (and key) to Zotpress, and after that point requests don't have to have the private key attached to them? (Now I understand what you mean by POST being the solution, since no data, e.g. in this case a URL, is recorded in, say, a log. I'm curious: are the logs not private?)

    Oh, and I'll be sure to update the notice and database given my new knowledge of the format of user IDs, too.
  • I don't know what the purpose of those requests are, so I don't know how this relates to authentication, but, basically, application code running on a server should almost never need to make HTTP requests to itself. (Calling other services—like, say, an external API such as the Zotero API or a separate service running on the local network—is another matter.)

    PHP already has the private key, because the user submitted and it's stored in the database. Anything the subrequest can do, the original request from the client can do itself. There's no reason your server-side code should be sending additional HTTP requests through the web server just to run code that you already have access to in the original request—that's what include files and function calls are for.
  • And to answer your question, whether web server access logs are private depends entirely on the server configuration, but they could easily not be private. That's somewhat tangential to the main issue, though, which is that web app code shouldn't be making HTTP requests to itself.
  • I also have problems with Zotpress 2.4. Inside Wordpress, there are no error messages anymore, but I want to display a site with Zotpress content, I get a requester saying "Error: undefined".
  • PHP already has the private key, because the user submitted and it's stored in the database. Anything the subrequest can do, the original request from the client can do itself. There's no reason your server-side code should be sending additional HTTP requests through the web server just to run code that you already have access to in the original request—that's what include files and function calls are for.
    I think that the design is to allow the citations to be loaded asynchronously and/or updated in real time (which mattered much more when the server was slower, and the citations were not cached). Since they're cached most of the time, it seems to me that cached citation content should be sent inline with the post, and provided asynchronously only if the citation was not cached already. Continuous updates also seem unnecessary, although they could be nice for -- say -- a group library that was frequently updated.
  • edited February 19, 2011
    That may have been the original design—and that was fine when the requests were coming from the client—but I'm saying that there's no justification for the server-side code to call itself via HTTP. Even if there was still a mechanism for asynchronous, JS-based updating, there would be no reason for the web server to call itself. It just causes Apache to do extra work. Whatever function generates the responses for JS-based requests can be called directly from the original client request.

    I didn't really mean for this to turn into a big discussion, since there's nothing at all specific to Zotpress here. Web app code just generally shouldn't make HTTP requests to itself.
Sign In or Register to comment.