syncing db?

bdarcus

So to sync a data between machines, where I can connect them via ssh, what do I need to sync? The entire profile, or just the sqlite file?

dstillman

You should sync the entire "zotero" directory within the profile directory (i.e. including the "storage" subdirectory, which holds snapshots and other attachments). Best would probably be to use rsync-over-ssh.

Syncing the rest of your profile directory isn't necessary for Zotero, though you'll want to make sure that any Zotero preferences you care about are the same on both systems.

Obviously this approach won't help if you change data on both machines (and the current approach to itemIDs isn't particulary helpful there either), but we'll be dealing with that in time.

zencat

I wonder if this would be a viable alternative to synching data -- what I've done is installed Firefox as a portable application on a USB drive. I can then run Firefox from there with the Zotero plugin and physcially move it to any machine I want to work on. Or how about this -- just export and import a collection? These aren't perfect solutions, but they might work in some situations.

rcrath

I backup and restore my data between two machines daily and would like to put the database in my main data directory instead of in the profile directory. Could you make the location of the database files settable in preferences instead of fixed?

noksagt

They plan to:
http://forums.zotero.org/comments.php?DiscussionID=36&page=1#Item_4

shel

I know nothing of extensions for firefox, but I was wondering if it would be possible at some point to have a sync option with a central web server. Sort of like the google browser sync extension that, once installed on various machines, keeps all the bookmarks, cookies, passwords in sync. Obviously, syncing a potentially huge database might be prohibitively slow, but if you could selectively sync collections, it would be hugely convenient.

snipe

I don't know if there's been more discussion on this topic or not (I did a forum search), but I just wanted to add that some manner of central database syncing would be tremendously helpful. My organization researches criminal animal cruelty cases, and because I'm on the move a lot, I use three machines as my primary work machines (not counting my Treo). This becomes problematic, since research would have to be imported and exported constantly in order to keep everything synced up. Plus, the ability to share research data with the rest of my team, or even the investigators working on open cases, would be turn this into a crime-fighting tool in addition to an incredible research tool.

Rauser

I tried to use .mac's iDisk as a way of syncing between my work and home computers' zotero database. Starting with my home computer, I uploaded my zotero files to a folder I created on iDisk and set the prefs to access that folder. Then, at the work computer's end, I tried to reset the prefs to access the databse on the iDisk. I got an error that declared that zotero did not see that folder as a zotero folder and that it did not recognize any of the data at zotero data (and, did I want to go ahead and use that folder anyway?). I assumed that if I elected to set this iDisk folder as my work computer's zotero folder, I would end up with two databases in the same folder, neither of which would sync with the other.

So is there any workaround to this?

Thanks!

bdarcus

Dan, I'd like to go back to your suggestion about rsync-over-ssh, as I'm just now setting up a new desktop machine, and was thinking to do just this.

My question is: I was planning to set up automatic syncing using the rsync daemon mode. Is there any problem doing this so long as I'm not modifying data on the two machines simultaneously (which would be impossible!)?

E.g. does it matter if Firefox is open on one or both of the machines when the syncing occurs?

sybille

bdarcus, I back up my files and configurations, including my Firefox profile plus the Zotero data, with rsnapshot, a Perl program that runs rsync periodically using the cron daemon.
http://www.rsnapshot.org/

I've experienced no problems whatsoever with backing up in this way. Scheduled backups have occurred at the same time that I was using Zotero on the machine being backed-up with no ill effect. I've been able to restore the backed-up Zotero data when needed.

So that's some anecdotal evidence for you. Incidentally, rsnapshot can be set up to use ssh, so it might be worth taking a look at depending on the kind of system you'd like to set up.

bdarcus

Thanks.

I first looked into using Carbon Copy Cloner (which is a really nice GUI that uses rsync underneath), but ultimately thought using rsync directly per this tutorial was a little more transparent.

I may look into rsnapshot later, as time allows.

Oh, and I just am syncing the backup directory for Zotero; not the entire profile. Hope that's OK.

sybille

Oh, and I just am syncing the backup directory for Zotero; not the entire profile. Hope that's OK.

I think that, for the most part, a file is a file as far as rsync is concerned, so there shouldn't be any problem with backing up whatever files or directories you choose.

If needed, there is a howto for rsnapshot here:
http://www.rsnapshot.org/howto/
And here's a blog post about using rsnapshot on OS X:
http://www.nearinfinity.com/blogs/page/jharwig?entry=checkpointed_backups_in_os_x

dstillman

E.g. does it matter if Firefox is open on one or both of the machines when the syncing occurs?

Firefox definitely shouldn't be open on the machine being backed up to, as that could probably result in corruption pretty easily if any changes (including automatic ones) were made to Zotero on that machine before restarting Firefox with the new database (and, at the very least, various bits of data in Zotero would be out-of-date due to in-memory caching). Zotero 1.5 will add a lock file to the Zotero data directory (to add a bit of protection when storing the database on a network share), so in the future you could check for its existence via an SSH command in a shell script before running rsync to know if the sync should proceed.

As for the source computer, officially I'd have to recommend that Firefox be closed on that as well before doing the backup, since theoretically rsync could copy the file while it's in an inconsistent state. But realistically, you're unlikely to run into any trouble doing this, as the times when the database is actually being updated are intermittent and brief. (For example, Time Machine backups with Firefox open seem to work fine, though as far as I know they're theoretically vulnerable to the same problem.)

However, to be safe, you might want to rig up something to automatically run an rsync once Firefox is closed. You could probably do this with an AppleScript wrapper, though there might be better ways.

Oh, and I just am syncing the backup directory for Zotero; not the entire profile. Hope that's OK.

That's fine, though you might want to keep your prefs.js file in sync too.