Advice or warnings about trying out using multiple databases?

dperkel

I am interested in wondering if anyone has any advice about maintaining multiple databases when using Zotero on one machine. You may be asking why anyone would want to do this. So here is what I am doing.

I am actually using Zotero for two purposes:

1. Like most people, I am using Zotero to manage academic references. This is what Zotero was designed for.

2. I am also using Zotero as a tool for doing web archiving as a part of doing online ethnography. This was not necessarily what Zotero was designed for, yet it seemed to make sense. Rather than have to collect screenshots of pages or do lots of "Save as..." in my browser, Zotero can save the entire look and feel of pages at the moment I saw them. I can annotate them on the fly. Tag/code them. Write up quick thoughts as I see them. In other words, I am experimenting with Zotero as a field notebook, yet one that is nonlinear, searchable, sortable, etc.

I was thinking that it might be a good idea to separate these two activities after I noticed today that I had a database error of some sort. I was going to use the Zotero Database Repair Tool and in the process of backing up my directory realized that my Zotero directory had ballooned to being about 6.1GB. I don't know if this is because of my web archiving or if because the database errors are causing something else to happen. When I collect references, I do store copies of the PDFs, but my friend who has way more PDFs than I do in his Zotero database only has a directory that is 0.5 GB.

Regardless, I was toying with the idea of trying to run two databases once I figure out the database error problem. I think that this would better help me evaluate whether or not Zotero is a good tool for task 2 above. I can imagine that this might benefit other researchers who are using Zotero in the same way (if there are any) or if I would want to recommend that people try to use it this way (which I have been doing and now am feeling a bit unsure).

In my head I imagined that I would have to just be really careful to point Zotero to the right directory depending on whether or not I am just gathering references or whether or not I am actually doing "field" research. It may be tricky.

Is this even possible? Does anyone have any thoughts on why this may or may not be a good idea?

Thanks in advance for any help,

Dan

dperkel

Oops. I realize now that this is now not only "possible" but even promoted on the Tips and Tricks page of the FAQ (http://www.zotero.org/documentation/tips_and_tricks). But I was still wondering if anyone has had any particularly good or bad experiences trying to actually manage multiple libraries.

Thanks again,
Dan

dstillman

For what it's worth, it's highly unlikely that the database is actually 6.1GB, and far more likely that the size is being reported incorrectly due to filesystem corruption.

dperkel

Hi Dan,
Thanks for the response. Just an update on this, because I may have been confusing.

zotero.sqlite.bak is about 118mb

When I put in: du -sh zotero into the Mac Terminal, it took a while to come back with a result and the result said "6.1G" I think that most of that comes from the "storage" directory, so the filesystem rather than the database is what is huge.

Since my post I backed up the Zotero directory onto an external harddrive and the little "Copy" window that Macs show people (the Copy progress window) says that it's only 4.7 GB (which is still huge, but bigger than 6.1GB).

There definitely might be a filesystem corruption issue somewhere. If you have any ideas on how to go about exploring that I'd appreciate it (I realize it might not have anything to do with Zotero... it could be Firefox, could be my Mac, no idea).

However, I still think there is something going wrong on the Zotero end as well. When I "Check Database Integrity" it fails. And, Zotero is suddenly really slow in almost every way. I am going to try to use the Zotero Database Repair Tool to deal with this.

Thanks again,
Dan

dstillman

Ah, OK. In that case, you might want to run du -h --max-depth=1 in the storage directory to see what's taking up so much space.

dperkel

Sorry, having trouble with that command. Should it be "-sh" and then are the two hyphens okay? I keep getting "illegal option" results from my terminal.

Thanks,
Dan

noksagt

My favorite method is:
du -s * | sort -n


(Dan's suggestion of using max-depth assumes GNU du, which isn't in the default FreeBSD userland (and so may not be in the Mac OS X userland))

dperkel

Thanks, nokstagt. Will try that.

In the meantime, Dan or anyone else, I just posted this to the troubleshooting forum:

http://forums.zotero.org/discussion/4275/database-repair-tool-fatal-error/

dstillman

Sorry, was in Linux mode. You probably want this:

du -h | sort -n

(rather than -s)

noksagt

Thanks, Dan. I did mean s, but lacked the '*' argument for all directories in the current one.

s is for summarize (size of subdirectories)
h is for human readable (MB, GB)

h is not easily sortable if the directories vary greatly in size (e.g . 200 MB would show up after 1 GB).

kieren

dperkel:

You need multiple firefox profiles - http://xrl.us/ostsw (Link to lifehacker.com) You'll need to install zotero separately in each profile.

I collect all my academic stuff in my main profile, and all my sociology stuff in a separate one. To run the sociology profile concurrently with my main profile (can get a bit slow on this slow old hardware) I have a shell script like this:

#!/bin/sh
/Applications/Firefox.app/Contents/MacOS/firefox-bin -P sociology -no-remote 

The -no-remote flag lets you run multiple profiles at one. Absolutely no problems at all with this approach.