some notes shrinking while syncing

emily michelson

I recently took extensive notes on a book using the 'Notes' section, while offline. Later I synced Zotero and half of my notes disappeared. Even though I was offline the whole time I was writing, only the first half of what I wrote now appears - this is true on the original device, on the new device, and in the online library. Is there any way to restore previous material? Thanks.

DWL-SDCA

What, if any, is the character limit within the notes field?

emily michelson

If there is one, I wasn't close to it. I've taken pages and pages worth of notes in the notes field in the past, far more than this.

adamsmith

giant -- I think 100k characters -- and it wouldn't delete them, it would throw a sync error when you exceed it.

The problem with getting the note content back is that the only way (and even that I don't really see) I could imagine them getting lost is if they were never written to the database, so they literally never existed for Zotero purposes, so I have no thoughts on getting these back.
Obviously this should never happen, but absent a replicable instance, I'm not quite sure how to go about troubleshooting what happened.

emily michelson

I don't see how, if I was offline the whole time, the first 2 hours' worth of notes were written to the database and the second 3 hours' worth of notes weren't. I didn't sync until a few hours after I had written the whole thing. I briefly wondered if I accidentally selected and deleted some sections, but no: the bits that stayed were the bits that were written first, even though some were at the top and some were at the bottom of the whole, and were integrated in with the parts I wrote later.

adamsmith

the Zotero database is local/offline, so being disconnected from the internet has nothing to do with this.
I really wish I had something better for you, but there's just no way to find a problem that's supposed to be impossible to occur at a technical level, that occurs incredibly rarely, and that can't be made to occur reliably.

emily michelson

I know - it's very random. I'll keep thinking about what could have happened. If I remembered every missing word it would help, but then I wouldn't have needed to take notes! Thanks for trying.

What does it mean for them not to be written to the database?

adamsmith

Dan may want to correct me on the details here, but basically:
as you type a note, you're typing in an editor and the text exists in that editor only.
Zotero saves all text content -- all metadata, all notes -- in a database file (an sqlite database file, to be precise).
When you you type into a Zotero field, it saves what you typed to the field the moment you switch to a different field (or item).
It also does that when you write a note: when you switch away from the note, Zotero will write the content to the database. But that's obviously not enough, since you might be spending significant time on a note, so it also writes what you have written to the database every couple of seconds. My only idea for how what you describe could be happening is if that process somehow stopped working (which, again, it never should).

dstillman

Yeah, I don't know how this would happen. I'd recommend checking your database integrity from the Advanced → Files and Folders pane of the Zotero preferences. Beyond that, I can't really think of anything — unless you received a sync conflict and chose the (shorter) remote version, there shouldn't be any way for this to happen. How sure are you that this happened when syncing, as opposed to, say, when switching to another item and back?

A couple other ideas for getting your data back (if you haven't rewritten it yet):

1) You can check the zotero.sqlite.bak and zotero.sqlite.1.bak files in your Zotero data directory. If there's a timestamp that looks at all promising, make a copy of that file immediately. Then see Restoring from the Last Automatic Backup.

2) If you're on a Mac and use Time Machine, even if you didn't have it plugged in, Time Machine might have a local snapshot of the database from the relevant time. So you could check that.

emily michelson

Many thanks, both. I've given up on this particular data but I've now backed up my zotero files to my external hard drive, having read through the sites you linked to. So some good came out of it.