My notes have been orphaned! And is the new merge sync functionally different from a standard sync?

So all the notes I've entered for many references have unexpectedly been orphaned -- what were notes attached to references now appear as standalone notes in my library. This occurred immediately after a "reset from server" operation, using Z 2.0 on XP. (I was testing this operation out at home after making significant library updates at work earlier in the day.)

Yikes. Please tell me I won't have to go through each note and find its parent. One option -- I know -- is to go back to my computer at work, where presumably my notes are still attached to the parent references, and "restore to server" from there. The problem with that option is that I didn't realize what had happened until after making further organizational changes to my library from my home computer. I.e., complicated changes I don't want to have to redo.

Is this a situation where the (apparently new) merge sync option on the sync tab, under preferences, will help? What exactly does this sync do anyway?
  • This was due to a bug in 2.0b6 and 2.0b6.1, and it's been corrected in 2.0b6.2. Those versions together were only out for about a day, but unfortunately any child items uploaded by them would've been promoted to standalone items on the server, and pulling them back down would show them as orphaned. This should be limited to items you modified with 2.0b6(.1), however. Using "Restore to Zotero Server" on your work computer—via Tools->Add-ons, as explained in Troubleshooting Sync Errors, so as to not trigger auto-sync—may be your best bet, though you will lose any changes on your home computer.

    An alternative might be to disable auto-sync at work (again via Tools->Add-ons, before opening the Zotero pane) and then generate a report of your entire library (or items in all collections that might have been affected) and then use that at home as a guide to (hopefully fairly quickly) reorganize the orphaned notes back into their parents.

    There's a slim chance that a merge sync from your work computer—again, avoiding a normal auto-sync—could correct your problem, if you chose Local for every attachment/note conflict that it displayed, but if you did a lot of collection-based reorganization at home it would probably fail. If you do decide to go the generate-report/reorganize-at-home path anyway, though, you might as well try it after generating the report (and making a backup of your database). Basically, merge sync forgets sync history and pulls everything from the server and compares it to everything from the client, displaying a conflict resolution dialog for anything that's different.

    Sorry for the inconvenience. We're taking some additional steps to ensure a bug like this doesn't slip through in the future.
  • edited July 9, 2009
    Thanks for the helpful comments. I have good news to report. A merge sync at work seems to have fixed the problem. My notes are back where they should be, and the changes I made at home were entirely preserved. Phew. Here's the weird thing: Upon syncing, the merge window produced only five conflicts, none of them having to do with my notes. In fact, none of the conflicts were patently conflicting, which is to say that every piece of data displayed as "local" vs. "server" (including time stamps), which I was then asked to resolve manually, actually matched already. So something is going on there beyond my field of vision. As for the notes, they seemed to work themselves out, as if by magic.

    I appreciate the logic that this problem had something to do with versions b6 and b6.1, which were out for a short time. But I'm not entirely convinced -- because when I got home I made it a point to upgrade to b6.2 before syncing. In fact, Firefox practically made me do it by presenting the option on startup. So... someone might want to take another look at 6.2 and make sure the issue is fully resolved.

    For others who might encounter this problem, I highly recommend backing up your library before trying a merge sync, since merging is still very much in development. For what it's worth, I think it's a great tool to have. At the very least users benefit from the increasingly clearer distinctions among various types of syncing -- they are not created equal! -- including the default sync.

    Initially I wasn't sure whether, in a default sync, certain local changes would be privileged when compared with the server version. That's what I think most users would expect; otherwise, they might see some unusual behavior -- for example, a reference removed from a collection, but not the library, might unexpectedly reappear in that collection following a sync. (Fortunately, this is not the case by default.) The mere availability of merge syncing has helped make this distinction clearer, but some related documentation would also help.
  • My notes are back where they should be, and the changes I made at home were entirely preserved. Phew. Here's the weird thing: Upon syncing, the merge window produced only five conflicts, none of them having to do with my notes.
    OK, well, that works... If you haven't yet, you should do a Restore to Server from work just to make sure the server is up-to-date with the correct data.

    It's possible that merge sync doesn't currently compare the parent item at all and only compares item data, in which case it wouldn't detect any change and might then resend the (correct) local data back up to the server. I'll have to look into this. Glad to hear it worked though.
    But I'm not entirely convinced -- because when I got home I made it a point to upgrade to b6.2 before syncing.
    Well, but the problem would've already happened with any items that you modified (or that were flagged for upload behind-the-scenes) at work using 2.0b6 or 2.0b6.1, assuming that's what you were using there. 2.0b6.2 at home would've just pulled those changes down.
    Initially I wasn't sure whether, in a default sync, certain local changes would be privileged when compared with the server version. That's what I think most users would expect; otherwise, they might see some unusual behavior -- for example, a reference removed from a collection, but not the library, might unexpectedly reappear in that collection following a sync.
    Actually, the items should reappear and I think it's preferable. In the case of a conflict involving collection membership or tag linkage, it doesn't really make sense to privilege one version over another another, and conflict resolution would be annoying and difficult (to develop and to use), so Zotero will automatically err on the side of more data and just display a warning. In other words, if you remove an item from a collection on one side and remove it on the other between syncs, there's no way for Zotero to know which change to keep, so it will restore the item to the collection and display a warning to that effect. Not ideal, but better, I think, than the alternatives (including keeping a log of every collection item removal). And, with a 15-second default sync delay, this should be pretty rare in normal operation.

    Anyhow, I've updated the sync troubleshooting documentation with an explanation of the merge sync, but it remains something that shouldn't really be used in normal operation, at least until some of the bugs(/magic) are worked out.
  • edited July 10, 2009
    Thanks again, and good advice. For the record, I had already upgraded to b6.2 at work, then synced by "restore to server" before going home and encountering the problem. So I'm still not convinced it has (solely) to do with a bug (solely) in the older version.

    But thanks for explaining more of the syncing nitty-gritty there. I'm not sure we're talking about exactly the same thing. Let me be more clear. Say I'm working on one local copy of my library. I copy a link from one collection to another, then I remove the link from the original collection. (In effect, I've just moved without copying from one collection to the next.) Then I sync again -- a standard sync, using the sync icon in the upper right -- or, say, one that happens automatically because I've enabled auto-sync. To my knowledge, the link I moved is not restored to the original collection, where I had just removed it. And that's precisely the behavior one would want, right?

    On the other hand, I agree that if I leave this local copy running, then make changes from a different location that somehow contradict previous changes, then Zotero should "err on the side of more data." I'm guessing, but not sure, that that's what we're saying.
  • Say I'm working on one local copy of my library. I copy a link from one collection to another, then I remove the link from the original collection. [...] To my knowledge, the link I moved is not restored to the original collection, where I had just removed it
    Correct. There's no conflict in this case, so the change is just sent to the server.
    On the other hand, I agree that if I leave this local copy running, then make changes from a different location that somehow contradicts previous changes
    It's fine to leave Zotero running. You only get a conflict when an object (item, tag, etc.) has changed on both the server and a client since the last sync of that client. So given two identical, synced libraries A and B, if you make a change to an item on A, sync A, make a change to the same item on B before syncing B, and then sync B, you'll get a conflict. Normally you'll get a conflict resolution dialog, except for the case of things like collection item membership and tag association, which can't be easily reconciled in a UI and are automatically merged to keep the extra data instead.

    This is why Zotero auto-syncs when you open the Zotero pane. If you always sync before starting work on one side, you should basically never get a conflict (except perhaps in a group library where people are working on the same thing simultaneously). Zotero also auto-syncs on idle to reduce the chance that remote changes haven't been synced before you start working.
  • Excellent discussion. Might be useful to add that or something like it to the wiki, maybe on a page like "more details about syncing" or something. Many thanks.
Sign In or Register to comment.