Upgrade Failed (Report ID: 1135535810)

szarka

I attempted to upgrade from (I assume) the latest Zotero 1.x to 2.0b7.6 and received the following error after about 30-45 minutes:

[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.executeSimpleSQL]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: anonymous :: line 485" data: no] [ERROR: constraint failed]

The system is running Windows XP and Firefox 3.5.6 and the sqlite file is about 150 MB.

Before I downgrade, anything I can do to recover from this problem? Any further info needed?

szarka

Hmm.. immediately after the upgrade attempt Zotero wasn't working. But after a restart of Firefox, I was surprised to find that it came right up and appears to be a fully-functioning install. Poking around a bit, my files and database appear to be intact. So now I'm not sure whether I should downgrade or give this install a go...

ajlyon

So now I'm not sure whether I should downgrade or give this install a go...

Since the 2.0 betas are so far ahead of the 1.0 series in functionality, I imagine you'll be much happier sticking with them for now. Just make and retain a backup of your Zotero data, and you'll be set. The Zotero developers keep hinting that the final release of 2.0 is on its way-- the current beta is very close to being final.

szarka

What I'm unsure about is whether all my data made it safely from one version to the next. I can see that all my files are still on the disk, of course, but not sure how to tell whether some data has gone missing from the database...

ajlyon

Do you have a backup of your data from before the upgrade? If so, you could set Zotero to start up with debug mode enabled (from Preferences-> Advanced), close Firefox, move the your upgraded data directory to somewhere else, replace it with the backup, and restart Firefox. If you get errors again, submit the error report and debug information and post here again. The Zotero developers should be able to look at the debug information and errors and determine whether any information appears to be lost, and also track down what is causing the errors.

If you don't have a backup of your data from before the upgrade, then there's probably not an awful lot that can be done to confirm that nothing was lost.

szarka

OK, so I turned on debugging, shut down Firefox, moved my old zotero directory back in place, restarted Firefox, and let it churn for an hour...

Once again:

[Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.executeSimpleSQL]" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: anonymous :: line 485" data: no] [ERROR: constraint failed]

This time it's report id 1270220850

As before, zotero failed to start until I restarted firefox, then it came up apparently ok. The database also passes the integrity check.

I told zotero to submit the debug file to the server and got id D861845428

The resulting converted database is the same size as last time. (It's slightly smaller than the v. 1.x database.)

Anyone have an idea what's going on? Short of tarring up 7 GB of data and moving it to a Unix machine where I can do a proper diff on it, it's hard to tell if anything went missing, but after casual inspection it does look OK. (And performance is noticeably better than under 1.x as well!)

dstillman

I'm not quite sure what that debug output is, but it doesn't appear to include updating of the database schema. (It includes other updates, but not the updates that occur while the upgrade wizard is showing.) Did you restart Firefox before sending in the debug output?

Relevant debug output should include a line beginning with "Updating user data tables".

szarka

Yes, I had to restart Firefox before sending in the debug output, since Zotero wouldn't run at all until I did so...

I take it the debug output is cleared on restart of Firefox?

dstillman

Yes, it's cleared on restart.

You can access the Zotero preferences via Tools -> Add-ons -> Extensions -> Zotero -> Options/Preferences even if you can't open the Zotero pane (though whether debug output submission works may depend on the error).

szarka

OK, I did the upgrade dance again, with the same result, but this time submitted what I hope will be more useful debug output..

Report: 2098773452
Debug: D803575741

Same error, but in context it's:

(3)(+0012794): Vacuuming database

(3)(+0001189): [Exception... "Component returned failure code: 0x80004005 (NS_ER
ROR_FAILURE) [mozIStorageConnection.executeSimpleSQL]" nsresult: "0x80004005 (N
S_ERROR_FAILURE)" location: "JS frame :: chrome://zotero/content/xpcom/db.js ::
anonymous :: line 485" data: no] [ERROR: constraint failed]

(2)(+0000000): Transaction is not in progress in rollbackTransaction()

(3)(+0004010): Backing up database 'zotero'

szarka

I should add that, this time, I first made sure that none of the files were set to read-only (Windows has an annoying habit of changing that, seemingly at random...) and I also told the stupid Windows indexing service to leave them alone. Didn't seem to make any difference, of course.

dstillman

Does your pre-upgrade database pass the integrity check in the Advanced pane of the Zotero prefs (if you load it in Zotero 1.0)? You may need to use the DB Repair Tool.

szarka

Yes, I checked the database immediately before shutting down Firefox, backing up the zotero directory, and attempting the upgrade. The database passed before the upgrade, and the upgraded database also passes after upgrading and then restarting Firefox.

dstillman

If you don't mind uploading your pre-upgrade database to the DB Repair Tool and e-mailing the Upload ID to support@zot....org, with a link to this thread, we can take a look. (You can ignore the download link the tool provides, which is unlikely to fix the problem.)

Basically, something in your database is in a state that's inconsistent with certain checks that were added in Zotero 2.0, and a database cleanup that takes place after the upgrade is catching that. The update process should take care of such cases, but something is apparently being missed. The upgrade itself seems to have completed successfully for you, so all your data should be intact, but you might run into an issue later if the problem isn't corrected.

szarka

Unfortunately, my database is about 160 MB--too big to upload that way. I guess I'll upload the file elsewhere and email support a link to the file...

dstillman

An alternative, if you were comfortable doing so, would be to open your post-upgrade database in the SQLite command-line client (download the Windows client, drag the .exe into a DOS window, press the spacebar, and then drag the .sqlite file in) and run this command:

VACUUM;
That would give us a more helpful error, which unfortunately Firefox itself doesn't pass along.

szarka

Hmm... I think I did the equivalent of what you suggested, but got the same terse error it was handing out before:

C:\Documents and Settings\szarka\Desktop\foo>sqlite3.exe zotero.sqlite
SQLite version 3.6.21
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> VACUUM;
Error: constraint failed
sqlite>

(I'll go back to plan A and upload my db while I sleep...)

dstillman

Ah, I guess that's all VACUUM produces anyway. (Constraint errors from sqlite3 are usually more informative.) We'll probably need a copy of the pre-upgrade DB, then.

szarka

Following up with a summary for future reference...

Dan wrote to let me know that "this looks like a bug in SQLite that allowed an inconsistency into your database without triggering an integrity check error." He had me follow the instructions at https://www.zotero.org/utils/dbfix/manual to rebuild the database, and I could see that SQLite caught several "constraint failures" that had crept in. Then I issued the VACUUM; command from the SQLite prompt and this time it completed without error.

So, I'm calling this a success and moving on to try syncing my database & files!