rdf+files import failed:ID:235982793

MartinWeidenhammer

Hi,

i try to import a 10 GB Database form another (zotero)Computer to mine.But on my Computer Zotero is always freezing.

Error Report:
JavaScript Error: "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFile.copyTo]" {file: "chrome://zotero/content/xpcom/attachments.js" line: 84}]

can anybody help me?

(sorry for my bad english)

adamsmith

while this should work in general, it isn't a recommended way to transfer a library:
http://www.zotero.org/support/kb/transferring_a_library

Do you get an unresponsive script message? If so: http://www.zotero.org/support/kb/unresponsive_script_warning

MartinWeidenhammer

thanks fr comment,..

yes, sometime i get an unresponsive script message, but in every case i clicked on "continue"

is it possible to copy the files form the " files"-folder into the storage-folder? not really hu?:-)

adamsmith

you can't just copy the files over, no - but you _can_ copy the entire Zotero folder as specified in the link on transferring a library.

MartinWeidenhammer

new report while import

Could not read chrome manifest file 'C:\Programme\Mozilla Firefox\extensions\{972ce4c6-7e08-4474-a285-3208198ce6fd}\chrome.manifest'.

Direct3D 9 DeviceManager Initialized Succesfully.
Driver: igxprd32.dll
Description: Mobile Intel(R) 965 Express Chipset Family
Version: 6.14.10.5218

[JavaScript Error: "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFile.copyTo]" {file: "chrome://zotero/content/xpcom/attachments.js" line: 84}]

[JavaScript Error: "uncaught exception: [Exception... "Component returned failure code: 0x80630003 (NS_ERROR_STORAGE_CONSTRAINT) [mozIStorageStatement.execute]" nsresult: "0x80630003 (NS_ERROR_STORAGE_CONSTRAINT)" location: "JS frame :: chrome://zotero/content/xpcom/db.js :: :: line 145" data: no] [QUERY: REPLACE INTO fulltextItems (itemID, version) VALUES (?,?)] [ERROR: insert on table "fulltextItems" violates foreign key constraint "fki_fulltextItems_itemID_items_itemID"]"]

[extra output removed — D.S.]

ID:1658654190

MartinWeidenhammer

at the moment, i have no possibility to get the original storage-folder

it must work with rdf+files

dstillman

Can you provide a Debug ID for an import attempt that fails?

MartinWeidenhammer

i can:-)


D1634985124

dstillman

I added a clearer error message for this to the 3.0 Branch dev XPI. Install that and try again. If you then go to Report Errors, you should see the file it's trying to import. Locate that file and inspect it. It's possible it's a Windows file alias, it has strange characters (though that shouldn't matter), the filename is too long, etc. Let us know what you find.

MartinWeidenhammer

ok, cool...

it shows me that file:
ould not read chrome manifest file 'C:\Programme\Mozilla Firefox\extensions\{972ce4c6-7e08-4474-a285-3208198ce6fd}\chrome.manifest'.

Direct3D 9 DeviceManager Initialized Succes..
Driver: igxprd32.dll
Description: Mobile Intel(R) 965 Express Chipset Family
Version: 6.14.10.5218

[JavaScript Error: "Failed importing file C:\Dokumente und Einstellungen\User\Anwendungsdaten\Mozilla\Firefox\Profiles\ek0n2btv.default\zotero\files\10256\Nagel Grewe Kopiez Altenmüller 2005_The relationship of psycho-physiological responses and self-reported emotions while listening to music.pdf" {file: "chrome://zotero/content/xpcom/attachments.js" line: 106}]



what can i do? it seems to long i guess... should i cut that out, or rename it?

adamsmith

yeah - that's just too long. Windows has a limit of 255(?) characters for a filepath. I think if you just move the file out of the folder it's in that will help - you can later re-attach it after shortening its filename.

MartinWeidenhammer

ok... i take this procedure.. just found the second file with overlength..

i believe it works... for every further out-cutted file the import-procedure makes real progress. (is that sentence understandable?:-) )

i let you know the end of the story

MartinWeidenhammer

Okay...

After i took out some bad files, zotero import for an half hour the libary and then MOZILLA crashed down.

The Error Report from Zotero says:

Could not read chrome manifest file 'C:\Programme\Mozilla Firefox\extensions\{972ce4c6-7e08-4474-a285-3208198ce6fd}\chrome.manifest'.

Direct3D 9 DeviceManager Initialized Succesfully.
Driver: igxprd32.dll
Description: Mobile Intel(R) 965 Express Chipset Family
Version: 6.14.10.5218

[JavaScript Error: "Livemark Service: feed processor received an invalid channel for http://fxfeeds.mozilla.com/de/firefox/headlines.xml" {file: "resource:///components/nsLivemarkService.js" line: 793}]


So what is that now?

dstillman

So what is that now?

Firefox running out of memory, probably.

Try after a fresh restart of your system with no other programs open.

You might also want to set the indexing preferences to 0 in the Search pane of the Zotero prefs to disable full-text indexing. You can always rebuild the full-text index later.

If that doesn't work, unfortunately there's not a whole lot we can recommend other than to try the import on a computer with more RAM and then transfer over your data directory.

We might be able to optimize memory usage for extremely large imports in the future, but that won't help you now.

Out of curiosity, how big is the RDF file itself?

MartinWeidenhammer

will it be better to take the standalone-zotero, to get more memory-ressources?
it has 12,5 MB:-)

dstillman

Not much, but it could help a little. You might as well try in that—and it's easy to do so and then switch back, since it will share the same data directory by default.

MartinWeidenhammer

okay, first i try to shut down all other programs and clear the ram..

dstillman

Just to check, is this library different from your server library? If not, if you do the import you'll get duplicates for every single item. Avoiding that would take a few extra steps.

MartinWeidenhammer

sorry.. i dont know something obout a server libary.This is a libary from a mate...

with restart and clear Ram Mozilla crashs down, when the storagefolder is about 13 GB. The filesfolder is about 10 GB, is that correct than?

Now i try to set the indexing preferences to O. you mean from 500000 to O?

dstillman

i dont know something obout a server libary.

You have over 9,000 items in your library on the server, which you can view at zotero.org. (You only have 100MB of synced files, though.)

If those were the same items you were trying to import (in order to transfer the files, presumably), you would want to take additional steps.

Now i try to set the indexing preferences to O. you mean from 500000 to O?

Yes.

MartinWeidenhammer

hm okay...

mozilla is crashing down too... damn. I have 3 GB Ram, the maximum Ram on a 32Bit System... i guess i must go to an friend with a better System...

when i input the Libary in Offlinemodus, i shouldnt have any more conlicts with the serverlibary,.. or do you mean something else?

dstillman

No, if it's the same library you would need to do more than switch to offline mode—you'd need to use the sync reset options in order to avoid duplicates. But again, this only applies if this was exported from the same library that's been synced to your online account. You're the only one who knows what's in the RDF file.

I'm afraid I can't really help you beyond this. Export isn't meant for transferring libraries, and you wouldn't be running into any of these problems using the recommended method. If at all possible, you should really get access to the original data files and transfer those.

adamsmith

If you know a little bit about XML you can open the .rdf file with an editor and split it into multiple pieces that would likely import w/o crashing.
Note that you'll actually have to do some editing, since .rdf, unlike RIS and bibtex for example, doesn't just consist of individual entries one after the other.

dstillman

Not sure, but with the cross-references in the RDF that might be tough.

dstillman

That said, I'm not really sure how a 12MB RDF file would use up 3GB of memory if the indexing settings (max characters and max PDF pages) were set to 0.

MartinWeidenhammer

splitting into peaces sound beautiful:-) but that sounds also like: okay, sit down for some days and learn how to hack microsoft!...
i never did something like that...

this weekend, i will have the time for this...

Note: i have two changeable harddrives for my laptop, one with xp sp3 and one with win 7. As i try to import in win7, it feels like under win7 the crashdowneffect comes much later. that makes me think that win7 can handle such big libaries in a better way.. but not better enough

@win7 mozilla was closed by itself without any errormessage or warning after 45minutes... when i restart mozilla the libaries is listed under "my Libaries" but without any entries... so its empty, no Tags or something else. BUT THE STORAGEFOLDER has 10,1 GB? so ... thats crazy

MartinWeidenhammer

splitting into peaces sound beautiful, because i always import libaries in this way, but even not such big one. The biggest database importing without such problems was @5GB

adamsmith

if you've never done something like that, manually splitting might be hard - see also Dan's note below mine.

I'm also still not sure why you wouldn't be able to use the recommended method for transfer? Why do you need to move giant RDF files around? That's really not what they're for.

dstillman

when i restart mozilla the libaries is listed under "my Libaries" but without any entries... so its empty, no Tags or something else. BUT THE STORAGEFOLDER has 10,1 GB? so ... thats crazy

Not crazy, actually, because of the way it works technically, but something I don't think we ever considered—that might explain the orphaned folders people sometimes see. Anyhow, it won't help the import, but you should delete those folders.

MartinWeidenhammer

okay... i try to explain:

first: i have soonest in 2 weeks the chance, to get the original-libary for the recommended import-method from my lecturer. I understand, that my actually matter is not the best way, but i always used the rdf+files Method for Libaries up to 5 GB... so my experience was: it works! My problem is, that i need this library to support my preparation of a study...

second: with: "the storage-folder has 10,1 GB after crashdown" i ment, that all files from the files-folder were imported:
files-folder:32521 Files
(NEW)Storage-folder:32512 Files
so thats what i want to say is: "all" files were imported, but zotero has no files listed in GUI.

___________
but although it dont work for me at the moment, i will thank you for your help and patience, i understand my favourite databasesoftware much better now, and i never will import such big libaries in that way....... i swear:-)

dstillman

so thats what i want to say is: "all" files were imported, but zotero has no files listed in GUI.

Yes, I understand, and I'm saying that's not surprising because of the way it works. Database entries are saved in a single transaction, all or nothing, but files are imported one by one. So if it crashes, the database entries won't be saved but the files will remain.

Short of getting the original database, your best bet at this point is to try in Zotero Standalone on a freshly restarted computer with as much RAM as possible—again, with the max characters and max pages settings set to 0 in the Search pane of the preferences. If you have access to a Mac, that might work better, since Macs tend to have better memory management.