Export -> Import Problem

jjweimer
To transfer my citations from one machine to another, I exported my library and imported it. The contents of the library were imported, however the imported items were all under My Library. The distribution that I had set up with citations in various collections (and sub-collections) was not honored. An integrity check of the database after import showed it was OK.

Firefox 2.0.0.10 + Zotero 1.0.1
Exporting Machine: Windows Vista -> Importing Machine: MacOS 10.4.10
  • Tjowens
    edited November 30, 2007
    Was there a folder called "Imported" with the time stamp of when you imported the collection? All of your imported collections and subcollections should be in that "imported" collection. You can drag and drop them out of there as you see appropriate.
  • dstillman
    edited November 30, 2007
    If you just want to move your entire library, see the FAQ entry, What is the best way to move my entire collection to a different computer?
  • jjweimer
    I understand and have not yet tested the option to move the entire library by copying the Zotero directory. My initial hope in using Export -> Import was actually to be able export/import just a collection (not the entire library) from one machine to another. Since this is apparently not possible, I suggest it as a feature request.

    Regardless, my disappointment in using the (seemingly easirer) export -> import process to transfer a library was because it did not honor the collection structure that had been set up for the library on the originating machine. In answer to Tjowens, yes, I did have an Imported + Time Stamp folder when the Zotero RTD was imported, and that folder (collection) had all the sub-folders properly labelled under it. What I lost was the division of citations into their original collections.

    In this regard, if the above mentioned behavior is not a bug, I suggest adding a clarification to the FAQ that Export -> Import transfers the entire My Library folder (not just collections) and that, when it is used as such to transfer a library, all citations are imported to the top-most level in the import folder (ie, citations have to be manually moved back into their originating collection folders after the import process is completed).

    Thanks!
  • dstillman
    If I export a collection containing one item and having a subcollection under it containing a different item and then reimport it, I get a new Imported folder with the top-level collection containing both items and the subcollection containing the item it originally had under it.

    So, the bug would seem to be that, after importing, items in subcollections show up in the top-level collection in addition to the subcollection. Is that the behavior you see as well?
  • jjweimer
    edited December 3, 2007
    As I recall (I am not at the MacOS right now), the import ended with all items at the top level.

    Original Structure
    ------
    My Library
    ..Collection 1
    ....Sub-Collection X
    .......Citation 1
    .......Citation 2
    ....Sub-Collection Y
    .......Citation 3
    ..Collection 2
    ....Citation A

    Imported Structure
    ------
    My Library
    ..Imported (date)
    ....Citation 1
    ....Citation 2
    ....Citation 3
    ....Citation A
    ....Collection 1
    ......Sub-Collection X
    ......Sub-Collection Y
    ....Collection 2

    --
    JJW
  • dstillman
    You're going to have to upload (somewhere) or send us (at support@zot....org) a Zotero RDF file, explain what you clicked on to export it (right-click on collection->Export Collection, right-click on items->Export Selected Items, or Actions menu->Export Library), what gets imported, and what you expected to happen.

    The RDF file should include as few items as possible, with Export Files not checked.
  • jjweimer
    I'll do so sometime in the next week or so.

    I also see now how right-clicking on a lowest level collection will give a menu to export just the collection. In my example above, this means I can export just Sub-Collection X, Sub-Collection Y, or Collection 2; however I cannot export Collection 1 as one unit (ostensibly because it contains sub-collections). This partially addresses my desire to be able to transfer just collections - they can be exported -> imported as long as they do not contain any sub-collections (a useful note perhaps to add to the FAQ).

    Thanks!

    --
    JJW
  • dstillman
    however I cannot export Collection 1 as one unit (ostensibly because it contains sub-collections)
    Well, you can—it just appears to exhibit the behavior I outlined above. But that's a bug, not something to add to the FAQ. I was just asking if you've seen different behavior. You can get back to us if you find you did.
  • algemmell
    I have been trying to export my bibliographic database from Biblioscape to Zotero. Using the Save As RIS-Reference Manager option I have saved the file as a txt. file, but when I come to import it into Zotero, I get an error message of the form 'No translator could be found for the given file'. I have tried several times, using alternative formats for the export file, but always get the same result. What should I try next? The file is quite large, c.4500 entries. Could that be part of the problem? Thanks.
  • noksagt
    Others have successfully imported from biblioscape, so it is hard to say why you can't. Given the number of records, I'd try to import a small subset first. Then, either import other subsets or try to run your whole exported file through another program to clean it up (bibutils (stand-alone or through refbase.org) might be a good bit--it is fairly forgiving on import, but rigid in export).
  • algemmell
    I have now managed to get import to work, but only on relatively small sets of records (<30). It turns out that part of the problem which I had not realised lay at the Biblioscape end of things. When I selected a file name to save the data exported from Biblioscape, the system by default sets to save the data as a Biblioscape tagged file. As I was setting it for RIS - Reference Manager before selecting a file name, I had not noticed that it was reverting to the Biblioscape tagged file structure. Once I set the dat style after the file name, then it works correctly. The problem with importing files in small batches is one of tedium, rather than technology.
    However, I have noticed that the export/import sequence seems to lose abstracts somewhere - are they saved in RIS -Reference Manager mode?
  • noksagt
    I have now managed to get import to work, but only on relatively small sets of records (<30)...The problem with importing files in small batches is one of tedium, rather than technology.</blockquote>What happens when you try to import larger batches? Do you get an error message or does it just take a while? The import can be slow & many just let it go overnight if they're importing a lot.
    However, I have noticed that the export/import sequence seems to lose abstracts somewhere - are they saved in RIS -Reference Manager mode?
    RIS stores this in 'AB -'. Does biblioscape export it? Which version of zotero are you using? The latest release will store it as an attached note.
  • algemmell
    I am using Zotero 1.0.1, and have just discovered that the abstract is stored as an attached note, as you say. I tried importing the full 4000+ references as a single exercise, and there were signs of activity for about 2 minutes, and then the entire system seemed to freeze. I left it for about 30 minutes, but nothing further happened, so I eventually closed Firefox (2.0.0.11) as I thought that Zotero had frozen. On restarting Firefox and opening Zotero, I found a subfolder in the Library labelled with the import date and time, but when I tried to look at the contents, it appeared to be empty. I think from what you say, the system actually works fine once the problem of setting up Biblioscape correctly to do the export has been solved - it was just my impatience which was causing the problem. However, I will try a large export/import operation and leave it running overnight, to see what happens.
    As a separate point, can Zotero cope with the database being on an web-based site such as Box.net, or would data transfer be too slow to be practicable?
  • bdarcus
    Hmm ... seems a little strange to treat an abstract as a note? It's a property of the resource, not commentary by a user/reader/note-taker? Moreover, DC and other formats support it out-of-box as a literal property.
  • dstillman
    RIS stores this in 'AB -'. ... The latest release will store it as an attached note.
    Actually, while it boggles the mind, according to the RIS spec N1 and AB are notes while N2 is an abstract. Zotero will store N2 in the abstract field—that is, with the rest of the metadata—and N1 and AB as attached notes.
  • dstillman
    edited December 13, 2007
    As a separate point, can Zotero cope with the database being on an web-based site such as Box.net, or would data transfer be too slow to be practicable?
    I'm not familiar with how Box.net works. If it's a live online drive that you'd run Zotero off of, then I'd say that neither SQLite nor Zotero's data layer are really designed for that sort of use. I believe some people have had success using it off a network share on a LAN (though that's still unsupported), but I suspect using it on a WAN link would be quite slow, and the integrity of the database might be compromised.

    Syncing to an online service after closing Firefox on one computer is another matter and should be possible with a decently fast uplink speed (though using a tool like rsync to copy only the part of the database that had changed would be better).
  • noksagt
    Actually, while it boggles the mind, according to the RIS spec N1 and AB are notes while N2 is an abstract. Zotero will store N2 in the abstract field—that is, with the rest of the metadata—and N1 and AB as attached notes.
    Ah--thanks for the correction.
    I believe some people have had success using it off a network share on a LAN (though that's still unsupported), but I suspect using it on a WAN link would be quite slow, and the integrity of the database might be compromised.
    FWIW: I've used both SFTP and WebDAV mounted drives over the WAN. It was usable with a small database that had no snapshots or PDFs. It definitely isn't the ideal way to run the tool & syncing through other methods or using a common local profile (on a usb stick, for instance) are much better.
  • algemmell
    The experiment I referred to earlier (importing a large database overnight) seems to have completed successfully, but has thrown up some problems in utilisation. When I first start Zotero, the middle panel now stays blank for a couple of minutes, and then a message appears in a box. This message says 'A script on this page may be busy , or it may have stopped responding. You can stop the script now or you can continue to see if the script will complete.' If I click 'Continue' the central panel still stays blank, and the script message will appear again after another couple of minutes. Clicking on 'Continue' again and the central panel becomes populated with the list of references, as per normal - the entire process takes about 5 minutes.
    Once the full database is in place, I can scroll through the entries and see details, but the Search facility seems to be unable to cope with the load of data. I started to put in a search term, but I could only enter two characters of the term before the system froze and the Script Error message referred to above appeared again. On clicking 'Continue' I got a Firefox error message (reported as Error #570930125) and had to restart Firefox.
    Having now seen the problems with a large database in Zotero (I suspect that lack of RAM and processor power may be a factor in those problems rather than the software), I am now wondering if there is an 'Undo Import' facility available for Zotero, and if not, could it be put under consideration for a future upgrade of the program?
  • Jon Rubin
    Quick tip on the search problem:
    type " (double quotation mark) before whatever you want to search for then press enter once you've typed the full search term.
  • sybille
    edited December 14, 2007
    I'm thinking that your problem may have to do with the full-text indexing in Zotero. You could try deleting the current index and rebuilding it from scratch. This may well produce some more "A script may be busy" messages, but these can be disabled in the Firefox preferences:
    http://forums.zotero.org/discussion/1415/

    Alternatively, you can turn off the full-text indexing altogether, by setting both "Maximum characters to index per file" and "Maximum pages to index per file" to 0 and then deleting the existing index, all in Zotero's Preferences for Search (the gear icon). I do this because I find the text cache slows down my Zotero too much (I have an AMD Sempron 2800+ processor and 1.5 GB of RAM), but also because I use another full-text indexing service for my files (Beagle search on Linux). I just had to add the Zotero storage folder to Beagle's preferences, and that works fine for me.
  • dstillman
    I'm thinking that your problem may have to do with the full-text indexing in Zotero.
    That may contribute to the slow search, but it also may be just the number of items (which includes notes and attachments) vs. the hardware being used. As we've said elsewhere, the size of library usable on various computers is highly variable, and we still have more work to do optimizing Zotero for use with larger libraries.

    We'll try to fix the unresponsive script warning, but in the meantime you can disable it by going to about:config in the address bar and setting dom.max_chrome_script_run_time to 0. We haven't seen that DB error behavior before, but it's probably a spurious error from accessing a large amount of data in the Firefox 2 storage system and will go away in Firefox 3 (though disabling the unresponsive script warning may also help).

    In the meantime, you can try clearing the fulltext index to see if that helps. If not, you can delete those imported items for now by going to the newly created import collection, selecting all items, right-clicking, and selecting "Delete Selected Items from Library".
  • jjweimer
    FYI, I have posted to Zotero support the Zotero rdf files illustrating the export->import problem I had.

    Thank you!

    --
    JJW
  • algemmell
    As the newly imported library contained a number of items which I had installed separately, so I actually have some duplication at present in My Library, will highlighting all items in the newly imported collection and selecting "Delete Selected Items from Library" only delete the copy of items that is in the imported collection, or will it clear out all copies of items, only some of which were actually imported with the collection?
  • dstillman
    As the newly imported library contained a number of items which I had installed separately, so I actually have some duplication at present in My Library, will highlighting all items in the newly imported collection and selecting "Delete Selected Items from Library" only delete the copy of items that is in the imported collection, or will it clear out all copies of items, only some of which were actually imported with the collection?
    It will only delete the items you have selected, not any duplicates. (Of course, you should make sure you have a backup of your Zotero database before making any mass changes, but ideally you're doing that daily anyway.)
Sign In or Register to comment.