macOS: Time needed to export large library

postorganic
edited April 28, 2022
I want to export my Zotero library containing 2753 items including all PDFs (the size of my Zotero folder being 15 GB) to Cita. To do so, I clicked "Export library" with BibTeX as the chosen format and activated the export files button.

How long should the export of this library take approximately? I'm asking because, weirdly enough, upon executing the export on Zotero, no window popped up indicating the remaining time left for the export.

Also, can I already import the library to Citavi before the library has been exported including all files or should I rather wait?
«1
  • postorganic
    Regarding the exported library: Out of the 2373 folders containing PDFs, 575 folders are empty.

    Does this mean that the export isn't done yet? Or is it normal to have a certain percentage of empty folders?
  • postorganic
    Anyone? I'd appreciate a response.
  • postorganic
    Here (https://www.zotero.org/support/reporting_problems) it says that providing a report ID is not possible on iOS.

    Does this mean that if there's an error, it will not be shown?
  • postorganic
    edited April 27, 2022
    Hi, I'm on a Zotero 6.0.5 (iOS), trying to export my entire library (including files) to Citavi, chosen format being BibTex. The exported library reaches 11.7 GB out of a total of 12.8 GB and stops.

    Here's the Debug ID: D959078784. Any help is much appreciated - and do let me know if you need any additional information.
  • dstillman
    edited April 28, 2022
    Please don't create duplicate threads for the same issue.

    I've merged this back into one thread.
  • dstillman
    edited April 28, 2022
    @postorganic: You're using macOS, not iOS.
    [JavaScript Error: "Error: Worker 'export' failed: {"annotations":[],"error":"{\"message\":\"bad ObjStm stream\",\"name\":\"FormatError\"}"}" {file: "chrome://zotero/content/xpcom/pdfWorker/manager.js" line: 183}]
    It looks like you're getting an error exporting one or more PDFs — probably because of something corrupted in the file — but you didn't generate the Debug ID correctly. It needs to actually cover the operation in question (in this case the export) for us to say more. See the instructions again.
  • postorganic
    Thanks for the response.

    I did everything according to instructions: enabled Debut Output Logging, performed the action (in my case: exporting entire library), waited about 4 hours until the export stopped copying the PDFs - reaching, like last time, 11.7 out of 12.8 GB - and finally clicked on Debug Output and clicked Submit.

    The Debug Output window popped up, but for some reason remained blank. As Zotero was no longer responding, I "force quit" Zotero. Afterwards I reopened Zotero and clicked on Debug Output. It worked this time: D264116354. But because Zotero crashed, I don't know whether if I need to redo the above steps all over again - please let me know.
  • dstillman
    No, sorry, the debug output is cleared on restart.

    Since this is macOS, you can log easily to a file:

    https://www.zotero.org/support/debug_output#logging_to_a_terminal_window

    See the part about "when Zotero is crashing or hanging".

    You can then zip the file created on your desktop (right-click → Compress) and email it to us at support@zotero.org.
  • postorganic
    edited May 1, 2022
    Just sent the zip file to support@zotero.org.

    Thanks in advance for your help!
  • dstillman
    Can you install the Zotero beta and generate another Debug ID with that? It looks like there's a single (likely corrupt) file that's causing trouble here, and that should tell us which one.
  • postorganic
    Did just that. Hope I did it right!
  • dstillman
    edited May 1, 2022
    zotero(3)(+0000001): Exporting PDF for item 1/66DSFA4C

    console.log: (new InternalError("too much recursion", "chrome://zotero/content/xpcom/pdfWorker/worker.js", 1839))
    zotero(1)(+0000796): Error: Worker 'export' failed: {"annotations":[],"error":"{\"fileName\":\"chrome://zotero/content/xpcom/pdfWorker/worker.js\",\"lineNumber\":1839,\"columnNumber\":5,\"message\":\"too much recursion\"}"}

    export/<@chrome://zotero/content/xpcom/pdfWorker/manager.js:185:21
    If you paste 66DSFA4C into the search bar in All Fields & Tags mode and try to export just the item that shows up, does it hang? Can you provide debug output for that?
  • postorganic
    It doesn't hang, but the export apparently fails as the exported file remains empty.

    Here's the Debug ID: D22464765.
  • dstillman
    Can you temporarily disable auto-sync in the Sync pane of the preferences and repeat the full export that hangs? Your log is mostly filled up with sync activity, which makes it very hard to read.

    (For what it's worth, you can also likely figure out what's causing this by doing a binary search: export half of the library, and if that succeeds export the other half, and then export half of the failing half, and so on until you hopefully find a single item that fails.)
  • dstillman
    edited May 1, 2022
    @postorganic: Oh, and could you send the PDF that resulted in an empty export above to support@zotero.org with a link to this thread? We'll definitely want to see what's going on there.
  • postorganic
    Ok, so I e-mailed the PDF that resulted in an empty export.

    I then disabled auto-sync and repeated the export of said item. Same deal - export folder stays empty. Here's the Debug ID: D1634162320.

    In any case, I really don't need that particular item, so I can delete it if that helps at all.
  • dstillman
    No, sorry, we need the full output with auto-sync disabled. Since the export goes through with just that file (even though the file is empty), that's not what's causing the hang.
  • postorganic
    Sorry, I'm not sure I got it. You mean that I should export the entire library and not just this one item?

    Also: did I correctly disable auto-sync or not?
  • dstillman
    Yes/Yes
  • postorganic
    Here's the Debug ID: D535158443 - Zotero didn't crash this time! :)
  • dstillman
    Oh, if it didn't crash there's nothing to debug for that.

    You're getting errors (and presumably empty files) for a bunch of files, though. Some of the items that are failing:

    66DSFA4C
    CQAV77W2
    HMXR22X6
    FGRPS43K
    3LIJEAMK
    QKWQ22I3
    CGGF38GC
    U6PUHK9M
    JFHWRRAJ
    QLY56HIF
    EDMSVFWY

    If you send some of these to support@zotero.org, we can take a look.
  • martynas_b
    console.log: (new InternalError("too much recursion", "chrome://zotero/content/xpcom/pdfWorker/worker.js", 1839))
    zotero(1)(+0000796): Error: Worker 'export' failed: {"annotations":[],"error":"{\"fileName\":\"chrome://zotero/content/xpcom/pdfWorker/worker.js\",\"lineNumber\":1839,\"columnNumber\":5,\"message\":\"too much recursion\"}"}
    The PDF file has untypical amount of internal references therefore crashes while trying to export it. We should fix that, but it'll take time.
  • postorganic
    @dstillman: sent all items in three batches about 9 hours ago.

    @matinynas_b: which PDF are you referring to?
  • postorganic
    Why will the fixing take a long time?
  • postorganic
    Why wouldn't deleting the items solve the problem?
  • dstillman
    It would, but you'd have to delete all the affected PDFs, and there appear to be a lot.

    Martynas was referring to the first PDF you sent. The other ones are failing in different ways, and we still need to look through those.
  • postorganic
    I understand.

    How much time will this take approximately? Are we talking days, weeks or months?
  • postorganic
    Also: I've been using the Beta version for a week now. Can I use it normally or do I need to revert back to the release now that the Debug ID has been successfully generated?
  • dstillman
    Up to you. Many people use the beta all the time, but if you have no particular interest in running beta software you should switch back to the release version.
  • postorganic
    What about the duration for fixing the PDFs? Will this take days, weeks, or months?
Sign In or Register to comment.