Suggestions for handling a very large project?

mhoerner
I am trying to create the bibliography for my combined dissertation. So far, I have handled the very large number of references and citations by working on the document in separate pieces. However, I have to provide a single bibliography for the complete dissertation, so I put everything in a single file this afternoon. However, there are evidently so many references/citations that Zotero simply cannot handle the combined document. I have tried repeatedly to get it to work, but I cannot even get the "Document Preferences" dialogue box to open, much less accomplish anything or add a full bibliography. I sat and waited for over half an hour for the dialogue box to open and nothing happened. Are there any options for me at this point so that I can finish my dissertation without having to start over with some other piece of software? Thank you in advance for your suggestions and help.
  • adamsmith
    which word processor, which OS?
  • mhoerner
    I'm sorry, that is important information! Word 2010 and Windows 10. Thank you for your help on this.
  • adamsmith
    (Word for Windows is good news, btw. It's one of the faster setups with Zotero, especially compared to the Mac version).
    It's certainly true that Zotero goes painfully slow in large documents, but a typical book length document really shouldn't be a problem to at least generate a bibliography. Also, the Set Document Preferences window, afaik, doesn't even load up all citation information, so I don't see why that would not come up.

    How exactly are you assembling this? Are you pasting into an empty document or are you pasting Chapter 2, 3, 4, into the Chapter 1 file?

    If you're doing the former, try selecting a citation style _before_ you paste anything into the document, see if that helps.

    If it doesn't, what if you go bit by bit, i.e. start with Chapter 1 and 2, see if that works, then do 1,2,3, etc. This will a) make sure the general merge process works and b) let you test at what stage this actually fails.
  • mhoerner
    Thank you for your response. I spent last night and today so far working with pieces of the document to make sure that everything was in the same citation format and then recombining the pieces into a single document. About an hour ago, I clicked the "Insert Bibliography" button, and the document is still sitting there. I am leaving it open in hopes that it might eventually complete the bibliography, but at this point I think I need to start looking for other options. I really do appreciate your feedback. I doubt I will have any other documents with nearly so many references in the forseeable future, but for now, I am just concerned with getting a bibliography generated so that I can submit my thesis.
  • adamsmith
    You didn't answer my questions above, though, so hard to say if something else isn't working or if it's really just an issue with the document size.
  • mhoerner
    Sorry, I am assembling it the second way- starting with a document with chapters 1 and 2 and pasting the rest of the chapters and appendices in. I took it bit by bit as you suggested. I'm not sure which questions you are referring to.
  • adamsmith
    OK, thanks. If you check in Task Manager, do you see Firefox/Zotero doing something?
  • mhoerner
    Zotero is doing something. Firefox is not installed on my computer and I just use the stand-alone.
  • mhoerner
    Word just finally put out a bibliography. It ended up taking a couple of hours to generate, but leaving both Word and Zotero untouched allowed it to complete. I just thought I would share this information in case it is useful to someone else with a similar problem down the line. I had almost 1400 separate references and many thousands of individual citations in-text over the course of what ended up being nearly 1000 combined pages, so for projects this large, it is probably best to find alternative solutions for reference management. Thank you, adamsmith, for your suggestions. They did end up working. I just needed to give it more time to complete.
  • bwiernik
    I'm not sure there will be any good solution for quickly generating a 1400 reference bibliography. Endnote will have the same problems as Zotero.
  • RintzeRintze
    Since this is not such an uncommon scenario, has anybody done some benchmarking to see where all this time is spent? Is it mostly Word/Zotero communications, or is Zotero's CSL processor having a hard time with the disambiguation in such large documents? Are there significant differences in speed between style types and disambiguation settings? Are there checks to make sure there are no large performance regressions between CSL processor versions?
  • adomasven
    @Rintze attempting to improve this is on my to-do list in some not too distant future, but may take a while as there's a lot of code to sift through.
  • bwiernik
    If the disambiguation processing is a big part of the bottleneck, an option to insert unformatted placeholder citations (that would ignore things like disambiguation) would be nice (Endnote has such an option).
  • RintzeRintze
    @mhoerner, if you or anybody else has a similarly large Word document that they would be willing to share (either publicly or privately), I'd be interested in running a few benchmarks by hand to see if different citation styles make much of a difference. Anything that takes more than a few minutes to render will do.
  • adamsmith
    If the disambiguation processing is a big part of the bottleneck, an option to insert unformatted placeholder citations (that would ignore things like disambiguation) would be nice (Endnote has such an option).
    Yes, Frank and I have talked about this and I think it'd be a huge help. Wouldn't actually help in the case above (since that's for the final format, but it _would_ help people working on medium large documents.
  • cdhankin
    I am having an identical problem with a dissertation in Word 365 for Mac. Just combined six different chapters into one document using add object feature of Word and bibliography won’t compile. Any updates? My doc is just over 300 pages, around 100,000 words.
  • bwiernik
    See https://forums.zotero.org/discussion/comment/333284/#Comment_333284
Sign In or Register to comment.