Attach Link to File... freezes firefox and zotero (Not Responding)

jeroenvg

WHAT I'M TRYING TO DO:
i'd like to store my library of programming books in zotero. for example: i have an Android.zip (435 MB) with 33 books (pdf, epub, mobi, source code zips). i unzip it, and for every PDF: Add Item by Identifier: ISBN; Add Attachment: Attach Link to File...: Android.zip; delete unzipped files when i'm done.

WHAT HAPPENS:
attaching a link to Android.zip to a book item takes about 5 minutes to complete, if it does. the title bar shows...

Zotero - Mozilla Firefox (Not Responding)

...for the firefox extension, or...

Zotero (Not Responding)

...for zotero standalone. the cursor changes to 'busy' (hourglass) and CPU usage is constantly high. this is 100% reproducible; it always takes too long and sometimes doesn't recover. the link to the file exists if firefox/ zotero is forcefully ended via task manager, when it doesn't recover.

WHAT I EXPECTED TO HAPPEN:
i expected the link to be created immediately.

MORE INFO:
because i suspected that zotero/ firefox/ system was 'doing something' to all 435 MB's of the file, i did some tests with a far smaller zip (2,56 MB), in firefox and zotero standalone.

* indexing set to disabled for both the extension and standalone, separately (Preferences...: Zotero Preferences: Search: Full-Text Cache: Maximum characters to index per file: 0).
* full reboot between each test.
* as little as possible applications running (see below).

time to recover from a 'Not Responding' state, and CPU usage during recovery:

firefox extension (in a separate tab):
* link to Android.zip, 435 MB: ~ 5 min., CPU usage ~ 40% continuous
* link to C.zip, 2,56 MB: 10 min., CPU usage 100% continuous (application forcefully ended via task manager)

standalone:
* link to Android.zip, 435 MB: ~ 5 min., CPU usage ~ 40% continuous
* link to C.zip, 2,56 MB: 10 min., CPU usage 100% continuous (application forcefully ended via task manager)

applications running during tests:
* avira antivirus (system boot)
* adobe gamma loader (account auto startup)
* openoffice.org 3.3 (account auto startup)
* pageant (account auto startup)
* dropbox (account auto startup)
* evernoteclipper (account auto startup)
* appigo sync for windows (account auto startup)
* xmarks for IE (account auto startup)
* notepad (started manually)
* task manager (started manually)
* firefox 16.0.2 with zotero extension 3.0.11, or zotero standalone 3.0.11.1 (started manually)

CONTEXT:
the zips i'm trying to link to reside in my home directory on a linux server, shared by samba to a windows xp professional (sp3) desktop computer, and mapped as a network drive, to a drive letter.

other installed firefox extensions are:
* evernote web clipper 5.4
* firebug 1.10.6
* java console 6.0.33
* java console 6.0.37
* web developer 1.2.2
* dutch dictionary 3.1.0
* xmarks 4.1.3
* java console 6.0.35 (disabled)
* java quick starter 1.0 (disabled)
* microsoft .net framework assistant 0.0.0 (disabled)

please let me know if you need any other info.

dstillman

Well that certainly doesn't sound right. Can you provide a Debug ID for one of these add attempts?

jeroenvg

yes, repeated first test: freshly booted, no notepad started, no task manager started, recovery after some 5 minutes, as expected.

debug id is D1677024847.

dstillman

So for some reason the ZIP file is being detected as application/pdf, and Zotero is then trying to run the PDF tools on it (despite the indexing prefs being set to 0?), which apparently takes a very long time with a file of that size (or at least does on a network share—you could see if it takes as long from a local drive).

I'll see if I can reproduce this.

dstillman

Can you post a file that triggers this somewhere? A zipped PDF is not detected as application/pdf for me.

jeroenvg

i'm not sure zip size matters. note the second test with C.zip of 2,56 MB; that one doesn't recover after 5 mins., but doesn't recover at all. i can email you that one if you like?

dstillman

You can send it to support@zotero.org with a link to this thread.

jeroenvg

done.

dstillman

OK, so a bunch of different things are going on here:

1) These ZIP files (or at least C.zip) appear to have been created with no compression, so they're essentially just PDF files with a ZIP file header. Both Zotero and the PDF tools skip past the ZIP header, find the PDF header, and treat the rest of the file like a PDF. Zotero could require the PDF header to be at the very start of the file, but since the PDF tools don't mind the extra header, I'm not sure Zotero should either (and I'm not positive there are no situations where a valid ZIP file could have other bytes at the beginning).

If I create a ZIP file with the -0 flag to disable compression, Zotero and the PDF tools treat it as a PDF file. With default ZIP compression, the file is treated as a ZIP file and not indexed.

2) Your C.zip file imports fine for me with PDF indexing enabled, so I suspect this is an issue with your network share. Can you generate real-time debug output for the C.zip file to see if anything is actually happening (and if not, what the last few lines are)?

3) Regardless, Zotero shouldn't be running pdfinfo/pdftotext if the indexing settings are set to 0, and doesn't for me. Are you sure you have those both set to 0?

jeroenvg

3) i have 'Search: Full-Text Cache: Maximum characters to index per file: 0', but 'PDF Indexing: Maximum pages to index per file: 100' on the same tab, in both firefox and standalone.

this results in 'Indexed: No' in the right (metadata) column for the file-link attachment 'Android.zip'.

i was under the impression this disables indexing completely. is this correct?

dstillman

OK, I can reproduce a freeze with C.zip with those settings. I'll look into that.

With both at 0, indexing is skipped altogether as expected.

jeroenvg

perfect, and thanks A LOT for your prompt help!

do you still want a real-time debug output for C.zip from me?

dstillman

No, that's OK, thanks.