Bulk upload or store 12K pdf files from a flat directory as new child files

bjbraams · 2024-09-09T02:18:45+00:00

I want to start using Zotero to manage my pdf articles collection, which consists of 12K files in a flat directory. I have the Zotero space for it through a work subscription. But I don't see how to upload more than 50 files at a time (the apparent limit for drag and drop through the browser). Any advice?

dstillman · 2024-09-09T06:46:28+00:00

What do you mean by "through the browser"? Are you using the web library or the desktop app? You want to be using the latter, and there shouldn't be a limit for dragging files in. (The desktop app will also run PDF metadata retrieval, which the web library doesn't support.)

bjbraams · 2024-09-09T08:23:52+00:00

Thanks for that advice! I was using the browser and now I am using the desktop app and within that app the Add File... command under the tab that carries the label Add Attachment. Now the upload limit seems to be about 300 files per batch and I need to wait some as yet undetermined time between batches. Am I doing it right?

dstillman · 2024-09-09T08:26:06+00:00

Wait for what reason?

There's no intentional batch size limit on our end, but metadata retrieval uses various upstream services, and they might start blocking you temporarily after a large number of requests. So if you just mean that it stops being able to recognize PDFs, that would be the reason. If you mean something else, we'd need more details.

bjbraams · 2024-09-09T08:31:30+00:00

Thanks for the quick response. So I use the Add File command as described. In the window that pops up when I do Add File I highlight the files that I want to upload. I found that if I highlight more than about 300 files then nothing happens, but (in the initial experience) when I highlighted fewer than 300 then it immediately starts uploading and after the upload it starts acquiring the metadata. I let it complete and I try to upload the next batch same way. But now nothing happens even at smaller batch size. I interpret that (without much confidence) as having to wait a bit before the system will allow further requests from my side.

dstillman · 2024-09-09T08:37:44+00:00

No, shouldn't be anything like that.

Can we see a Debug ID for an attempt that does nothing?

bjbraams · 2024-09-09T08:40:45+00:00

My apologies, I think something else is going on. Interspersed among my 12K pdf files there are also a handful of directories. (Linux here.) If a directory is in the highlighted list then the interaction just hangs without response. Probably I just need to circumvent the directories when highlighting.