Zotero 10.0.2 Find Full Text queue stops after an HTTP 502 error

SCI_Group123

Hello,

I am experiencing a reproducible problem with the Find Full Text function in Zotero 10.0.2 on Windows 10.

Environment
Zotero version: 10.0.2 (64-bit)
Operating system: Windows 10 (19045)
Locale: zh-CN
Zotero was tested in Troubleshooting/Safe Mode with all extensions disabled
extensions.zotero.findPDFs.resolvers is set to []
The Zotero library contains approximately 17,000 items.
Problem

When I select a batch of approximately 20–22 items and use Find Full Text, Zotero can successfully find the PDF for the first item, but then the process stops progressing on a subsequent item.

For example:

Select 22 items.
Right-click → Find Full Text.
The first item is processed successfully.
After that, no further items are processed for more than 10 minutes.
Restarting Zotero and repeating the exact same operation produces the same behavior: the first item succeeds, and the queue then stops progressing.

This makes large-batch Find Full Text operations impractical. I would like Zotero to skip a failed PDF request after a reasonable timeout and continue processing the remaining items.

Debug output

Debug ID:

D191531454

The debug output contains repeated errors such as:

Request timed out after 30000 ms

and repeated HTTP 502 errors from IEEE Xplore:

HTTP GET https://ieeexplore.ieee.org/stampPDF/getPDF.jsp?... failed with status code 502

The same IEEE request appears repeatedly in the debug output.

Additional observations

I previously had custom PDF resolvers configured, including several Sci-Hub resolvers. After removing them and setting:

extensions.zotero.findPDFs.resolvers = []

the performance of single-item Find Full Text improved substantially.

However, the batch-processing problem described above still occurs.

I have also tested the following:

Manual PDF downloads in a web browser are fast.
Dragging a manually downloaded PDF into Zotero works immediately.
Zotero's WebDAV server verification succeeds.
Disabling automatic sync does not resolve the Find Full Text problem.
Changing the proxy setting temporarily did not resolve the problem.
The problem persists with Zotero extensions disabled.
Expected behavior

If a PDF request fails or repeatedly returns an HTTP 502/timeout, I would expect Zotero to mark that item as unsuccessful and continue processing the remaining items in the Find Full Text queue.

Actual behavior

A failed request appears to keep the Find Full Text process from progressing to subsequent items for an extended period of time. In a batch of 22 items, the first item succeeds but subsequent processing can remain stalled for more than 10 minutes.

Could you please advise whether this is expected behavior, or whether there is a setting or fix that would allow failed PDF requests to be skipped so that the Find Full Text queue can continue?

Thank you for your help.

JaapDo

I can reproduce the same issue in Zotero 10.0.2 with IEEE Xplore. Find Full Text repeatedly receives HTTP 502 from ieeexplore.ieee.org/stampPDF/getPDF.jsp and does not proceed to the next selected item. The retry delays in my debug output increase from 2.5s → 5s → 10s → 20s → 40s → 60s → 120s → 240s. I can download the same PDF normally in my browser.

Debug ID: D760745318

bruce-willis

I am also seeing the IEEE 502/retry pattern, and have isolated a more general difference in cookie and referrer handling between Zotero.HTTP.download() and Zotero.HTTP.request() (XHR).

Environment tested: Zotero 10.0.5, Gecko 140.15.0, macOS 26.7.1.

The self-contained MWE below calls the actual core downloader against a temporary local server. It needs no IEEE subscription, publisher URL, Python, or plugin. The server separately requires a dummy cookie, the supplied Referer, or both.

Observed results:

Server requirement          HTTP.request (XHR)   HTTP.download
None (download sanity check)  not tested           200, file saved
Dummy cookie only             200                  403, no file
Supplied Referer only         200                  403, no file
Both                          200                  403, no file

For all three protected cases, XHR sent the dummy cookie and the exact supplied Referer. The core downloader sent neither, even though both methods received the same Referer header option. Cookie and referrer requirements therefore fail independently in this fixture.

Important: the fixture deliberately returns 403 for missing test context. This reproduces the request-handling difference, not IEEE's particular 502 response or its access policy. In my real IEEE test, XHR and fetch with explicit credentials/referrer downloaded the PDF successfully while the default fetch request returned 502; an IEEE-only XHR compatibility wrapper restored normal updates.

To reproduce: in Zotero 10, open Tools → Developer → Run JavaScript, check “Run as async function”, paste the script, and run it. Everything stays on localhost. It creates and cleans up temporary files, its uniquely named dummy cookie, and the local listener; it does not change library items or preferences. The output includes only test-cookie presence and referrer-match flags, not real cookie values.

// Zotero: Tools > Developer > Run JavaScript; select "Run as async function".
// No publisher, Python, or plugin required. No library/preferences change.
// Creates a loopback fixture, a dummy cookie, and temporary files; cleans them.
const { HttpServer } = ChromeUtils.importESModule("chrome://remote/content/server/httpd.sys.mjs");
const server = new HttpServer();
const token = Services.uuid.generateUUID().toString().replace(/[{}-]/g, "");
const cookieName = "zotero_mwe_" + token, prefix = "/zotero-mwe-" + token + "/";
const files = [], results = [];
let started = false, base, article;
const cookieHeader = age => `${cookieName}=1; Path=${prefix}; Max-Age=${age}; SameSite=None; Secure; HttpOnly`;
function endpoint(name, needsCookie, needsReferrer) {
  server.registerPathHandler(prefix + name, (request, response) => {
    const cookie = request.hasHeader("Cookie") && request.getHeader("Cookie")
      .split(";").some(value => value.trim() === cookieName + "=1");
    const referrer = request.hasHeader("Referer") && request.getHeader("Referer") === article;
    const ok = (!needsCookie || cookie) && (!needsReferrer || referrer);
    // Synthetic 403 tests missing context; this is not a publisher response.
    response.setStatusLine(request.httpVersion, ok ? 200 : 403, ok ? "OK" : "Missing test context");
    response.setHeader("Cache-Control", "no-store", false);
    response.setHeader("Content-Type", "text/plain", false);
    response.setHeader("X-MWE-Cookie", cookie ? "present" : "absent", false);
    response.setHeader("X-MWE-Referrer", referrer ? "matches" : "missing-or-different", false);
    response.write(ok ? "Local fixture download succeeded\n" : "Required dummy context missing\n");
  });
}
async function probe(method, name) {
  const options = { headers: { Referer: article }, timeout: 10000,
    errorDelayMax: 0, noRetryOnThrottle: true, successCodes: false };
  let file;
  try {
    let response;
    if (method === "XHR") {
      response = await Zotero.HTTP.request("GET", base + prefix + name,
        { ...options, responseType: "arraybuffer" });
    } else {
      file = Services.dirsvc.get("TmpD", Components.interfaces.nsIFile).clone();
      file.append("zotero-http-mwe-" + token + "-" + name + ".tmp");
      files.push(file.path);
      response = await Zotero.HTTP.download(base + prefix + name, file.path, options);
    }
    const header = name => response.headers ? response.headers.get(name) : response.getResponseHeader(name);
    results.push({ method, test: name, status: response.status,
      dummyCookie: header("X-MWE-Cookie"), suppliedReferrer: header("X-MWE-Referrer"),
      ...(file ? { fileSaved: file.exists() } : {}) });
    if (response.status !== 200 && response.arrayBuffer) await response.arrayBuffer();
    return response.status;
  } catch (error) { results.push({ method, test: name, error: String(error) }); }
}
try {
  server.start(-1); // Bundled API binds loopback only on an unused port.
  started = true;
  base = "http://localhost:" + server.identity.primaryPort;
  article = base + prefix + "article";
  for (const [name, age] of [["seed", 300], ["cleanup", 0]]) {
    server.registerPathHandler(prefix + name, (request, response) => {
      response.setHeader("Set-Cookie", cookieHeader(age), false);
      response.write("Dummy cookie only");
    });
  }
  endpoint("public", false, false);
  endpoint("cookie", true, false);
  endpoint("referrer", false, true);
  endpoint("both", true, true);
  await Zotero.HTTP.request("GET", base + prefix + "seed", { timeout: 10000, errorDelayMax: 0 });
  await probe("HTTP.download", "public");
  for (const name of ["cookie", "referrer", "both"]) {
    if (await probe("XHR", name) !== 200)
      results.push({ test: name, note: "XHR control failed; this case is inconclusive." });
    await probe("HTTP.download", name);
  }
} finally {
  for (const file of files) await Zotero.File.removeIfExists(file)
    .catch(error => results.push({ cleanupError: String(error) }));
  if (started) {
    await Zotero.HTTP.request("GET", base + prefix + "cleanup", { timeout: 10000, errorDelayMax: 0 })
      .catch(error => results.push({ cookieCleanupError: String(error) }));
    await server.stop();
  }
}
return JSON.stringify({ zotero: Zotero.version, gecko: Services.appinfo.platformVersion,
  fixture: "Loopback only; synthetic 403 means required dummy context was absent", results }, null, 2);

Could this be a request-context regression from the switch to streaming fetch? Would a core PR preserving streaming while restoring the appropriate cookie/referrer behavior, with regression tests, be welcome?

dstillman

Yes, thanks, this was a regression in 10.0. We've fixed it for the next beta.

dstillman

This should be fixed in the latest beta.

dstillman

Fixed now in Zotero 10.0.6