Import Annotations for multiple PDFs
When importing a large PDF library into Zotero is there a way to have Zotero automatically convert all embedded annotations of the PDF files to Zotero annotations?
I know that in Zotero's "Export Library ..." dialogue, when choosing to "Export files" I have the option to "Include annotations", which according to the documentation should embed Zotero annotations in the PDF (to be read by external PDF readers).
I want to achieve the opposite. I want to have Zotero move all embedded PDF annotations into Zotero's database (deleting the embedded annotations in the PDF files).
So essentially I'd like to do in bulk what I can do file by file by
- opening a PDF in the Zotero PDF viewer and then
- choosing "Import Annotations ..." from the File menu
Is this possible? Or possible to implement?
I know that in Zotero's "Export Library ..." dialogue, when choosing to "Export files" I have the option to "Include annotations", which according to the documentation should embed Zotero annotations in the PDF (to be read by external PDF readers).
I want to achieve the opposite. I want to have Zotero move all embedded PDF annotations into Zotero's database (deleting the embedded annotations in the PDF files).
So essentially I'd like to do in bulk what I can do file by file by
- opening a PDF in the Zotero PDF viewer and then
- choosing "Import Annotations ..." from the File menu
Is this possible? Or possible to implement?
Upgrade Storage
Any chance this could be considered in a future release?
annotationsImport. Can be run from Tools -> Developer -> Run JavaScriptExplanations are in the script. It has a dry-run option for testing first. Also it has an option to turn all highlights to yellow, to avoid the issue discussed here where the imported color is too bright:
https://forums.zotero.org/discussion/126747/color-of-imported-highlights#latest
The option is set to "false" by default.
IMPORTANT: It only works on highlights and underlines, not on freehand/ink or other elements. This is because the annotation engine of Zotero can only import freehand etc. that was previously created in Zotero. See here for discussion:
https://forums.zotero.org/discussion/126434/import-annotations-doesnt-work-for-freehand-markup-and-text-notes#latest
Here's the code. Paste it into the window in Tools -> Developer -> Run JavaScript and run as async function.
// Import the annotations embedded in PDFs into the Zotero database, for all items tagged "annotationsImport".
// Equivalent to File -> "Import Annotations..." in the reader, run for every tagged PDF.
//
// Run in Zotero: Tools -> Developer -> Run JavaScript
// (tick "Run as async function" if the window offers that checkbox).
// Start with DRY_RUN = true and LIMIT = 3, read the output, then widen.
//
// Based on Zotero.PDFWorker.import(itemID, isPriority, password, transfer) in
// chrome/content/zotero/xpcom/pdfWorker/manager.js (main branch). That function does the
// whole job itself (reads the PDF, saves the annotations, and with transfer = true removes
// them from the PDF file); it returns only a Boolean, not the annotations.
const TAG = "annotationsImport";
const DRY_RUN = true; // true: only report which PDFs have embedded annotations
const LIMIT = 3; // 0 = all attachments; small number = test run
const TAG_DONE = "annotations_imported"; // replaces TAG on items whose PDFs were all imported cleanly
const SWAP_TAGS = true;
const RECOLOR_HIGHLIGHTS = false; // set all editable highlights to Zotero's yellow
const YELLOW = "#ffd400"; // Zotero's default yellow (underlines are left alone)
const log = [];
let nAtt = 0, nDone = 0, nNone = 0, nErrors = 0;
// 1. all items (parents or standalone attachments) carrying the tag
const s = new Zotero.Search();
s.libraryID = Zotero.Libraries.userLibraryID;
s.addCondition("tag", "is", TAG);
const ids = await s.search();
const items = await Zotero.Items.getAsync(ids);
// 2. collect their PDF attachments
const attachments = new Map();
function addAtt(att) {
if (att && att.isPDFAttachment()) attachments.set(att.id, att);
}
for (const item of items) {
if (item.isAttachment()) addAtt(item);
else if (item.isRegularItem()) for (const id of item.getAttachments()) addAtt(Zotero.Items.get(id));
}
log.push(`Items tagged "${TAG}": ${items.length}; PDF attachments: ${attachments.size}`);
// 3. import
const counts = (att) => {
const a = att.getAnnotations();
return { editable: a.filter(x => !x.annotationIsExternal).length,
external: a.filter(x => x.annotationIsExternal).length };
};
// Highlights (editable ones only) whose color is not yet yellow
const wrongColor = (att) => att.getAnnotations().filter(x =>
!x.annotationIsExternal && x.annotationType === "highlight" &&
(x.annotationColor || "").toLowerCase() !== YELLOW);
let nRecolored = 0, nRetagged = 0;
const okAtt = new Set(); // attachments imported without errors and without leftover external annotations
for (const att of attachments.values()) {
if (LIMIT && nAtt >= LIMIT) { log.push(`(stopped at LIMIT = ${LIMIT})`); break; }
nAtt++;
const parent = att.parentItem;
const label = `${parent ? parent.getField("title") : att.getField("title")} [${att.attachmentFilename}]`;
try {
if (!(await att.fileExists())) { log.push(`MISSING FILE: ${label}`); nErrors++; continue; }
if (DRY_RUN) {
const has = await Zotero.PDFWorker.hasAnnotations(att.id, true, "");
const c = counts(att);
log.push(`${has ? "WOULD IMPORT (PDF has embedded annotations)" : "no embedded annotations"}` +
` | now in DB: ${c.editable} editable, ${c.external} external` +
`${RECOLOR_HIGHLIGHTS ? `, ${wrongColor(att).length} highlight(s) to recolor` : ""}: ${label}`);
if (has) nDone++; else nNone++;
continue;
}
// transfer = true: annotations become normal, editable Zotero annotations and are
// removed from the PDF file in Zotero storage (exactly like the menu item).
const any = await Zotero.PDFWorker.import(att.id, true, "", true);
let recolored = 0;
if (RECOLOR_HIGHLIGHTS) {
for (const a of wrongColor(att)) {
a.annotationColor = YELLOW;
await a.saveTx();
recolored++;
}
nRecolored += recolored;
}
const c = counts(att);
if (any) nDone++; else nNone++;
if (!c.external) okAtt.add(att.id);
log.push(`${any ? "imported" : "nothing to import"} | now in DB: ${c.editable} editable, ` +
`${c.external} still external${c.external ? " <-- CHECK" : ""}` +
`${RECOLOR_HIGHLIGHTS ? `, ${recolored} highlight(s) recolored` : ""}: ${label}`);
} catch (e) {
nErrors++;
log.push(`ERROR: ${label}: ${e}`);
}
}
// 4. swap the tag on every tagged item whose PDF attachments were ALL handled cleanly
if (SWAP_TAGS && !DRY_RUN) {
for (const item of items) {
const pdfs = item.isAttachment() ? [item]
: item.isRegularItem() ? item.getAttachments().map(id => Zotero.Items.get(id)).filter(a => a && a.isPDFAttachment())
: [];
const title = item.getField("title") || item.key;
if (!pdfs.length) { log.push(`TAG KEPT (no PDF attachment): ${title}`); continue; }
if (!pdfs.every(a => okAtt.has(a.id))) { log.push(`TAG KEPT (not all PDFs processed cleanly): ${title}`); continue; }
item.removeTag(TAG);
item.addTag(TAG_DONE);
await item.saveTx();
nRetagged++;
}
}
log.push("");
log.push(`${DRY_RUN ? "DRY RUN. " : ""}Attachments looked at: ${nAtt}, ` +
`${DRY_RUN ? "with embedded annotations" : "imported"}: ${nDone}, ` +
`without: ${nNone}, highlights recolored: ${nRecolored}, ` +
`items retagged "${TAG}" -> "${TAG_DONE}": ${nRetagged}, errors: ${nErrors}`);
return log.join("\n");