Upgrade StorageZotero as a collaborative corpus annotation tool
Hello,
Zotero is a great tool to handle PDFs. It contains almost every corpus annotation tool's functionalities, including Label Studio, Argilla, etc.
-> The most important missing feature is the ability for a group administrator to customize the annotation panel for the entire group.
For example, instead of displaying "Red" as the label for a red highlight, the interface could display a user-defined label such as PERSON. The administrator should also be able to define custom colors and labels and make them available to all group members.
This would allow research teams to define a shared annotation scheme directly in Zotero.
The following plugin explored customization of PDF annotations, but this functionality does not appear to extend to Zotero Groups:
https://groups.google.com/g/zotero-dev/c/ezD5963yApQ
Other features:
- Annotation guidelines: Provide a way to display annotation guidelines and instructions while annotating. Ideally, users could open two different documents simultaneously in a horizontal or vertical split view.
- Document assignment and access control: Allow group administrators to assign documents to specific members and restrict access so that members can only see the documents assigned to them.
- Independent annotation mode: Allow group administrators to enable a mode that hides other members' annotations from all annotators, preventing them from influencing one another's decisions. The administrator should be able to control this setting for the entire group. Related discussion: https://forums.zotero.org/discussion/102696/highlighting-filter-in-group-library
- Structured annotation export: Allow administrators to export all annotations, including the highlighted text, color or label, page number, comments, document metadata, and annotator identity. Annotations made independently by different users should remain distinct, even when they cover the exact same text. Support for standard corpus annotation formats would be particularly valuable.
Inter-annotator agreement metrics: As a longer-term goal, provide tools to compute agreement metrics such as Cohen's kappa or Krippendorff's alpha, helping researchers assess annotation consistency and identify disagreements.
I believe these additions could make Zotero a compelling alternative to dedicated corpus annotation platforms, especially for research teams working with scientific literature.
They would build on Zotero's existing strengths rather than require researchers to adopt an entirely separate annotation workflow.
Cheers,
Zotero is a great tool to handle PDFs. It contains almost every corpus annotation tool's functionalities, including Label Studio, Argilla, etc.
-> The most important missing feature is the ability for a group administrator to customize the annotation panel for the entire group.
For example, instead of displaying "Red" as the label for a red highlight, the interface could display a user-defined label such as PERSON. The administrator should also be able to define custom colors and labels and make them available to all group members.
This would allow research teams to define a shared annotation scheme directly in Zotero.
The following plugin explored customization of PDF annotations, but this functionality does not appear to extend to Zotero Groups:
https://groups.google.com/g/zotero-dev/c/ezD5963yApQ
Other features:
- Annotation guidelines: Provide a way to display annotation guidelines and instructions while annotating. Ideally, users could open two different documents simultaneously in a horizontal or vertical split view.
- Document assignment and access control: Allow group administrators to assign documents to specific members and restrict access so that members can only see the documents assigned to them.
- Independent annotation mode: Allow group administrators to enable a mode that hides other members' annotations from all annotators, preventing them from influencing one another's decisions. The administrator should be able to control this setting for the entire group. Related discussion: https://forums.zotero.org/discussion/102696/highlighting-filter-in-group-library
- Structured annotation export: Allow administrators to export all annotations, including the highlighted text, color or label, page number, comments, document metadata, and annotator identity. Annotations made independently by different users should remain distinct, even when they cover the exact same text. Support for standard corpus annotation formats would be particularly valuable.
Inter-annotator agreement metrics: As a longer-term goal, provide tools to compute agreement metrics such as Cohen's kappa or Krippendorff's alpha, helping researchers assess annotation consistency and identify disagreements.
I believe these additions could make Zotero a compelling alternative to dedicated corpus annotation platforms, especially for research teams working with scientific literature.
They would build on Zotero's existing strengths rather than require researchers to adopt an entirely separate annotation workflow.
Cheers,