What format to use when exporting bibliography for permanent storage in rep.

athelas

Hi,
I am just finishing my dissertation and thank you thank you Zotero. We have been together since the early days, through many bugs, smaller or bigger catastrophes, last moment surprises, but that all is forgotten and all that will be remembered are the good times. So, this is also cudos to developers for remarkable stability improvement over time. I feel quite confident about the Zotero not letting me and my students down moments before submitting. My paper has 20+ pages of references, all in live citations in 200 pages long document in Word. Updating the document, even inserting citation takes some time but it works!

QUESTION BEGINS HERE>>>
I am trying to make a difference and besides attaching full datasets to my paper I would like to include my bibliography also as a digital attachment (separate file) to my dissertation.
1. Which of the many formats offered in Export Library would you recommend to use for this purpose? I'd like that people could import the bibliography in their managers if they want. (I don't want to export the full texts stored along because of copyright issues.)
2. Is there a smart way how to export to RDF/etc... format references in the document (as not all references in my Zotero collection dissertation folder made it to the paper?


Thank you for your advice and for all the great work!

bwiernik

You can extract the cited items from your document using https://rintze.zelle.me/ref-extractor/ and export them. BibTeX format would probably be the most flexible for authors using different systems.

emilianoeheyns

I'd say keep them in both BibTeX and CSL-JSON for maximum compatibility -- internally, Zotero, Mendeley, Pandoc etc all use CSL-JSON and all make different (necessary) compromises when they convert from bibtex to csl-json for processing.

athelas

Oh, amazing people!

@bwiernik excellent tool, saved me a ton of time, thank you!
@emilianoeheyns thanks for the advice, I will go with that!

adamsmith

(FWIW, I'd have gone with RIS and CSL JSON instead; I think BibTeX's lack of standardization is a big downside for storing, but that's a matter of taste and also of which tools you most expect to be used for importing)

Another innovative solution you could consider is to simply create a public Zotero group with your dissertation bibliography. That allows export in any format that Zotero supports.

emilianoeheyns

I'm working on a tool that will populate a Zotero collection based on the ref-extractor results

emilianoeheyns

@adamsmith I agree but then I'd say csl-json, RIS, *and* bibtex.

Rintze

I'm working on a tool that will populate a Zotero collection based on the ref-extractor results

Doesn't Zotero already automatically create a new collection as long as you don't import the CSL JSON via the clipboard import function?

emilianoeheyns

Yep, it will, but I'm working on something that allows scanning the docx from Zotero and just creating a collection from the items already in your lib, it doesn't actually do anything except look at the item ID and add it to a new collection. ref-extractor does more (gets you the actual CSL) but would also create duplicates on import. What I'm doing is hitching a ride on something else I'm working on, also based on ref-extractor -- a way to replace the in-text citation fields with the BBT citekeys to ease porting a document to latex. Between that and the existing aux-scanner it was pretty much free for me to do.

adamsmith

What I'm doing is hitching a ride on something else I'm working on, also based on ref-extractor -- a way to replace the in-text citation fields with the BBT citekeys to ease porting a document to latex

I hope you're not putting too much work into that? I figured we'd get that as a freeby if we add citekey in Zotero and as a CSL variable in hopefully not too distant future.

emilianoeheyns

It isn't much effort, but good point -- a standard style could just do that at that point.

Rintze

Yep, it will, but I'm working on something that allows scanning the docx from Zotero and just creating a collection from the items already in your lib, it doesn't actually do anything except look at the item ID and add it to a new collection. ref-extractor does more (gets you the actual CSL) but would also create duplicates on import.

You're aware of the new feature in Reference Extractor to select existing items in your Zotero library? You don't get duplicates with that option.

What I'm doing is hitching a ride on something else I'm working on, also based on ref-extractor -- a way to replace the in-text citation fields with the BBT citekeys to ease porting a document to latex.

What would be really nice is if you could update the item IDs in the in-text citation fields with those of the (duplicate) imported items. Since Zotero doesn't retain items IDs on CSL JSON import, the imported items currently aren't connected to the items cited in the extracted Word document.

emilianoeheyns

I was not aware of this -- I did not grasp it could be used this way. Cool! @athelas, this seems like your ideal option, combining points of @adamsmith and @Rintze; scan your docx using the ref-extractor, select those items in Zotero, copy them to a public group, boom, done, and anyone who wants them can just sync in that Zotero collection.

@Rintze I don't fully understand the 2nd point. After import, the duplicated items would have new item IDs -- how would I tie these to the original ones?

Rintze

I was not aware of this -- I did not grasp it could be used this way.

It's a new feature introduced last month, made possible by an extended protocol handler introduced in March in Zotero 5.0.61: https://www.zotero.org/support/changelog#changes_in_5061_march_9_2019 and https://twitter.com/danstillman/status/1141812891542347781

I don't fully understand the 2nd point. After import, the duplicated items would have new item IDs -- how would I tie these to the original ones?

Right. You'd first have to figure out what the old and new ID is for each imported item, and then replace the old IDs with the new IDs in the Word document for all in-text citations.

Matching up items between the import file and the imported items could be done by comparing their CSL JSON representation (assuming Zotero doesn't tweak it). Alternatively, Reference Extractor could easily add a separate tracking ID upon extraction, e.g. in Zotero's Extra field. Then you could match on that.

emilianoeheyns

I'm trying to understand the use-case still. Is it so that if someone hands you a docx with items, you can take over editing it by importing its references? That seems useful.

If that's the case, the ideal path would keep the item URIs already present in the citations somewhere so that after import the match can be done deterministically. The ref-extractor could perhaps optionally add them to the extra field; a matcher running either inside Zotero or outside with a mapping from extra-URI to actual-URI (which could be generated from inside Zotero) could then very easily swap them.

Alternately it could all be done inside Zotero in one go -- user hands a docx to Zotero (through a plugin or not), which scans for items the same way ref-extractor does it, imports them, and modifies the docx as it goes.

edit: ah I see you just thought of the same thing.

adamsmith

(and yes, that's exactly the use case; can be quite the hassle if you don't plan on collaborating on a doc from the start and therefore cite from your personal library and then want to collab)

emilianoeheyns

Just out of curiosity -- wouldn't it be easier for those users to sync to a group and be done?

edit: ah no wait -- personal library, and if you copy to a group lib, keys would change.

emilianoeheyns

How disruptive would it be to actually move the items from personal to group? In the Zotero DB it'd be as simple as changing the libraryID for those items but it'd probably mess up sync something bad.

fbennett

This sounds a lot like the Jurism cite extraction feature. It is indeed useful for collaboration. We use it for thesis supervision all the time, it saves a lot of hassle.

emilianoeheyns

It sounds exactly like it. Wouldn't it give users the best experience if this was either ported to Zotero or extracted to a plugin?