Upgrade StorageLooking to store plugin-specific, syncable reference data
Hi,
I maintain the better-bibtex plugin, and I'm looking for a clean (or at least as clean as possible) way to store data for references in a way that doesn't disrupt regular Zotero usage or other plugins, and that syncs cleanly and reliably. I also need a way to quickly search through this data, preferably by SQL.
I've tried raising the question on zotero-dev, but did not receive a response, so I'm hoping I'll find more activity here.
Here's what I do now: I stick my data at the end of the "extra" field of the reference. Pro: every reference type has this. Con: it's not easily searchable, at least not really reliable, as the extra field can contain anything the user (or other plugins) pleases.
I could solve the searchability by parsing out the data on every reference save and sticking it in a temporary table. This seems a bit fragile, as I would depend on this derivative data to always be up to date, and the data I would rely upon would not itself sync.
Attempt #2: I tried to hide my data in records I created in the fulltext tables for the (non-attachment) item, but this data does not seem to sync over.
Attempt #3: I tried sticking my data in an attachment to the item in an analogy of the Google Books link attachments; this works reasonably well, but it means that the twisty in the itemTreeView loses its indication status as "has attachment", which I think pollutes the UI too much.
Attempt #4: Putting my metadata in the "tags" table with a new "type" number. That really screwed things up during sync, to the point I had to abandon the account I used back then.
Attempt #5: sticking the data in annotations/highlights. These do not seem to sync (anymore, perhaps).
Option #6: Putting the metadata in regular tags, optionally filtering them out of the tagcloud. This would have the added benefit of making my metadata searchable, but I've been strenuously advised *not* to stick extra data in the tags table for performance reasons.
Option #7: putting my metadata in the relation table with a custom "predicate". I'm not too keen on just trying this out without feedback from someone who knows the Zotero sync infra after my experience with attempt #4.
Option #8: adding new field/itemTypeField records. Not going to do this without discussion of the ramifications, also because attempt #4.
So far it seems only #1 and #3 are safe. Any options I have missed that I should be looking into before I invest too heavily in either #1 or #3?
I maintain the better-bibtex plugin, and I'm looking for a clean (or at least as clean as possible) way to store data for references in a way that doesn't disrupt regular Zotero usage or other plugins, and that syncs cleanly and reliably. I also need a way to quickly search through this data, preferably by SQL.
I've tried raising the question on zotero-dev, but did not receive a response, so I'm hoping I'll find more activity here.
Here's what I do now: I stick my data at the end of the "extra" field of the reference. Pro: every reference type has this. Con: it's not easily searchable, at least not really reliable, as the extra field can contain anything the user (or other plugins) pleases.
I could solve the searchability by parsing out the data on every reference save and sticking it in a temporary table. This seems a bit fragile, as I would depend on this derivative data to always be up to date, and the data I would rely upon would not itself sync.
Attempt #2: I tried to hide my data in records I created in the fulltext tables for the (non-attachment) item, but this data does not seem to sync over.
Attempt #3: I tried sticking my data in an attachment to the item in an analogy of the Google Books link attachments; this works reasonably well, but it means that the twisty in the itemTreeView loses its indication status as "has attachment", which I think pollutes the UI too much.
Attempt #4: Putting my metadata in the "tags" table with a new "type" number. That really screwed things up during sync, to the point I had to abandon the account I used back then.
Attempt #5: sticking the data in annotations/highlights. These do not seem to sync (anymore, perhaps).
Option #6: Putting the metadata in regular tags, optionally filtering them out of the tagcloud. This would have the added benefit of making my metadata searchable, but I've been strenuously advised *not* to stick extra data in the tags table for performance reasons.
Option #7: putting my metadata in the relation table with a custom "predicate". I'm not too keen on just trying this out without feedback from someone who knows the Zotero sync infra after my experience with attempt #4.
Option #8: adding new field/itemTypeField records. Not going to do this without discussion of the ramifications, also because attempt #4.
So far it seems only #1 and #3 are safe. Any options I have missed that I should be looking into before I invest too heavily in either #1 or #3?
-
dstillmanedited July 4, 2014This belongs on the dev list, not here. If you want to transfer any of this to a follow-up post in your dev list thread, do so — I'll be deleting this post from here.
This discussion has been closed.