Beta won't start

bwiernik
Just updated to the latest beta (5.0.85-beta.6+c3e390eb5) and now Zotero won't start.

This is the startup error message.


ReferenceError: item is not defined
Zotero.Item.prototype.migrateExtraFields@chrome://zotero/content/xpcom/data/item.js:4634:23
Zotero.Schema</this.migrateExtraFields@chrome://zotero/content/xpcom/schema.js:633:21
From previous event:
ZoteroService@file:///C:/Program%20Files%20(x86)/Zotero/components/zotero-service.js:352:7
@chrome://zotero/content/include.js:4:14
  • dstillman
    Sorry about that — should be fixed now.
  • bwiernik
    Still not starting for me:


    There was an error starting Zotero.

    Error: No creator exists at position 1
    Zotero.Item.prototype.removeCreator@chrome://zotero/content/xpcom/data/item.js:1103:11
    Zotero.Item.prototype.setType@chrome://zotero/content/xpcom/data/item.js:505:11
    Zotero.Item.prototype.migrateExtraFields@chrome://zotero/content/xpcom/data/item.js:4632:5
    Zotero.Schema</this.migrateExtraFields@chrome://zotero/content/xpcom/schema.js:633:21
    From previous event:
    ZoteroService@file:///C:/Program%20Files%20(x86)/Zotero/components/zotero-service.js:352:7
    @chrome://zotero/content/include.js:4:14
  • dstillman
    What sort of type: values are you using in item Extra fields?
  • dstillman
    Actually, can you provide a Debug ID for Zotero startup?
  • dstillman
    edited March 17, 2020
    I have a fix for the bug here, but it looks like it would be triggered if you have a type: set to note or attachment on an item with creators. Currently, on a schema update (or, soon, on an item save), it will actually change the item type to the one specified by type: if it's valid, but that might be a little dangerous — here it's actually trying to wipe out the creators, which aren't valid for the new item type. But the goal here is that if you have, say, type: standard and that becomes a valid item type, the item will be updated automatically to use that type, so I'm not totally sure what we want to do.
  • bwiernik
    edited March 17, 2020
    I think in My Library it's just type: dataset and type: pamphlet. Hard to check without Advanced Search in the web library.

    In the Test Items Library group (https://www.zotero.org/groups/2205533)—type: dataset, type: treaty, type: figure, type: pamphlet, type: review-book, type: review, and type: entry are all used.
  • bwiernik
    I don't think I have any type: note or type: attachment.

    Startup debug won't finish submitting. Here is the output log: https://gist.github.com/bwiernik/ceabf6b97ce89455e2eaf1dbba277505

    Looking at the log, it is erroring in https://www.zotero.org/groups/2368853, but that library doesn't have any type: at all.
  • dstillman
    Can you provide another log from the latest beta?
  • dstillman
    @bwiernik: Actually, wait, sorry, don't upgrade yet.
  • dstillman
    @bwiernik: OK, sorry, could you provide a log from the latest version? It should stop with a "Disabled" error.
  • bwiernik
    In 5.0.85-beta.10+8cd04e5d9, startup finished normally.

    Here is the Debug ID for startup: D1606248865
  • dstillman
    edited March 18, 2020
    That wasn't the first startup, though, right? (Or you first ran 5.0.85-beta.9.)

    Are you able to generate a Debug ID for startup — which should fail with an error — with a backup of the database from beta 8 or earlier? If you were getting this error and now you're not, it means that some creators were cleared from at least one of your items due to a type: entry, probably incorrectly, and I'd like to figure out how that was happening.
  • bwiernik
    Okay, have two recent backups.

    This one I think is first through beta.9, so started without errors: D897021134

    This one was before any of these updates (was from 2020-03-13): D144061227

    Also starts without errors using beta.10.
  • dstillman
    edited March 18, 2020
    Hmm, I'm not sure why you're not getting an error in the beta. I don't think it should've happened if you didn't have type: note or type: attachment, and that doesn't appear to be the case.

    Out of curiosity, when you have a line like this in Extra (see item WDZS4WXV in one of your libraries for the actual line), is that a mistake, or is that being used by something else:

    author: Smith || ,, [@abc]

    Because in the latest beta, that will be converted to ,, [@abc] for firstName and Smith for lastName.

    It looks like you also have a lot of creators in Extra that are already valid and being converted into real creators (e.g., 7TKDNA94).

    And there are others where you have a type: for a valid item type, such as UC5CTW9W, which had type: broadcast. That was converted into item type radioBroadcast, with an existing creator type updated to be valid for that item type.
  • bwiernik
    edited March 18, 2020
    Ah, that is just leftover testing data where I was trying to find a way to get a way to get Mayo Clinic [@mayoclinic] in the bibliography (no comma) but Mayo Clinic in the citation. It didn't work. The parsing you describe would be the correct behavior.

    The other valid creators being converted are fine. Those test items were a bunch of slight variations on the same item, and it was easier to generate the various iterations/combinations of creators working with Extra.

    type: broadcast would have been for podcast from before the CSL type mapping was changed from `song` to `broadcast`. Could you perhaps give a list of the other items for that case?
  • dstillman
    Sorry, do you mean other items for type: broadcast or for other cases of type: with a valid item type?

    For the former, I see J2LATH6C, WF85WQLC, NHLHA9AP, and HD6IJ3Z9.

    For the latter, there's a type: article-journal that became journalArticle (3PUSWMZZ) and a type: personal_communication that became instantMessage (EVMGSQDH).
  • bwiernik
    edited March 18, 2020
    The personal_communication migration looks like a problem. EVMGSQDH seems to have lost Archive, Archive Location, and Type/Medium. That migration should probably be to letter to preserve more fields.

    In general, I worry about field loss from these migrations. Is there a way to ensure that fields that would be lost in the transfer are instead moved to Extra?

    Migrating issued is also problematic. Zotero's date parser is less flexible than citeproc-js's (e.g., citeproc-js supports date ranges, BCE dates, uncertain dates), and storing issued in Extra has been a way to get access to citeproc-js's more flexible date parser (e.g., EVMGSQDH had a date range provided as 1930_1967; that renders correctly as 1930–1967 when pulled from Extra, but incorrectly literally as 1930_1967 when entered in date. Perhaps before this rolls out, a more flexible date parser from citeproc-js or BetterBibTeX should replace Zotero's current parser (BBT's is the most flexible)?

    For type: broadcast, this will be a rare occurrence in general I suspect, but I imagine the most common function application would be to remap podcast, so perhaps that migration would make more sense? Items of type podcast would already have been mapped to type: broadcast anyway, so perhaps this function should check the current item type before migrating?


  • dstillman
    I've disabled Extra field migration in the beta for the moment. I actually enabled it by mistake in an unrelated commit, and it clearly wasn't ready to go. Sorry about that.
    The personal_communication migration looks like a problem. […] That migration should probably be to letter to preserve more fields.
    Yeah, that's a bug — that should be to letter. The schema has an ordered list of possible Zotero types, and it's supposed to use the first one.

    We also should check to make sure the current Zotero item type isn't one of the other possible types for that CSL type, and remove the redundant type: line if so. (Is that what you're suggesting re: podcast?)
    Is there a way to ensure that fields that would be lost in the transfer are instead moved to Extra?
    Yes, if Extra specifies a different type, we can move existing fields that would be invalid in the new type to Extra. Alternatively, we could just ignore type: altogether and have people migrate those manually.
    Perhaps before this rolls out, a more flexible date parser from citeproc-js or BetterBibTeX should replace Zotero's current parser (BBT's is the most flexible)?
    @emilianoeheyns: BBT is using the EDTF library @inukshuk wrote, right?
    For type: broadcast, this will be a rare occurrence in general I suspect, but I imagine the most common function application would be to remap podcast, so perhaps that migration would make more sense?
    We can do that. That would mean moving podcast above tvBroadcast in schema.json.
  • bwiernik
    That’s what I was suggesting re:podcast yes.

    Type migration is probably fine? So long as there is no data loss.
  • emilianoeheyns
    edited March 19, 2020

    @dstillman inukshuk's EDTF.js is part of my date parsing strategy, but BBT does a lot more:

    • parses out uncertain/approximate indicators for partially-specified dates and applies that to everything below
    • tries a number of informally-formatted regex recognizers for dates that are not valid EDTF; if that fails
    • try parsing with EDTF.js; if that fails:
    • parses the older EDTF format by converting it into the new EDTF format before feeding it to EDTF.js; if that fails:
    • retry EDTF.js with the results from edtfy; if that fails:
    • try to replace month names found in citation-style-language-locales with their English counterparts, then pass through edtfy, then to EDTF.js; if that fails:
    • try to break it up in two parts with some sensible delimiters, and retry the above on each part; if that succeeds, return it as a date range, if it fails:
    • return the input indicating it should be treated as verbatim (will catch stuff like Forthcoming)

    As you can imagine this date parser isn't particularly fast, but it does what I consider to be the right thing on a wide range of input.

Sign In or Register to comment.