Weird corruption from re-import of database

procrastinator

Don't really know, how it came to be, but the majority of my Zotero items (the older ones) have corrupted PubMed entry links.

Rather than having a normal PubMed entry link and a proper publication date (e.g. Nov 1979), most older entries give the publication date like this

2005-01-01T00:00:00

and has replaced the normal PubMed entry link with a note with this format

aug, PubMed ID: 15677703

Links to my externally stored PDF's are fine, but I'll need a way of repairing the entries, hopefully without having to re-add all my references manually.

I've been looking at the sqlite database using SQLiteDatabaseBrowser, but I cannot find out the format of a PubMed entry link from the info visible there.

adamsmith

Are you positive these ever looked different? It'd be incredibly unlikely for a database to get corrupted in a way that an attached link gets converted to a note.

What do those say in the library catalog field?

Did you import your database from a different reference manager?

procrastinator

I am positive the corrupt ones used to be PubMed entries, and that that's where the notes containing e.g. "nov, PubMed ID: xxxxxxxx" come from.

The newly correct entries have "PubMed". Most of the corrupted entries have a blank library catalogue field, A few have someting else in them, such as "DOI.org (Crossref)"

I have built the database i Zotero from scratch, but abandoned it a few years back trying out Citavi instead - but I haven't made a data round trip. I simply re-downloaded my database from the Zotero cloud as I remember it.

dstillman

You're just misremembering this — there's no way these changes happened automatically. "DOI.org (Crossref)" appearing in the Library Catalog field instead of "PubMed" just means the item was saved via DOI rather than from PubMed. That's all. This isn't database corruption.

It's possible you corrected some things locally when you used Zotero previously but never synced those changes, so when you set up a new version it downloaded the older versions from the online library. But these are not changes that were made to your database.

dstillman

@procrastinator: The items in question were all both created and last modified on a specific date in May 2020, and appear to have been created via an import — they all have the exact same timestamp, and none of the parent items have Library Catalog values. So you imported these into Zotero via a file, and that's the data you ended up with. They were never in a different form, and this isn't database corruption.

procrastinator

Thanks for looking into this.

I have misremembered, because I have now found a citavi file (cv5) dated May 3rd 2020, and importing from its BAK-file as detailed elsewhere on this site, I get similarly badly formatted results.

Setting aside the definitory niceties for a brief moment, would you have any suggestions for me on how I might correct these faultily imported items in a less hurtful way than having to manually re-import them all via PubMed and merging them?

adamsmith

Do they have PMID: 123456 in the Extra field?

procrastinator

No, not the faultily imported ones. Only my recent additions have that.

The Extra field in my bad items contain "Type: JournalArticle" instead. I only have the PMID number in a notes field along woth the month of publication as I showed above.

procrastinator

Because of the upcoming v7, I recently had another look at my corrupted entries. They are mostly still present, and I’ve not found any really good way of correcting them. If I re-add the PubMed entry, I get a separate citation that I have to merge with my existing one with the link to the locally stored source file.

If a new solution/plugin for mass-editing has emerged, I’d appreciate a pointer.