Upgrade StorageFeature request: reject all changes to Zotero fields
At present, during refresh, if Zotero detects that a manual change has been made to a field, it shows a pop-up asking if the user wants to keep the change, warning that saying 'yes' will stop Zotero changing that field in future, but nevertheless giving 'yes' as the default.
In my case, I want to always say 'no'. Any changes to my fields are unintentional, but can occur due to being caught in global replacements, or index automarking, or just be changed accidentally by slippery fingers.
[Aside, for those unfamiliar with index automarking: it is not possible to stop automarking marking (changing) Zotero fields. It just matches strings in the whole document. A string like "Hegel", say, could well be found in a Zotero reference. It marks text in WORD-generated tables too, such as the Table of Contents, but the marks are silently removed next time the table is refreshed. I have 'update all fields' set in my print options. For referencing, I use the MHRA3 standard, which puts all references in footnotes. My workaround is, after indexing the document, to remove automarks from all footnotes. That will cover all Zotero fields, but also footnote parts not controlled by Zotero that I might want to mark. This trick would not help anyone using an in-line referencing standard like Harvard. Anyway it adds a confusing and easily-forgotten step to the indexing process. The price of forgetting is having to press the "no" button 500 times during refresh, or a force quit with possibly unforeseen consequences. It would be nicer just to automark everything and have Zotero take out the marks in Zotero fields. End of aside].
So, whether indexing or not, I don't want to keep any changes that have been made to Zotero fields by any method other than Zotero itself. I want Zotero to stay in control. If I want to change a citation, I will do it in Zotero. I don't expect to do any real manual changes to Zotero citations at all, but if I must, I will do it right before publishing after unlinking from Zotero.
I also don't want refresh to be interrupted. I would like to be able to start refresh and then leave the computer to process it while I do something else. I don't want to sit staring at the screen for an hour in case a dialogue pops up.
My request, then, is that there be a new setting in preferences that I can tick to say I always want to reject all changes to Zotero fields, That would allow Zotero to retain control and refresh to run unattended without interruption.
In my case, I want to always say 'no'. Any changes to my fields are unintentional, but can occur due to being caught in global replacements, or index automarking, or just be changed accidentally by slippery fingers.
[Aside, for those unfamiliar with index automarking: it is not possible to stop automarking marking (changing) Zotero fields. It just matches strings in the whole document. A string like "Hegel", say, could well be found in a Zotero reference. It marks text in WORD-generated tables too, such as the Table of Contents, but the marks are silently removed next time the table is refreshed. I have 'update all fields' set in my print options. For referencing, I use the MHRA3 standard, which puts all references in footnotes. My workaround is, after indexing the document, to remove automarks from all footnotes. That will cover all Zotero fields, but also footnote parts not controlled by Zotero that I might want to mark. This trick would not help anyone using an in-line referencing standard like Harvard. Anyway it adds a confusing and easily-forgotten step to the indexing process. The price of forgetting is having to press the "no" button 500 times during refresh, or a force quit with possibly unforeseen consequences. It would be nicer just to automark everything and have Zotero take out the marks in Zotero fields. End of aside].
So, whether indexing or not, I don't want to keep any changes that have been made to Zotero fields by any method other than Zotero itself. I want Zotero to stay in control. If I want to change a citation, I will do it in Zotero. I don't expect to do any real manual changes to Zotero citations at all, but if I must, I will do it right before publishing after unlinking from Zotero.
I also don't want refresh to be interrupted. I would like to be able to start refresh and then leave the computer to process it while I do something else. I don't want to sit staring at the screen for an hour in case a dialogue pops up.
My request, then, is that there be a new setting in preferences that I can tick to say I always want to reject all changes to Zotero fields, That would allow Zotero to retain control and refresh to run unattended without interruption.
We could offer a cancel button to abort the document update, but for 99% of the cases you only get a few of these dialogs at the time, and we don't want to confuse users with yet another button, especially since if you'd be able to abort the update you'd likely just get the dialog again. If you get a lot of these dialogs, bar your exceptional use case, it usually means greater document corruption, which also shouldn't be ignored.
We could consider including an Always Reject hidden pref, but I'm not sure if even that is a good idea, since it would prevent you from spotting doc corruption if it ever occurred.
For your use case, you can kill/close Zotero safely when you get the popup, perform whatever changes you need to the doc and attempt the refresh again.
I did not suggest 'cancel' in this feature request, but in my parallel request to show the progress of refresh. For that, there would have to be a small open window of some kind open throughout refresh (not a pop-up) but please continue that discussion in the appropriate thread. https://forums.zotero.org/discussion/86458/feature-request-refresh-progress-indicator
The 'Always Reject' preference setting is my preferred solution. I am not sure why you are calling it 'hidden'? I suggest it be in the Zotero Document Preferences pane, easily accessible from WORD, where one selects the citation type, footnotes or endnotes, whether they are created as fields and whether they are automatically updated. I don't know how hard that is technically, but for the user it is a logical place.
I am not sure what you mean by 'document corruption'? I cannot think of any corruption that I want Zotero to spot for me. If Zotero is unable to refresh a citation due to a document problem, then it can log it and report it to me at the end of the refresh. The citation itself would be left unrefreshed with dotted underline, as usual. Otherwise, with the 'Always Reject' preference on, I WANT it to remove everything inside a Zotero field and fill it correctly. Anyone who wants to see if a field has changed would turn off the 'Always Reject' preference.
Your last comment about kill/close is not applicable because there would be no popup, but perhaps you meant this for https://forums.zotero.org/discussion/86458/feature-request-refresh-progress-indicator?
I am aware that I am writing this from a particular use case. I invite comments from other users who have documents with several hundred citations and therefore use manual refresh. (It does not matter if you are not using indexing, just ignore my 'aside').
1. They've changed a citation in the document, when they should have updated the item metadata or the cite itself instead of editing it in the text and those edits shouldn't be removed silently, but rather be done appropriately.
2. Something has gone corrupt in the document and Zotero thinks that the citation was changed, when in fact it hasn't (this isn't that common, but it does happen especially as Word versions change and minor changes behind the scenes are introduced).
In the 1st scenario there will only be a few popups rarely, so there's no need for a reject-all feature, and for the 2nd scenario it is imperative for the user to see this and go troubleshooting. We do not want to have an easily accessible way to disable these warnings, because it is prone to creating much bigger issues for our users and eventually ourselves when we need to help in troubleshooting, which is why I'm only considering this as a hidden pref. This is purely because it's too easy to set something like this and forget, even if you only want it for a single operation. Moreover, we don't want to overwhelm new users with too many options for basic Zotero operations, and there shouldn't be any unclear corner-case checkboxes in something as prominent as the doc prefs dialog (which we have gone through some lengths to simplify and declutter a few years ago). The best option here would obviously be a "Reject All" button for an individual refresh, but we don't have that as an option right now. This would be great in an ideal world, but not all corruption is detectable automatically. Each user knows best what edits they've done to the document since the last time they cited and what citation updates to expect. If Zotero retrieves field codes from word and for some reason they do not match what's expected, it cannot automatically know whether those changes were user-made or part of some unexpected word processor behavior. My last comment was regarding your aside in the intial post:
As always, if there's major interest from other Zotero users/the community expressed on this thread (or other forum threads) we can reconsider some version of a reject all changes option when refreshing docs.
1. The "reject all" preference could be turned on or off, so anyone wanting to be alerted to changes can turn it off. That would also be the default. It would always be on during automatic refresh.
2. I gave examples of scenarios where a large number of changes can occur: global replace and indexing. I also showed that workarounds are not entirely satisfactory.
3. In the case of a corruption (which in 7 years I have never experienced in the way you describe), an error log would be more appropriate. I don't know what I could do with a popup anyway.
4. If the "reject all" preference is off by default (and the automatic refresh on) new users would experience the system as it is now and not need to be aware of the new option. "Reject all" is only needed with manual refresh, and that is only needed with documents with a large number of citations. Users with a large number of citations are probably not beginners.
I am not saying this as a hypothetical either. We have made some changes to the Word for Mac plugin recently to fix Word updates breaking with autoindexing that you have reported. The changes had to do with the order in which we update citations. Since then we have had a couple of users report getting the dialog for changed citation text where they hadn't done any changes and all the citations had effectively "shifted" by one position. This is obviously very bad and we still haven't managed to figure out whether it's part of the recent update or unrelated (e.g. the user had edited the document while Zotero was updating it, which could also cause something like that). If those users also had an option to automatically reject all citation changes checked, they wouldn't have even known that something had gone wrong, and if it had happened on their final refresh of the document, they could even submit it in such a form (being used to Zotero never doing the wrong thing).
This isn't an isolated event either. We troubleshoot an user with either some sort of corruption or editing citations in text in the unintended way that manifest as multiple unexpected "citations changed" dialogs about every month and these troubleshoots are usually long and confusing for our users. We cannot really rely on thinking "advanced users will do the right thing" when designing the software, because that's simply not the case for even the developers of software that usually know it in-and-out. This is doubly so for a feature that can have very time-consuming consequences. Everyone forgets, no matter how advanced they are.
So to summarise the counter-point:
1. We do not want to clutter the UI for any, but especially new, users.
2. More than a few "changes to citations" buttons is extremely rare and if a user gets that most of the time it is related to document corruption.
3. Given 2., we do not want to prominently add an option to reject all changes which will solve the extreme corner case and could potentially silently hide Zotero writing wrong citations for many more users.
Once again, if there's more vocal support for the feature (i.e. we decide this isn't such an extreme corner case as I believe it to be) we will reconsider it.