Clean-up of auto-complete lists

I'm not sure if this is a bug or not. I believe that the auto-complete list for "Publication" automatically updates itself to include only those titles used in my library. In other words, if I delete my last record with a certain publication title, that title is removed from my Publication auto-complete list. So far so good. However, I believe that the same behavior is not true for the "Creator" auto-complete list. I have deleted all instances of a certain author in my library (verified with an Advanced Search), yet this author's name still appears in my Creator auto-complete list. I would like all of my auto-complete lists to automatically update so I don't have lots of unused items to scroll through. Any help you can provide is greatly appreciated. Thank you for making Zotero such an awesome program!
  • Close and reopen the Zotero pane, and deleted creators should be purged.
  • Hello Dan,
    Thanks so much for your prompt response. I tried closing and reopening the Zotero pane, and I tried quitting out of Firefox entirely and reopening Zotero. In both cases, the creator item was not removed from my auto-complete list.

    Here's a little more info. I created an author like "Dan Stillman". Later, I added a middle initial like "Dan J Stillman". Later still, I edited the middle initial to include a period, like "Dan J. Stillman". Now my auto-complete list contains all three versions, even though only "Dan J. Stillman" is used in my library.

    I'm using Firefox 3.5.3, Zotero 2.0b7.1, and Mac OS X 10.6.1 (Snow Leopard). Any inklings about what's going on? Thanks again.
  • Currently, it only purges after you delete a creator, not if you modify one in such a way that causes the old versions to be left behind (which shouldn't really happen if the creator is only used on one item, but under certain circumstances it might).

    If you create and then delete a creator, and then reopen the pane, it should purge. You can also go to about:config in the Firefox address bar, look for extensions.zotero.purge.creators, and toggle it to true, and then close and reopen the pane, but that certainly shouldn't be necessary.

    This can obviously be improved, though there are some trade-offs involved.
  • Dan,
    The mystery deepens. First, I have several articles by this author, let's say it's Herbert Simon. When I change the creator of one article to "Herbert A Simon", it's clear that a new row is being inserted into the auto-complete table. If I proceed to change all of my Simon articles to "Herbert A Simon", would my original creator record of "Herbert Simon" ever get deleted from the auto-complete table? It does only when I toggle extensions.zotero.purge.creators to 'true'.

    But here's the weirdness. If I now "edit" the author on all my articles to "Herbert A. Simon" (with a period), then the same toggling of extensions.zotero.purge.creators does *not* delete the record "Herbert A Simon" (without the period) from my creator auto-complete table.

    This is not a big deal. Certainly I can live with skipping over "Herbert A Simon" to select "Herbert A. Simon". But how about adding a button somewhere (perhaps in preferences) that would allow the user to drop and rebuild the entire creator auto-complete table as needed. Such a button would be much simpler and would sidestep developers' headaches about when and how to insert and delete from the auto-complete table. Just a thought.

    Much obliged to you for your help.
  • edited September 22, 2009
    1) There's no autocomplete table, per se. It's just the set of creators in the database, which may or may not be linked to items. We likely just need to update the purge flag in a few additional cases, and maybe pick a different purge trigger.

    2) Thanks for the suggestion, but developers' headaches are better than users' headaches.
  • Dan,
    Even more weirdness. I was able to recreate the problem with another author, adding a middle initial, then adding a period. Toggling extensions.zotero.purge.creators to 'true' and reopening the Zotero pane, however, *did* remove the orphaned author (the one with the middle initial and no period). In other words, it seems that the toggling trick cleans up my auto-complete list.

    However, I have my legacy "Herbert A Simon" that persists, despite being orphaned. I created this author record in an older version of Zotero (back in the 1.0 days). Perhaps I will look into removing this pesky record manually via SQL -- with extreme caution, of course.

    Thanks again. I shall henceforth hold my peace on this issue.

    P.S. My problematic author isn't actually Herbert Simon, but because this forum is public, I'm protecting an innocent author by using the late Simon illustratively.
  • If it were truly orphaned, purging should delete it. You certainly shouldn't need to go into SQL, regardless of when it was created. Have you done a Creator search in the Advanced Search window?
  • Dan,
    Yes, I've been using the Advanced Search window to check at every turn. I, too, would have expected a purge to delete it, exactly as with my recent experiment.

    I also thought that maybe it isn't an orphan after all. But a thorough search of my library turned up nothing. Database integrity, checked via the Preferences pane, comes up clean too. I'm stumped.
  • This is likely a different problem but I've been seeing quite similar behavior with tags - mystery tags that are not in the tag cloud but DO show up in the auto-complete list, and can't be deleted.

    Eventually I figured it out. I have auto-tagging turned off, and I previously deleted all auto-tags. The mystery tags are auto-tags that somehow slipped through.

    For example, in the auto-complete list I kept seeing "Sociolinguistics" but it wasn't in the tag cloud, so I couldn't find any items tagged as such. However when I did an advanced search for tag = Sociolinguistics, there it was. The item had auto-tags (icon is orange-ish not blue). I deleted the tags from that item, problem solved. Some mystery tags were identical to my own, so I had items tagged "18th century" but there was one with an auto-tag "18th century" as well.

    I suspect I had auto-tagging enabled for a brief period - the problem items were all created around the same time.
  • Dan,
    I like closure so I thought I would wrap up this thread by writing to say my auto-complete problem has disappeared. I hope the credit is due to your work between Zotero version 2.0b7.1, which I was using at the time, and version 2.0b7.4, my present one. You have my thanks.

    Also, I am greatly impressed with your attentive and genial way of responding to what must be a deluge of user questions, requests, and complaints. CHNM is lucky to have Zotero development in your hands.
  • Dan,

    Let me begin by saying, "I LOVE your work!"

    In my ignorance, I've been modifying incorrect places (publisher cities) instead of deleting and replacing them. Is there a way to purge incorrect ones. I'll know better in the future.

    For creators, you wrote: "it only purges after you delete a creator, not if you modify one in such a way that causes the old versions to be left behind (which shouldn't really happen if the creator is only used on one item, but under certain circumstances it might).

    If you create and then delete a creator, and then reopen the pane, it should purge. You can also go to about:config in the Firefox address bar, look for extensions.zotero.purge.creators, and toggle it to true, and then close and reopen the pane."

    Is there a way purge "places" too. I've searched for the aberrant ones, but can't locate them to delete since they've been modified.

    Thanks
  • In my ignorance, I've been modifying incorrect places (publisher cities) instead of deleting and replacing them. Is there a way to purge incorrect ones. I'll know better in the future.
    There's nothing at all wrong with what you're doing. "Place" is just a field—it's not at all the same as creators or tags, which are separate objects. (I'm not even really sure what you mean by "deleting and replacing" when it comes to Place, but it's not something you need to do.) The autocomplete list for Place is just the set of existing values in the database. Search for the wrong ones and fix them.

    Eventually there'll be batch editing (i.e., find and replace) to make this easier.

    (And an obligatory note: Zotero is the work of many people, not just me.)
  • Thanks,

    The irony is that I can't find the offending records; they don't turn up in any searches. It's not a big deal. I just tend to be a perfectionist sometimes. ;-).

    For example, I want simply "Grand Rapids." It still keeps offering "Grand Rapids Mich" and "Grand Rapids, Mich." even though I don't have any records with that. I got rid of "Grand Rapids, MI" by emptying my trash. Groups?

    Anyway, it's a very minor issue, and I'm certainly happy to live with a few extra "offers." I just thought I'd ask in case I was missing an easy solution. I'm a huge fan of the program and have been trying to get all our students and faculty at Africa International University in Nairobi to start using it.

    I know there are tons of people working on Zotero, but I've seen your name on enough nice forum responses to get the impression that you are a really great guy.
  • As a workaround until batch editing can be implemented, can this be cleaned via the sqlite tables? If I find an orphaned creator in the createrData table and delete this, along with the matching record in the creators table, will the orphaned creator then disappear from the autocomplete list? Or will this corrupt the sqlite database? Same with places or publishers (though it isn't clear what tables these are in)? What about tags? Can I just delete problem ones or modify tag text?

    (NB: there is a very nice sqlite extension for firefox).
  • If you follow the steps described by benbyerly above, orphaned creators should be purged okay for you. That's a much safer way to go than tampering with the database directly. Individual database rows are inter-connected with other rows in other tables in ways that are not immediately obvious, and sometimes quite complex. Direct modifications to the database is likely time-consuming in the long run, and can be kind of risky as well.
  • Well, I tried the trick in about:config and like benbyerly and tdquinn above found, it doesn't purge. I even shut down and reopened Firefox.

    I recently went through all my own papers to make my name identical in all so as to avoid disambiguation problems. I laboriously changed my name to C Michael Barton (no period) in all cases.

    Yet, I still get C. Michael Barton (with period) offered in autocompletion and more mysteriously, if I do a search on "C. Michael" (with period), a bunch of my papers are found--I think all the ones I changed from C. Michael to C Michael. So some of my papers are still linked to the unused version of my name and that version is still in the autocompletion list--even though it is not used in any of my records (at least not obviously).

    Just to note, when I changed from "C. Michael" to "C Michael", I deleted the entry in the first name field and then used autocompletion to pick "C Michael".

    As the other folks noted, this is not a huge deal, but it can be problematic for disambiguation in citations and with the lack of batch editing, it is laborious to change.

    Thanks for your work on this great program.
  • edited December 30, 2010
    Just stabbing in the dark, but you have emptied the trash? Not sure if that's relevant to this, but just in case. (Oops, I see there are references to that in the thread, so no need to mention.)
  • Sometimes a stab in the dark is on the money. In this case, I didn't empty the trash but did look in it and there is nothing related to Zotero. In most cases, purging the sql data tables should not send anything to the trash I would think.

    Michael
  • The Zotero trash, not your OS trash.
  • OK. But it made no difference--even though I quite firefox and reopened it.

    This entry is still in the createrData table. And I can find its itemID in the itemCreators table too. But I can't figure out how to see which records these are to see what is going on.

    Michael
Sign In or Register to comment.