The entire verb panel is recreated when refreshed #2520
Description
Right now RefreshVerbs() removes every UI element and recreates them. This leads to some adverse effects that had to be worked around in #2513, and it would probably be better to just remove the buttons that are no longer in use.