Unfilter?
 

Currently, once you've constructed a filter, you have 3 spots in the ribbon to change to get rid of the filter (and maybe you have to empty the searchbar as well). What about just having an 'unfilter' button? I know that remembering filter is in the work, which will probably solve the problem anyway.