You should be able to remove Growl. Let me look around their site a little…
Ah ha, according to the
Removing Growl page, there's an "Uninstall Growl" script on the Growl disk image. Maybe download another copy from their website, then open it and double-click on that script?