I almost never agree with the language police (hereinafter referenced as the "LP"), but I have to admit that literally in a nonliteral sense usually makes my left knee go numb. (I also have to admit that I used reference as a verb hoping that any LP reading this would get annoyed.)
