Dave was out with his pal, John Mayer, Wednesday night -- hitting Craig's and Delilah in WeHo -- when we asked him about Olivia calling him "tone deaf" for his jokes about Louis C.K. and sexual harassment. If you missed it ... Dave said Louis' alleged victims had a "brittle spirit," among other things.

The comedian had a very direct and positive response for Olivia.

Despite the criticism over his latest Netflix stand-up specials, Dave was still swarmed by fans and paps at the end of the night. As he said, "I'm very famous."

Fun Fact: John, who toured with Dave, didn't say anything about Olivia's criticism -- but happens to have a pretty cool song about Olivia. Wilde though, not Munn.