Sources with direct knowledge tell TMZ ... the TLC singer-songwriter isn't giving a second thought to a new story that came out today, where Usher revealed he proposed to Chilli way back in the day -- while they were still dating -- and instead ... she's enjoying a vacay in Jamaica with her new man, celebrating his 44th birthday.

We're told Chilli and Matt are down in Sandals Dunn’s River where they hit up local hot spots, enjoying the island's natural beauty and even partaking in an intimate candlelight dinner. Check out these videos, obtained by TMZ, that shows 'em cuddling, laughing and even smooching ... clearly, these two are very much in love.

More importantly, her mind is a world away from all this Usher stuff. Speaking of which, our sources tell us that Chilli, generally, takes marriage very seriously ... and she's not just going to hop into a commitment like that until she's totally convinced she and her partner are both ready for marriage.

So, even when it comes to Matt -- who she's been with for a couple years now -- it's pretty obvious she takes her time and won't just say yes to anyone.

Clearly, she didn't feel Usher was the one to say "I do" ... 'cause he told People he got his heart smashed by Chilli two decades ago after she rejected him when he asked for her hand in marriage -- something he said gave him trust issues when it came to women.

It seems Usher's over those issues though ... as tied the knot with his longtime partner Jennifer Goicoechea on Super Bowl Sunday with his mother acting as a witness.

As for Chilli ... she and Matthew Lawrence are pretty dang serious at this point -- with Chilli telling us last March that she was really hoping to marry ML ... but they ain't there just yet.