Oscar just broke his silence on the lawsuit filed by David Gonzalez -- famously known as "Tattoo" -- who accused De La Hoya of doing some seriously messed up stuff in his role as head of the company.

In his suit, Gonzalez -- who was hired to do announcing and PR stuff for Golden Boy -- claims Oscar would force him to run personal, non-work related errands ... sometimes while Oscar was "highly intoxicated and under the influence of various substances."

So, when we saw Oscar at LAX over the weekend, we asked him for his response to the accusations ... and that's when the boxer swung back.

"It's all frivolous and no merit," De La Hoya said ... "No merit, whatsoever."

Despite the scandalous allegations, Oscar says he doesn't feel betrayed by his former employee ... he just thinks it's all bogus.

After the legal stuff, Oscar talked about his #1 fighter, Canelo Alvarez, and explained why he's still hoping to get Canelo a rematch with Floyd Mayweather.

"That's the fight everyone wants to see, that's for sure," Oscar said.

The original fight went down in 2013 -- and Floyd won by majority decision.

"When Canelo fought Mayweather, he was young. He was inexperienced and Floyd Mayweather toyed with him. That's the truth."

"But, it might be a different story after a few years, so we'll see."