He's not joking -- telling TMZ Sports, "Let's be real. Canelo moving up to my weight? You ain't gonna win that fight."

Diaz says he's sparred with top-flight boxers like Andre Ward, and honestly believes he can compete and defeat Canelo, widely regarded as one of the top pound-for-pound fighters in the world.

"You're just basically one of them little guys I spar before I fight," Diaz says ... adding, "Nothing personal. Motherf**ker don't speak English so he ain't knowing what I'm saying anyway."

So, we went to Oscar -- Canelo's promoter -- to ask if there's a chance he'd make it happen.

Oscar ain't exactly saying "no."