It all went down Saturday afternoon at the Nike 3ON3 tournament at L.A. Live ... where Horry's son was playing.

Witnesses tell us a man affiliated with the opposing team was heckling Horry throughout the game when he approached R.H. and pushed him.

Horry responded with his fists ... throwing several punches at the man. It's unclear if any of the punches landed.

We spoke to Horry who insists he was simply defending himself.

"The guy was trash talking the whole game. He shoved me. Where I'm from, you protect yourself."

R.H. -- a 7x NBA champion -- says he ultimately decided to walk away rather than continue to fight, knowing he was "the only one who was gonna lose in this situation."

Horry claims this isn't the first time the guy has been verbally abusive ... saying he's trash talked his son's team in the past.

A rep for the tournament had no comment on the incident.