Albert Haynesworth and his baby mama, Brittany Jackson, went nuclear on each other during an argument about Albert's alleged cheating ... and it got so bad, cops were called. 

TMZ Sports broke the story ... Albert claims he's the victim of domestic violence -- alleging his baby mama abused him physically and emotionally during their rocky relationship. 

We did some digging and found multiple incidents with couple involving police -- including one altercation in Ft. Lauderdale, Florida in which the ex-NFL star says Jackson was berating him in front of their 3-year-old son. 

The officer who responded to their home said, "Brittany told me that she was upset that Albert has been talking to other women and they had an argument over his infidelity."

Brittany told the cop the alleged cheating was "an ongoing issue between them."

Brittany denied physically attacking Albert. Albert denied physically attacking Brittany. 

Albert admitted they did argue over "claims" of infidelity -- and he tried to ignore her but she wouldn't drop the issue. Albert says he just wants to be left alone. 

The cop advised both parties to cool off. No arrests were made. 

As we previously reported, cops were also called to a home in Tennessee the very next month -- where Haynesworth claims Brittany kicked him in the groin twice during an argument. 

We reached out to Brittany for comment. So far, no word back.