Remember, stars like T.I., Soulja Boy and Spike Lee called for a Gucci boycott over a sweater that resembles blackface ... with T.I. calling the fashion item "racist and disrespectful."

But Broner seems to think the boycott is a massive overreaction -- and on his way into 1 OAK in L.A. he told us, "You can wear whatever you want to wear."

"I don't care! It's just a sweater! Why's everyone getting so bent out of shape about it?!"

Broner also said he's NOT wearing Gucci though -- explaining he's more of a Dolce & Gabbana kind of guy.

The decision seems to be motivated by fashion, not social justice.

AB's pal, Floyd Mayweather, also refuses to boycott Gucci -- in fact, he went on a MASSIVE shopping spree at the Gucci store in Bev Hills this week and proudly proclaimed that NOBODY tells him how to feel or what to wear.