Scott spoke about the incident at Sixers Media Day on Monday ... admitting he shouldn't have fought, but makes it clear the fans at Lincoln Financial Field started it.

“I definitely should've walked away before it got to that point," Scott told the media. "I’m a professional, you know, I gotta be the bigger person, but once you keep going ... he was poppin’ hella sh*t, he was going off.”

Initially, there were rumors that Scott went off because he thought a coffin at the game was to mock late Redskins star Sean Taylor (the fans claim it wasn't). Scott didn't get into specifics on the slur that sparked the fight ... but it clearly pissed him off.

“He was talking crazy. He took it to the next level so I had to see if he matched that energy ... and y'all saw what happened.”

When asked if he felt bad for the way he acted, Scott stands by his fists.

“Sympathy for him? I don’t feel any for those individuals ... yeah, what a day,” he laughed.