The story is insane ... around 4 AM on Sunday, Smith's wife heard a man in the house "screaming at the top of his lungs."

31-year-old Smith -- who lives in the home with his wife, 3 young daughters (8, 5 and 2)  and mother-in-law -- moved everyone to a room and decided to confront the intruder.

"I'm expecting that I'm gonna hear a gunshot or he's gonna stab me," Smith told ESPN's Ariel Helwani. ... "I figure I've got about two minutes before whatever he's got takes me out."

So, Smith charged hard and tried to subdue the man in his computer room -- but to Anthony's surprise, the man fought back ... HARD.

"No normal human is able to fight like that ... I'm by no means the baddest dude on the planet. But, he's a regular Joe and I had a hard time dealing with him."

"And he took everything that I gave him -- every punch, every knee, every elbow. He took every single one of them and kept fighting me."

During the scrap, Smith says his mother-in-law brought him a kitchen knife to use on the intruder -- but it didn't completely stop the guy.

Eventually, cops arrived and helped to subdue the intruder, identified as Luke Haberman -- who was already bloodied, beaten and cut up.

The craziest part of the story ... Smith says once Haberman was in custody, he apologized -- saying, "Hey man, I'm sorry."

What?!

"I'm not lying when I said it was one of the toughest fights I've had in my whole life," Smith said ... "I went into that fight ready to die."

As for Haberman, he was hospitalized and hit with a citation which requires him to appear in court in the near future.

Smith says the guy had been wreaking havoc in his neighborhood earlier in the night -- and broke into his home through an open garage door.