We got the rapper out Wednesday at LAX and he tells TMZ ... a lot of damage was done and is gonna keep his mom and sister at a distance following their altercation. He says it's whack how his fans turned on him without hearing the full story ... but it is what it is.