Kim claimed she took ecstasy twice -- once when she married her first husband back in 2000 and again when she filmed the tape in 2003 -- and she says both experiences were bad for her.

Our Ray J sources say Kim took no hard drugs nor did she drink alcohol during the filming of the sex tape. They do say, however, she smoked pot around the time the tape was made ... and it was from a pipe shaped like a penis.

As for Kim saying her lip quivering was evidence she was on ecstasy, the Ray J sources fire back ... "her lip was quivering because she was getting ******."

Sources close to Kim tell us she stands by the claim she made on the show -- she did in fact take the drug before the tape was shot -- and she has nothing to gain from lying about it at this point in her life. We're told Kim is thankful to have learned early on from the experiences with drugs and attributes those lessons to her decision to not get heavily involved in drugs or alcohol through adulthood.

Our Ray J sources add he's offended that Kim would suggest she did the sex tape because she was in a significantly altered state. They say she was down with it from the beginning and all the way through.