9:07 AM PT -- A rep for Aaron tells TMZ ... he didn't storm off the set of "The Doctors" after having rehab recommended, but rather left "quietly and professionally" after hanging out for at least half an hour when taping finished.

Aaron Carter is willing to talk about his prescription drug abuse, but bolted as soon as the topic of rehab came up on "The Doctors" ... according to family members.

Aaron agreed to a drug test when he taped the show about a month ago -- it airs this week -- and results came back positive for benzodiazepine (Xanax-like drugs) and hydrocodone (generic Vicodin). He claimed the doctors were helping him more than any therapist had, but a Carter family source tells us Aaron drew a line at the suggestion that he gets treatment.

We're told the doctors made the rehab recommendation after the show was done taping -- and they even said the show would pay for it, but Aaron got defiant and insisted he would NOT go.

Our sources say he stormed out of the studio, hopped in his car, and drove from L.A. to Florida the next day.