Law enforcement sources tell TMZ ... Charlie Sheen's ex-wife says she's being blackmailed by her former drug dealer, who claims to have videos of her buying drugs and other compromising actions.

We're told the dealer told Brooke he'd go to the media or make the videos public to ruin her if she did not pay him off.

Law enforcement sources say Brooke had already tried to keep him quiet with gifts and money ... totaling tens of thousands of dollars. A source close to Brooke says she has never given him a dime.

We're told this went on for months ... until Brooke finally decided it was time to call the authorities.

No arrests have been made in the case ... but the investigation is ongoing.

This comes amid an already rocky year for the actress ... who has been linked to the Matthew Perry death case.

As TMZ previously reported, Brooke was at the same L.A. rehab as the late "Friends" star and Jasveen Sangha, who has been dubbed "The Ketamine Queen." Sangha was later arrested and charged in connection with MP's death.

Our sources also previously shared that Brooke was close friends with Erik Fleming, one of the men who cut a deal in the death case.

Specifically, Erik pled guilty to conspiracy to distribute ketamine and distribution of ketamine resulting in death and serious bodily injury ... after the prosecution painted him as a drug broker who delivered ketamine to the actor's assistant, Kenneth Iwamasa.

As we previously reported, Brooke cooperated with authorities in their investigation ... she was never arrested or charged with any crimes.

Brooke also appeared on the "Hollywood Raw" podcast a month ago and spoke about being extorted a few times.