Stuart Townsend's facing a felony charge for allegedly getting physical with his wife and refusing to let her leave during an argument ... TMZ has learned.

Law enforcement sources tell us ... the actor's neighbors called 911 complaining of loud screaming coming from his home in Santa Monica, and when cops showed up Townsend and his wife could be heard arguing from outside the house.

We're told the officers separated the two, but Townsend allegedly warned his wife not to say anything. Our sources say she complained of pain in her arms, but didn't need medical care.

Cops continued questioning the couple and learned they had a verbal fight that escalated. We're told Townsend allegedly grabbed his wife by the arms and at some point ... refused to let her leave when she tried.

The actor was arrested and booked for felony domestic violence. He's also facing 2 misdemeanor counts -- false imprisonment and dissuading one from reporting a crime.

He was released after posting $50k bail.

We got Stuart and his wife out Friday night and they brushed off the incident and walked out of a restaurant hand in hand saying everything's fine.

Townsend's best known for his role as Dorian Gray in the 2003 action flick, "The League of Extraordinary Gentlemen." Of course, he also dated Charlize Theron for 9 years until they split in 2010.

We reached out to Townsend and his wife. Stuart tells us they did have a loud argument, but claims they had already resolved their issues when police arrived. He adds, they both "look forward to this being resolved speedily and favorably and returning to their quiet lives as spouses and parents."

His wife tells us she asked cops not to arrest him and agrees -- she'd like the matter dropped so they can move on together as a family.