Paps spotted the pair holding hands as they left a gym sesh together Saturday in the East Village in NYC. As you can see by their workout gear ... things are getting physical between them.

We already know Joe's super athletic -- he boxes and whatnot in his spare time. Looks like he's getting Sophie into that ultra fit life too.

Plus, she's already made it clear Joe's giving her that good good. So again ... get it, Sophie!