The 'Late Show' host made the Royals one of his topics on Tuesday night's opening monologue -- and he completely embraced unfounded speculation and innuendo that Bill is, perhaps, cheating ... a rumor that has been floating around for a while in tabloids, etc.

The alleged "other woman" in question is Lady Rose Hanbury -- the Marchioness of Cholmondeley -- who's been in William and Kate's inner orbit, as a friend, for years.

Like we said, this has never been confirmed ... but it's been gossiped about for a long time -- and now, amid Kate's mysterious absence, it's kicked into high gear again ... with some suggesting that it somehow has something to do with why KM's been out of the spotlight.

Colbert didn't seem all that interested in connecting the dots -- but he was happy to poke fun at the headlines that have theorized this ... cracking jokes at Will and Kate's expense.

He even made fun of a report that suggested Kate had confronted her husband about this ... to which the Prince allegedly "laughed it off" and said it was much ado about nothing.

As we reported ... William has been keeping calm and carrying on with his Royal duties as Kate continues to recover from her abdominal surgery behind closed doors -- although, slight glimpses of her have been seen here and there ever since her procedure in January.

Of course, her failed Photoshop attempt over the weekend -- which she has explicitly taken the blame for -- only added more fuel to the fire on what exactly is going on with her.

Fact is ... hardly anybody believes the picture was taken when Kensington Palace says it was -- and with this new affair rumor now making its way into the mainstream ... it's just another layer that Will and Kate have to combat as people keep digging into theory after theory.