The singer spilled the alleged beans to DailyMail.com -- telling the outlet he's furious the Kardashians have essentially accused him of holding an unreleased sex tape over their heads ... and that he was the one who leaked the OG tape in the first place.

He insists the truth is the exact opposite -- namely, that the "leak" wasn't that at all ... but a systematic deal brokered between himself, Kim and Kris Jenner and Vivid Entertainment.

Here's how it allegedly went down, according to Ray J ... he says that while he and Kim were dating, he jokingly floated the idea of putting out a sex tape of theirs in the wake of Paris Hilton's video leaking -- which elevated her status and celebrity.

While he claims he didn't really mean it, Ray alleges Kim jumped on the idea ... going to her mom, talking it out and then getting the wheels in motion on the whole thing, which he says he was not really privy to until it came to signing away the rights.

Ray says he and Kim both signed a contract, and the rest is history.

Here's the fascinating thing ... the guy claims all the original sex tape footage -- including a SECOND one he says they shot in Santa Barbara that is unreleased thus far -- has always been in Kim's possession ... in a Nike shoebox under her bed.

Ray claims he's never been in possession of any copies, and that he's been fuming over the fact the Kardashians have made it seem of late he had more footage and that he was on the verge of releasing it. He did, in fact, give Kanye West a computer and hard drive with some personal content between him and Kim ... but argues there was nothing explicit on it.

One other thing ... it sounds like the Kardashians themselves might've triggered his rage here with their new Hulu series. In one of the first few episodes, Kim and co. apparently joke about Ray putting a dildo in her butt, which he's irate over ... saying they made him look bad.