Ray J's manager, David Weintraub, tells TMZ Ray went into the hospital 4 days ago and doctors learned from the jump he was battling pneumonia. David looped Ray in on the call and we spoke to him for several minutes, and it was apparent he wasn't out of the woods.

Ray told us ... when he was admitted doctors assumed he had COVID, so they placed him in the COVID wing of the hospital, which he said was frightening, telling us, "I thought it was over. I started praying and knew God is good so I just kept praying."

He says he was given 5 COVID tests, all of which came back negative, and after 2 days he was moved out of the COVID wing.

He's currently getting oxygen but he still has difficulty breathing and talking. He says when he wakes up at 6 AM he feels like he could go home, but by 6:30 things deteriorate and he fears the worst.

Ray says all he thinks about is getting better so he can see his 2 kids. Weintraub says Ray will hopefully be out by week's end.