The first time country singer Jelly Roll was arrested, he was only 14 years old. Since then, he’s been in and out of jail 40 times. He knows a thing or two about jail life, but it took him awhile to change his life for good.

It’s been years since Jelly was arrested, but he’s actually been to jail quite recently. He’s been there to play his music for the inmates.

When Jelly was arrested, it was always for drug charges. He had to kick his addiction to drugs.

In an exclusive interview with PEOPLE, Jelly explained, “I had to learn that you could drink alcohol without doing cocaine. It took me a long time to learn that.” He added, “I’ve never said that, but that’s real. There was a long time where I just assumed, when people told me they drank without doing cocaine, I was like, I thought we only drank to do cocaine.”

Jelly used to believe that drinking alcohol “was to make us not feel like drug addicts.” He explained, “Nobody wants to snort cocaine sober, then you’re a drug addict. But I had to re-look at my relationship with alcohol like that.”

While Jelly has successfully kicked his drug habit, he does still drink alcohol from time to time, but he no longer drinks it so that he doesn’t feel guilty about doing drugs. He explained that he’ll “have a cocktail, but very, especially this year, very seldomly.”

In addition to drinking alcohol, Jelly also smokes marijuana from time to time, but it’s sparingly, and he feels that he pretty much has it under control. One reason for that is due to his family.

Jelly explained, “I’m very anchored in my family.” He added, “Being a father is so important to me. The single most impactful event of my entire life was having my daughter. It changed everything.”

Jelly has two children, a 15-year-old daughter named Bailee, and a 7-year-old son named Noah. In addition, he co-parents the children of his wife Bunnie Xo.

It was Bailee who made Jelly decide that he needed to change. She was born when Jelly was in jail at age 23 years old. He compares missing his daughter’s birth because he was in jail to “the Christian scripture of when Saul turned into Paul on the Damascus Road. It was kind of that moment for me.”

Watch the video below to learn more about Jelly’s past and how his daughter’s birth forever changed his life.