Whole Foods CEO John Mackey achieved success by building a grocery empire, but he occasionally regrets shelving one particular item.

In an interview with Freakonomics Radio, the 67-year-old businessman said he often thinks about how different his life would have been if he’d had children.

“I just love children,” Mackey told host Stephen Dubner on a Nov. 4 episode of the show.

Given the trajectory of his life, starting a family wasn’t in the cards. Mackey always had an ingrained passion for food, leading him to become one of the first investors in Whole Foods. In 1978, Mackey raised $45,000 from family and friends, and went on to open the store’s first location in Austin, Texas.

The single storefront eventually expanded to a high-end retailer with locations across the country. Mackey later met his wife, Deborah Morin, a yoga practitioner and teacher. According to Mackey, Deborah did not want children — and he accepted that fact because she was the woman he loved. 

“If I could do it all over again, would I make a different choice? And the answer is no, because I married the most amazing woman,” Mackey said. “She has helped me so much. She’s made me so happy. It’s been such a great partnership.”

Another aspect that played into Mackey’s decision was his complicated relationship with his own father, Bill Mackey, who died in 2004. A hospital chairman and CEO, John says his father had high expectations and could be difficult to please, often causing him to question his adequacy.

“I remember when I was growing up, for example, bringing in a report card one time, all As and one B, and I was really proud of it. And my father said, ‘What are you going to do about that B?’ And that was devastating, because I thought he’d be proud of me,” Mackey told Dubner.

His father’s exacting tendencies impacted Mackey’s self-esteem, and resulted in feelings of inferiority, he said.

“In the short run, I was first hurt, and then I was angry. It’s like, I can never make him happy,” Mackey said. “I experienced him mostly criticizing me.” 

Mackey told Dubner he can’t say for sure if he would have held the same high standards for his own children. Instead, he speculated his parenting style would have more likely been impacted by when in life he had kids.

“I’m a lot more conscious as I’ve gotten older than I was when I was younger,” Mackey told Dubner. “I’m sure if I started at age 20 or 21, I’d have had more energy, but I’d lacked the wisdom that I would have had when I got older.”

Despite having some nagging sadness over not having a family of his own, Mackey says he wouldn’t change the course of his life, nor does he take his relationship with his wife for granted.

“I would not go back and replace my wife with a woman who wanted to have kids, because I have really scored well on that one,” he said.