He’s been a household name for over four decades, but Tom Selleck, who shot to fame with Magnum P.I. in the 1980s, there has never been one to seek fame and celebrity.

“I’m a fairly private person,” says the 75-year-old actor in this week’s issue of PEOPLE. “And I’ve always treasured the balance between work and time with my family. It’s always about them.”

Indeed, when Selleck is not shooting Blue Bloods, the police drama now in its tenth season, in New York City, he’s at home with Jillie, his wife of 33 years, on their Ventura, California, ranch.

“My relationships and my ranch keep me sane,” says Selleck, who is dad to Hannah, 31, and Kevin, 54, his son with first wife Jacqueline Ray.

For more about Tom Selleck, pick up this week’s issue of PEOPLE, on newsstands FridayThe ranch was a working avocado farm until the plants were decimated by drought and is home to over 1,500 native trees. “I do grunt work and I make the rounds. I like watching things grow. It’s a retreat.”

A retreat that Selleck has called home since 1988, after the actor quit Magnum P.I. at the height of his fame, in order to lead a quieter life with his family. Fame, he had quickly learned, could be stifling.

“I knew intellectually what it would mean in terms of being a public person, but until you’ve lived it, there’s no way to understand it,” says Selleck. “I had a feeling of, ‘I don’t think I’m cut out for this.’”

Ultimately, “I quit Magnum, not because I didn’t like it or I was tired of it,” he explains. “I was tired from it. And I wanted a three-dimensional life because I didn’t have one.”

And so Selleck chose to retreat from Hollywood on a self-imposed hiatus. One year off became several when Selleck found he “didn’t like what I was being offered.” Says the actor: “I put up with the articles that said he’s disappeared, he’s done. And you do get done, I’m well aware of that. It was a big lull, but it put a lot of things in perspective.”

Selleck began working steadily again in the ’90s, with roles on Friends and The Closer, as well as his Jesse Stone TV movies, before landing the role of police commissioner Frank Reagan on Blue Bloods.These days, the actor says he’s grateful for him career longevity and equal appreciative of the life he’s built away from the cameras.

“I’m proud of my work, I still love what I do, and I have my family,” says Selleck. “I’ve been enormously fortunate.”