One tends to associate Robert Wagner with the 1970s, flared safari suits and television pap such as Hart to Hart. But he has also been in Titanic (1953), The Pink Panther (1963), The Towering Inferno (1974), and the Austin Powers movies, in a film career spanning nearly six decades, while his on-off-on-again life with Natalie Wood is one of Hollywood’s greatest love stories. He’s also a tremendous gossip, a perceptive analyst of other people, and quite a ladies’ man. His memoirs cannot help but be entertaining.

His first love was Barbara Stanwyck, as he reveals here for the first time. “I was 22, she was 45,” and she was recently divorced. This handsome toy boy obviously did her the world of good. Not that she was his first conquest. Joan Crawford came before that, seducing him by jumping naked into a swimming pool with him. And indeed he lost his virginity at 12½, to a girl who passed him a note saying, “I would like to make love to you.” The result, he found, “was a lot better than masturbation”. She told her friends, and soon young Robert had “easy access” to several others. Evidently, teenagers in 1940s California were just as bad as today’s.

Stanwyck gave him an instant entrée to Hollywood royalty, and his pen portraits are exceptional. There’s the fearsome director John Ford, winner of six Oscars and a Rear Admiral in the US Navy, surely something that will never be repeated; he put the fear of God into even John Wayne, and once encouraged the young Wagner to a better performance by simply pushing him over. Of Steve McQueen, Wagner writes, “I think he trusted me as much as he trusted anybody, which wasn’t all that much”, and he is brilliantly laconic on “\ Fonda’s remoteness, which always translated as integrity”.

After Stanwyck, Wagner had affairs with both Anita Ekberg and Elizabeth Taylor, among others. “Some beautiful women are passive in the bedroom. Elizabeth was not one of those women.” But while evidently fond of her, he doesn’t gush. With Taylor, “there was a great deal of maintenance. This is not a woman who gets up in the morning and fixes breakfast. By the time she comes downstairs for breakfast, it’s time for dinner”.

Outrageously indiscreet, he tells us that Peter Sellers “liked amyl nitrate when he was getting oral sex”, and in a similar vein recalls his second encounter with Errol Flynn. He opened his door, saying, “Mr Flynn?”, only to find the old roué sitting down, and “between his legs . . . a blonde girl”. Flynn looked up, and solemnly shook his head. Young Wagner backed out and shut the door behind him.

He first set eyes on Wood when she was 10 and he was 18, whereupon she announced to her mother that she was going to marry him. They did, nine years later, and then four years after that, they divorced. She put her acting first, and she had an affair with Warren Beatty. But she and Wagner were made for each other, and they remarried years later, older and wiser. He evokes beautifully what made her so loveable. She wasn’t only gamine and gorgeous, “she seemed to live in a perpetual state of joyful discovery”, passionate without being difficult or moody. Everyone was drawn to her “spectacular sense of life as well as humour”. “God, she was fun,” he says. You can almost hear the sigh.

And then one night in 1981, a little tipsy, Natalie went out on the deck of their sailing boat, anchored off Santa Catalina island in California, slipped, hit her head, fell in and drowned. She was 43. Wagner barely survived the heartbreak. For seven days he couldn’t even wash, shave or get out of bed.

David Niven proved one of his greatest friends. Wagner recalls Niven, the kindly gentleman, beside his swimming pool, “fishing out bees and wasps so they wouldn’t drown”. When Niven himself was dying of sclerosis, a paparazzo took a photograph of him in his garden, “looking mortally ill”. “A group of us made sure that it would be a long time before that son of a bitch was able to take any more pictures.”

It’s the only harsh moment in the book. Wagner was always too sunny and uncomplicated, perhaps, to be a really great actor, but he’s an extremely amiable man. Now nearing 80, he has no intention of retiring. Retirement is a collusion between “life insurance companies and cruise companies”. Near the end, he quotes a good line from Eugene O’Neill: “Man is born broken. He lives by mending.”

Pieces of my Heart by Robert Wagner
Hutchinson £18.99 pp336

Times Recommends

• The 100 Best Books of the Decade
• The Times picks the 50 best paperbacks of 2009
• The conversation: James Ellroy