William Baldwin was always the prettiest of the Baldwin brothers. His intense come-to-bed eyes, lascivious smile and coiffed hair made him an obvious 1980s pin-up.

For a decade no film was quite complete without a Baldwin - William, Alec, Stephen or Daniel. But no acting dynasty has had so many ups and downs. William added eye candy to Flatliners and Backdraft, and dated Madonna, but then a couple of bad choices - the Sharon Stone bomb Sliver and Cindy Crawford's disastrous debut Fair Game - relegated him to the slush pile of 1980s pretty boys looking for work. Alec was the chiselled alpha male in Working Girl and Glengarry Glen Ross, married Kim Basinger, but fell from grace during their bitter divorce when his phone rant to his teenage daughter ricocheted around the internet. Stephen had his biggest success with The Usual Suspects and his biggest flop with the Kylie Minogue film Bio- Dome, then became an evangelical Christian in the wake of 9/11. Daniel became a star in Homicide Life on the Street before yo-yoing in and out of rehab, with assorted scrapes with the police thrown in.

Now, in light of his turbulent family background, it seems that William has finally landed the role that he was born to play - at the heart of a family of dysfunctional multimillionaires. In Dirty Sexy Money, one of the biggest hits of the US TV year, he plays Patrick Darling, a married politico with a penchant for transsexuals. His brother is a vicar with a secret love child, his sister a serial marrier, and his youngest sister could be Paris Hilton's best friend.

Twenty years on from when he started, the dimples and the jet black hair are still there, along with a mischievous glint in his eye when he talks about comparisons between the Baldwins and the Darlings.

“How do we measure up? The Darlings are rich, famous and highly dysfunctional, so fairly well,” he shrugs. “When we were filming the pilot people were arguing that some things wouldn't happen, someone wouldn't do that to their own brother. I was like: ‘Oh, yeah? I've got the scars to prove it, buddy'.”

Unlike the Darlings, the Baldwin brothers weren't born with their own platinum cards; their father was a teacher who raised six children on $20,000 a year. At first William wrestled (he has been inducted in the US wrestling hall of fame) and studied politics, but then he got his first taste of Hollywood glitz when his looks landed him a contract modelling for Calvin Klein. Then his oldest brother started acting and he decided to give it a try, too. His two other brothers felt the same and also followed Alec into the business. It was inevitable that, with so many Baldwins washing around Hollywood, there would be some crossover. When they started out there was a more delineated pecking order, but now William admits that there is a good-natured rivalry between them over roles.

“There are several times when I've said to Alec that I'd got a meeting about a part, only to find out that he'd already been offered it. It's happened to me with Stephen - they've offered the role to me - but I'd really love to get one over on Alec.”

We're sitting in the poolside cabana of an elegant hotel on Sunset Boulevard on a blazingly hot LA day. William is wearing a Panama hat at a rakish angle, which emphasises the air of frisky eccentricity that he seems delighted to cultivate. Having followed Alec into film acting, the brothers, ironically, now seem likely to follow him on to TV. Alec, with appearances in the Oscar-winning movies The Departed and The Aviator behind him, is now enjoying success with the TV comedy 30 Rock, for which he won a Golden Globe, and William is fiercely proud of his big brother. He was Alec's “date” at the Emmys last year, when he was nominated for Outstanding Lead Actor.

“Alec has always been really funny and I've always wanted people to know that about his work,” he says. “It wasn't until he had done Saturday Night Live eight times that Hollywood started to catch on and say we have to find him some fun roles.”

While Alec has the bigger career success, his bitter divorce from Kim Basinger and their furious custody battle means that William is probably the happier man. He and his wife Chynna Phillips, of the girl band Wilson Phillips, have just celebrated their 12th wedding anniversary. She and their three children - Jamison, 7, Vance, 5, and Brooke, 2 - were the main reason that he accepted the role in Dirty Sexy Money.

“I've never been one of those actors that was doing a film and had my next three films lined up,” he admits. “Now I know when I roll out of bed that I have a job and I have the security to provide for my family at this stage of my career.”

With so many famous people rattling around one family tree, William admits that his kids have taken a while to realise that not all families are like the Baldwins. For the first time, all four have appeared on prime-time US TV shows at the same time.

“The kids think TV is like home video because every time they turn on they see one of their uncles. But I have a pretty good head on my shoulders, and my wife even more so, which is incredible when you think she was raised by Warren Beatty, Jack Nicholson and Dennis Hopper and all these Hollywood lunatics who lived with her mother [Michelle Phillips, from the Mamas and the Papas]. We are trying to take all the things from our parents that were wonderful and beautiful, and discard all the other stuff.”

He smiles that big, dimpled Baldwin smile and launches into one of his practised anecdotes about family life. A passing publicist's eyebrows shoot sky high and, egged on by the reaction, he launches into a selection of tales from the Baldwin bedroom.

“The other day the kids were in bed so we decided to go for a roll in the hay, but we get in there and she takes off her underwear and she puts it on my head like a shower cap. She says: It's such a hassle afterwards, I never know where my underwear is and this way at least I know where it's going to be when I'm done'.

As I walk away from the cabana, William calls after me. “I bet you can't get the picture of me with my wife's underwear on my head out of your mind.”

You know what? I can't.

Dirty Sexy Money, Friday, C4, 9pm

Editor's choice

• The 100 Best Films of the Decade
• 70 facts you didn't know about Marvel
• The 50 Biggest Movies Of 2010