Last Saturday, when my wife Kate Garraway was finally voted off BBC1’s Strictly Come Dancing, I was emotionally ripped in two. Part of me was upset, but another part was relieved.

In the previous six weeks the show had taken over our lives while capturing the imagination of the nation. Kate had barely been out of the papers since day one, and the paparazzi were camped outside our door.

There was a fuss in the newspapers because the audience was voting to keep Kate in while sending other, arguably better, dancers home. As the controversy grew, the News of the World ran the full-page headline: “[Steve] McClaren is the Kate Garraway of football.”

Last Tuesday there were three pictures of Kate in The Sun alone. One to accompany a piece ridiculously alleging she had made up her latest injury; another on the television listing pages; and one where they had Photoshopped her “dancing” with Peter Crouch.

Like Rod Stewart – whose wife Penny Lancaster was also on the show – I was dubbed a “Hab” by the media: one of the husbands and boyfriends. The legendary rocker summed up the feelings of all of us when he cried, a week after the show began, “I want my wife back.”

Penny was voted out the week before Kate. A few days later she was keen to show Rod that she had got over her upset and drove to the airport to pick him up in his favourite Ferrari. As he hopped in the car his first question was: “How you doing, love?” – at which she opened her black velvet coat to reveal a set of skimpy underwear.

Back at the Draper house for our first Sunday morning in two months, I waited in bed for Kate’s treat. It eventually appeared. Sporting a mismatched tracksuit and a face mask, she brought me a bacon butty. Well, we can’t all be rock stars.

I was relieved that Kate was no longer being mauled in the media – though we’d long taken the view that all publicity is good publicity in this case, and a lot of the coverage was positive – and I was glad just to have my wife back.

As we finally got our old life together I found myself reflecting on Strictly Come Dancing’s unique popularity. It outperforms ITV’s The X Factor, drawing up to about 9m viewers each Saturday and millions more on Sunday and throughout the week for its BBC2 spin-off. Just what is it about Strictly, as its fans call it, that proves so absorbing?

The show’s reach is phenomenally broad. Sir Michael Lyons, the chairman of the BBC, singles it out as his favourite programme, and each week we had to share out the four tickets each celebrity is allocated between political bigwigs, newspaper executives and Kate’s family, especially her young cousins.

Those are the lucky few. At the less privileged end of the spectrum there is a plethora of Strictly forums on the internet where real people intensely debate the dances, the dresses, the judges and the way the show is put together each week.

The diversity of stars is, of course, one of its attractions. There’s something for everyone: EastEnders fans, buyers of lad mags such as FHM, sports supporters and music lovers. It is also brilliantly and expensively produced.

Admittedly the sandwiches in the green room are a throwback to the bad old days of British Rail, and Kate would get irate phone calls if she kept a cab waiting for more than five minutes; but this penny-pinching sits alongside a real investment in quality.

The trailer for this year’s show cost more than the entire first series and all those fabulous dresses are made in advance – though you can, if you have £1,500 or so to spare, buy them in the new year from Chrisanne’s in south London. They always sell out, I’m told.

The show’s popularity isn’t fully explained by its quality, however. Nor is it explained by the echoes of ancient Rome, Punch and Judy, British music hall, Hollywood studio glamour and pantomime. (The judges are the villains, Bruce is Buttons and the winner is Cinderella.)

I think that it appeals, on a fundamental psychological level, to three of our most deep-rooted instincts: sexuality, aggression and the need for belonging. It even taps into ancient mythology and the theory of dreams.

This makes it as fascinating to the armchair psychoanalyst as it is to the armchair viewer.

Freud’s notion of sexuality was not just about sex. It was about what absorbs and excites us. Strictly Come Dancing allows us to enjoy the spectacle of dance, with its close physical relationship to sex and displays of flirtation and colourful plumage that wouldn’t be out of place in a wild-life documentary. The dances – especially Latin ones such as the samba – ooze sexuality.

Fans know of the rumours that previous couples have taken the heat with them off the dance floor. In this series there have been stories suggesting that handsome Matt Di Angelo from EastEnders is falling for his dancing partner Flavia Cacace. True or not, they add a frisson of excitement to their dances every week.

Luckily there was no such rumours about Kate and Anton Du Beke, her partner, although they did grow incredibly close – so much so that they have been having regular coffees to manage their withdrawal.

Anton is remarkably popular with women. One girl fainted when he twirled in front of her at the National Television Awards. He seems to have changed a bit after his experiences with Kate, though, and I suspect he may be ready to put aside his reputation as a ladies’ man and settle down.

Strictly’s other basic psychological drive is aggression, which gives the show its kick. While enjoying the achievements of the stars, we get to see them suffer from the gruelling training regimes, the injuries and the caustic comments of the judges. Kate was compared to a dancing quail and Tutankhamun, and she was told she was as “sexy as a coconut”.

The studio audience is encouraged to boo and hiss at these insults. It is a potent emotional mix: they can admire, envy and enjoy the humiliation of the contestants.

Even that does not explain the depth to which the show has entered the national psyche, however. This is down, I think, to something vital that is missing from most of our lives today: a real sense of community.

Evolutionary psychology shows that for tens of thousands of years we lived in villages of around 150 people. We knew everyone and they knew us. We shared their trials and tribulations, and they shared ours.

In our modern lives, we often do not even know the names of our closest neighbours. But we crave to be part of something big that can unite the nation.

I think that we thirst to be part of a collective story, like our ancestors were in their villages, and that this thirst drives the gigantic interest in celebrities – even talentless ones.

We admire the most beautiful, pity the least fortunate and argue over the most eccentric in the new global or virtual village brought to us by television, newspapers and the internet.

Strictly perfectly provides for this thirst. But unlike the overt manipulation involved in shows such as Big Brother there is something authentic about it.

Yes, it allows us to identify – in our fantasies – with Athenas and Adonises like Kelly Brook and Di Angelo; but it also allows us to identify, on a more realistic level, with the likes of Kate.

People at home understandably think that they too – with a good teacher, a glamorous dress and a bit of confidence and luck – could dance like her or do even better.

This taps into the oldest and greatest archetypal myth: that of the ordinary soul who goes on a journey and comes back a hero.

It also binds people together in a conversation about talent. A truck driver can share a conversation with a lady of the manor about a skill that until five years ago had largely disappeared from Britain’s consciousness.

Compare this with Big Brother. Try to strike up a conversation about whether Chanelle dumped Ziggy or vice versa, and in polite company the conversation will be fairly short. Strike up a conversation about Kenny Logan’s paso doble and the chances are that people will have an opinion – and an informed one at that.

Strictly brings out the best, not the worst, of humanity: it demonstrates our need and desire to master a talent, and the hard work and disappointment that goes with it. Despite the glamour, its emotions are not contrived.

When Gabby Logan cried for two days after being voted off Strictly it was not because she had been manipulated by a production team or due to any personal flaws. It was her genuine, heartfelt response to what had happened. The public, to use a psychotherapy cliché, felt her pain.

Other contestants also revealed their inner vulnerability. Gethin Jones’s struggle to be sexy; Letitia Dean’s initial lack of confidence; Penny Lancaster’s occasional insecurity – these were not contrived but showed the reality behind the usually airbrushed photoshoots and finely spun publicity campaigns.

People have been getting to see their favourite stars as they really are and have enjoyed discovering that they are more like them than it often appears. In an age of artifice, people crave such authenticity, especially if they can share their appreciation of it with others.

So what does this have to do with dream theory? Consider this: the two most recurrent dreams people mention in psychotherapy are finding yourself naked in a public place and sitting an exam without having prepared. These symbolically represent the two great human anxieties: vulnerability and failure.

Each week the Strictly celebrities enter the lion’s den at Television Centre and act out those terrifying dreams in front of some 9m people. We watch transfixed as they live out, simultaneously, our greatest fantasies and our worst nightmares – all to a backdrop of a live big band, glitterballs and sequins.

Putting aside my psychotherapist’s handbook, I asked Sue Nye, Gordon Brown’s right-hand woman, her opinion. She joined us in the studio audience one week with her nine-year-old son Matthew, who made friends with everyone as he filled his handmade album with autographs.

Sue believes the secret of the show’s success is down to the fact that the whole family can watch. Matthew, his teenage brother and sister and their brainbox economist father Gavyn Davies gather round the telly with Sue every Saturday; and each has his or her own valid view about what’s going on.

“It’s basically good old-fashioned family entertainment,” Sue said.

Moreover, the lessons that Strictly teaches are the right ones for us all – especially kids – to learn. If a 6ft 6in rugby player can glide around the dance floor in a pink sequinned top then any of us can do something we never felt capable of before.

Derek Draper is a psychotherapist with diy-therapy.com and a former Labour spin doctor

Times recommends

• Jennifer Aniston, time to be a real bitch
• It may be hard to say so, but conversation is back
• News Review interview: Georgina Baillie