FOR A FEW GOLDEN YEARS, IAN HISLOP, Rosie Boycott, and myself were the BBC’s Literary Dream Team. We were the TV discussion panellists on their coverage of every Booker and every Whitbread during the millennial period, eventually becoming a sort of bizarro literary version of the Have I Got News For You crew, or an even more bizarro version of the Founding Mothers of Spare Rib. Our proudest moment, I feel, was at the 1999 Whitbread Award when, on being asked to plump for the overall winner, we unanimously went for the third instalment of the series whose sixth instalment is being published next week: to put it more simply, we all went for Harry Potter (and the Prisoner of Azkaban, in this particular case). We were all of course wrong: the real judges voted the overall winner as Seamus Heaney’s translation of Beowulf.

This remains, for me, a supreme example of everything that’s nonsensical about literary prizes. It is of course totally absurd that the Whitbread even has an overall winner, because you might think that comparing Harry Potter with Beowulf is like comparing Chas & Dave with Beethoven — but as long as there is an overall prize, the judges need to accept that in the category of Rumbustious Cockney Singalongs Chas & Dave knock Ludwig into a cocked hat. In other words: what should have been assessed is not which is the “greatest” work of literature in this prize, but is Harry Potter a greater children’s book than Heaney’s translation of Beowulf is a piece of poetry? And then there is no doubt that the nerdy, possibly gay, bespectacled hero flies away on his Quidditch Broom with the Brewery-Sponsored Statuette.

Because even the previous winner of the Whitbread Prize — Ted Hughes’s Birthday Letters — is a more significant moment in poetry than Heaney’s Beowulf. Just a year earlier; but I doubt that within a radius of 50 years either side we will end up talking about a book that has created a dent in the culture of childhood like Harry Potter. He is an enormous, iconic figure, the Alice in Wonderland of our day, and when these moments happen they need to be celebrated, not downgraded to second-place precisely because of the immensity of their success.

I actually had a row with one of the judges that night — in my memory, it’s Rose Tremain, although this can’t be right, as she was nominated — who told me that the decision had to go to Heaney because, she said, the poem was still so relevant: with everything going on in Bosnia, she told me — this of course was pre-Iraq — the bloodiness, the treatment of violence, the pain in the poem, all this still spoke urgently today. Well, I replied, in that case, you’ve given the prize to the wrong man. Because Heaney, however good his verse, is only the translator: it wasn’t his inspiration, the bloodiness, the pain. If there’s a prize for still having relevance today for an Anglo-Saxon poem, then the person who should be opening the champagne and posing for the paparazzi with Jerry Hall is Anon. Is it too late to phone his agent?

I believe that one of the judges — not Jerry Hall, by the way — even threatened to resign if Harry Potter was awarded the overall prize, saying that it would bring the competition into disrepute. Think about that. Every category in the Whitbread is meant to be in equal contention for the overall prize. So what children’s book would this judge have felt didn’t bring the prize into disrepute? Ricky Gervais’s Flanimals? That salutary tale of the small underground mammal that finds one day that some unidentified faeces have landed on his head, The Story of the Little Mole Who Knew It was None of His Business?

Children’s books, you see, if they are any good, are not high literature. They’re for children. They’re not meant to be War and Peace. And if you’re going to put them in competition with War and Peace, then each needs to be considered on its own terms, not on some universal standard of artistic correctness. What this award was about, of course, was fear: intellectual fear, which always seems to infect literary judges at prize-giving time. It has forever been my contention that properly clever people never have to prove themselves so. The middlebrow, however, are continually afraid of being found out, so they are quick — too quick — always to choose the highbrow choice. They may have really enjoyed that thriller, but come the crunch somehow they always find themselves arguing fervently for the big f***-off tome set in 17th-century Tasmania.

So one day I hope that Harry Potter does receive the prizes. Perhaps, when the day comes, Rosie, Ian and I will re-unite and have a celebratory threesome, a kind of weird parallel universe version of the Have I Got News For You? team, or an even weirder parallel universe version of the Founding Mothers.

David Baddiel will be presenting Heresy, a programme that dares to question received opinion, on Radio 4 on Wednesdays, 6.30-7pm

Julie Burchill has moved to the main paper

Explore Books

• Book Extracts

• Book Reviews

• Books Group

• Audio Books

Times Recommends

• Superfreakonomics by Steven D Levitt and Stephen J Dubner
• The Times picks the 50 best paperbacks of 2009
• Vincent van Gogh’s letters show the soul of the artist