Afew weeks ago, bored one evening and with a shortage of Hitler porn to watch on the history channels, I dug out Mel Brooks’s 1968 “classic”, The Producers, and watched that instead – anything for a late-night Adolf-fix. I seemed to remember it contained only one decent laugh. But I had quite forgotten the full panoply of sheer awfulness – the painfully witless jokes telegraphed five minutes in advance; the excruciating, ham-fisted acting; the histrionic gurning and mugging of Brooks himself. And, dear Lord, Gene Wilder. It’s by no means the worst film Brooks has made: check out Spaceballs or History of the World for an evening of perfect mirthlessness. The Producers is one of his best, in fact. Imagine that.

It’s hard to believe, from this vantage point, that in the late 1970s there was a debate in film circles as to who was the better director, Mel Brooks or Woody Allen, with many critics tilted strongly towards Brooks. By 1979, Allen had long escaped the exuberant, anarchic comedy of Bananas and Sleeper and Play It Again, Sam, and seemed determined to make films that (a) were in black and white, and (b) weren’t remotely funny. Interiors, in all its well-mannered, boring gloom, had just been released to decidedly mixed reviews. Brooks, meanwhile, had scored a critical and commercial hit with the genuinely witty Hitchcock spoof High Anxiety – the only Mel Brooks film you could watch today without gouging out your own eyes in misery. The debate was posited as a sort of lifestyle choice. If you opted for Allen, you were middle-class, pretentious, humourless, self-important. A vote for Brooks meant you were sunny side up.

It was a false dichotomy, of course, and a grave insult to Allen, who, in the 1980s, managed for a time to square the circle and direct films that succeeded in being amusing without being that terrible thing, that artifice he had begun actively to despise: comedy. “I had the courage to abandon... just clowning around and the safety of broad comedy,” he remarked, after making the fine Annie Hall in 1977. Would you had been a little more cowardly, some of us thought at the time. But, still, what we got during the 1980s was a succession of delightful films, several of which must count among the best Hollywood movies of the past 50 years: Zelig, Stardust Memories, The Purple Rose of Cairo, Hannah and Her Sisters, Crimes and Misdemeanours. Yet with each successive film, the stuff that made you laugh diminished in quantity until, in the end, you weren’t laughing much at all. Later still, the humour was sucked out altogether: we had gone, in less than 20 years, from the hilarious Love and Death to the impenetrably glum Shadows and Fog. But then, Shadows and Fog, that was serious, wasn’t it? You don’t laugh during serious films.

Last weekend, Allen wrote a sweetly self-deprecatory encomium upon the death of his greatest hero, Ingmar Bergman. “Bergman... couldn’t help being entertaining even when all his mind was dramatising the ideas of Nietzsche or Kierkegaard,” he observed. The choice of words suggests that Bergman’s knack of making intellectually complex films watchable was almost accidental; as if the thing that drew the audience in was the Nietzsche and the Kierkegaard stuff, and the entertaining just came along for the ride. Allen so craved high seri-ousness that somewhere along the way, he forgot to be accidentally entertaining at the same time. Somewhere around 1990, I’d reckon. Having worried that he would too often succumb to the easy joke, he stripped out the jokes altogether. It’s a familiar problem for the cleverest of our humorous artists, this throwing out of the baby with the bath water in the hope that they might be taken more seriously, might find true posterity. Mike Leigh and Martin Amis have been similarly afflicted. Hubris, I suppose.

The present decade hasn’t been too kind to Allen, either; his films, with one exception, have made little headway at the box office and were scarcely better received critically. I recently watched Match Point, his impeccably well-mannered examination of luck as it impinges upon the life of a retired tennis pro. A young former tennis pro, mind, who reads Dostoevsky and is an opera buff: you come across many of them? Allen reportedly believes it to be one of the two finest films he has ever made (along with Stardust Memories), and it is certainly what we know to call “intelligent”, with its halting, naturalistic dialogue and long single shots. But I would not wish to watch it again in a hurry, nor wish it upon you – and there is less to be gained from it in terms of genuine revelation than can be acquired, laughing all the while, from Everything You Always Wanted to Know About Sex But Were Afraid to Ask. John Updike was once cattily dismissive of the English comic novel: why just make your story funny when there’s so much else to do besides, he asked. But the reverse is true also: a film or novel expunged of all humour has no roots in the real world. Watch Match Point and your lips will not so much as twitch.

Allen has done a book, too, his first collection of humorous prose in 25 years, entitled Mere Anarchy – and it is anything but. There are occasional moments of that old comic inspiration, such as Mickey Mouse, Donald Duck and co in court testifying against the Walt Disney Company. But mostly it is terribly hard going, the learning worn as heavily as a coat of chain mail, the prolix prose occasionally verging on the unreadable, the jokes arriving with a limp, or in bandages. “Epistemology renders dieting moot. If nothing exists except in my mind, not only can I order anything, the service will be impeccable.” Well, strap those ribs up. One remembers wistfully the carelessly tossed-off joys of those much earlier collections, Without Feathers and Getting Even. The problem is maybe one of retribution: Allen decided way back that the comic muse was perhaps a little beneath him and was therefore to be spurned. Now he finds it can’t easily be wooed again.

This may seem unkind, but it is not intended to be. Regretful, maybe, but not unkind. We have plenty of reasons to be grateful to Woody Allen. During that piece on Bergman, he was emphatic and unusually concise in rejecting the notion that, like his mentor, he himself was a genius. So emphatic, in fact, that you wonder how often the question preys upon his mind. But if Allen’s other cinematic hero, Fellini, can be called a genius, then Woody Allen certainly can. It is an overused term, “genius”, but it does not diminish its weight to append it to his name. The problem is that Allen himself has scant regard for those very qualities that enable him to be thus defined. Humour need not be disposable, crass or cheap. It need not be Mel Brooks. Woody Allen showed us that, whether he likes it or not.

Allen’s next film, Cassandra’s Dream, will be shown out of competition at Venice

Explore Film

• Film Reviews

• Trailers

• DVD

• London Film Festival

Editor's choice

• The 100 Best Films of the Decade
• 70 facts you didn't know about Marvel
• The 50 Biggest Movies Of 2010