Why hasn’t Terry Gilliam given up? The director of Brazil, Time Bandits and Twelve Monkeys is regularly described as the unluckiest man in cinema. His movies are frequently beset by problems and, occasionally, catastrophes. He had to abandon one film, The Man Who Killed Don Quixote, within a few days of the start of shooting after the actor Jean Rochefort was taken to hospital and a freak rainstorm washed away a large part of the Spanish countryside where he was planning to film. And after the death of his star Heath Ledger last year, Gilliam came very close to giving up on his latest film, The Imaginarium of Doctor Parnassus. A lesser man would have crumbled into retirement by now. So what is Gilliam’s secret?

We meet in a tiny garret room in the Soho editing studio where he is putting the final touches to the movie, which has its world premiere at the Cannes Film Festival (running until May 24) . The room, with its slanting ceiling and Dickensian view across chimney stacks and ramshackle roofs, seems too small for him as he clatters in, shedding possessions. The laugh is a force of nature — it represents something of his unstoppable spirit and gives a hint at what has kept Gilliam going through the career vicissitudes that have plagued him.

Although, to be fair, Gilliam has also had his share of good fortune. An early childhood in Minnesota’s rural heartland imbued him with the vivid, almost hallucinatory imagination unique to children left to their own devices. Later, while studying at Occidental College in Los Angeles, Gilliam’s art work for the college magazine caught the eye of his childhood hero, Harvey Kurtzman, the founding editor of Mad magazine. Kurtzman, then at Help! magazine, offered him a job. It was at Help!, making one of the magazine’s trademark “fumettos” or photo strip cartoons, that Gilliam first worked with John Cleese.

When Gilliam moved to the UK, his meeting with Cleese provided an “in” with Monty Python’s Flying Circus, who shared his anarchic, absurdist sense of humour and offered him creative freedom as an animator and later a director. In the UK, Gilliam not only found creative partners but also his life partner. He has been married to the make-up artist Maggie Weston since 1973. Their three children have all been involved in Gilliam’s films, either as actors or behind the scenes. Amy, the eldest, produced her father’s most recent picture, and it was she who had to break the news of the death of the film’s star.

“We were devastated,” Gilliam says. “We spent the whole day — Amy, Nicola Pecorini, the director of photography, and myself — lying flat on the floor. He’s dead, and you don’t quite get over that. I suppose I’m in an interesting position because while I’m cutting the film I’m basically working with him every day and he’s fine; he’s in good shape.”

The film is a fantasy about a group of travelling players, led by the immortal Doctor Parnassus (Christopher Plummer), who have to rescue Parnassus’s daughter from a deal he made with the Devil. Ledger’s role, of which a third had been shot when he died, is as a mysterious outsider called Tony who joins the troupe on their quest.

Gilliam’s initial reaction to the news was that he would have to call a permanent halt to the production. “I thought, ‘There’s no way I can make this film without Heath’.” But Amy and other collaborators lobbied hard to keep the film alive to honour the memory of its star.

“Ideas are floating around. Then finally we decided, ‘OK, let’s get three other people to take over the part’. And we were lucky because we have a magic mirror in this movie. Not every movie has a magic mirror. So you can very genuinely say that these other actors are different aspects of the character that Heath plays. And it works. The point was, we’ve got to keep going. It was a bit like half being there, but apparently on autopilot I can still do a few things.”

In casting the actors who would step into the role, Gilliam says that a key considerations was that they had to have been close friends of Ledger. Fortunately, Ledger’s friends included Johnny Depp, Colin Farrell and Jude Law, names that no doubt helped to persuade the film’s financiers that Doctor Parnassus was still viable. “It’s a hard thing to do, walk into a character and take over. But they did it,” Gilliam says.

He has no doubt that Ledger fully deserved the Oscar for his chilling portrayal of the Joker in The Dark Knight. “But wouldn’t it have been nice if they had noticed before? Even before Brokeback Mountain he was doing brilliant work. Everything he has done has been solid, even the earlier silly things. I first saw him in The Four Feathers and he just takes over the screen. He was what, 21 years old?”

The planned release date for Doctor Parnassus is in the autumn. “We want to be in that last third for the Academy Awards! Maybe we’ll get another award for Heath. We’re going to get as many awards as possible for him, long after he’s gone.”

Gilliam, who has a history of taking his audience out of their comfort zone, suggests that the film’s viewers might find some elements of the movie tough going. “I think there are going to be moments in Parnassus; I’m just waiting to hear what the audience does when they see certain shots. There are lines that we refused to change after Heath died. It’s like the script was prescient. It’s really spooky.”

Explore Film

• Film Reviews

• Trailers

• DVD

• London Film Festival

Editor's choice

• The 100 Best Films of the Decade
• 70 facts you didn't know about Marvel
• The 50 Biggest Movies Of 2010