IMAGINARY WORLDS are enjoying a cinema boom thanks to the wizardry of computer-generated images. And the film adaptation of Philip Pullman's trilogy His Dark Materials promises to be the greatest treat of all.

With its witches, talking bears and daemons (human souls visible as animals), the trilogy could become as successful as The Lord of the Rings or Harry Potter. But what is it about Pullman's vision, or J.R.R. Tolkien's, or J.K. Rowling's that has not only won their authors respect but inspired whole generations to embrace them? Why does Lyra's world convince us, while Christopher Paolini's Eragon does not? And why do authors build other worlds when the real one is so interesting and complex, and when children need to know more about real history, art and geography?

Sally Gardner, the prize-winning author of I, Coriander, thinks that it has to do with comfort. Her heroine, escapes from a cruel Puritanical stepmother by living in the world of Faerie while locked in a chest, emerging to triumph over her adversary when she is long believed dead. “Imaginary worlds are the playground for the mind. They allow a child or young adult to explore the things that worry and concern them most, without causing injury or harm,” she says.

Harry Potter — and Pullman's boy hero, Will — escape into parallel worlds when they face unbearable stress or danger. And although Lyra and Will part and return to their own worlds, some of the most interesting new fantasies have protagonists who reject returning from Neverland.

“Yes, we do need that ‘otherness',” Pullman says. “Why? Perhaps because in an invented world we can exaggerate and isolate things we want to write about, so they stand out more clearly — as I did with religious power in His Dark Materials.”

Successful imaginary worlds ask complex questions about identity and moral choice. One criticism is that they are too Manichean — there are no good orcs in Tolkien. But later authors are more sophisticated; in Rowling's world, good characters are often transformed into evil, and vice versa. Pullman's characters are even more ambiguous, with Lyra's selfish, ambitious parents finally sacrificing themselves for her; but there are no good priests or benign nuns.

The idea of a parallel world (whether Heaven, the Hereafter or Faerie) is as old as the fairytale itself, and making it original is part of the fun.

One of this year's most striking fantasy novels, Catherine Fisher's Incarceron, has a whole world that is a prison, absorbing and regenerating dead matter. The magic of Eoin Colfer's futuristic fairies owes as much to technology as spells; Rowling's wizards are amazed by what “Muggles” have achieved without magic. Sally Prue's hero in The Truthsayer, who travels from a magical world to ours, is struck by our freedom. Seeing our own world through their eyes makes us realise what is special about it.

Discovering differences is a big part of the appeal. Terry Pratchett's Discworld, which not only parallels but satirises our world, has morphed into modernity, as Ankh-Morpork's social infrastructure is found wanting. (The most recent book, Making Money, introduces drains and banking.) Where Tolkien looked back to high deeds and chivalry, modern writers such as Pratchett, Colfer and Rowling examine the present. The great fantasy writer Ursula Le Guin suggests a vast history in A Wizard of Earthsea, where lesser authors get bogged down in small details.

For a reader to fall in love with an imaginary world, the place must not simply have tides that behave according to two moons; it depends more on character and plot. Diana Wynne Jones, one of the most gifted and entertaining of children's writers, wrote the Tough Guide to Fantasy Land in 1996, poking fun at the tired tropes of medieval costume and superstition. None of this has stopped authors such as Robin Hobb, Colfer or Rowling from following such archetypes, but they renew them.

Done badly, fantasy is more risible than any other genre, perhaps because there is such a fine line between heroic endeavour and bathos. Success isn't just a matter of consistency (Tolkien despaired of C.S. Lewis when he introduced Christian myths such as Father Christmas into a world with nymphs and satyrs). A gifted writer makes the mundane magical and the magical mundane. We believe in everything they tell us because the ultimate magic is to make us think that what they describe is true.

Explore Books

• Book Extracts

• Book Reviews

• Books Group

Times Recommends

• Avro Lancaster: Owner's Workshop Manual
• iD: The Quest for Identity in the 21st Century by Susan Greenfield
• Fatal Misconception by Matthew Connelly