Next week the winner of the Booktrust Teenage Book Prize will be announced. It is the only national prize for teenage fiction and will be judged by a mixture of teenagers (winners of a short-story competition) and adults (an author, a librarian and myself, a journalist).

Why do we need such a prize? After all, there's an argument that “teenage fiction” is a needless genre: when you're done with Dahl and company, why not move straight on to Salinger or Sylvia Plath, Daphne du Maurier or Dickens?

Some do. But most teenagers have to cross a huge no man's land: a place from which the Alex Riders and Tracy Beakers of childhood seem silly and irrelevant, and the world of adult literature looms, vast and forbidding, in the distance. It's here that many lose their way, condemned to endlessly reread the Harry Potter corpus or, worse still, abandon books.

Good teenage fiction reduces the reading casualties. It challenges and inspires in a language that isn't patronising or abstruse. It tackles the crucial themes: identity, sex, family, faith. It keeps the story glands going.

And some of it is among the best fiction being written today, regardless of the target age range. An adult reading Mark Haddon, Meg Rosoff or Mal Peet doesn't need to make any intellectual adjustments: once the book begins, the genre distinctions end.

For teenagers to benefit from this rich pool of literature, they need to know about it — and that's where Booktrust, a charity dedicated to promoting books and reading, comes in. By the end of the year its Booked Up scheme — in which pupils choose from an eclectic list of 12 titles, including Sally Gardner's novel of 17th-century London, I, Coriander, and Joshua Doder's zippy canine adventure A Dog Called Grk — will have provided half a million 11-year-olds with free books.

The Booktrust Teenage Book Prize brings a similarly diverse range of books to older readers: this year's shortlist (below) veers from present-day Luton to Renaissance Italy, featuring vampire hunters, cross-dressing orphans and manic depressives. All six novels are compelling and thought-provoking: like lifebuoys on an open sea, they should be grabbed with both hands.

The Shortlist:

The Medici Seal by Theresa Breslin

Corgi, £6.99

Ignore the cover, which makes this look like a half-baked rip-off of The Da Vinci Code: in fact, it is a brilliantly written story about a gypsy boy taken in by Leonardo, rich in its detail and epic in its sweep.

Leaving Poppy by Kate Cann

Scholastic, £5.99

Amber, feeling suffocated by her disturbed sister Poppy, escapes to a relaxed student house on the coast. But odd things start happening at night, and Poppy is never quite far enough away ... A vivid supernatural and psychological thriller.

The Penalty by Mal Peet

Walker, £6.99

A South American sports journalist gets embroiled in a kidnapping, while several hundred years ago a young man is taken from his home and made a slave. A sophisticated and beguiling tale of the real and spirit worlds.

Here Lies Arthur by Philip Reeve

Scholastic, £12.99

An orphanmgirl named Gwyna gets taken in by Myrddin, a crafty old storyteller working for a power-hungry warlord: one Arthur. Myths are turned inside-out in this dark and ingenious historical novel.

Just in Case by Meg Rosoff

Puffin, £6.99

Fifteen-year-old David Case thinks fate is out to get him, so he changes his name to Justin, adopts an imaginary dog, and attempts to seduce an eccentric older girl — but fate is not so easily fooled. Witty, wild, and aching with existential angst.

My Swordhand is Singing by Marcus Sedgwick

Orion, £6.99

A Transylvanian forest in the 17th century is the setting for this thrilling and tautly written fable that gets to the root of the vampire myth. Not one simply for horror-loving boys.

Times Recommends

• The 100 Best Books of the Decade
• The Times picks the 50 best paperbacks of 2009
• The conversation: James Ellroy