A box-office assistant whose first novel was rejected by 20 agents and publishers before one editor spotted its potential has won one of Britain’s most prestigious literary prizes.

Catherine O’Flynn, 37, joined the likes of H. G. Wells, William Golding, Graham Greene and J. K. Rowling by finding spectacular success after a string of rejections when her mystery story What Was Lost took the First Novel prize at the Costa Book Awards last night.

While agents and publishers had all found ways to tell O’Flynn that her book was “not one for them”, Joanna Trollope and her fellow judges were unanimous in their praise. They called it “an extraordinary book”, describing it as “a formidable novel blending humour and pathos in a cleverly constructed and absorbing mystery”.

O’Flynn, from Birmingham, was one of the five category winners in the Costa awards — previously known as the Whitbread prize. The “most enjoyable” first novel, novel, biography, poetry and children’s book were chosen from 553 entries and will compete against each other for the title Book of the Year and a £25,000 prize, to be announced on January 22. The category winners each win £5,000.

Beatrix Potter would have been familiar with O’Flynn’s struggle for recognition. The Tale of Peter Rabbit, now considered to be one of the most important children’s book of the 20th century, was rejected by numerous publishers before it finally came out in 1900.

Kenneth Grahame’s The Wind in the Willows, Wells’s The War of the Worlds and Golding’s Lord of the Flies all faced initial rejection.

Rowling, whose Harry Potter books have now been read by more than 350 million people, was turned down by a dozen publishers before she was picked up by Bloomsbury; Wilbur Smith, whose books have sold tens of millions of copies, almost gave up writing when every major publisher in London spurned his first attempt more than 40 years ago.

The record is held by the crime novelist John Creasey who suffered the indignity of receiving an unbroken succession of 743 rejection slips. He went on to enjoy some divine justice, selling tens of millions of books.

Jack London received at least 600 rejections before selling his first story. These are collected in an exhibition at the House of Happy Walls, London’s estate in Sonoma County, north of San Francisco.

Another frustrated author confirmed last year what other aspiring writers have long suspected — that even Jane Austen would have difficulty finding a book deal today.

David Lassman, who had tried in vain to interest a publisher in his own novel, discovered that only one of 18 publishers and literary agents spotted Austen’s writing when he retyped the opening chapters of three of her classics — Pride and Prejudice, Northanger Abbey and Persuasion — and submitted them under a false name. He changed only the titles and the names of the characters. O’Flynn’s break came when a a small independent publisher, set up in 1998 by writers tired of endless rejection, recognised her potential.Alan Mahar, publishing director at Tindal Street Press, said: “What Was Lost doesn’t fit neatly into the standard pigeonhole that publishers like to put their books into. Publishers are hidebound by those categories, which are largely ruled by the book trade and Waterstone’s.”

Ten of Tindal Street’s titles have been listed for national literary prizes. They include the Booker in 2003, with Clare Morrall’s debut novel, Astonishing Splashes of Colour, which had been turned down repeatedly by other publishers. “Again we could see the potential,” Mr Mahar said.

But other publishers seem to be getting something badly wrong, to judge by the latest figures released to The Times yesterday by the Nielsen Company, which tracks all book sales nationwide.

Of 200,000 books sold last year, 190,000 sold fewer than 3,500 copies. More damning still, of 85,933 new books published, as many as 58,325 sold an average of just 18 copies.O’Flynn found the inspiration for her book when she was working at a record store in the Merry Hill shopping centre near Dudley, West Midlands. The novel tells the story of a disenchanted assistant manager and a security guard at a shopping centre who seek out the truth about a child who disappeared 20 years ago.

Mr Mahar said: “It is an unusual book — very intelligent, very funny, and very well plotted, saying something quite new about our consumer ways, our obsession with shopping but it is also a mystery story of great emotion.”

Winners

First Novel Catherine O’Flynn for What Was Lost
Novel A. L. Kennedy for Day
Biography Simon Sebag Montefiore for Young Stalin
Poetry Jean Sprackland for Tilt
Children Ann Kelley for The Bower Bird

Win the shortlist

Enter our competition to win all 20 books in the Costa Book Awards 2008 shortlist and a Costa card with £20 credit to spend at any UK Costa store. We also have two runners-up prizes of a Costa card credited with £25.

Answer this question:

Which author won the Book of the Year award at last year's Costa?

Email your answer to books@timesonline.co.uk and make sure to include your name, address and daytime telephone number. Entries must be received by 5pm on January 17, 2008.

Explore Books

• Book Extracts

• Book Reviews

• Books Group

• Audio Books

Times Recommends

• The Sunday Times 100 best holiday reads
• What will we be reading this summer: the booksellers predict
• Literary festivals: a survivor’s guide