Assembled from the journal of Harrison Shepherd, occasional newspaper cuttings (both real and imagined) and odd interjections from his archivist and stenographer, the redoubtable Mrs Violet Brown, Barbara Kingsolver’s The Lacuna recounts the life of Shepherd, a forgotten figure of postwar American literature that never was.

The novel begins, however, with the opening chapter of his incomplete memoir and his recollections of a time on Isla Pixol in Mexico when he was 13. The reasons the memoir remained unfinished emerge only later, with the same creeping melancholy that permeates much of the prose. The Lacuna, then, is a complex novel of devices and hide and seek, even a kind of palimpsest: who is Shepherd as both man and author and, indeed, the writer behind the writer? Repeatedly we are told (first by Frida Kahlo, no less) that “the most important thing about a person is always the thing you don’t know”, while the lacuna of the title is both a submerged cave into which the young Shepherd swims to test his lungs and, literally, the missing section of a manuscript. Even the cut-and-paste formation of this novel, therefore, builds its meaning — we can learn most from what is unsaid, unwritten, unreported or lost. This structure is a dizzyingly clever conceit. Kingsolver’s command of her material counters any giddiness and ensures that the narrative grips from the start.

Shepherd has arrived in Isla Pixol in 1929 with his Mexican mother, recently split from his American father in the hope of snaring an oil baron. Every morning they are petrified by the island’s howler monkeys that shriek a terrifying dawn chorus. These howlers will come to signify a more sinister kind of hubbub. Mother’s pragmatic ambitions soon lead them to Mexico City on the promises of another tycoon, and the young Shepherd secures employment mixing plaster for Diego Rivera’s historical murals in the National Palace. But he is sent back to America at 16 to suffer two years at Potomac Academy, a Washington prep school. Here, he first experiences the latent brutality of his father’s country in the suppression of the “Bonus Army” riots of First World War veterans in 1932. He also experiences the first stirrings of his nascent homosexuality.

Firmly established as an outsider, Shepherd returns to Mexico and the Rivera-Kahlo household in 1935. Now, however, the eccentric couple have house guests — Lev Trotsky and his wife Natalya. On the run from Stalin’s assassins, Trotsky is composing the intellectual foundation of the international workers’ movement. Kingsolver paints him as a figure both pitiable and heroic; though part of her skill is to ensure that his bloody demise comes as something of a shock. It is a period that provides some of the most effective and affecting passages in the book — Trotsky’s relationship with Kahlo, his relationship with his wife, Kahlo’s with Rivera, and Shepherd’s with them all.

After Trotsky’s murder Shepherd has little choice but to flee and settles in North Carolina. America is at war and he is temporarily thrilled by the effort. “But you should see the Yanks now,” he writes to Kahlo. “Swearing unity with people from across borders just as you and Diego used to do.” It is thanks to Kahlo’s intervention that Shepherd’s writing begins to take shape. His novels are tales of conquest and colonialism in Ancient Mexico that tap public appetite for the former and fail to impinge on their ignorance of the latter; at least at first. Shepherd’s decline into persona non grata, caught on the wrong side of early anti-communism and, ultimately, the House Committee on UnAmerican Activities, feels too inevitable, though Kingsolver’s empathy with her protagonist never wavers.

A decade ago Kingsolver published The Poisonwood Bible, which with its challenging subject matter of neocolonialism in Congo became something of an unlikely bestseller. In many ways The Lacuna, her first full-length novel since, is a companion piece, another indignant and erudite tale of past American folly and present denial that sidesteps a hectoring tone thanks to its author’s ability to create truthful, fullbodied people. Addressing more current issues than it is possible to illuminate here (most notably the climate of fear as hollered by a howling media), this is urgent, political storytelling that confirms Kingsolver as one of the most important contemporary writers. Ultimately, she allows Shepherd to disappear into his lacuna. But, in doing so, she stands up for the enduring and redemptive power of a good story to teach and, above all, to give us reasons to believe in the possibility of change.

The Lacuna by Barbara Kingsolver (Faber and Faber, £18.99; Buy this book; 528pp)

Times Recommends

• The 100 Best Books of the Decade
• The Times picks the 50 best paperbacks of 2009
• The conversation: James Ellroy