Since his arrival in England, Clive James had felt awkward and misplaced. The young Australian spent the 1960s trying to find a niche in his new home, but by the turn of the decade he was working anywhere he could; as a literary critic, media commentator, songwriter, scriptwriter and humorous pundit. The variety of jobs testified to his versatility and intelligence, but offered little security to the young expatriate.

But things were about to change, as James found when he arrived for an audition as a presenter on Cinema, ITV’s film show. “Luckily, being a misplaced person was the thing to be,” James says, as he recalls the break that changed his life. “I got into television because of one of the great servants of the British media, Lord Bernstein (the former chairman of the ITV station Granada),” says James. “He decided that I would be just right for his channel because nobody knew where I came from. He was sick of the British class system and here was a guy who didn’t fit into it anywhere.”

There is an irony that James’s entry into television owed much to his ambiguous identity. Television work brought him fame and financial stability, allowing him to pursue interests in high art and pop culture. Over the next three decades, his CV expanded to include roles as critic, novelist, satirist, poet and memoirist. But even as he became one of the best-known cultural figures in Britain, combining swashbuckling wit with metropolitan sophistication, an ambivalence surrounded his career. While a new volume of his memoirs testifies to James’s enduring popularity, he still gets irked at the sniffy confusion about him.

“I have a permanent blurred image problem,” he says. “Because wanting to talk about all these things was fundamental to me, it inherently produces what appears to be a divided personality. I think it’s united, but it looks divided. It became a perennial question: who does he think he is?” North Face of Soho, James’s new book, was not written as a rejoinder to such doubters, but still provides an eloquent insight into the impulses that have driven his varied career. The fourth (and penultimate) instalment of a series that began with 1979’s Unreliable Memoirs, it covers the years between 1968 and 1982, as James abandons the showbusiness ambitions of the Cambridge Footlights to become a heavyweight literary critic, a populist television columnist for The Observer and, finally, a rising television personality.

In person, James is engaging, laconic and articulate. He also knows his audience, conscious of his Irish surroundings, quoting WB Yeats and linking his own progress with that of his Irish peers: “It’s often underestimated, the freedom that the Irish have in the British media — Terry Wogan is one of the biggest broadcasters in Britain. You’ve got to be from somewhere else. The Aussies share that privilege.”

In the book, James chronicles his rise vividly. The narrative is driven by lethally funny vignettes and enlivened by at least one howler in the mistaken assumption that Conor Cruise O’Brien has passed away: “Oh my God, what a catastrophe — I’ll have to cover my arse on that one, pronto,” he says, visibly taken aback. But such distractions aside, James’s urge to explore creativity in all its manifestations emerges as a driving force behind his endeavours.

“I was as interested in championship ice skating as I was in Beethoven’s late quartets. They were works of art of different intensity and on different levels, but I think they came from the same impulse,” he says. Above all, though, the image emerges of a man unsure of his direction. It is a surprising portrait of someone so sure-footed on the page and on the screen, but a deliberate one.

“It’s a portrait of the artist as a young man,” James says. “But I’m a man who’s not Stephen Dedalus, because the only thing I don’t like about Stephen Dedalus is his air of superiority: I can’t stand the prick. And I always thought, if I ever do one of these, I’ll do the other thing: I’ll do trial and error, an air of inferiority.

“I haven’t got any natural wisdom, anyway. I had to learn everything by trial and error. But it’s simply true to life, and especially in complicated things like the arts and the media, that you feel your way forward mainly by getting things wrong and admitting it. In fact, without that capacity for self-criticism, you crash and burn quite early. That was my principle for writing the books from the first day. It’s not a formula, it’s an actual vision of life.”

James’s belief that his life owes as much to chance as to natural talent and hard work is not the false modesty of the successful star, but the hard-won credo of an intelligent, reflective man who experienced ill fortune early on. Born in Sydney in 1939 into “a bog-standard proletarian background”, James lost his father at a young age. Having been captured by Japanese forces early in the Pacific war, his father survived the notorious prisoner-of-war camps, only to die when the American plane carrying him home crashed. It is an event that still haunts James, now more than ever.

“When I was young, I didn’t think much about it,” he says. “And there’s a danger later on when you do start thinking about it that you’re consciously developing a theme, maybe even forcing the issue. The truth is that it was the effect on my mother that determined my life.

“It wasn’t so much that my father never came home — he was just an absence. I saw, as a presence, the devastating effect on my mother and I still feel, even today, that I can’t live in a society that denies opportunity to women. My whole relationship to the Islamic revolution which is now upon us depends on that. And that continues to be so. First of all, I feel like I owe it to him, because I’m leading this life, to get something done, and, second, I’d like to make it up to my mother. She only died a couple of years ago.”

He pauses for a second, then continues. “I’m not giving a good answer here, because you’re right in the centre of the furnace; this is the driving element. But one of the reasons I go on working isn’t just that I haven’t finished expressing myself yet — I’ve got more things I want to do — it’s also that it would be criminal not to live a productive life when their lives were blasted by this. It was mainly just an accident, but I saw it, and it made me loath luck. I’ve always had far more than my fair share of it, but I don’t like luck.”