A few years ago, American wunderkind John Mayer played his first gig in London in a tiny, sticky-floored, smoky club. His (mostly American) audience, almost certainly under 30 and very definitely female, knew every word of every song from Mayer’s first album, Room for Squares. When he sang his mega-anthem Your Body is a Wonderland, he was all but drowned out by the hysterical screaming. Back then, Mayer was a cute campus pin-up from Connecticut with a knack for writing catchy, sexy songs that spoke of teenage longing and twentysomething angst. His fame was born of word-of-mouth on the campus circuit, not via a full-blown, multimillion-dollar marketing campaign.

How things have changed. Last month, Mayer, now a five-time Grammy winner with three albums under his belt, was back in London playing to a jam-packed Albert Hall. The audience (ranging from teens to fortysomething Brits) also knew all the songs. This time, though, Mayer wasn’t playing them alone. Sprinkled throughout the set were bluesy Stratocaster guitar jams with his band – a nod to the fact that critics are variously calling Mayer “the new Clapton”, “the new James Blunt” or, less flatteringly, “the new Paul Young”.

Mayer seems unbothered by the plaudits and the brickbats. By his own admission, he has a split personality. “I represent two factions of the same life,” he says. “On the one hand, I want to mess up. To slip, you know?” He emphasises the word with sincerity. “But then the other part of me runs John Mayer Incorporated, the business, and I’m incredibly controlled about how people see me.”

In person, Mayer is every inch the star. Six foot three, and dressed all in black, he has a sleepy-eyed handsomeness that has helped win him a legion of female fans (and aided and abetted much-publicised liaisons with singer Jessica Simpson and actress Jennifer Love Hewitt). He doesn’t hold back about his talent either, to say the least. “It’s a huge gift, like any minute you can hover off the ground. It’s like having a superpower.”

Evidence of the other John Mayer – the one who wants to “slip” – edges out in conversation when he’s candid about the insecurities of getting older. “Getting to 27 was like turning 30 for me [he’s 29 now]. That was my year of crying uncontrollably. I’d sob in the line at the bakery because I’d heard a song from my childhood.” He’s also frank about having suffered from anxiety attacks in the past. “To this day, I carry a Xanax in my back pocket. It’s an insurance measure.” During his recent Albert Hall gig a couple of “screamers” in the audience managed to make Mayer blush by shrieking that he should disrobe. Mayer demurred. “I’m getting a bit of a belly these days,” he said bashfully.

This combination of arrogance and disarming vulnerability mirrors the abilities that have made Mayer such a success. He’s undoubtedly a virtuoso musician – his blues guitar and tender vocals are lauded by everyone from Elton John to Eric Clapton. But his willingness to reveal his insecurities, to hang his dirty washing out to dry, is what makes him such an effective songwriter. And nowhere is this more in evidence than on his latest album, Continuum. “For me, this record is the crossover. I’ve learnt not to care too much what other people think about me, and out of that perfect carelessness you can make the perfect record. This is the first endeavour in my entire life, musical or otherwise, that I did not cop out for a second on.”

It’s certainly a progression from the straightforward acoustic balladeering on his first two albums – funkier and more influenced by his long-term love for old-school blues. But Mayer had already made moves away from his sensitive image when he recorded a fiery blues/rock live album, Try!, with the John Mayer Trio, a group he formed with elite session musicians Steve Jordan on drums and bassist Pino Palladino. Along the way, Mayer, who had never really been much of a drinker, developed a taste for scotch. Yet he doesn’t drink that much. “I’m a lightweight. You have to swallow so much to be a drunk. I just go till I feel funny and then I stop. I’ve never vomited from alcohol.” He pauses. “The thing is, if I started drinking heavily then I think I’d probably be the last person at the bar every night.”

It’s certainly true that there’s an obsessive quality to Mayer. You get the sense that when he does something he has somehow to do it more intensely, or bigger, than anyone else. The way he collects watches is symptomatic. “I love hunting them down and I keep them in a safe at home. I must have 60 vintage Rolexes, which are just so beautiful. I go online and find out which dial variation I’ve got of each model. I have a huge appetite for information. I’m a geek really.”

Once again, his spot analysis is spot on. But he admits to being a geek with a rock star’s swagger, his smile wreathed in his own particular mix of cockiness and angst. “I’m so tired of thinking about credibility it makes me sick. I’m only just learning how to loosen up.”

Continuum is released on October 16

Explore Music

• CD Reviews

• Live Reviews

• Classical

Editor's choice

• The 100 Best Films of the Decade
• 70 facts you didn't know about Marvel
• The 50 Biggest Movies Of 2010