Spend an hour in the company of Jeff Tweedy and you would still need only one hand to count the moments of pure, unguarded relaxation. This, however, might count as one.

“It took me years to work out what went into a great pop song,” he says. Then comes a brief pause, in which Wilco’s frontman realises that such talk may inadvertently raise expectations for his own band’s new album, Sky Blue Sky. “But even if I know the recipe it doesn’t mean I have the ingredients. The way I see it, Abba made Dancing Queen and, from that moment on, every musician who has heard it faces the struggle to come to terms with their own imperfection.” It’s not as though he doesn’t have a point. At the same time, given that Tweedy is a 39-year-old man from Chicago and Abba were, well, Abba, you half expect such a statement to come with the mischievous twinkling of an eye. In fact, the singer is just getting started. “ Dancing Queen is just so glorious. After some time I have finally figured out how to play it on an acoustic guitar. If nothing else, it means that Abba and I can share just a little common ground.”

Getting Tweedy to admit that he has made his most accessible album since the sprightly aberration of Summerteeth (1999) was never going to be easy, but his opening utterance is as near as we might get to a tacit admission. What was once an aversion to categorisation has, in recent years, become closer to a fatal allergy. As the years go by, those who have met him on previous occasions learn from their mistakes. Those who haven’t lob terms such as “experimental” or “rootsy” his way and feel slightly embarrassed when Tweedy holds forth on why, actually, the opposite is the case.

It doesn’t take too much psychological excavation to find the cause of this condition. In the early 1990s, Tweedy formed Uncle Tupelo, and jump-started the Americana/alt.folk vanguard. It wasn’t meant to be this way, he maintains. “It was more that, after you get punk out of your system, you ask yourself how to write a song. And one way to start is by writing songs you hear the easiest. Invariably that would be like a Woody Guthrie or Hank Williams song.” By the time Uncle Tupelo recorded their final album in 1993, the superb Anodyne, Tweedy felt like an imposter. He didn’t know what Wilco was going to be when he started trading under that name. The years since then suggest he still doesn’t. To this day, there remain Tweedy-watchers who gauge each album according to how much it sounds like his old band. Wilco’s 1995 debut A.M. fared well in this respect. But by the time Yankee Hotel Foxtrot appeared in 2002, he had strayed beyond any recognis-able roots-rock comfort zone. Febrile meditations such as Radio Cure prompted their record company, Warners, to drop them – only for Tweedy to stream the entire album on the band’s website. In doing so, he kicked off a buzz that propelled Wilco from theatres to stadiums. And, in a deliciously ironic twist, the American Radiohead (as they were now touted) were signed up by Warners subsidiary Nonesuch.

To Tweedy, the hitherto unexplored musical detours all fed into a truer sense of who he was. Country music was something his parents had listened to but, as a child, he shared his elder brothers’ taste for leftfield Krautrock pioneers such as Amon Düül and Kraftwerk. “Was I considered unusual by my peers? I don’t know if that was the only reason people considered me unusual. The records were probably the least of it.”

If his brothers had been two or three years older they would have been fighting for territory on the same cultural battleground. But as the youngest by ten years, he was indulged by them.

“One time,” Tweedy recalls, “my brother switched colleges and brought all his stuff home. When he walked into the house he found me filling out a Columbia Record Club questionnaire – one of those promotional things where you get 13 albums for a penny. He ripped it up and said: ‘Look, I’ll just leave my records here.’

“It was actually very enlightening and exciting to have access to this world that wasn’t available to me, a world beyond mainstream radio.”

I suggest to Tweedy that, years later, it must be nice for his brother to correlate that early act of generosity to the effect that it had on his musical development. On the other side of a thoughtful pause, Tweedy chooses his words carefully. “I haven’t had a chance to be very close to him for a long time, because of alcohol and addiction. So I don’t even know what credit he’s aware of.”

Tweedy himself knows a thing or two about addiction. He has been teetotal for more than a decade, but harder to deal with is the struggle with the painkillers used to control his severe migraines. It was presumably in a bid to convey to the wider world how those migraines felt that Tweedy recorded one song from 2004’s AGhost is Born. Much of Less Than You Think is a wordless eight-minute hum of temple-throbbing density.

Tweedy’s memories of the sessions are also coloured by the depression from which he was suffering. Once the album appeared he said good-bye to his wife and two children and checked himself in to rehab. When he came out, his most immediate worry was that of having to revisit songs that, emotionally, stemmed from an intensely troubled place. “I was in such a dark state of mind when it was happening. Then a month later, when I had finally gotten healthy, that record ended up feeling like I wrote it to myself ahead of time.

“I was making significant changes in my life, so I had to ask myself: ‘Are there going to be songs that I don’t want to sing any more because they remind me of how bad I felt?’ Actually, it was a pleasant surprise to encounter these lyrics that were comforting to me and said a lot of things that I couldn’t have understood before I had gotten better.”

Tweedy was also bemused to discover that his problems had been reported in the nonmusic press. “I didn’t mind it, because it didn’t feel like it was really about me. It could have been anyone. I just happened to provide a certain set of circumstances that fitted into a preexisting template that the news needs. I mean, I’ve never gotten so much press in all my life.”

Speaking on the phone from his Chicago home, Wilco’s drummer Glenn Kotche commends Tweedy’s determination. “I’ve got nothing but admiration for what he has done. To go through that you have to face a reality that you can never go back to that place. I don’t mean this as a putdown, but he had to grow up a little bit.”

Five years ago, Kotche’s arrival in Wilco was deemed emblematic of the group’s detour into more experimental waters – Kotche is one of America’s foremost leftfield percussionists. But Sky Blue Sky is by some distance the most straight-ahead songs album of Wilco’s collective life – and it’s a development that the drummer is swift to embrace. “I think that in the past, Jeff got inspiration from different places, and in hiding the truth a little bit. But there’s a directness here that has benefited the songs.”

The directness is, indeed, startling. Had Elliott Smith survived to beat his demons and then sent his band packing he might have divined a song as exquisite as Please Be Patient With Me. Songs such as Sky Blue Sky and What Light have a folksy warmth that recall the best work of Tweedy’s old band. “I survived,” he sings in the former, with childlike fragility. “It’s good enough for now.”

When you’ve just made your most beautiful album and your most reconciled sounding album, whither the cliché about genius and pain? It’s a topic that Tweedy has had plenty of cause to ponder. “What was that quote about great art?” he smiles, “Younger artists sit around and wait for inspiration, while the rest of us get busy. I’ve been pretty busy lately.”

— Sky Blue Sky is released by Nonesuch on Monday. Wilco play Shepherds Bush Empire (0870 771 2000) on May 20 and 21. www.wilcoworld.net.

From margins to mainstream

R.E.M. The guitarist Peter Buck said it best: “We’re the acceptable edge of unacceptable stuff.”

RADIOHEAD Proof that introspection and experimentalism is no barrier to world domination.

FLAMING LIPS Odd-rockers are now as doted on as Kylie.

JOANNA NEWSOM A Marmite voice and classical harp a barrier to the Top Ten? Not any more.

Explore Music

• Festivals

• CD Reviews

• Live Reviews

• Classical

Editor's choice

• U2 at Nou Camp Stadium, Barcelona
• The 50 Best Movie Villains
• The 15 golden rules of theatre etiquette