There was one dreadful moment during my interview with the artist and film-maker Matthew Barney when I thought I’d lost him. I was asking him why he has felt it necessary to perform, variously, as Gary Gilmore, General MacArthur and a kind of mythical Manx satyr. And why, in his latest film, Drawing Restraint 9, he attends a traditional tea ceremony alongside Björk (his real-life partner with whom he has a four-year-old daughter, Isadora) while on the mothership of the Japanese whaling fleet, with a shell on his back and the shoulder blades of elks clamped to his feet.

“I was educated to avoid . . .” he says, and pauses. He does this often. “. . . to avoid the mannerisms of . . .” he says, and pauses.

Barney has come to London this week from the riverside studio he inhabits in Brooklyn, New York, for the British premiere tomorrow of Drawing Restraint 9, which will then have a short run at the Gate Picturehouse in Notting Hill, West London. It’s an ocean love story, but one played out alongside a more esoteric, emblematic narrative in which a sculpture made of petroleum jelly is formed and reformed on the ship’s deck. It’s the first big film that he has produced since the conclusion of his epic five-film Cremaster cycle, a series of narratives exploring the biological process of creation, made between 1994 and 2002, and for which he is best known.

It’s also the first screen role and soundtrack that Björk has worked on since Dancer in the Dark,the Lars von Trier film released in 2000. And it was while Björk was in New York promoting that film that the couple first met.

As well as attending the film premiere, Barney will open an exhibition at the Serpentine Gallery of all his work to date on the Drawing Restraint series. It’s a body of art about creativity and form, freedom and restriction, and at the Serpentine it will unfold through a range of materials, from documents relating to the first instalment in the series to the 16th, which will actually involve Barney clambering about among the Serpentine’s exhibits. There will be more films, drawings and artefacts and also sculptures such as Ambergris, a recreation of the long, stony, undigested matter that is vomited up by whales and that plays a symbolic role in Drawing Restraint 9.

All this might seem strange territory for a boy who grew up in the American heartland of Idaho. In fact, his mother is an abstract painter, yet it was only after Barney gave up football, and then a medical degree, that he moved to Yale’s art department. He matured fast, and when he arrived in New York in 1989, he attracted instant attention; by 1991 he was international news.

The last instalment of his Cremaster cycle was screened in London in 2002, accompanied by a display of related sculptures. Barney says it was his work on those sculptures that seeded ideas for his latest film. “I made large cast petroleum-jelly pieces which collapse under their own weight,” he explains. “They were very like the Drawing Restraint sculptures in that they were put into a completely uncontrollable situation – you had no idea how they would behave. They each had a different ratio between the jelly and the microcrystalline wax that it’s cut with and they’re affected by the temperature of the room – all these factors change the behaviour of the sculptures. It was then that I began to think about making one of these sculptures on a boat, where the movement of the ocean would affect the behaviour of the casting.”

For some, Barney’s success is to have found ways to talk about subjects such as sex and gender through fabulous allegories and slick films.

Others are attracted to his persuasive yoking of film and sculpture. But for others, Barney’s work is tediously hermetic and self-indulgent. Had he been at all concerned about such accusations, though, he surely would not have engaged Björk to work on this new film, but he says that it simply made sense. “The subject matter was something we were both interested in – Japan, the relationship to Shinto that whaling has. Also, making a love story out of a Drawing Restraintwork seemed preposterous to me at first, but when I decided I wanted to do that, it seemed logical to work with Björk. The collaborative aspect of working with a composer is familiar to me, and that aspect of working with a visual artist is familiar to her – so it felt natural.”

Drawing Restraint 9 is actually less revealing of the couple’s relationship than it might promise. It is a love story, and a beautiful one, but it is also abstracted. Cremaster was far more revealing, Barney says. “It’s much more autobiographical. For instance, Cremaster 1 features the town I grew up in and the field I played football on from the time I was 12 to the time I left Idaho.”

It may be autobiographical, but that’s not the same thing as being revealing, and Barney remains intriguingly distant about some matters. It was noticeable that that lengthy pause he took came when I asked him why, being someone who seems reticent as a personality, he is drawn to such flamboyance in performance. He appeared ready to respond to this – and say something about his upbringing – but then wouldn’t, and rather wilfully bent his answer around once again to address his sculpture.

Even in talking about that, Barney can be cautious, but he did reveal that travel is what always spurs new work. He likes island cultures: hence he was drawn to Japan for Drawing Restraint 9, and in the past he has filmed in Ireland and the Isle of Man. So, any holiday plans?

“Well, I will say this about what I’m working on. I’m planning to go down the Nile soon.” I think he’ll find some material there.

— Matthew Barney, Serpentine Gallery, Kensington Gardens, W2 (020-7402 6075), tomorrow until Nov 11. Drawing Restraint 9 is at the Gate Picturehouse, W11 (0871 7042058), from Sept 28

Editor's choice

• U2 at Nou Camp Stadium, Barcelona
• The 50 Best Movie Villains
• The 15 golden rules of theatre etiquette