Opera has always suffered from one basic problem: the eternal struggle between the librettist's words and the composer's music. It's a battlefield scattered with the bodies of cumbersome librettos and over-long arias, frustrated composers and depressed poets. Quite why anyone would want to enter the fray, let alone a successful novelist or an Oscar-winning composer, is a mystery.

But Scottish Opera has press-ganged the cream of Scotland's literary talent - including Ian Rankin and Alexander McCall Smith - into the service of a brave new 21st-century operatic movement: 15-minute operas. And it has worked: so great has been the demand for tickets that Scottish Opera has had to release 100 more this week alone.

“We're not trying to create a new art form,” insists the dramaturg Michael McCarthy. “Opera doesn't really function in 15 minutes. The idea is to explore the genre with a view to commissioning one full-length opera from the series.”

Is it a competition then? “No,” says McCarthy, somewhat wearily, “although there will almost inevitably be that element for audiences.”

Alongside Rankin and McCall Smith, the diverse new voices include those of the author Suhayl Saadi working with the composers Nigel Osborne and the sarod virtuoso Wajahat Khan in a semiimprovised, cross-cultural Glasgow romp; the novelist Bernard McLaverty and young composer Gavin Williams; and the team of poet/novelist Ron Butlin and composer Lyell Cresswell, who already have one chamber opera under their belt.

“I suspect that most people actually want to write an opera,” McCall Smith says. Dream Angus (adapted from his own short story) is an excerpt from a full-length opera he is working on with the composer Stephen Deazley and director/dramaturg Ben Twist. “One should have an X-ray which would reveal whether there is one there - and is it any good?” muses McCall Smith. The former professor of medical law at Edinburgh University is also a bassoonist and founder member of the Really Terrible Orchestra.

But opera is not an easy thing for authors, much less those whose every word becomes a cult as soon as it's on the page. The librettist's art is traditionally considered a literary backwater, subservient to the whim of the composer. Even the brilliant Lorenzo da Ponte, who wrote the librettos to Mozart's masterpieces Don Giovanni and Così fan tutte, was originally a defrocked priest on the run, who essentially became a librettist only because he told enough people in Vienna that he was one.

For Rankin, it's a completely new way of working. “Usually the first inkling my publisher gets of a plot is when I present the finished book, but with opera you've got to be open to other people's ideas. You can't be monomaniacal in the way that authors usually get to be monomaniacal.”

But if composer-led partnerships dominate the history of opera, there are plenty of great operas spawned by fiery relationships, and plenty of less successful ones endangered by the wordy - just think of Mozart driven to distraction by all those Metastasio rhyming couplets. Strauss and Hugo von Hofmannsthal, one of the great operatic relationships, spent two decades working “towards perfection”, from their smash-hit starting point of Der Rosenkavalier, by way of rarely meeting, never calling each other by their first names, and a mutual antipathy to “overblown” Wagnerian pomp. Wagner himself showed either the brilliance of just what can be achieved if a composer writes his own libretto, or the unmitigated horror - depending on your taste.

So what awaits these 21st-century librettists? The modern equivalent of the composer-librettist relationship is influenced by technology. Now two towering egos need no longer slap heads like bull elephant seals, but can work at either end of a very long wire. Thousands of kilometres long, if need be.

“We communicated mostly by e-mail,” Rankin admits, although it was for reasons of schedule rather than antipathy, even if his “very good” relationship with Craig Armstrong, the composer best known for his scores for Baz Luhrmann films Moulin Rouge and Romeo + Juliet, survived an early change of subject when Armstrong suggested a more narrative-driven alternative to Rankin's initial “technological, overcomplicated” police-station libretto.

“I didn't have a problem with changing the subject matter. It's the job of the music to get to the hearts of the people, and the words to get to the brain,” adds Rankin, before reading an extract from his thumpingly visceral libretto for Gesualdo, on the life of the 16th-century composer - a tale of murder and chromaticism - which opens with Gesualdo frenziedly stabbing his wife to death. “It's not all Reservoir Dogs, though,” says Rankin with a glint. “There's a bit of humour, too.”

If Rankin had a steep learning curve in writing for music, his friend Ron Butlin had a head start on the musical demands of his long-term collaborator, Lyell Cresswell, with whom he's working on The Perfect Woman, a cautionary tale of evil scientists and plastic surgery.

“I just provide the raw material for the composer to flesh out. I enjoy changing the word rhythms, putting in lots of action, shouting and screaming,” Butlin says. “Mine will be the one with the sweary words - Lyell always wants more vowels. Once I got a note from him saying, ‘Can you give me two lines of deep despair and a rollicking drunken song by the end of the week?'”

“For a writer,” McCall Smith says, “it's like doing exercises - I've never felt proprietorial about the words.” He is collaborating with the composer Stephen Deazley and dramaturg Ben Twist, and the team chose to work on McCall Smith's kitchen table rather than the internet. “I'm enjoying the rare delight of joint labours,” he adds.

“Theatrical directors have a very different way of telling a story, and ask pertinent questions such as ‘what is the point of that?' We don't write by committee, but in all other respects we've got a completely equal collaboration. Ours has been a most - the most - harmonious team!” he asserts, laughing, as if to allude to less tranquil pairings elsewhere.

Rankin, his next-door-but-one neighbour, alludes to a lot of secretive “flanking and jousting” among this newly formed community - reminiscent of a Handelian operatic heyday - in the race to complete their librettos.

McCall Smith elaborates: “In opera, unlike fiction, you don't have to defend against the idea of fantasy and artistic exploration. One must remember that opera is over the top anyway. There's an inherent unlikeliness about most operas, and that's probably the great attraction - you can indulge yourself.

“I didn't do this with a feeling that I had to create something new, to reinvent the genre,” he adds. “I was simply in search of beauty, in a way, and I think the music has brought that out.”

The five 15-minute operas will be performed at Òran Mór, Glasgow, Feb 29 to March 2 (0844 8471645) and The Hub, Edinburgh, March 8 and 9 (0131-473 2000)

Editor's choice

• The 100 Best Films of the Decade
• 70 facts you didn't know about Marvel
• The 50 Biggest Movies Of 2010