That wealth and good taste do not always go together is nowhere better illustrated than in the Middle East. There, oil-slicked rulers have built gaudy monuments to their billions. But now the gold-plated palaces, revolving skyscrapers and seven-star underwater hotels have a tasteful challenger. The Museum of Islamic Art, in Doha, the new national symbol of Qatar, is an understated Gulf icon. And it might just be the best new museum or gallery building anywhere.

IM Pei, the Chinese-American architect behind the Louvre’s glass pyramid, agreed to undertake the project — his last, at the age of 91 — on condition that the Emir of Qatar reclaimed an island in the Gulf on which the new institution could sit like a citadel, without being encroached on by any other building. Pei’s stubbornness paid off. As you arrive by dhow, the vast, tiered limestone cubes of the 400,000 sq ft structure tower over you, giving the building colossal, instant impact. Yet thanks to the soft detailing, notably the Islamic geometric patterning and traditional Arabic arched windows, the structure is not severe.

Inside, the huge atrium is illuminated by a 150ft-high glass curtain wall looking out over the emerald sea. The galleries, with exhibitions of calligraphy, textiles and ceramics from the 7th century onwards, shoot off in every direction. Standing under the silver dome, it’s hard to disagree with the director, Oliver Watson (formerly of the V&A and Oxford’s Ashmolean), that it is “the most spectacular space”. When it opens in November, it will, at the very least, do for Doha what the Guggenheim did for Bilbao — put it on the arts map — but the project is about more than Qatar, modern architecture or Islamic art. It is the first shot in a cultural revolution that could transform the arts across the Gulf.

Reformist sheikhs have spent the past decade using their wealth to snap up financial institutions and retail and leisure brands in an effort to transform the city-states they rule from one-camel towns into global business and tourism hubs. Now they want to use it to acquire something that is impossible to price and may, in fact, be impossible to buy at all: culture.

Like latter-day Renaissance aristocrats, the rulers of Qatar, Dubai and Abu Dhabi are sinking £100 billion into grandiose galleries and museums. As well as the new Museum of Islamic Art, the Emir of Qatar, Sheikh Hamad bin Khalifa al-Thani, has four new museums on the drawing board, including an extension to the National Museum designed by the French modernist architect Jean Nouvel.

Abu Dhabi is betting £20 billion that it can transform barren Saadiyat island, just off downtown Abu Dhabi, into a 21st-century version of the pyramids of Egypt.

The city-state’s ruler, Sheikh Khalifa bin Zayed al-Nahyan, has persuaded Frank Gehry to design the latest — and, at 450,000 sq ft, the biggest — branch of the Guggenheim. Nouvel is behind the first offshoot of the Louvre outside Paris. Zaha Hadid, the London-based Pritzker prizewinning architect, is designing a performing-arts centre, and Norman Foster the national museum. The new institutions are scheduled to open in 2012.

Dubai is opening galleries and an opera house, as well as hosting arts festivals, but it is devoting most of its efforts to becoming an art entrepôt. Sheikh Mohammed bin Rashid al-Maktoum, the emirate’s ruler, recently lured Sotheby’s and Christie’s to the city.

Clearly, the sheikhs believe that by mixing their black gold with the genie of western architectural and artistic expertise, they can create a cultural oasis where a new generation of art-lovers and artists will grow. The long-term aim? To make the Gulf, a potential nexus between East and West, as much of an emerging artistic power as a financial one.

With oil at record prices, the estimated £100 billion bill for all the new developments is small change. The question is, will these museums attract local and international visitors or turn out to be white elephants, like the palaces Arab rulers built during the last oil boom?

Even their backers acknowledge that luring westerners will be tough. It’s not just that travellers are reluctant to fly on airlines whose names they cannot pronounce, to cities they cannot find on a map, to look at art, much of which they will not understand. With the conflict between the West and Muslim extremists still raging in neighbouring Iraq, it’s hardly a good time to be marketing “brand Islam”. Watson concedes that many overseas visitors will be “more curious than passionate”.

That is one reason both Qatar and Abu Dhabi are spending so much money on the museum and gallery buildings themselves. Pei has already proven his worth in Doha. If the scale models displayed in the Emirates Palace Hotel, in Abu Dhabi, are even half realised, Saadiyat island will become an architectural destination in its own right.

Editor's choice

• U2 at Nou Camp Stadium, Barcelona
• The 50 Best Movie Villains
• The 15 golden rules of theatre etiquette