Superficially, the substance of my first two books couldn’t be more different. The first, Moondust, describes a search for the nine remaining men who walked on the moon between 1969 and 1972, while the second, which I’m in the process of writing, concerns a search for the soul of the internet visionary Josh Harris, Manhattan’s Great Gatsby through the wild years of the Silicon Alley bubble in the late 1990s. Nevertheless, both focus on a few years in which a group of creative young people promised an evolutionary leap for humanity, then watched their extravagant dreams turn to dust. One point of divergence, however, was the way music resolutely demurred from entering the new story, because after the experience of Moondust, I’d assumed it would; imagined it always did.

In retrospect, I shouldn’t have been surprised that music forced its way into the narrative of that first book. The Space Age ran from the launch of the Soviet Sputnik satellite in 1957 to the end of 1972, when the last two Americans stepped off the surface of the moon, thus neatly coinciding with the first age of rock (which might be viewed as running from the release of Elvis’s Heartbreak Hotel in 1956 to the rise of its first postmodern sub-genre, glam rock, in the early 1970s). Both music and space exploration were driven by the same demographic, because at the time of the first landing in July 1969, three weeks before Woodstock, the average age of staff at mission control was 26. “These people had never tasted failure,” the flight director, Gene Krantz, commented. “So they had no fear of it.”

All the same, I’d imagined myself to be writing a book about humanity’s relationship with space, and to life on Earth, themes that don’t necessarily imply the intrusion of music. Yet it seemed that every time I turned around, there was another song, album, artist or performance waiting — until all I could do was relax and let it come, with educative and often amusing results. By the time I’d finished writing, I already knew that I wanted the book to have an album companion.

Sometimes, these musical incursions were weirdly direct. Frank Zappa’s father was a missile systems engineer at Edwards Air Force Base, in the Mojave desert while Neil Armstrong was testing the first supersonic X planes there; Martha and Rufus Wainwright’s grandfather, Loudon II, was one of the Life magazine reporters embedded with the families of the astronaut corps. “Oh, he was a doll-baby,” recalled Rene Carpenter, ex-wife of the Mercury Seven astronaut Scott. “So sensitive, always crying.”

Equally weird and direct was Armstrong’s revelation by e-mail that the “strange, electronic-sounding music” his crew-mate Mike Collins reported him taking into space was, in fact, Music out of the Moon, an album of theremin pieces by Dr Samuel J Hoffman. The first man on the moon was a music fan who had played Dixieland jazz at school, and had taped that album from his own collection, choosing a soundtrack most often associated with 1950s sci-fi B-movies for his own real-sci voyage into space: eccentric, yet somehow perfect. By contrast, on the knockabout mission, Apollo 12 (featuring lightning strikes and joy rides round the back side of the moon, where Nasa couldn’t see them), the lunar-module pilot Alan Bean took The Girl from Ipanema and the Archies’ Sugar Sugar to bounce around to, while the captain, Pete Conrad, drove him crazy with a whole cassette of country and western.

When I called Brian Eno to inquire into his majestic Apollo album, which he wrote to soundtrack the Oscar-nominated documentary film For All Mankind, he noted his discovery that a lot of the lunar astronauts had chosen to take country music with them, and said he’d used this in his own work.

“I thought (the choice of country music) said something interesting about the way they saw themselves, which was as frontiersmen,” he explained. “All the harmonic pieces I wrote for the film have a kind of unearthly country-and-western feel.” Eno added that he loved the way one astronaut observed that while the moon is grey, “until you’ve been there, you have no idea how many shades of grey there are” — a wonderful metaphor for the whole Apollo programme.

Other music entered the story in less direct, though no less interesting, ways. The BBC commissioned Pink Floyd to jam in real time as the first spaceship dropped to the lunar surface; Creedence Clearwater Revival’s pessimistic Bad Moon Rising was No 1 during this stargazing period; the hippified sleevenotes to Strawberry Alarm Clock’s debut album, Incense and Peppermints, spookily resemble words delivered in a lecture by Edgar Mitchell, the Apollo 14 moonwalker who experienced an “epiphany” on the way home and is now a new-age guru in Florida. A meeting with Mitchell provided my first hint of a link between the space race and the counterculture, which found echoes in some of the astronauts’ trippy descriptions of what they’d felt and seen. The wonderfully named Story Musgrave even described hearing music during a space walk, saying: “It was a noble, magnificent music. I was a little on the margin. . . I was walking the edge.”

Of course, with no atmosphere to transmit sound, Musgrave couldn’t have “heard” music in the usual way, but perhaps he sensed it. A few years ago, Cambridge scientists claimed to have found the deepest note ever detected, a B flat 57 octaves below middle C, emanating from a black hole.

I often work to the sounds of radio interference over Neptune, as recorded by Nasa probes, and wanted a few snatches of this on the album. Believe it or not, it’s relaxing.

Conversely, the Byrds’ Eight Miles High was banned by American radio stations on the assumption that it was a drug song, when it was, in fact, about flying. Like their Californian contemporaries Jefferson Airplane, they were air and space nuts, captivated by the possibility of finding life up there.

Other musicians entered my thoughts through things they said, such as Wayne Coyne, of the Flaming Lips, who remembered the first landing chiefly for his brother’s concurrent admission to their mother that he had been experimenting with acid. Wayne then riffed on a line from one of my favourite Lips songs, Do You Realize?, that simply asks: “Do you realize / We are floating in space”, which I’d taken to be relaxing. “No!” he exclaimed. “I don’t mean that line to relax you! We really are. We’re floating on some insignificant speck in an endless cold vast sea of nothing. I mean, it’s just hanging. . .

Explore Music

• CD Reviews

• Live Reviews

• Classical

Editor's choice

• The 100 Best Films of the Decade
• 70 facts you didn't know about Marvel
• The 50 Biggest Movies Of 2010