SHAKESPEARE WROTE of “black night” as “Death's second self, that seals up all in rest.” And Hamlet rejected this. “To die, to sleep, no more...it is a consummation/ Devoutly to be wished. To die, to sleep: perchance to dream, aye there's the rub...”

Human beings are creatures who think about their own nature, obsessively. And the fact of sleep and dreams, of the dark hours of helpless unconsciousness and involuntary happenings and images, has always profoundly affected the way in which we do this thinking. Black Dog has published a scholarly and complex set of essays to accompany the Wellcome Collection's exhibition “Sleeping and Dreaming”. It covers philosophy, psychology and scientific research, and ends with a brilliant essay by A.L.Kennedy on the writer's fate: No Sleep - Only Dreams.

Ralf Konersmann, writing on philosophical dream criticism, remarks that Descartes, who was suspicious of dreams, was said to have managed by purposeful practice to have only reasonable dreams. In an eventful night in 1619, Descartes dreamt that he was offered two books, a dictionary and a book of poetry, which he took to be a choice between science and philosophical wisdom. This dream was redreamt by “a friend” of William Wordsworth, who saw, in a dream, a figure on a dromedary who was both an Arab (from the Arabian Nights) and Don Quixote, riding along a beach, pursued by “the fleet waters of a drowning world.”

This composite poetic figure offered the dreamer a stone or a shell - these were said to be science and poetry. Both dreams have the characteristic of thinking dream images - compressed metaphors, ideas which are things. A French dream researcher wrote to Freud, asking him for an interpretation of Descartes' dream. Freud replied that this was a “Great Dream”, in which the dreamer's own interpretation was valid.

There are various discussions in this book of Freud's The Interpretation of Dreams. In the beginning Freud appears to have insisted that the events and images in a dream were only the stuff of the waking day, rearranged and sorted out - “dream-work”, with symbols. There is a good essay by Daniel Pick on Dreams in Contemporary Psychoanalysis, in which he discusses how both analyst and patient might dictate the subject of dreams. I have always thought that Jung's patients dreamt to please Jung - their dream drawings are narrowly similar and repetitive.

One of Freud's greatest works is Beyond the Pleasure Principle, written when he came to see that there were dreams which did not correspond to his belief that dreamers were always in search of pleasure, however much they disguised it. Soldiers damaged in the First World War did not dream to evade their horrors, but obsessively dreamt them, again and again. They, and their minds, were trapped in time. Freud attached an analysis of trauma, or shock, to an observation of the compulsion to repeat.

Tamara Fischmann's essay in this book, Collapse in the Night, is a gripping discussion of nightmare, anxiety dreams and sleep terrors. They are there in Shakespeare's rhythm again. “Macbeth hath murdered sleep and therefore Cawdor/Shall sleep no more, Macbeth shall sleep no more.” And Lady Macbeth is reduced to a repetitive walking nightmare, compulsively repeating the washing of her hands.

Freud said that fairy tales were the common dreams of a people. I am less convinced by Walter Benjamin's belief that the cinema was the collective dreaming of the public. A film has a very conscious director, and very conscious actors. It is at best a manipulated dream. Konersmann discusses this, and “the hermeneutics of suspicion” subtly, and at length.

When the neuroscientists first started studying sleep and dreaming they thought that they had disproved Freud once and for all. There are phases of sleep in which the eyes move rapidly and the sleeper moves the body - animals also have such REMs, and dogs may be dreaming in answer to a physiological stimulus. It was thought that dreams took place only in REM sleep, which originates in the pons, a part of the brainstem. Further research, as Mark Solms shows in the fascinating The Interpretation of Dreams and the Neurosciences has shown that “higher” parts of the brain are active at other times during dreaming, including those which validate Freud's ideas - parts of the brain concerned on one hand with “arousal, emotion, memory and motivation”, and with “abstract thinking and visual perception” on the other. What remains inert is the part of the brain which provokes and enables actions.

There is much casual discussion of scientists who solve problems during sleep by finding an image - Kekulé's dream of whirling snakes which revealed the structure of benzene. But work is now being done on how we need sleep to improve our memory and thought processes. Ulrich Wagner, in The Brain Never Sleeps, writes lucidly on this aspect of our whole minds. It is a process analogous to deliberately not thinking of a lost word, and waiting for it to surface. I met a physicist who says he looks at a problem with his front brain, thinks it out with his middle brain, then puts it into his back, or unconscious, brain, to solve itself. He says that he lies down to sleep in the midst of thinking out something difficult - and I have found this works, with writing novels as it does with science.

Conscious and unconscious co-exist in one brain.

Sleeping and Dreaming edited by Helga Raulff, Michael Dorrmann, James Peto and Ken Arnold
Black Dog, £19.95

Explore Books

• Book Extracts

• Book Reviews

• Books Group

• Audio Books

Times Recommends

• Superfreakonomics by Steven D Levitt and Stephen J Dubner
• The Times picks the 50 best paperbacks of 2009
• Vincent van Gogh’s letters show the soul of the artist