Brain Hormone Linked to Maternal Instinct

A mother knows innately how to feed, nurture and protect her young; yet, these instincts—ingrained deep in the brain—have been known to misfire. When inherent tendencies fail to kick in, a neurotransmitter system in the brain may be to blame, a recent study says.

“Changes in brain serotonin levels are related to several mental health disorders, such as depression and anxiety,” says Evan Deneris, Ph.D., of Case Western Reserve University’s School of Medicine. “However, despite the prevalence of postpartum depression and the use of serotonergic drugs to treat it, the link between maternal serotonin function and nurturing had not been convincingly demonstrated—until now.”

Deneris is the senior author on a study published in September in Nature Neuroscience that tested genetically altered mice carrying a mutated version of a gene called Pet-1. Pet-1 controls the development of the brain’s serotonin neurotransmitter system, but in mice carrying a mutant version of the gene, the brain serotonin system fails to develop properly. The study shows all mouse mothers carrying the mutated Pet-1 gene had normal pregnancy rates and gave birth to normal numbers of offspring, but none of their offspring survived to 5 days old.

“Rodent pups are not able to maintain body temperature on their own,” says Jessica Lerch-Haner, a doctoral student at Case Western Reserve and co-lead on the study. “Mothers have to build proper nests and huddle their offspring. Although the Pet-1 mutant mice in this study nursed their offspring, they often failed to build suitable nests and never organized offspring in a huddle.” The baby mice died from cold exposure, she said.

The study didn’t just show negative results; it also suggested what might be done to fix them.

The researchers partially restored the mutated gene’s function during fetal brain development in the mice, which partially restored the rodents’ serotonin levels and improved their maternal behavior later in life. The finding may mean that subtle changes in embryonic formation of the brain serotonin system can affect the care females later provide to their offspring.