Genetic memory is the concept, unsupported by modern science with regard to humans, where an individual may possess the memories of their ancestors, hidden away within their genetic code.
Core concept
The idea behind genetic memory is that an individual's experiences would become imprinted upon their cells, and would in turn be hidden within the cells of their descendants. This theory is also used to explain the mystery of benevolent mutation.
While an individual's genetic memory was not normally available to them, exposure to various awareness spectrum narcotics, such as the spice melange could cause an individual's genetic memory to surface.
Awareness
Members of the Bene Gesserit Sisterhood became exposed to their genetic memories (on the female side only) after their consumption of the Water of Life. The Bene Gesserit called this Other Memory, and used it as a source of immense knowledge that had otherwise been lost to history. Through their breeding program, the Bene Gesserit hoped that they could one day bear a male who could possess the genetic memories of both female and male ancestors. Such an individual would be called the Kwisatz Haderach.
The Bene Tleilax used psychological trauma to restore the memories of gholas up until and including the time of death of the original individual. This would not be possible were it not for genetic memory.
Hazards
Dangers existed with the possession of genetic memory. A Bene Gesserit adept could possibly become overpowered by the ego-memory of one of their ancestors. Their personality would ultimately become supplanted by that ego-memory, and become possessed by them. This was called Abomination. Bene Gesserit general policy was that anyone found to be Abomination, should be killed, regardless of how benevolent or powerful the ego-memory was considered to be.