The Reverend Mother Gaius Helen Mohiam tells Paul when she first meets him that the spice melange allowed the Bene Gesserit Reverend Mother to unlock genetic memory, but only that of their female ancestors. The masculine side of their ancestry represented a place in their consciousness that repelled and terrrorized them. A Kwisatz Haderach would be a male Bene Gesserit who would have access to the memories of both his male and female ancestors as well as an ability to bridge space and time with prescient ability.
The Bene Gesserit desire access to this knowledge and associated powers, driving them to initiate a long-running breeding program to produce such a being. He would be Bene Gesserit Reverend Mother, Mentat, and Guild Navigator, all in one. The initial Bene Gesserit plan was to breed the daughter of Duke Leto Atreides to a Harkonnen male, which would produce the Kwisatz Haderach. This was to be the culmination of more than 10,000 years of careful breeding. This plan would have seen the end of the centuries-old feud between the Great Houses Atreides and House Harkonnen and placed a prescient, Bene Gesserit-controlled male on the Golden Lion Throne. However, because of her love for Duke Leto, the Lady Jessica disobeyed her fellow Bene Gesserit and gave birth to a son instead of a daughter, to give him an heir, Paul.
It seemed fairly certain from early in Paul's life that he would in fact be the Kwisatz Haderach, since he showed an ability to see into the future. It was when he was fifteen years of age that the Bene Gesserit sent a Reverend Mother to test Paul's prescience and his training in the Bene Gesserit ways. This event appeared to be a significant catalyst for the events that would befall the universe for the next several thousand years. Paul's testing with the Gom Jabbar, as well as the Sisterhood's silent complicity in his father's death, proved to instill significant negativity in Paul against the Sisterhood. As a result, when he reached young adulthood, and ascended to the Golden Lion Throne, he vowed that he would never be under the control of the Sisterhood. Because of this, the Sisterhood lost control of their breeding program, their Kwisatz Haderach, and the possibility of placing a Bene Gesserit on the throne.
Paul Leaves the Golden Path
Indeed, it appeared that the majority of people did not realize that to know the future is to be trapped by it. Paul could see that he would have to lead humanity onto a drastic course in order for it to escape its own annihilation. As a result, he chose to escape the Golden Path by allowing himself to be blinded, so that he could walk into the desert and not be a burden on his tribe. The mantle of the Golden Path was subsequently taken up by Leto II.
As Kwisatz Haderach, both Paul and his son Leto would at times experience significant grief because they knew they were trapped by their own destinies. One of the few people who understood this was in fact one of Paul's enemies. It was through the failed Tleilaxu Kwisatz Haderach 'experiment' that the Tleilaxu Master Scytale could in fact sympathize with Paul and the decisions he had to make.
When Leto II, the next Kwisatz Haderach, ascended to the throne, not only did he manage to avoid Bene Gesserit control, but he also took control of their breeding program. He then relegated them to a relatively insignificant role in the universe throughout the 3500 years of his reign.
After the Famine Times prompted by Leto II after his death, the remaining power brokers in the Old Empire either forgot about the concept of the Kwisatz, or vehemently guarded against the rise of another of the super-beings. The Bene Gesserit would go so far as to terminate the life of humans that showed aspects of enhanced abilities. But as was shown with the "wild" Atreides line, the heightened human abilities continually showed themselves.
After the The Scattering, both Miles Teg and his daughter Darwi Odrade, descendants of the Atreides, showed super-normal actions. Teg was able to move at lightning-fast speed and disabled his kidnappers on Gammu. Odrade had limited prescience that enabled her to know that the misguided Honored Matres and Bene Gesserit should merge to form one single organization.
But it was the ghola of Duncan Idaho which caused the most concern for all involved. The conservative branch of the Sisterhood, led by Schwangyu, would go so far as to disobey the order of the Mother Superior Alma Mavis Taraza, and terminate the life of a young Duncan ghola -- even as she has sworn to be his protector from the Tleilaxu, who had allied themselves temporarily with the returning Honored Matres.
Behind the scenes
It's likely that the term was borrowed by Frank Herbert from the Kabbala. The compilation of Jewish mysticism the core of which is entitled HaZohar (The Book of Splendor) ascribed to Rabbi Shimon bar-Yohai, ascribed by Jewish people to have been written in the 1st century C.E. Modern scholars ascribe it to a sephardic Jewish author around the 13th century. The term itself is Hebrew, "K'fitzat ha-Derekh", (קְפִיצַת הַדֶּרֶךְ) literally, "The Leap of the Way," by means of which an initiate may travel some distance instantaneously, appearing to be in two or more places at once.