I agree, basically, with your assessment: afterwards it was more dangerous, because of the radical people.
But, originally, slavery had nothing to do with "color" or "race". Georgia was founded by slaves, prisoners, from England. They were white. My own ancestors came to the Colonies as "indentured servants". Indentured servants could be beaten, killed, robbed, in the case of women, raped, repeatedly. In other words, they were slaves.
Most people today are slaves; slaves to fashion, to drugs, but, even more, slaves to the filthy rich.
But, as to what you say, which was worse, the period of slavery or after, definitely afterwards was more dangerous to the blacks. Regrettably.
I needs also must point out that the KKK was anti-Jewish, anti-Native American, and anti-decency.
They were low-lifes, despicable, and to be pitied.
The American version of Nazis.