"But if you're Negro, the only way you can vote"