Yes, it can feel that way. I have learned to disassociate myself with racist in my romantic life. And if you don't have empathy for people of color you are just as guilty as the racists.