And who would be "filled with guilt" except complacent racists?
Let me break it down for you.
You said the following:
1. It is not the responsibility of black people to diminish racism by "behaving."
2. Making oneself palatable to white people does not qualify as combating racism.
3. You should take responsibility for your own racism by understanding that fundamentally you are wrong to think such and such a thing about black people, regardless of how you perceive them as behaving. Then you would not hold the "bad" examples against the entire group.
P.S.- Don't paint yourself as the victim by crying about "he/she called me racist and I'm not! I swear!" It's old and pathetic.