Zarathustra had a great post on what it was like to be a "tall white guy wearing a suit" that does a pretty good job of explaining much of it.
I'm decidedly terrible at discussing those kind of fuzzy social, and cultural issues, to the point where all I know is what it feels like to be me, but have no ability to translate those feelings into anything resembling a cogent point of view.
Don't worry you didn't come off as mean spirited, just (justifiably) annoyed.
I could tell you about starting High School (public) and feeling so estranged from any sort of culture, that I just hung out with the Asian kids for the first year and a half.
It's that not having a place feeling of being white.
Anyway thanks for being chill about the whole thing.