Interesting way of putting it. What if some people look great in hot pink, though? As an example, I would probably notice that first haha
haha actually, i'm one of those people. a colour like that will always be noticeable, but if you are close enough to get a good shot of the persons face, the colour should be complimenting the facial attributes of him/her. If the colour dominates then it shouldn't be worn because it will make the face look washed out or ashy and make you want to look at the colour instead of the face.
when i click back and forth between this picture and another window, i'm first am drawn to Reese Witherspoons face because the colour compliments her skin tone, hair colour, etc. This is what you want to aim for.
I'm told I have a good sense of fashion...but like you, I'd rather wear my Star Wars/Doug/Nerd T-shirts with some sweatpants or shorts than to take the effort in being 'stylish'...but I'm trying I guess.
I'd recommend wearing warmer colors with your cool skin tone. Warm purples and blues, orange, and red are good (yellow can be really hard for people to look good in...so it's up to you whether or not to wear it). If you want to wear more than one color, refer to a color wheel to see which combinations go best...but don't go out looking like a Matisse, only little kids can pull that off. I'd recommend never wearing gray on gray, brown with black, or silver with gold...I find it sort of tacky personally.
Don't wear anything really low cut unless you have a shirt underneath (that also isn't really low cut) if you're more well-endowed. People won't look at your face as much.
That's all I can think of for now. Wear whatever you think is cool.