And if everyone is constantly staring at your boobs instead of listening to you, it's probably because you're not saying anything worth listening to. You think they'd be paying more attention if you were ugly? If you weren't physically attractive you'd probably be more insecure than you are now. The other side's grass always seems greener, but in this case it's not.
Just consider the opposite end of the spectrum for a minute. What's worse on your self-esteem: too many people constantly thinking you're attractive, or nobody ever thinking you are?