Are you afraid that if you assert yourself to certain people and in certain situations, you will be showing your true self?
And then, even more importantly, that they will reject that true self (which is much worse than them rejecting your scary mask.)
Everyone is afraid of this. The only thing that truly abates this fear is to completely accept your self (your strengths, weakness, as noted above, but also all of your past successes, failures, fears, the things you wanted to do but didn't growing up, everything).
When you accept yourself, you "are comfortable in your own skin" and everyone who interacts with is able to see this. Freedom from your fear of rejection allows you to become more interested in other people -- not so wrapped up in yourself --- and that makes people more interested in you.
When you live without fear, you actually become a refuge and a comfort to other people.
Uber, many people here can feel that there is fear and insecurity behind your bravado. And they care about you whether you think you want them to or not. There is a lot of wisdom in this thread.