Yeah, people seem to relate it to shyness. But it's not shyness.
It usually gets uncomfortable when they start drilling you as if you're shy though.
I just don't make eye contact, but if asked, I can stare deadly long into someone eyes until it scares them too, no problem.
I just enjoy having my eyes pointing 'at my thoughts' as opposed to pointing 'at other people'.