I used to think most of my issues stemmed from an E/I divide as well, I'm definitely looking at it as an N/S thing at this point. Most of the people I was around in high school typed as S, and I'm pretty sure my parents and a wide array of my family are sensors as well. There can definitely be problems when you tend to think in abstractions and live outside the moment, while those around you value groundedness, practicality, and "common sense." If your own perceptions and ways of thinking are never supported/mirrored by those around you, it can definitely lead to alienation and feelings of inadequacy - you will think it is you who are defective and need to change, when really you just need to meet a few folks who think like you so you can realize that you're not crazy.
Intelligence is another issue that can certainly leave you feeling like a misfit.