I was reading blink last night and it reminded me of something... In the book, he talks about how strangers were able to make better predictions for three traits based upon looking at their "dorm" - or living space - than their long time friends.
It's very easy to fall into certain stereotypes, in this case his ease of being around people. The trick, however, is walking into his room and having more books than I have... the movies, the computer... Yes, the guy is social, but he doesn't fit into the biological definition of E (positive emotions) and he has some serious recharge time - far more than his E time... and he likes solitary time, or small group time, a whole lot more.
He just likes being around people. Every other E trait isn't present... but I did judge him based upon that. Someone who looked at his life as a whole would not.
I compare it to my mom, who dances three times a week, plays sports at least twice, often more, a week... who is always looking for something to do... and is always out with friends/etc. Same thing applies - if you judge her based upon that, you'd say that she's an E. She's not, however. She hates groups, hates doing anything social that isn't part of what she wants to do. Same as my friend.
I find that SPs can be a bit harder to see because they often have the "do do" attitude.