However, that's not to say it's always completely annoying. If we have on a happy face (read: not the "angry" thinking face) and are looking at you in a funny way, feel free to ask... probably be pleasantly surprised if you can drag it out of us.
Also, on a related note, if you ask "what's wrong" outside the proper boundaries (i.e. you didn't actually did something to annoy/hurt/confound us) it gets supremely annoying. We won't show it, and usually just answer with an exasperated "I'm fine." (I know several E-types who got my thinking or neutral faces confused with angry or frustrated, and that's just ridiculously silly. I always look that way...) Eventually, we'll avoid talking to you out of sheer frustration. Though, again, that's prolonged use. A few times, here and there, is kind of endearing.
(I just realized I use a lot of parenthetical statements when I'm thinking... So many asides in this conversation it could be a Shakespeare play.)