^ Nice info, @Esoteric Wench. Helpful!
Personally - my brother's an ISTP. He can be challenging for me to read. Mostly I've learned that him being super, super quiet and not really responding to anything is a good signal that something's not right. But my ESFJ mom and ISFJ boyfriend are much better at reading him than I am.
I suspect that you guys put out some sort of Fe-available signals that we xNFPs struggle to pick up on. We really read mood, so if you don't do a lot of "mood" changing, that's probably why he's not getting it. I'm quite good at responding to nuance in mood but it's challenging for me to understand how to address feeling where there's not much emotional overtone. I guess because my ISTP and I are siblings, I usually figure out enough to not be shocked, but sometimes I'm surprised by the gravity of something weighing on him. He can get really grumpy over basically nothing (he's a teenager, after all) but if it's a serious thing then he's often more subdued. I usually end up asking the super-blunt, Fe-stupid "errr so what can I do for you?"