well, I don't have an extensive experience with MBTI but since I am an INTJ and I befriended both ISTJs and INTPs I can tell you about the differences I've perceived between us.
1) my ISTJ friend is always attentive, never misses up any detail or something and is like my human agenda: whether I have an exam, class or just to hang out with her, she's gonna remind me that, and that's good. we overall get on pretty well since my ideas and theories combine quite well with her perfectionist way of carrying out things. therefore, they are the perfect partner to work with: efficient, precise, quite humble, are not going to argue, still smart enough to be able to handle themselves without the need of telling them everything.
so I'd say the difference is big when it comes to behaviour, and smaller when it comes to thinking (in fact, we relate to people and the world in a similar way, although I tend to make more mental plans than her).
2)my INTP friends, on the contrary, are much more dreamy that me. they seem to spend all their time coming up with inpractical ideas just for the sake of enjoying the cultural feeling. they tend not to schedule anything, and this can be easily upsetting when you have to work with them (don't do it, just don't. they are smart, but only when it comes to stuyding dead languages, making weird hypothesis and losing your time in general. still, if find them pretty fascinating, and hell, even sexy at times, and maybe the coziest of NTs, and can really make you have fun and feel at ease)
the surface difference is small (I had problems distinguishing my J/P trait myself), but once you get the difference you understand just how differently they perceive the world.
this is my short limitated opinion

if you want a general statement as an answer, I'd say all the differences matter, since 16 types are quite few to classify all the people and thus, needless to say, if you aren't of the same type they are not going to behave like you (they wouldn't even if they were of the same type, though, but the more so if they aren't).