Female INTJ *waves*
No one is annoying as long as they take the time to achieve mutual understanding with an INTJ; what annoys me is the flouncing off with a 'oh, you're just incapable of understanding' when in fact I'm actually tying myself into knots trying to do so.
As a career lurker, I'm always relieved when someone stands up and says that they have INTJ friends who have a deeply emotional side; it means that there is someone out there who has actually allowed an INTJ the time they need to establish trust and a safe relationship. God knows, it happens little enough.
Like all the remarks about INTJs cutting people off. Yes, I do this, but only when fearsomely hurt by something someone has done. All the E types jump up and down about how hurt they are in this situation; strangely it rarely occurs to them to wonder if the INTJ has first been hurt in some way, or realise that if this is the case then the INTJ would bite their own tongue out rather than admit it. After all, you hurt me -> I withdraw from you -> I no longer value your opinion -> therefore I have no logical right to my emotional reaction -> admitting I may be having an illogical emotion gives you the power to hurt me further -> therefore the way to deal with this collision of logic is simply to maintain the distance. Once an INTJ has tied themselves into this circle they can often only be rescued by a high-level sensing type

who forces them to validate their feelings and helps them with behaviours that can effect a reconciliation. Alternatively the other person simply has to wait it out and then effect a rapprochement, which (to their surprise) frequently works - the logical mess has simply been swept away by the passage of time.
I do begin to wonder if we are the most disliked of all the types, not least because society defines 'normal' as openly emotional. Being a female INTJ I'm probably more aware of this than most; I have resigned myself to life alone, though the emotional pain that brings me is something no one, friends or family, would ever have any idea of. On the few occasions I have shown emotion they have got angry/shocked; I'm clearly not expected or allowed to behave like this.
The perfectionist traits we're accused of are often overblown, too, and in my experience actually represent other problems. If you don't do something to my satisfaction I'm likely to ask myself about all the other things I value in you, and conclude that the damage/hurt I'm likely to cause by making a fuss far outweighs my annoyance, and keep my mouth shut. However, that balance changes if we are professional colleagues and you are messing things up (very important for me as I'm an emergency doctor) or this is a friendship that is heading for the rocks because of lots of other problems. Then you might see me as demanding or arrogant, because that's the emotional response to my annoyance. In fact, I don't think INTJs are perfectionist; they want a system to WORK; that's their first criterion. Everything else is just frills.
Oops...have wandered off point and ranted a bit. Sorry.