I'm an INFP, and my best friend is an INTJ, and we get along famously. But she's a healthy INTJ--an unhealthy one can be toxic, and I find them nearly impossible to be around.
Unhealthy INTJs have a way of presenting themselves deliberately to make you feel like a moron--their tone, their attitude, their air of intellectual superiority, even if it's unwarranted. Only a fool thinks they know everything.
Additionally, some of them can be manipulative.They think you're too dumb or blind to see through it, but in reality, it's easy to spot--manipulators are hardly ever as subtle and brilliant as they think they are.
But do not mistake me--I don't hate on INTJs. Like I said, my best friend has that personality type, and I love her dearly.
When I make disparaging INTJ remarks, I'm referring only to those who are mentally and emotionally unhealthy.
Really, any unhealthy personality type is going to be difficult to get along with.