As mentioned above, it's in the eyes. I realize this and can't really do anything to change it so I've just learned to roll with being thought intimidating.
What's really bizarre is that my present supervisor becomes visibly nervous (swallowing hard, sweating) when he talks to me. All I do is look at him with a neutral expression and respond where I feel I have something of note to add.
This is just something we INTJs do. We can't turn it off and when we try to disguise it we look and act odd. What's more, it's not our faults that some people are so lilly-livered that a little eye contact makes them lose their composure.