I've had problems with people of various types - some that spring to mind immediately are ESFP, ENFP, ESFJ, ESTJ, INTx, and possibly INFP.
I would put this down more to these people not being particularly mature or "healthy", rather than it being specifically type-related. I'd say anyone of any type can be an ass, or can be wonderful. Another problem has sometimes been that we have had very different expectations for our relationship (whether that's a friendship or "relationship"), and sometimes that's been partly my bad. And it's not like I've been "healthy" 100% of the time in my life, either. Although, people I've had huge problems with have also tended to be those who others have huge problems with, I've noticed.
I have problems particularly with people who are any or all of the following: mean, inconsistent, flaky, hypocritical, arrogant. Beyond individual interactions where I've had problems, I really don't like people who behave like stereotypical ESFPs, ie. exhibiting the worst, most stupid qualities of that type. (I think a healthy/kind ESFP can be fantastic and probably has a lot of qualities I'd like to have.) I also find that many NTs can be highly arrogant, and INTJs (or perhaps again I should say those behaving like stereotypical INTJs) can be infuriating.
Although we can have gaps in understanding each other, and I sometimes find them too rigid, and they seem to find me too whimsical/dreamy...I basically love SJs.