It is quite hard for me to describe why I cannot get along with INFPs. I know three (two female, one male) And while we can communicate fairly well, our opinions are almost opposite on the topic in question. For example, we both think we "get it" when talking about a philosophic issue. When can never come to a consensus. I can only agree with people who have the Fe function. I tend to use it as a means to and end to fit into the framework. The Fe thinks about the situation as it applies to others where the Fi sees it applying to themselves as an end in itself. I have heard an INFP utter those same words when arguing about something. Since this is their dominant function and I do not understand it, we usually do not get along too well. There is this unacknowledged friction. We should just agree to disagree once and for all but we end up arguing too often for my liking. Since both of our types don't like conflict we don't usually end up being friends.
Although INFPs share the same first three letters, our functions are opposites in regard to them being either extroverted or introverted.
The way we use our functions are opposite, which all has to do with the P and J.
I think INFJs would get along better with INTJs because they have Introverted Intuition as a dominant function. They are supported by different yet still extroverted functions (Fe and Te). I think they complement each other well. I think you tend to get along with those that have functions in common with you.