Everything "other" people do and say is questioned for motive, yet what goes under-evaluated is an honest, deep evaluation of their own inner motives. For example, I will be ascribed a certain motivation in this thread by my mere presence, not really by what I am saying. Then, my words will be twisted pretzel-like to fit the desired conclusion as to why I am here saying what I am saying.
It's just the way it is. The INFJ level of emotional self-examination can be surprising shallow.
So, you have to decide this: do you want to fit into what the average-health-level INFJ sees you as? Is it that important to you? Until they have gained some rational, critical emotional self-evaluation skills, interaction will primarily be on their singular terms, not collaboratively.