You're not wrong, though. l really don't view it in the same way and am not an ENTP who employs the best use of Fe.
l can tell that when l'm socializing with an INFJ, the give and take is very important...even if what's being exchanged isn't very meaningful to me. l think it's difficult for them not to take that personally.
l wouldn't say l hate small talk but this could be called small talk, just conversation that l don't find especially appealing and it drains me. So l do tend to put them on an ignore mode when that happens.
The difficulty comes in when my feelings are analyzed by them during that period, l honestly do not possess the ability to argue with someone about things l may or not have been angry about (when l just needed space) but of course, not every NFJ will react this way.
l don't mean to be harsh by doing so, but FJs usually have more acquaintances to speak with than l do, because they LIKE chatting more lol.