I think that points to the limitations of Fi-based empathy. Rather than starting with the kind of global notion of the good that Fe-users seem to hold, we give in apportion to what we hope to receive. It may mean that we will feel incredibly deeply for another, but it's equally possible that we will feel nothing at all and feel equally justified in either case. I wouldn't mind feeling more externally accountable. (In fact, a big part of my personal ethical development was having the ENFJ I was seeing take me to task on what was admittedly some flabbily relativistic thinking on my part.) But I think that for an Fe-user to get me to that point, it takes an appeal to reason as opposed to pathos, otherwise Fi will balk.
I'm curious what you think of that as an Fe-user.