I agree that would be the way to go. It is important to assess whether both people have what it takes right then to provide the building blocks for a successful and healthy relationship. I wish that people had in mind before what that requires and what it is/is not when they see it. Someone who is a great person may not be prepared at that time in their life to be the kind of partner they are needed to be.
I expect the bitterness comes in when the INFJ has given up a lot to make it work and figures that maybe they've burned some bridges or if they feel that the other person wasn't up front from the start about those problems. Maturity level and security also plays into it.
I don't think I'm generally very angry and vindictive, so you're right that it seems foreign. I met an INFP once with qualities that were wonderful and whom I had grown to care deeply for, but he was wading through all kinds of unresolved stuff (finances, addiction, previous relationships having failed, child whom he couldn't see, choice of friends, business failing) and we ended up going our separate ways before it really came to anything. It was fortunate I was leaving town when I was, or I think it would have been hard to resist the temptation to want to help save him, or to enter a relationship that did not have the potential for success.