I'm not usually one to discuss intimate details of my life on the internet, but for the sake of this thread, I can say that this situation is one that I can unfortunately say I relate to far too well. You said you want to hear from an INFJ with similar experiences, and that's what you're going to get.
I've been in the same situation you have at least five or six times. I've gotten close to people, gotten into relationship with them, and its when things get really close and intense that I find out the inevitable - they're been cheating on me.
The most recent examples go back five years, three years and two months respectively.
About five years ago I got into a relationship with a girl. It was nice, close, intimate, and it lasted for over a year and a half. Half way through it, I found out she was getting her jollies off with some guy on the internet. She begged and cried for me to not leave her, and for some reason, I didn't. Some time later, I criticised her for treating someone in a way that I thought was mean and cruel to them.
The next day she texted me a picture of her kissing some other guy. We haven't talked since.
I kept to myself for a while after that, and about three years ago I fell in love with a very old friend of mine. It had been building over a couple of years, until we just came out and said it to each other. For the time it lasted, it was perfect.
We had almost everything in common, we could spend 12 hours in each other's company and still not run out of things to talk about. She was thoughtful, inspiring and emotionally uplifting, and it was the happiest I've ever been. It's the closest I've ever been to anyone, before or since.
About seven months in, a friend of hers who had a not-even-remotely-secret dislike of me decided to spend lots of time with her, whilst refusing to even acknowledge my existence. This caused some obvious tension. I was angry because she refused to help resolve the situation or stand up for me. He didn't react at all.
A few weeks later, I was dumped. A few weeks after that, they were a couple.
The whole thing threw me off balance, a balance I don't think I've ever been able to get back. Just like you, it made me feel like the friendships I had with people were convenient little lie, without any real integrity. All I've learned from these experience is that people will do whatever they think they can get away with, which has completely killed me ability to trust people.
I shut myself off from interacting with people completely for over a year, until I started spending time talking to a woman by the name of Nickey. She was dealing with a lot of stuff herself, and she was in a similar place to me. It was a very slow process, but we got closer over the space of a year.
Past experiences have made me very unwilling to trust people or get close to them, even as friends, and this wasn't an easy experience. But she constantly tried to reassure me that she understood, that she couldn't even contemplate cheating on me, that it wasn't in her nature, blah blah blah.
About two months ago, while attending E3 in Los Angeles, she let me use her phone so I could get calls during the event. Some guy texted her with words that were... lets just say... not the sort of thing a friend says to a friend. When I confronted her about it, she attempted to kill herself. I haven't seen her since that day. The sad thing is... I still want her back.
My self worth is shot from the experience, and I haven't socialised with anyone in months. I don't want to be around people, or get close to anyone whose interested, and I know there people who are.
Why you would want to stay with this man is beyond me. Cheating is a choice, a willing action by a person that can act upon of their own free will. If he cared, he never would have done it in the first place. And it will probably happen again.
You deserve better. This feeling of loneliness and worthlessness you have right now is entirely his fault, and it will not go away unless you give yourself the respect and love you deserve, and find someone who can give you the same. Take it from someone with far too much experience with these situations, because I'm a complete and utter mess from putting myself through the same thing a half dozen times.
That will probably be the first and last time I'll ever post something so personal about my life on here. I hope you can acknowledge the point I'm trying to make, and do the right thing for yourself.