I think whether he called or not is not important.
You write that you want to see him and speak to him. So....just pick up the phone and call him. Make no mention of the mystery call. It does not matter.
If you don't want to see him or speak to him, then don't contact him. Simple as that.
Your actions here need not have any reference to his actions or what he might do. Or what he might think. Also, any attempts at us trying to decipher what he feels is not a good use of time. Believe me, I've done that kind of thing before and it goes no where!
Anyway, good luck. A lot of relationship issues revolve around communication, and communication styles. It's important to put the gun down and address each other in a kind way. Guys especially have a low tolerance of emotional stress.
I hate to make generalities, but this is from my experience. If you're a source of emotional stress, they'd rather just not be around you. Often, guys will cocoon a little, lick their wounds and get better before subjecting themselves to stress again. INFPs especially, since we are pretty sensitive.
I think there is actually something chemical about it. I read somewhere that emotional stress triggers a chemical reaction. It's like their being out in the wild and facing a tiger. They actually need to get away from the stress trigger so their chemistry gets back to normal.
(Please excuse what sounds like crazy talk, ha, but honesty, I read this somewhere! And it's been a long day!).