I should think that an INFP would be eventually horrified and feel extremely put upon if suddenly the world noticed them the same way as it does the ENFP. Being noticed is a big responsibility and gets tiresome as hell. I could see an INFP, after some time passing, suddenly feeling quite smothered and wishing for some desperately needed privacy and time to themselves to just think/feel. Maybe even coming to the conclusion eventually that it's impossible to trust/truly know who likes them for what quality and why.
It's just as possible to be lonely and feel misunderstood in a crowd.