I believe that female xNTx's and male INFx's probably have it the hardest within each gender.
As an INTP myself (and also thinking about close NT family members and friends of mine and their respective experiences), I would say some of the benefits and drawbacks are as follows:
Benefits:
-there's a sort of unspoken (sometimes spoken) respect that we get because people see us as super intelligent. A close family member of mine treated me disrespectfully for many years. Years later I found out (during an outburst of his) that he thought I was so smart that I literally had the world at my fingertips. People always coming to me for help in writing a document, if something "mathy" comes up, or sometimes if they need an informed opinion on something.
-this is weird, but I think when I get mad (or firm shall we say) and get tough with people - i.e., there are certain rare instances where I am able to turn my Ti into Te and I can just pick a situation apart like there's no tomorrow or pick someone's ridiculous BS apart (an example might be in Good Will Hunting in the Harvard Bar scene where Affleck tries to pick up on Minnie Driver and the Harvard snob starts schooling him on the colonial economy - but he was talking out of his butt. Damon stepped in and just ripped him apart. I don't necessarily take pride in this because I have hurt some feelings for sure(although at times it has served me well), but I have found when this happens, people *sometimes* respect me more for it. As an INTP, I exhibit Ne and Fe (which are the 2 most "feminine" traits in this society - Se and Te would be more masculine), so people see me as a softy at times - just happy to go with the flow and live in peaceful tranquility. That sometimes results in a lack of respect in this society - from the ESxx's mostly. But, when I'm able to flip my precise thinking inside out and extrovert it, people are amazed and have this new respect. It's weird.
Some drawbacks:
-an outcast. Very difficult to connect with people. We see the world WAY differently than the vast majority of people. We don't fit in and that's tough sometimes. I see ENTx friends of mine who experience this same thing, but they still fit in - in a general sense. An ENTP friend of mine is actually seen by our circle of friends as "the most awesome dude on the planet". Sometimes I just wanna say, "Hey, over here! I'm just as likeable as him and very similar to him - I just don't want to hang out every night of the week. Maybe just once a week." But, it doesn't seem to work as easily for the INTx's.
Another way to describe this is imagine everyone in the world thinks almost the complete opposite as you. That's what it feels like sometimes. I'm not throwing a pity party - I'm glad that I am who I am - this is just an attempt to answer the OP. When I have a new business idea or an idea for a book or want to talk and have a mildly in-depth discussion about the movie we just watched, it only works with an "N". Hardly no S's that I've met in my life want to hear my ideas (or if they do, they're not interested and the discussion just dies on the spot).
-Another thing as an INTx is that I tend to "dress down" as to not bring attention to myself. I have found that the more dressed up I am, the more I am approached and expected to be outgoing. Most of the time, when I go out, I don't want to be approached. So, I dress down.
Example: If I'm wearing slacks, a dress shirt, and have my hair combed - just about any public establishment I'm in, I get a lot of hello's and small talk type of stuff. If I'm not in that mood at all (which is more than half the time), I will instead put on a pair of jeans with a t-shirt and a baseball cap and just play "the quiet guy" role. I think people probably see me as either snobbish, super quiet/shy, or maybe think I'm just in a hurry to get home. I'm left alone and I feel more "in my element". When I'm in the other scenario, I feel like a fish out of water. Not only am I a heavy introvert and don't really desire small talk, but if I do start talking and get into a real conversation with you, most of the time you're not going to be interested in what I'm talking about anyways. One almost starts to feel like "Really, what's the point of engaging anyone in conversation?" Just stay to myself! But, we all need interaction and so that's what makes it hard. It's hard to connect on any meaningful level. I like it most of the time, but sometimes it makes me sad.