I have never been uncomfortable with eye contact. It was just the way I was taught. My mother and my teachers would always say you should look at someone when they're talking.
But, I sometimes have a tendency to stare if I find someone interesting. So I try to time how long I look at someone when in a conversation and I look them in the eye every once in awhile to make sure they're still engaged.
About the whole guy/girl thing. I find that guys are more squeamish. I can look at a girl's eyes for longer without her getting uncomfortable. With guy's it seems more aggressive. I think when I make eye contact with a guy it says something like this, "Know your place or I'll put you in it. Don't mess with me and I won't mess with you."