If the laws/rules are equal, it's really irrelevant what other people think. Everyone dislikes something.
Men are attracted to lots of women. Doesn't mean anything will come of it. Gays merely having an attraction, isn't the same as trying to do something about it. The homosexuals of both sexes that I know, don't even date people that aren't out as gay, much less straight. Most aren't predators. Just like most heterosexual people aren't predators.