Yeah, it's just a stereotype thing. It annoys me when people think of feminists as extremist lesbian vegans.
For e.g., I fully believe in gender equality in my field (academic biological sciences), where >60% of postgrad students are female, but <10% of tenured University positions are held by females. But I would not support instituting a quota system to address this inequality. I believe that a cultural revolution is necessary, because a quota system would result in lack of academic respect for the females who do make it. After all, isn't respect what women are fighting for? This is shooting ourselves in the foot.
Does that make me anti-feminist? As you can see, the label currently means absolutely nothing in the larger context of complex social interaction.