Definitely annoying.... this question is so timely with the U.S. elections coming up. Anti-intellectualism in politics totally bothers me.
Another thing that really bothers me is when people think being aggressive/criticizing others means being an "expert" and that this trumps knowledge.
Finally for academic stuff, I used to worry I wasn't a real intellectual because half of the stuff people talked about in articles just annoyed me.
But I think you can be skeptical and independent and not in love with every intellectual idea but not against intellectualism. That's what the mind is for anyhow...