It normally feels like it creates a hierarchy I don't want or else is trying to buy me. Sometimes it seems to translate to "I'm so ditzy" or "I'm so stupid" also, especially when it is said about something that I figure wasn't that genius to begin with. In those cases I seem to assume the conversation has little potential and we probably can't discuss things in any meaningful way anyway. I normally still try to find common ground somewhere, but I prefer people who show an interest and try to learn, rather than label something as above them (which the comment feels like it does). So it could depend on what follows. "Wow, you're smart. Teach me!" would probably pass and get you some way. "Wow, you're smart. I could never do that." equals I feel like flattering you while taking zero interest in what you love or do.
Whoa, it seems to me like you're reading way too far into this.
If someone says to me "whoa, you're smart", I take it as a compliment, because it is. It doesn't make things unequal whatsoever, since my intelligence has nothing to do with my success in life. I doubt I'm higher in that category than most of the people I'm talking to. My intelligence might give me a general advantage in certain areas, but definitely not others. It's not the only spectrum to rate a person's worth or even that important of one.