Look, the office next to mine is occupied by one very flamboyant gay man. He comments about what I wear every single day. He thinks that I dress better than everyone he knows, including himself. So jealously creates a situation wherein he would not only notice such a fashion faux-pas, he would attempt to attack me in order to feel better about himself.
. . . And he's not the only one. People seek flaws in me no matter how small they might be - or non-existent, really - in order to feel less inferior. Most fail, of course. But others would use your sock scenario to lift their spirits to a level that would make me uncomfortable. And I would never willingly facilitate such a situation.