Usually I become immediately blunt and unsmiling towards the person as well as making sure others are alerted. Inappropriate behaviour thrives on secrecy. The really good ones at it though send out an inappropriate message/action that neither of you can mistake, but that unless you were there could be read in more than one way. My mother was very good at imparting trust, yet appropriate wariness towards other people so that we learned to send out fairly clear signals about anything physical. I have never had anyone try anything (other than one night at Hallowe'en when a random drunk stranger decided to reach out and give me a pat on the butt).
I think it's more when it comes to emotional boundaries that things get messier. I don't generally put as much stock in emotional interactions as sometimes the recipient does and therefore they misread my intentions. I'm good at being kind but firm when someone starts trying to emotionally mooch on a regular basis, as well as shutting down clearly inapproprate verbal stuff. It's more that needy, I need to pour out my life story and fears kind of thing where I am not spending excessive amounts of time or overexerting myself emotionally that I sometimes am unclear about.