I don't think making some people feel uncomfortable about what some people are doing down the street that has nothing to do with them is taking anyone's rights away or keeping them from having nice things. Are African slaves nice things? I guess in that case you could at least argue a more direct loss, that the plantation owners were losing their nice slaves. There's no law mandating that anything that gays do will interfere in any way with straight people's lives. If you minded your own business, you could potentially even pretend like gay people didn't exist and live your entire life without being affected by whether or not they were married. There's no law saying your small business has to serve married gay couples, or that your church must bring them into the fold. There is no tangible effect at all. The people who care so strongly about it are actually infringing on other people's rights, and no one is doing jack shit to them.
Boohoo, some people feel uncomfortable thinking about something that may be going on in some stranger's bedroom two streets away. What the fuck. You really are stretching.
It might make someone uncomfortable that I own cats because they fear cats. But I have a right to own cats as pets as long as I don't inflict those cats on the people frightened of cats.