the only human right I would support as inviolable is to have a choice over the things that happen to you, within the boundaries of known physical possibility. But this has great implications on many other things.
If a member of a BDSM community, certified sane, makes a free and informed choice to give up or surrender their freedom in certain areas to another person and become their slave, then it is not wrong for them to do so, nor is it wrong, IMO, for their 'master' to take on that role, if that's what they wish. However, if a person is captured by soldiers and taken against their will to a place where they're sold without any say in the matter to a master not of their choosing, then that is wrong. It's not 'freedom' that I see as being the human right that's been violated there, but choice.
If a person of sound mind makes a free and informed choice to die, then IMO it is not wrong for someone to assist them (youthenasia). But if a person has not chosen to surrender their life before the natural time, but somebody takes it nonetheless, then it is wrong.
If a person chooses to knowingly take their drinking water from a river that contains natural deposits of a toxin that causes brain damage, then that person, despite having somewhat incomprehensible motives perhaps, is free to do so. However if a person decides to pollute a river so that thousands of people who rely on it for their drinking water are forced against their will to either drink poisoned water or move to a place with a cleaner water supply, then the person polluting the river is IMO doing wrong.
If a person chooses to surrender a portion of their income to a government that promises to, in return for the taxes paid, care for and protect that person, then the government is not wrong to take that tax. However if governments of nation states monopolize the entire planet so that nobody has any choice but to live in a nation state and pay taxes to a government that provides services they would prefer not to partake in, then that's wrong, IMO.
In summary, as I said, it's all about choice. For me, moral relativism boils down to this: something is only wrong if the person it's being done to does not freely consent without coersion of any kind. Nobody has the right to tell another person what they can, can not, should or should not choose to allow to be done to their own person.