There are two aspects to the use of these terms. The first is accuracy. Does the person you are calling "retarded" really have this mental disability, or are you using the term to insult someone for doing something you disagree with? Does the "socialist" really hold political views consistent with that philosophy, or is he just not a dyed-in-the-wool Republican?
Assuming the term is accurate, it becomes a question of sensitivity. There are no objective criteria here, just the sensibilities of those using and hearing the terms. Reasons for finding one term or another offensive vary, and as someone else already observed, it is not always those in the labelled group that take greatest offense.