People who say free will is a contradiction are just trying to muddy it up by arguing semantics, which is annoying to do, so people just decide to cede the point.
Whether or not freedom and will have anything to do with causality and causality is a moot point (though I disagree with you, I don't think will has anything to do with causality). Free will as an idea is based on ownership and control of one's own actions. You're talking about freedom like it makes us intangible from the real world. Being free to do something means we act under our initiative, and having the will to do something is that initiative, so semantically the two aren't contradictory. Free will is us having the means (our initiative/will) within ourselves to have ownership over and control of our own actions. If I wanted to stab myself in the leg, as stupid and illogical as it would be, I can do so, because I have free will, and it was me who stabbed myself in the leg, not some other being that made me do it. Even if I'm not acting in my right mind, I'm still not being controlled by some other entity, but rather by own (compromised) self, so I'm still demonstrating free will, though not self-control.
The antithesis to free will is the idea that we're slaves to social systems and/or upbringing, that we ultimately have no ownership over who we are, and therefor cannot have control over what we do. It's the idea that we aren't responsible for our own actions because -- subconsciously -- we have no ownership over our selves, and therefor the faculties our actions stem are pre-defined and not truly free. It's basically that our actions are defined the mold society formed us from. If that's the discussion you want, that's another thread entirely.
There are realistic limitations, of course. I can't fly because I don't possess the capability, and free will doesn't grant me capability, only the control of what I'm capable of. And I can't go out and make a million dollars tomorrow, because there's no action within my control that lets me do so. Free will doesn't let me control probability like the lottery and it doesn't grant me omnipotence to control others into giving me their money. But free will does
grant me the capability of going out and trying
to do those things.