It's very easy to hurt an INFJ if you try to hurt their feelings, but very hard if you're trying to be nice (even if you're very emotionally clumsy). Intent is a big deal, especially when INFJs are dealing with INTJs, because we can sense it. You guys are more transparent (at least to INFJs) than you think, believe me.
It's not so much whether you're blunt, so much as why you're blunt that matters to INFJs. If it's for their own good, they'll appreciate it, and if it's just because you disagree with them and want to be right, then they won't so much.
But wait... I thought you were already married and everything! What happened?