I think my problem is that I might skip over what I consider to be stupid details, expecting that the listener will understand what I am saying.
Basically, I think that our problem isn't that we have trouble putting our thoughts into words, it's that our speaking manner can only be understood by Ns.
Edit to add: I have noticed some weird problems at work, though (I'm a waiter at Chili's). Sometimes I will walk up to a table without having any idea what I am about to say, and just start talking. Normally, this works out ok, but occasionally some random crap bursts out that doesn't make any sense. I'm past caring, though.