I can sense he gets a bit frustrated because Ricky and Stephen don't really give him a chance to articulate his ideas and opinions--this makes for good humor but it often backfires and makes them look like asses themselves. I honestly don't think either of them are smarter than Karl, just better educated and more articulate speakers, but they often sound like closed-minded pricks who are unwilling to consider different ways of looking at things--this is a form of stupidity in itself.
I also think Karl plays up his character a bit. He's quite deadpan, in the most airheaded, aloof way, but he's obviously putting on a show. That takes a pretty good deal of intelligence to so convincingly portray a moron.