It's probably because INTs, in general, tend to apply analytical reasoning to everything in a vicious manner that we tend to dislike church. Plus, as you said, we're not very social or generally optimistic (as in fact, we tend to be very cynical). Thus, church and the superficial smiling and false emotions which go along with it just tends to throw us off. We INTPs generally aren't made to see life from some false rosy glow; we want to know what is truly driving the world. Hence, logic tends to be used like a knife to cut away at anything and everything which doesn't add up. Therefore, we can totally reason away all of the happy aspects of life which simply ultimately lack real meaning. This may lead to nihilism, or something similar, but always something realistic and based on what logically exists. So we just tend to want to be around intelligent persons, as we are—not a bunch of rather "mindlessly social" drones who actually do tend to believe anything they are told.
At least that's how I tend to view the INTPs general relationship to society. I've been there, done that. I destroyed the concept of God, internally, through logic. I've reasoned away all of the fancy wishful thinking hopeful perceptions a human being could have toward this world (which I've come to conclude is absurd and meaningless). I've decided people are generally morons (as George Carlin did). So a church is the LAST place I ever want to be, truly wasting my time. I could masturbate, or play a few video games, instead. And life's more pleasurable and less logically dissonant that way.