In a way, the conscience is selfish -- it needs to assert its instincts to keep it from gnawing at itself; i.e., to maintain internal security and comfort. Conscience can become overwhelming and manifest externally as unhealthy egocentrism. To over-simplify, there are two basic reasons for why this is.
1. Transferring values to others
People with strong values tend to feel the need to proselytize, which can be pretty difficult and stressful.
Result: The urgency of conscience can overwhelm a person's ability to slow down and consider other perspectives and plans of action. They resist new modes of thought that either question or oppose their values. If a person's values don't include a concern for developing the rest of their cognitive functions* -- or if it does, but they're having difficulties doing so -- this results in poor rhetorical/social skills, intolerance, etc.
*obviously not a conscious thought for most people
2. Applying values to self (growing relationships, refining skills, maintaining emotional/spiritual vitality, gaining new experiences, etc.)
People with strong values tend to feel a need to live them out as fully as they can. This is a pretty heavy burden to carry around all the time. Not that Fi-users are necessarily altruistic, but they'll still be highly concerned with sticking by their own personal standards, or dreams, or whatever else. The weight of strong values may lead to laziness (or failure), which leads to shame, which leads to self-absorption*. The cycle goes on. Baggage is constantly accrued and, in worst cases, made obvious in interactions with others.
*I'm not sure how to explain the link between shame and self-absorption, but its existence is clear if you read anything about e4s, most of whom are FPs. I guess the way I would explain it is that Fi-users, especially 4s, deal with shame on a very inward level. They don't look for external validation as much as they try to renew enthusiasm for their own values and to jump-start personal growth via introspection. It's easy to forget that a good deal of personal growth needs to take place in the external world, especially when the ego is bruised. Even if the person is mostly oblivious to external judgment, having their shortcomings exposed makes them seem more real.
Apologies if this was bullshit, I just felt like getting my current understanding of Fi into words. (Also, I hope that it doesn't seem like I'm trying to say that Fi-users are privy to unique and honorable burdens or anything idiotic like that. It's just that these specific stressors are obviously most common, or immediate, to FPs.)
edit: I really like @OrangeAppled 's description of Fi as "forming value concepts using the self as a prototype for the human condition, not personal beliefs." Essentially what I meant by "values."