In the words of a good friend of mine...
"I've had a much worse time coming off of caffeine than when I went from 1-2g of weed a day all year to not smoking at all abruptly."
(This individual is a pure math major at a fairly well known college, BTW, and one of the most intelligent people I've ever known... So much for the typical stoner ethos.)
IMO, smoking weed habitually is a waste of money and time.
I find the effects far more profound and enjoyable when completely lacking tolerance.
If you're quitting because you need to, because you're at risk of losing your job or something of that nature, then it probably is worth quitting completely. Ditto goes for if you just feel that it is somehow holding you back in an easily quantifiable way. Because weed lacks any serious negative affects, it's very hard to say when one can even be considered to be abusing it. I'd say, if it seriously impacts your motivation to do other things, then it's a problem.
Otherwise, lacking the two above conditions, there's no particularly compelling reason(s) to abstain totally, if it's something you genuinely enjoy doing.
Moderation...
And as stated by others in this thread, if your friends really give you bunk for quitting or drawing down, you probably have some serious thinking to do about the people you choose to associate with.