That's an extremely simplified view of the situation. Some of the problem here is fallout from the Citizen's United ruling. Now that 501(c)(4) groups can engage in political spending, it becomes very difficult to determine whether a group qualifies for 501(c)(4); that is, whether the group exists primarily for "social welfare" and doesn't have a primary purpose of engaging in electoral advocacy.
Note that qualifying as 501(c)(4) gives one both tax exempt status and removes the requirement to disclose donors. This, in turn, has led to the rise of a lot of "dark money" in the last couple of elections, with 501(c)(4) groups outspending SuperPACs (which have their own issues).
A real solution would be to have clear guidelines and procedures for determining whether someone qualifies for 501(c)(4) status, and enough funding so that such procedures could be designed and actually followed (with some means of external review). It seems likely that such efforts would pay for themselves, since we are no doubt improperly granting tax-exempt to groups that are clearly primarily political (on both sides of the fence), and thereby losing tax revenue. At the very least, placing a percentage cap on the amount of political spending a 501(c)(4) organization would help (since if you are spending more than 50% of your outlays on political spending, claiming that your primary purpose isn't engaging in electoral advocacy seems absurd).
Looking at the IRS 501(c)(4) guidelines
, one can see that the guidelines are extremely murky and vague. It's pretty much inevitable that bias (individual or collective) is going to creep in without objective, quantifiable standards.
Politically speaking, I think the IRS videos are also damaging to the IRS's reputation, given that they are both ridiculous and were produced at tax payer expense. Certainly private corporations sponsor such silliness often (go to any major trade show, user conference, etc), but people become (fairly justifiably) upset when their taxes are paying for it.