While this is true about the budget, the increase in resources the space program needs to really attract great minds and get to the level that we desire can be higher than I think some people would want.
It's also hard to justify when things locally aren't exactly ship shape. People are going to ask, why spend 10 billion dollars to study some space rocks when you got this going on across the pond:
Because the many outweight the few, and the fact that the 'many' are not born yet isn't an excuse when we're talking about major timeline altering choices on mind boggling scales.
Feel good solutions only take you so far, and even without accounting for the extremism of the example you used I think it selfish to make that choice if it only leads to more suffering in the end once you're dead and buried.
1. It's not like we're spending trillions on dollar on the poor african kids already and that the mean space agencies are going to take the bread out of their mouthes.
2. Ever heard of the parable of the talents?
3. As I said before, I'm not talking about 'moral choices' or whatever, we're talking about hard facts. If you don't want space exploration, then you have to accept heavy population reduction, and in countries where people just won't listen and 'll keep at making kids it can only lead to hunger, death and wars.
4. It's borderline stupid to try to force people to go against human nature (growth, expansion etc.) when the most logical solution allows for our natural tendencies to exist and florish.
5. (logical consequence of 4) The other solution (timeline), so no space exploration and stagnation would pretty much mean an unique totalitarian regime with heavy use of morals, sociological 'solidification' etc.