To get one's goat: It implies taking something valuable (?) from someone. It's suggested that it comes from the French. To steal someone's goat was to take their only source of milk.
Another suggestion from a quick google was that racehorses used to be kept with goats to calm them. Don't ask why. If the horse's goat was taken away the horse became unhappy and less apt to win the race.
"No ray of sunshine is ever lost, but the green which it awakes into existence needs time to sprout, and it is not always granted to the sower to see the harvest. All work that is worth anything is done in faith." - Albert Schweitzer