After the Mexican-American war ended in 1848, Grant remained in the army and was moved to several different posts. He was sent to Fort Vancouver in the Washington Territory in 1853, where he served as quartermaster of the 4th Infantry Regiment. His wife, eight months pregnant with their second child, could not accompany him because his salary could not support a family on the frontier. In 1854, Grant was promoted to captain (one of only 50 still on active duty) and assigned to command Company F, 4th Infantry, at Fort Humboldt, California. However, he still could not afford to bring his family out West. He tried some business ventures, but they failed. Grant resigned from the Army with little advance notice on July 31, 1854, offering no explanation for his abrupt decision. Rumors persisted in the Army for years that his commanding officer, Bvt. Lt. Col. Robert C. Buchanan, found him drunk on duty as a pay officer and offered him the choice between resignation or court-martial. Some biographers discount the rumors and suggest Grant's resignation, and his drinking, were both prompted by profound depression. According to this view, Buchanan hated Grant and concocted the drunkenness story years later to protect Buchanan's action in removing the man who became one of the most famous generals in history. The War Department stated, "Nothing stands against his good name."
A civilian at age 32, Grant struggled through seven lean years. From 1854 to 1858 he labored on a family farm near St. Louis, Missouri, using slaves owned by his father-in-law, but it did not prosper. Grant owned one slave (whom he set free in 1859); his wife owned four slaves (two women servants and their two small boys). In 1858-59 he was a bill collector in St. Louis. Failing at everything, in humiliation he asked his father for a job, and in 1860 was made an assistant in the leather shop owned by his father and run by his younger brother in Galena, Illinois. Grant & Perkins sold harnesses, saddles, and other leather goods and purchased hides from farmers in the prosperous Galena area.
Although Grant was essentially apolitical, his father-in-law was a prominent Democrat in St. Louis (a fact that lost Grant the good job of county engineer in 1859). In 1856 he voted for Democrat James Buchanan for president to avert secession and because "I knew Frémont" (the Republican candidate). In 1860, he favored Democrat Stephen A. Douglas but did not vote. In 1864, he allowed his political sponsor, Congressman Elihu B. Washburne, to use his private letters as campaign literature for Abraham Lincoln and the Union Party, which combined both Republicans and War Democrats. He refused to announce his political affiliation until 1868, when he finally declared himself a Republican.