I have yet to talk to one person who has traveled to France who said they only spoke English & refused to try and speak French - most said they attempted French & only spoke English if the person they were speaking with initiated it. I agree it is a massively incorrect stereotype that Americans are arrogant about only speaking English & do not attempt to learn any phrases in the language of the country they are visiting.
It seems everyone has one story of the loud, arrogant American they encountered once (or more likely heard about - seems hearsay is often the source) and they latch onto it to justify their existing prejudice; meanwhile, they've ignored the countless Americans who do not fit that stereotype, but who disprove it.
So I disagree with people who say that Americans are more or just as rigid about foreign visitors speaking English poorly. In CA, we have TONS of tourists and if they make any attempt to speak English, people usually are very nice to them & don't correct errors. The irritation with Mexicans who don't speak English at all has to do with irritation over illegal immigrants (but I'm not defending nor attacking those views, although I notice most people aren't even irritated with Mexicans as long as they try to speak some English.), which is a very different issue from foreign tourists visiting the US.