I remember now that you and your husband are the same type as my parents, INFP mother/ESTJ dad.
I can confirm that INFPs do not like to get bossed around. They are willing to give, be flexible, determine which ones are not worth the hassle to challenge, seem quiet, and easy-going, and then, BAM, you just hit an immovable, inflexible ball of stubbornness, and its throbbing with pent up frustration and outrage at being pushed and pushed.
As I see with my dad, when he's trying to get my mom to do something (FOR EFFICIENCY!!), it can turn into a spectacle of herding cats.
A caricature of a fight/conversation between them.
Dad: I want to be done with replacing the chandeliers. Need to go look at some at the store. Let's go this afternoon.
Mom: Great, okay.
Dad: Good, so you'll get ready then?
Mom: Yes.
Dad: Okay, I'll mow the lawn, and fix the hose, in the meantime.
Mom: Okay.
*1 hr later*
Dad: Are you ready?
Mom: Yes, going to get ready now.
Dad: What do you mean going to get ready now? What have you been doing for the past hour?
Mom: *lists of random things, plus, needing to rest for a bit.*
Dad: *loses his cool* *rants* I can't work like this. You are making it seem like my time is not valuable, and I'm to wait around for you? You're just making me waste my time!
Mom: You didn't wait around, nor waste time. You got the lawn mowed, and the hose fixed. Didn't you?
*argument escalates*
Dad: You know what? Forget it. You can figure out the chandeliers on your own. It is officially OFF my to-do list. Not my problem anymore. I want nothing to do with this!!
Mom: *coolly responding* Fine, I'll manage it on my own.
*a day later*
Dad to mom: So, what's happening with the chandeliers?
It's like it bothers him, like a psychosomatic pain, not being able to cross something off his to-do list, and the fact that others are not getting in line/cooperating to get the task finished, just puts him on edge, while for my mom, things on the to-do list don't always bother her/gnaw at her, in terms of needing to stick to a strict schedule, so she's good to let it be for a while.