Getting in or out of a car is actually pretty complicated. We normally do it 'on autopilot' just as walking up or down stairs but if your autopilot is broken and you have to plan each movement separately, it becomes extremely difficult.
Getting my mum, who was by then walking with a frame, from standing to sitting in a chair became increasingly hard. I had to tell her every little step - 'No, turn round so you're facing away from the chair, no, all the way round, yes, ok, stop! Right, now take a step backwards, no not that way, backwards! Yes, now another step until you can feel the edge of the chair against your legs. No don't turn round again!' And so it would go on, even with much assistance she simply couldn't work out how to get there.
So maybe with your husband it's a combination of that and perhaps fear. Whilst in the car he at least feels safe. It is a known thing. But getting out makes him anxious. Perhaps he no longer recognises those familiar places like his brother's house or even the church. It might feel like stepping into the unknown. But when you are outside your house, it still looks familiar to him and his autopilot kicks in again.
Just my thoughts - it's all very difficult.