I only took him when carers at the Home suggested it. Every one's situation is different. It was right for us, but I would definitely not have done, if adviced against it. He had got into his head all sorts of strange reasons why I didn't take him home, so going there was very reassuring for him, as he could see that nothing had actually changed.When you took him home for visits how did you get him to go back to the facility? My husband begs to come home and I have been advised not to. I take him to church and lunch after on Sundays and out to llunch some other days and he goes back to the facility without a problem but I don't know about home visits.
I only take him when he is in the right mood. The Home are great at talking to him about it being a short visit and they are expecting him back. Equally they are great at backing me up if I don't take him. This support is vital. I always leave my bag at the Home so that he knows that I need to go back for it!
I don't think there is a right or wrong way, and it's very waring to be constantly asked by him. I put him off a lot, so he knows it's not going to be an everyday occurance.
You sound as if you do lovely things with your OH and that might be enough for you two.