When OH had 2 weeks respite, I didn't visit at all, the first time I was away, anyway. When he went into his 3rd respite, which morphed into permanent, I didnt go in the first two weeks, it was meant to be respite, and I got a tummy upset. The third week he got a chest infection, so I was going in every day for a while. When that cleared and he stayed there, I was going in 3-4 times a week. I lived about 10-15 mins away ( though the road was closed for a while, the official diversion more than doubled the trip, the unofficial one wasn't so bad. It was so well used it had to be repaired in places!) btw.
Then I went back to 3 times a week, he couldn't walk by now, talking was not good. I used to go in late morning, so that I could help him with his dinner if necessary. The day he told me to get lost, I left immediately. He never did that again!
When he was obviously failing, I went in more often, daily for the last few days.
I think you adapt to what's good for you, what you can manage, and how you both feel about it. Please don't feel guilty! In many cases the pwd doesn't remember, feeling guilty is self imposed, in some cases to ease one feelings. IMHO.