Gwendy, I don't have an explanation for you, just sympathy. My mother (Alzheimer's and no short term memory) is also in a care home and sometimes when I visit, I cry in the car afterwards, sometimes I don't, and sometimes I feel out of sorts and upset a day or two later. If there is any pattern or way to predict my reaction, I've not figured it out. I know the only way to handle things is to have no expectations and just go with the flow of whatever happens, but I can't always manage to do that. Often I feel that whatever I do is the wrong choice. I'm sorry if you feel that way as well.
Things that have helped me:
-always have a plan of where I will go and what I will do after a care home visit (even if I change my mind, it gives me something to look forward to)
-I do best if I do something physical afterwards, like go for a walk or running errands, but a walk outdoors is best
-sometimes favourite music in the car afterwards helps (depending on if you're driving and the length of your drive)
Would it be possible for you to consider, even temporarily, cutting down on the length and/or frequency of your visits? I know that's not always an option, but if you find yourself feeling particularly upset or frustrated, there is no reason not to give yourself a break.
Also if I were in your situation with him not being verbal, I would take something to read or do with me. Sitting in silence must be very hard. Although I imagine you've already thought of that!
I imagine you're angry about the disease, or the situation, or any number of things. Feeling angry can often be just the tip of the iceberg of other emotions (something someone said to me long ago that has frequently been accurate), if you don't mind the silly mental image and phrase. So maybe the angry feelings are just masking other feelings, or not.
I know how much I hate this wretched dementia and what it's done to our lives.
Very best wishes to you.
p.s. comparisons are odious and not permitted! We are all struggling with this disease and you are not to berate yourself that others have it harder than you and so you mustn't grumble. We are all just coping the best we can, and we are all different in how we do that.