I suspect you're right, it's your Mum's damaged brain sending out wrong signals.
I know I observe this frequently in my son (he has severe learning difficulties and autism, he's 27)
With him it tends to be related to the digestion of his food, he is hyper-sensitive to his stomach and bowels.
When he needs a bowel movement, this sort of behaviour is displayed. 24 hours later he uses the toilet, without distress but no amount of reassurance in the previous 24 hours will help.
He seems to simply hate the internal feelings and noises (he has issues with his nose and ears too) of his body.
But it can also be that he's heard a siren in the street or on the TV.
It can also be that the batteries have run flat on his remote control.
I KNOW it's not connected with dementia and my Mam who had Vas Dem and Alzheimer's (86) never displayed the same behaviour, nor does my FIL (88) Vas Dem. but I do still think it's linked to the brain and wrong messages or wrong interpretation of messages.
The things that cause my son grief would take an hour to list.
I think, to a certain extent, he prepared me for my Mam's dementia.
But it isn't good that your Mum is hitting herself, she could and will hurt herself, I would ask for all physical reasons to be discounted...
UTI infections. Constipation. Itches. Sores. Crumpled bed sheets. Lace on edge of nighties. Bright lights. Loud noises. Bad people...etc.
When all of these and probably more that you and others can think of, have been discounted then I would seek drug therapy to lessen her distress.
Calm and peaceful is what I would want for myself as long as it wasn't just the lace on my nightie driving me mad.
I'm not being flippant...this is how basic life can be if one's brain functions in a certain way.
Best of luck.