I'm not actually sure that you can force the issue with GP ( I stand to be corrected on this) , as you don't have Health & Welfare POA. The GP can ask your mum for consent, but if she doesn't agree, I don't know where you stand - even with loss of capacity.
Before my mum had dementia I spoke to her GP, who would only discuss her with me once Mum had given a verbal consent.
Even if they refuse to discuss the situation with you, you can at least explain that your Mum's default setting is that she is fine, and perhaps they can word their questions better e.g "Mrs X, we need to pop in to check your wound after the hospital visit" rather than "How are doing after your operation".
I can quite understand why you want to throw in the towel, and I don't think anyone would blame you.
All I would say is, if you do, call SS and spell out very clearly (perhaps back up with an email confirming your call) that your mother is completely unable to care for herself, and a medical professional has confirmed lack of capacity; that you are no longer prepared to attempt to intervene; that your brother is not communicating with you and you have no idea what, if any, care he is prepared to provide; and that she is their responsibility.
And see what happens.