If the maker of a Power of Attorney document (be it for health care or general management of business affairs) cannot consent to a change in the named Attorney-in-Fact, no other person has the authority or right to ‘transfer’ that responsibility to another, even another family member.
Power of Attorney can be granted only by the person for whom it is to be exercised; it cannot be assigned to someone not named in the document. (Otherwise, Power of Attorney could be assigned to a total stranger, or someone with interests adverse to the person making the document, etc.).
The family could seek to have a guardian appointed over the father and the father’s estate; such a ruling would be issued by a Probate Court or family court.
Such an action can be filed pro se (do-it-yourself) with the court, but Standard Legal recommends contacting an experienced local attorney to assist in the process. You can find a local attorney for free here.