'This is a completely unfounded caving to moral panic,' said one trans rights group