The former cricketer eventually decided that he could not give both careers the time they needed