You can, of course, retire whenever you like, but retiring too early may impact your Social Security payments.