A federal judge on Wednesday ordered Google to retool the system powering its monopoly in digital advertising, sparing the company from a wrenching breakup sought by the U.S. government.