The City of Yarra has halted plans to build at least 50 affordable rental apartments for workers in Abbotsford.