'It's beneath their office,' Carney grumbled to reporters in Ottawa on Tuesday