Hundreds of the most vulnerable people in Norfolk could be asked to pay more than £600 a year for a crucial service they currently get for free.