pwm: vt8500: Only unprepare the clock after the pwmchip was removed