Does the framework support evaluation of quantized models, such as AWQ models?

[watermelon-lee]

I attempted to test the quantized Qwen (AWQ) model, but the performance dropped significantly. Even after downloading the official quantized model, the results were still far from satisfactory. Does this framework support the evaluation of quantized models?

my script is:
python run.py --datasets ceval_ppl_578f8d
--hf-type base
--hf-path $MODEL_PATH
--model-kwargs torch_dtype='torch.float16' trust_remote_code=True device_map='auto'
--max-out-len 1024
--min-out-len 1
--hf-num-gpus 1
--batch-size 1
--max-num-workers 1
--stop-words "<|im_end|>" "<|endoftext|>"
--w "outputs/${SAVE_NAME}/p1"
--debug