madcat-os/lora
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: patch is_mllm_model for Qwen3.6 text-only model

Browse Source 
AutoRound misidentifies Qwen3_5ForConditionalGeneration as a VLM
and tries to load a vision processor. Patch to force LLM mode.
This commit is contained in:
marauder-actual
2026-06-01 04:26:15 +02:00
parent 367ed705ab
commit 465e74f49e
1 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
quantize_awq.py
+6
View File
@@ -9,6 +9,12 @@ Uses domain calibration data from substrate_v5.jsonl.
import json import json
import torch import torch
from transformers import AutoModelForCausalLM, AutoTokenizer from transformers import AutoModelForCausalLM, AutoTokenizer
# Qwen3.6-27B uses Qwen3_5ForConditionalGeneration which AutoRound
# misidentifies as a multimodal model. Patch before importing AutoRound.
import auto_round.autoround as _ar
_ar.is_mllm_model = lambda *a, **kw: False
from auto_round import AutoRound from auto_round import AutoRound
MODEL = "/workspace/substrate-qwen36-27b-merged" MODEL = "/workspace/substrate-qwen36-27b-merged"