Transformers v5.16.1 adds GLM-5.3-Flash multimodal MoE support
libraryHugging Face Transformers 5.16.1 lands support for GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series (320B total / 18B active MoE, hybrid sparse+linear attention, Manifold-Constrained Hyper-Connections, 30T-token multimodal pretraining). Reportedly outperforms GLM-5.2 and approaches Claude Opus 4.8 on coding/agentic benchmarks. Also restores BC for the tensor-parallel API and pins an ESMFold2 kernel commit for security.
An agent can now load and run GLM-5.3-Flash (multimodal, 320B/18B-active MoE with hybrid sparse+linear attention) directly via `transformers` for cheaper long-context reasoning, coding, and agentic tasks.
Headline is the GLM-5.3-Flash architecture add; small patch fixes (TP BC, ESMFold2 kernel pin) are secondary. High agent relevance: new multimodal MoE usable via standard Transformers API.
pip install -U transformers==5.16.1from transformers import AutoModelForCausalLM, AutoProcessor
import torch
model_id = "THUDM/glm-5.3-flash"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
messages = [
{"role": "user", "content": [
{"type": "image", "url": "https://example.com/chart.png"},
{"type": "text", "text": "Summarize this chart and suggest next actions."},
]},
]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(out[0], skip_special_tokens=True))