Hackers Could Weaponize GGUF Models to Achieve RCE on SGLang Inference Servers

A critical vulnerability in SGLang, a popular framework used to run large language models (LLMs), has raised serious concerns across the AI and cybersecurity communities.

Security researchers have identified the flaw, tracked as CVE-2026-5760, which allows attackers to achieve Remote Code Execution (RCE) by exploiting malicious GGUF model files.

The issue stems from how SGLang processes model metadata, specifically within its reranking functionality.

When a server loads a specially crafted GGUF model, attackers can execute arbitrary commands on the host system.

This makes it possible to fully compromise inference servers simply by tricking users into deploying a poisoned model from public repositories such as Hugging Face.

At the core of the vulnerability is a Server-Side Template Injection (SSTI) flaw in SGLang’s reranking endpoint.

The framework uses the Jinja2 templating engine to process chat templates embedded in models.

However, instead of using a secure sandboxed configuration, the vulnerable code relies on the default Jinja2.Environment() function.

This oversight allows templates to execute unrestricted Python code during rendering.

A recently published proof-of-concept (PoC) exploit demonstrates how easily this flaw can be weaponized.

In the attack scenario, a threat actor creates a malicious GGUF model file containing a crafted tokenizer.chat_template.

This template includes a trigger phrase such as “The answer can only be ‘yes’ or ‘no’,” which activates SGLang’s Qwen3 reranker detection logic.

Once the victim downloads and loads the compromised model into their environment, the attack is primed.

When a request is sent to the /v1/rerank endpoint, the application processes the malicious template through the insecure Jinja2 engine.

The embedded SSTI payload then escapes the template context using known Python bypass techniques, ultimately executing arbitrary operating system commands on the host machine.

This vulnerability highlights a growing supply chain risk in AI infrastructure. As developers increasingly rely on third-party model repositories, insufficient validation of model metadata can introduce severe security gaps.

CVE-2026-5760 is categorized under CWE-1336 (Improper Neutralization of Special Elements in Template Engines) and CWE-94 (Code Injection).

Notably, this issue shares similarities with past vulnerabilities such as the “Llama Drama” bug (CVE-2024-34359) in llama-cpp-python, as well as recent flaws impacting vLLM frameworks.

These recurring patterns indicate a broader systemic risk in how AI frameworks handle dynamic template rendering.

Security experts strongly advise administrators using SGLang version 0.5.9 to avoid downloading untrusted GGUF models until a proper patch is released.

Implementing sandboxed template rendering, such as Jinja2’s ImmutableSandboxedEnvironment, is considered a necessary mitigation step.

As AI adoption accelerates, this incident serves as a reminder that model files should be treated as untrusted input, requiring the same scrutiny as executable code.

Follow us on Google News , LinkedIn and X to Get More Instant Updates. Set Cyberpress as a Preferred Source in Google

The post Hackers Could Weaponize GGUF Models to Achieve RCE on SGLang Inference Servers appeared first on Cyber Security News.