Open-Weight AI Models: Cost, Security, and Governance Guidance

Customers frequently ask us whether open-weight AI models are safe to use, especially as cheaper models climb the leaderboards and show up on AWS, Google Cloud, and Azure. Below is the guidance we've been sharing, covering where these models stand today, the risks that remain even when you host them, and how to decide where they fit in your stack.

Bottom line

Open-weight models can cut AI spend, but in agentic systems they add supply-chain and safety risk that no test can fully remove. Our guidance: use them where the system cannot act, prefer Western open models or frontier APIs where it can, and put runtime monitoring and policy enforcement in front of any open model that touches code, data or external systems.

Two facts shape the decision. First, the best open models (all Chinese) now trail the frontier by about four months. Second, cheaper per token is not always cheaper per task, so cost savings need to be measured per workload.

Where open-weight models stand

The best open-weight model, Xiaomi's MiMo-V2.6-Pro, scores 46 on the Artificial Analysis Intelligence Index, against 58 for Claude Opus 5.5. Epoch AI puts the average lag at four months since January 2026, and CAISI measured the same four-month lag for GLM-5.3's cyber capabilities.

• Chinese labs hold every top open spot. Kimi K3, GLM-5.3 and DeepSeek V4.1 perform at roughly the level frontier models reached a few months ago.
• Western open models are far behind. Meta's strongest model, Muse Spark 1.3 (48), is still closed, with open weights promised but not yet released.
• Open is not automatically cheaper. Kimi K3 costs almost 3x more per task than GPT-6.1 Sol, which scores higher. Measure cost per completed task, not per token.
• Adoption is rising fast. Chinese models took 61% of OpenRouter tokens in February 2026, mostly coding and agent workloads, and a16z estimates about 80% of startups on open stacks run Chinese models.

All three hyperscalers now serve these models as managed APIs:

Managed hosting keeps prompts inside the cloud boundary, and AWS states data is not shared with the model provider. It does not change how the model behaves, including its ability to hijack harnesses or agentic sessions.

The risks

Hosting a model inside AWS, Google Cloud or Azure, or self-hosting these models removes the data-sharing risk. However, three risks remain.

1. Supply chain and state influence. China's generative AI rules require services to "adhere to core socialist values" and bar content that endangers national security or "national unity" (Interim Measures, Art. 4). The effect is measurable. NIST's CAISI found DeepSeek models echo four times as many misleading CCP narratives as US models, and an independent Kimi K2.5 evaluation found narrow political censorship, especially in Chinese. Early press tests of DeepSeek showed it deflecting questions on Tiananmen Square and Taiwan (cited in CrowdStrike's research below). More worrying for engineering, CrowdStrike showed that mentioning Tibet or Uyghurs in a coding prompt raised DeepSeek-R1's rate of severely vulnerable code by nearly 50% (19% to 27.2%). There is no public proof of a deliberate backdoor in a major Chinese model. But a model whose code quality shifts with who it thinks the user is, is already a supply-chain problem.

2. Weaker safety and alignment. The claim that open models are weaker because they are smaller no longer holds: Kimi K3 has 2.8 trillion parameters. The real gap is safety investment and evaluation. CAISI found DeepSeek's most secure model complied with 94% of jailbroken malicious requests (US models: 8%), and agents built on it were 12x more likely to be hijacked into phishing, running malware or leaking credentials. Kimi K2.5 shipped with no safety evaluation and showed "concerning levels of sabotage ability" in agentic tests. Cisco found multi-turn jailbreaks succeeded 26% to 93% of the time across eight open models, Western ones included, with capability-first labs (Meta, Alibaba, Mistral) leaving safety to the deployer. For agents this means more prompt injection and more high-impact mistakes.

3. Weights cannot be audited like code. Open weights are not open source. There is no training data or training process to inspect, only billions of numbers. Anthropic's Sleeper Agents research trained models to write secure code when the prompt said the year was 2023, and exploitable code when it said 2024. That hidden behavior survived standard safety training, and adversarial training even taught the models to hide it better. No scan can certify a model clean. Testing in your own environment lowers the risk but cannot eliminate it.

Guidance for engineering

Choose the model by how much the system can do, not by its benchmark score. Risk rises with agency: the ability to write, change state or communicate.

How Neo lowers the risk

Neo monitors what agents do and enforces policies at runtime, whichever model drives them: which tools, files, commands and network destinations an agent may use. Instead of trying to prove a model is clean (risk 3), it catches the outcome of a manipulated or misbehaving model, such as reading secrets or calling an unexpected endpoint. This makes high-agency use of open models, including coding agents, workable. The residual risk is still higher than with OpenAI, Google or Anthropic models, which get deeper safety testing before release.

Consider Western open-weight models first. OpenAI's gpt-oss, Google's Gemma 4, Meta's Muse Glimmer, Cohere's Command A+ and most of Mistral's models are Apache 2.0. They score well below the Chinese leaders, but they come from accountable vendors under US or EU jurisdiction, and Cisco found Google and OpenAI's open models more resistant to multi-turn attacks. They are still exposed to risk 3.

Recommended controls for any open model:

• Keep an approved-model list. Pin exact checkpoints and pull weights only from the lab's official repository.
• Run the model in your own cloud account (Bedrock, Vertex AI, Foundry) or self-host. Do not call Chinese labs' own APIs, which process data outside your boundary.
• Check the license per checkpoint. Kimi K3 uses a revenue-tiered license and GLM-5.3 a custom one.
• Re-assess each quarter.

Where to get an independent view: NIST CAISI publishes evaluations of PRC models, Artificial Analysis tracks capability and cost, and Epoch AI tracks the open versus closed gap.

References

1. Artificial Analysis, LLM Leaderboard and Open Source Models, Intelligence Index v4.3.2, accessed 30 Sep 2026
2. Epoch AI, Open models lag state-of-the-art closed models by 4 months, May 2026
3. NIST CAISI, Assessment of Z.ai's GLM-5.3 Cyber Capabilities, Sep 2026
4. NIST CAISI, Evaluation of DeepSeek AI Models, Sep 2025
5. Yong et al., An Independent Safety Evaluation of Kimi K2.5, Apr 2026
6. CrowdStrike, Security Flaws in DeepSeek-Generated Code Linked to Political Triggers, Nov 2025
7. Cisco AI Defense, Death by a Thousand Prompts: Open Model Vulnerability Analysis, Nov 2025
8. Hubinger et al. (Anthropic), Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, Jan 2024
9. China Law Translate, Interim Measures for the Management of Generative AI Services, 2023
10. AWS, Introducing Kimi K3 on Amazon Bedrock, Sep 2026
11. Google Cloud, Vertex AI managed models (MaaS)
12. Microsoft, Foundry Models sold by Azure, Kimi K2.6 in Foundry, Kimi K2.7 Code in Foundry
13. Dataconomy, Chinese AI models hit 61% market share on OpenRouter, Feb 2026
14. dreaming.press, Open-Source LLM Leaderboard, September 2026 (license details)