AI ModelsTencent HunyuanOpen Weight ModelsAI AgentsCoding AI

Tencent Hy4 Preview: 770B MoE Architecture and 1M Context Window

Tencent Hy4 preview has 770B total and 49B activated parameters, a 1M context window, published FP8 checkpoints, deployment recipes, and an Apache 2.0 license.

Tencent Hy4 Preview 770B MoE Architecture and 1M Context Benchmark Analysis
Analyzed Foundation Models (3)All Models Directory→

Tencent announced Hy4 preview on August 28, 2026, describing it as an open-weight model for productivity tasks. Tencent’s model repository documents a 770B-parameter Mixture-of-Experts backbone with 49B activated parameters per token and a 1M context length. These are architectural specifications; active parameter count is not the memory required to load the weights.

Architecture and release

The official model repository documents a 78-layer backbone: one dense feed-forward layer followed by 77 MoE layers. Each MoE layer has 256 routed experts and one shared expert; the top eight routed experts plus the shared expert are activated for each token. The repository also describes Gated DeepSeek Sparse Attention with IndexCache and identity Hyper-Connections. It lists a native multi-token prediction layer separately from the backbone parameter count.

Tencent publishes standard and FP8 checkpoints, deployment recipes for vLLM and SGLang, and an Apache License 2.0. Follow the repository’s current setup instructions for runtime support and hardware configuration. Its example vLLM deployment uses tensor parallel size 8; that is a documented recipe, not a universal minimum hardware requirement.

Published specificationValue
Backbone parameters770B total; 49B activated per token
Context length1M tokens
MoE layers77 after one dense layer
Routed experts256 per MoE layer; 8 activated per token plus 1 shared
CheckpointsStandard and FP8
LicenseApache License 2.0

What the evaluation shows

Tencent says it ran a blind side-by-side evaluation in which 163 internal experts rated model outputs on 203 engineering tasks. It reports a mean score of 2.99 for Hy4 preview against 2.92 for GLM-5.3 and 2.94 for Kimi K3, with pairwise win/tie/loss rates also published in the repository.

This is an internal vendor evaluation, not an independent benchmark. The source does not provide enough detail in the overview to treat the small mean-score differences as a general performance ranking. The repository’s benchmark appendix currently provides no detailed benchmark score table, so readers should use its published evaluation description as the limit of what can be verified from this source.

Known limitations and access

Tencent describes Hy4 preview as an early model and says it can spend longer than necessary reasoning through complex tasks and over-verify its work. These caveats matter for agent workflows, where extra tool calls can add time and cost.

At launch, Tencent said the model was available through Tencent products and APIs, and that WorkBuddy and CodeBuddy would offer two weeks of free access. That was a launch-period offer, not a current availability guarantee. Check the official product pages for current access.

A 1M context window is a maximum capacity, not a promise that every prompt fits comfortably or that the model will use every document accurately. Test long-context retrieval against known answers, and measure latency and memory with the actual serving stack.

The Tencent release notice and Hy4 preview repository are the primary sources for release details, license, deployment recipes, and limitations. Explore maintained RankLLMs data in the AI Model Leaderboard.

Frequently Asked Questions

What is the Tencent Hy4 preview model?

Hy4 preview is Tencent Hunyuan's open-weight Mixture-of-Experts (MoE) foundation model featuring 770B total backbone parameters, 49B active parameters per token, and a 1M-token context window.

Is Tencent Hy4 preview open source?

Yes. Tencent released the Hy4 preview base and FP8 quantized checkpoints under the permissive Apache License 2.0, available on Hugging Face, ModelScope, and GitCode.

How many parameters are active during Hy4 inference?

Hy4 contains 770B total parameters across 78 layers, activating 49B parameters per token (6.36% activation ratio) via 8 routed experts and 1 shared expert out of a 256-expert pool.

What attention mechanism does Hy4 preview use?

Hy4 utilizes Gated DeepSeek Sparse Attention (Gated DSA) combined with IndexCache for layer-wise sparse index reuse and Identity Hyper-Connections in its residual stream.

Where can developers try Hy4 preview for free?

Tencent offered two weeks of free WorkBuddy and CodeBuddy access at launch. That was a time-limited launch offer; check the services for current availability.

Sources & disclosures

Source links for data and claims discussed in this article are listed below. Refer to each source for its current figures and methodology.

Share Article

Recommended Reading

Related by topic and recency.

All articles→
ReviewsAI Models

Alibaba Model Studio Token Plan: Pricing Structure and Claude Code Integration

Review of Alibaba Model Studio Token Plan terms, Singapore availability, Personal and Team quotas, API compatibility, and billing checks before subscribing.

Sep 18, 2026 · 3 min read
ComparisonsAI Models

GLM-5.3 Flash vs Muse Spark 1.2: Coding Agent Access and Cost

Compare access, privacy checks, API billing, and a reproducible coding pilot for GLM-5.3-Flash and Meta Muse Code with Muse Spark 1.2.

Sep 2, 2026 · 2 min read
NewsAI Models

Ox Alpha and GLM-5.3-Flash: What Is Known and Unverified

Z.ai identifies GLM-5.3-Flash as the first native multimodal model in the GLM-5 series. Review its documented architecture, context limits, current API prices, and the limits of claims about Ox Alpha.

Aug 27, 2026 · 2 min read