Kimi K3: Moonshot AI’s 2.8 trillion parameter model

On 16 July 2026, Moonshot AI unveiled Kimi K3. It is by far the company’s largest model: 2.8 trillion parameters, a one million token context window, and native multimodality. The stated goal is to compete with the best proprietary models on the market.

Early results are strong, especially on software development that runs for hours, agent-driven research, automation, and complex document analysis. At the time of writing, K3 sits first on Arena.ai’s WebDev leaderboard, ahead of several leading proprietary models.

But K3 is not just a bigger K2. Moonshot changed several things in the architecture to handle long contexts, add depth to reasoning, and keep compute cost from exploding along with model size.

Who built Kimi K3?

Kimi K3 comes from Moonshot AI, a Beijing-based company founded in 2023 by researcher Yang Zhilin. Before Moonshot, he studied at Carnegie Mellon and worked on language models and foundation architectures.

The company made its name with Kimi, an assistant that could already digest very long documents in its earliest versions. Several open or open-weight models followed: Kimi K2 in July 2025, then the multimodal and specialised variants K2.5, K2.6 and K2.7 Code.

With K3, Moonshot moves to a completely different scale. The model is presented as the first « open » system in the three trillion parameter class. The timing is not accidental: DeepSeek, Alibaba, Z.ai and Moonshot are closing the gap with American labs fast, and they usually do it with models that are open or cheaper to run.

One caveat is worth stating. As of 27 July 2026, the full K3 weights are not in the official public repositories we were able to check. Moonshot had announced a 27 July release, and Artificial Analysis still lists K3 among proprietary models. For now, this is a model with announced open weights, not an open source model you can download today.

Kimi K3 key specifications

Four numbers sum up the model.

2.8 trillion parameters. That is the total size, not what runs on each token. K3 uses a sparse Mixture of Experts (MoE) architecture that activates only a small share of the network at each step.

A one million token context. On paper, the model can swallow a very large code repository, several hundred documents, or a very long work history in one go. The target is clear: autonomous agents and tasks that stretch over hours.

Native multimodality. Text and images go through the same model. Moonshot also highlights workflows involving video, screenshots, interfaces, animations and 3D environments. The public API, however, currently documents text and image inputs with text output.

Reasoning on by default. K3 is designed as a reasoning model: it can spend more compute planning and checking its answers. The API offers three effort levels: low, high, max.

A very sparse Mixture of Experts architecture

The core of K3 is called Stable LatentMoE. It is a Mixture of Experts architecture with 896 experts, of which only 16 are selected for each token. That is roughly 1.8 % of the routed expert pool.

In an MoE, each expert is a specialised sub-network. A router looks at the token and picks the experts best suited to process it. This lets you inflate the total parameter count without a proportional increase in the compute needed for each inference.

Do not over-read that number, though: K3 does not use 1.8 % of its parameters. Some components of the model are shared and always active, and Moonshot has not yet published the exact number of parameters actually activated per token.

Routing that many experts is a problem in itself. Moonshot uses a technique called Quantile Balancing, which spreads the load based on the quantiles of the router’s scores. The point is to stop a handful of experts from grabbing most of the tokens and creating bottlenecks on the accelerators.

Kimi Delta Attention: cutting the cost of long contexts

A standard Transformer uses quadratic attention. When context length doubles, the compute and memory needed for certain operations can climb very fast.

K3 introduces Kimi Delta Attention (KDA), a linear attention mechanism based on state updates and a delta rule. According to a preliminary analysis published on Hugging Face, three attention layers out of four use KDA, with the fourth using a gated variant of Multi-Head Latent Attention.

The mechanism did not come out of nowhere. Moonshot had already described it in the paper Kimi Linear: An Expressive, Efficient Attention Architecture (arXiv, October 2025), on a far smaller model of 48 billion parameters with 3 billion active. KDA is presented there as an extension of Gated DeltaNet with finer-grained gating, and the KDA plus MLA hybrid cuts KV cache usage by 75 % compared to full attention while outperforming MLA alone. This is the paper to read if you want the mechanism in detail rather than the press release version.

The hybrid design aims to combine two advantages. KDA layers do most of the work cheaply, while global attention layers keep the precision needed to retrieve information from far back in the context.

Moonshot claims the whole package (architecture, new MoE routing, revised training recipes) delivers scaling efficiency roughly 2.5 times better than K2. That figure comes from the company and still needs the full technical report to be substantiated.

Attention Residuals: attention applied to depth

The other headline change is called Attention Residuals, or AttnRes.

In a standard Transformer, each block adds its output to a shared residual stream, and information stacks up layer after layer. AttnRes does something different: blocks can reach back and select specific representations from earlier layers.

So the model applies a form of attention not only across the tokens in a sequence, but also across the depth levels of the network. The stated benefit is better information flow in a model with a very large number of layers, and less dilution of important signals along the way.

A model trained to run in low precision

K3 uses quantization-aware training starting at the supervised fine-tuning stage. Weights are in MXFP4, activations in MXFP8. The goal: less memory and better compatibility across accelerator families.

That does not make K3 usable on a workstation. Moonshot recommends supernode configurations with at least 64 accelerators to deploy it properly. Even quantized, a 2.8 trillion parameter model is datacenter infrastructure.

K3 is therefore an open model built first for cloud providers, large labs and companies that already run distributed infrastructure. Not for your machine.

Kimi K3 benchmark scores

Two things need separating: the scores Moonshot published, and independent evaluations.

Results published by Moonshot AI

The pattern is clear. K3 is strong when a task mixes planning, tools, browsing, long context and a finished deliverable. It is far less consistently dominant on knowledge or pure reasoning benchmarks.

Comparisons that are not all on equal terms

Moonshot says so itself: the models were not all evaluated in the same agentic environment. Depending on the test, K3 runs under Kimi Code or Claude Code, OpenAI models under Codex, and Anthropic models under Claude Code, Terminus or fallback mechanisms.

These scores are not comparing raw models. They compare bundles: model, agent, tools and execution environment.

Another point worth remembering: every K3 score published by Moonshot was obtained with reasoning effort set to « max », temperature 1 and top-p 1. Those settings push quality up, but also latency, token count and cost.

What do independent evaluations say?

Artificial Analysis gives K3 a score of 57 on its Intelligence Index. As of 27 July 2026, that puts it 7th out of 190 models in its category, well above the median of 32.

On agentic tasks the results are better still. In its first evaluation, K3 reached an Elo of 1,668 on GDPval-AA v2 and first place on AutomationBench-AA with 53 %. It also ranks among the best on AA-Briefcase, a knowledge-work benchmark combining presentations, spreadsheets and interface mockups.

On AA-Briefcase specifically, K3 scores an Elo of 1,543, with analytical quality comparable to Claude Fable 5 but presentation quality below GPT-5.6 Sol. The evaluation also reports an average cost of 10.57 dollars and 56.4 minutes per task, driven by the number of turns and tokens generated. That matters if you plan to run this at scale.

Human evaluations from Arena.ai confirm the frontend strength. K3 is first on the WebDev leaderboard with a preliminary score of 1,682 across 3,776 votes, ahead of Claude Opus 5 High and Claude Fable 5.

General conversation is another story. On the Text Arena leaderboard, K3 sits 11th with a preliminary score of 1,485. Competitive, but nowhere near the top spot.

Where is Kimi K3 actually good?

Four areas stand out.

Long-running software development first. K3 looks comfortable when it has to explore a large repository, use a terminal, edit several files, run tests and hold on to a goal across a long series of actions.

Frontend and visual development next. Its first place on WebDev Arena, combined with vision, lets it alternate between generating code and reading screenshots. It looks at how an interface or a game renders, then fixes its own code.

Deep research and agentic browsing too. The BrowseComp and DeepSearchQA scores show real ability to search, consult many sources and build an answer out of a complicated browsing path.

Finally, end-to-end document work: reading PDFs, analysing tables, building a presentation, producing a report, handling files. The results on OmniDocBench, SpreadsheetBench 2 and AA-Briefcase all point the same way.

Kimi K3 limitations

Infrastructure first. 2.8 trillion parameters, even very sparse and quantized, remain extremely hard to host. The 64 accelerator minimum rules out the vast majority of local deployments.

Operational performance next. Artificial Analysis measures around 32 tokens per second and a high time to first token once reasoning time is counted. The model is also more verbose than average.

Then behaviour. Moonshot acknowledges that K3 can be overly proactive, make decisions nobody asked for, or become unstable when the agentic environment fails to return its full reasoning history. Hence the recommendation to use a compatible tool and to define precisely what the model is allowed to do.

Finally, Moonshot admits the overall user experience still trails the best proprietary models. K3 can nail a long, complex task while being slower, less predictable and less pleasant in a simple conversation.

Is Kimi K3 on par with the best models?

Not everywhere. It still sits behind the leaders on several general reasoning, knowledge and user experience evaluations.

But for a model meant to be open, this is a real step up. K3 competes head-on with proprietary systems on tasks that matter in a business: software development, automation, document research, interfaces, spreadsheets, deliverable production.

For a digital agency like Fenxi Technologies, the point is not to build one more chatbot. K3 becomes relevant when you need agents that can work on large technical projects, analyse a lot of data, and produce complete applications or documents with limited human supervision.

The takeaway from K3 is not its parameter count. It is the direction. Open models are no longer just answering questions: they are learning to plan, use tools and carry a complex project through to the end.