On 29 October 2026 Mistral AI published the open weights of Mistral Large 4 on Hugging Face, a multimodal mixture-of-experts model with 1 trillion parameters, 49 billion active per token (52 billion counting embeddings and output layers). The licence restricts some model-as-a-service offerings. Mistral Large 4 self-hosting comes down to three decisions: memory, a serving stack and one clause in the licence. We run GPU infrastructure, so this is the order we would work through them.

The model was announced in early October as an API preview and trained further afterwards. Mistral says it tested it for three weeks with customers and cybersecurity partners and added reinforcement learning. Every benchmark figure so far is vendor-reported, and the sources disagree on several of them, so run your own tasks before you buy hardware.

MoE model GPU sizing: total parameters decide memory

The 49B active figure tells you what each token costs in compute. It does not tell you how much memory you need, because the router can pick any of the 128 routed experts (4 active per token, plus 1 shared expert) on the next token. Every expert stays resident, so sizing from 49B would be off by a factor of about twenty.

We found no official VRAM requirement, so this is our arithmetic. FP8 is roughly one byte per parameter, about 1 TB of weights; the NVFP4 checkpoint is roughly half that, about 500 GB. The KV cache sits on top of the weights, and we found no KV-cache figure at the full 1M-token context.

Active parameters set your speed. Total parameters set your hardware bill.

The checkpoint you can run depends on your GPUs. The NVFP4 checkpoint targets Blackwell GPUs and has the smaller memory footprint; if your GPUs are not Blackwell, plan on the FP8 checkpoint or a GGUF quant.

The FP8 reference on 8x H200 runs a 131K context cap and up to 16 concurrent requests, which points to limited room once the weights are loaded. The model card says the same thing in its advice to lower --max-model-len and --max-num-seqs if GPU memory runs short. The context cap and the concurrency limit are the first two numbers to set.

What the reference configurations show

Hugging Face’s launch post gives three measured data points and one reference recipe:

  • FP8 on 8x NVIDIA H200 with vLLM, tensor and expert parallelism, 131K context and up to 16 concurrent requests: about 65 tok/s single-stream and about 480 tok/s aggregate.
  • NVFP4 on a Dell B300 node with 4 GPUs, given as a recipe; we found no measured figure for it.
  • llama.cpp on 4x RTX 6000 Pro, NVFP4 at 32K context: about 10 tok/s generation.
  • One 512 GB Apple Silicon Ultra with 2-bit quants: about 15 tok/s on Q2_K.

Read the first as a production shape and the last two as ways to experiment. Reasoning cannot be switched off and the default effort is high, so each request spends output tokens thinking before it answers. Budget for that when you turn tok/s into response times.

Our AI services run on GPUs in Amsterdam, usually serving open-weight models such as Qwen, GLM and gpt-oss. If you want the weights on hardware that is yours alone, our Bare Metal GPU tier is single-tenant, uses the Nvidia RTX PRO 6000 Blackwell Server Edition, is available in all four sites, and comes with free hardware replacement. The results above are not measurements on our servers, so confirm the node count with us before you order: it has to hold the NVFP4 weights plus your KV cache.

vLLM deployment: the flags to set first

The model card recommends vLLM and also lists SGLang, Transformers and llama.cpp. It points to the image vllm/openai:mistral-large-4, and its FP8 example uses --tensor-parallel-size 8. The NVFP4 example in the Hugging Face post adds expert parallelism and caps context at 131,072 tokens.

A sketch assembled from those flags:

vllm serve mistralai/Mistral-Large-4-1T-A52B \
  --tensor-parallel-size 8 \
  --max-model-len 131072 \
  --max-num-seqs 16

The block assumes the FP8 checkpoint on 8x H200 and uses the base repository id from the model card, so confirm the exact id of the variant you pull. The value 16 for --max-num-seqs is our placeholder, taken from the concurrency of the Inference Endpoints reference; 131072 comes from the NVFP4 example and is used here as a context cap. We omit the expert-parallelism flag; copy its exact form from the Hugging Face example.

The FP8 checkpoint ships an embedded DSpark speculative-decoding draft head (3 tokens). Mistral measured up to 1.9x throughput from it and we found no stated conditions, so measure it on your own prompts. For self-hosted LLM inference the useful property is simpler: you download the weights once, and after that inference runs in the site you chose. For a vLLM deployment in the EU on Bare Metal, that site can be any of our four datacenters, all in the EU.

Open-weight model licence: where the model-as-a-service clause applies

The Mistral Ecosystem Open-Weight License is a modified MIT licence. Section 2 requires a separate commercial licence if you or an affiliate operate a Model as a Service business. The licence defines that as giving a third party access to the model or derivatives for inference or post-training, in a way that lets them exercise meaningful control over inputs, parameters or training data.

The text excludes two things: end-user products with embedded capabilities, and merely relaying requests to models hosted by others. Section 3 exempts internal use, meaning any use that does not make the model available to third parties, and use through Mistral’s official products. We found no revenue cap, no field-of-use clause and no EU-specific term in the file.

The edges are where you need counsel. We could find no legal analysis of the clause, and third-party summaries paraphrase it inconsistently. These are the questions we would put to a lawyer:

  • What counts as “meaningful control” when you expose an endpoint with configurable parameters?
  • Does serving subsidiaries on an internal multi-tenant platform make them third parties, and how far does “affiliate” reach?
  • Where does “relaying” end and hosting begin?

This is a reading of the licence text, not legal advice.

Checklist before you order hardware

  1. Size on total parameters. NVFP4 targets Blackwell; on other GPUs plan on FP8 or a GGUF quant. Fix a context cap and a concurrency limit from your real workload.
  2. Evaluate the model on your own tasks; t3n notes no independent evaluation exists yet, and vendor numbers conflict.
  3. Decide the tok/s per stream you need. The 8x H200 reference gives about 65 single-stream and about 480 aggregate; measure your own at the concurrency you will run.
  4. Map who gets access to the model, then read Sections 2 and 3 with counsel against that map.
  5. Compare cost against the API using your own contract price. API prices differ between sources, so do not plan on a quoted figure.

Sources