DeepInfer install: DeepSeek-V4.1-Flash offline inference
Step-by-step path for the open-source DeepSeek DeepInfer engine — production offline inference for DeepSeek-V4.1-Flash. Commands and versions below track the official README only. Prefill and Decode are two phases of the same engine, not separate installs.
1. Path choice: install DeepInfer offline vs official API
DeepInfer is the inference engine. deepseek-recipe is the protocol/API layer (prompts, tokenization) — it does not run model inference. Agent workflows belong on dshbase.com (DeepSeek Harness), not this install path.
| Path | Choose when | You need | Skip DeepInfer install? |
|---|---|---|---|
| DeepInfer offline | Self-host DeepSeek-V4.1-Flash with DeepSeek’s open engine | SM100 or Ascend 950 + build env below | No — follow this page |
| Official DeepSeek API | Hosted Chat Completions; no GPU ops | API key + client (optionally recipe) | Yes — skip self-host |
| deepseek-recipe only | Protocol conversion / prompts without local weights | Python/Rust; a backend or API elsewhere | Yes — see /recipe/ |
| vLLM / SGLang / other | Different accelerators or multi-model fleets | Their stack | Usually — see vs vLLM |
2. Hardware prerequisites
| Accelerator | Toolchain (README) | DEEPINFER_ACCELERATOR | Status |
|---|---|---|---|
| NVIDIA SM100 | CUDA 13 | nvidia |
Supported |
| Huawei Ascend 950 | CANN 9.2 | ascend |
Supported |
| Other NVIDIA / Ascend SKUs | — | — | Not listed in README |
Upstream Prefill/Decode charts use 4× SM100 (prefill) and 64× SM100 (decode) within one scale-up domain — context for capacity planning, not a second product. Reproduce via examples/sweep/run_sweep.sh.
3. Software dependencies checklist
Versions from the official Getting started section:
- Linux
- GCC ≥ 13.0
- Python 3.12 — empty / dedicated virtualenv before build
- CUDA 13 — NVIDIA SM100 only
- CANN 9.2 — Ascend 950 only
- Git clone of deepseek-ai/DeepInfer
- Disk + HF access for checkpoint
deepseek-ai/DeepSeek-V4.1-Flash-1017(or local dir)
4. Platform steps
4A. NVIDIA SM100 (CUDA 13)
- Confirm SM100 inventory + CUDA 13 toolchain
- Create and activate an empty Python 3.12 environment
- Clone the official repository
- Export accelerator and build
- Run offline smoke example
git clone https://github.com/deepseek-ai/DeepInfer.git
cd DeepInfer
# empty Python 3.12 env first (venv/conda — your choice)
export DEEPINFER_ACCELERATOR=nvidia
# Keep this exported wherever you run DeepInfer
./build.sh# Optional: local weights instead of HF cache download
export DEEPINFER_HF_MODEL_DIR=/path/to/DeepSeek-V4.1-Flash-1017
python examples/offline_inference.py4B. Huawei Ascend 950 (CANN 9.2)
- Confirm Ascend 950 + CANN 9.2
- Empty Python 3.12 environment
- Clone official repo
- Export
ascendand build - Smoke with the same offline example
git clone https://github.com/deepseek-ai/DeepInfer.git
cd DeepInfer
export DEEPINFER_ACCELERATOR=ascend
./build.sh
# Same smoke path as NVIDIA once the build succeeds
python examples/offline_inference.pyIf a flag or script name differs on Ascend beyond DEEPINFER_ACCELERATOR=ascend, follow the live official README — do not invent flags here.
5. Protocol layer: when to wire deepseek-recipe
Skip recipe for smoke
Offline examples/offline_inference.py does not require deepseek-recipe. Get the engine green first.
Add recipe for API shapes
Need Chat Completions / Responses conversion or V4.1 prompt rendering? Install recipe separately and point it at your backend.
python3 -m pip install deepseek-recipe
# See /recipe/ for conversion examples — recipe does not download DeepInfer weights6. Troubleshooting
| Symptom | Check | Action |
|---|---|---|
| Build fails on CUDA | CUDA is 13 on SM100; DEEPINFER_ACCELERATOR=nvidia |
Align toolkit version; re-read README prerequisites |
| Build fails on Ascend | CANN is 9.2; env is ascend |
Fix CANN install; keep env exported in the run shell |
| Wrong Python / GCC | Python 3.12; GCC ≥ 13 | Recreate empty venv; upgrade compiler |
| Permission / driver errors | User access to GPU/NPU devices | Fix device group membership / driver stack (platform-specific) |
| OOM during smoke | Card count vs model; other processes | Free devices; try README examples; see official issues — no invented memory numbers here |
| Expecting online multi-tenant API | Release scope | KV cache, Prefill/Decode disaggregation service, DP load balancing are not included — use hosted API or bring your own serving glue |
7. Offline parallel examples & sweeps
From examples/README.md — secondary to the build steps above; do not skip the hardware gate.
Parallelism flags
python examples/offline_inference.py --ep 8
python examples/offline_inference.py --ep 2 --dp 8
python examples/offline_inference.py --ep 2 --cp 8Benchmark & sweep highlights
python examples/benchmark.py— defaults include decode_num_requests=64, context_length=32768bash examples/sweep/run_sweep.sh decode ce-128k 131072 3 0.75bash examples/sweep/run_sweep.sh prefill pe-128k 131072./build.shafter empty Python 3.12 env +DEEPINFER_ACCELERATOR
8. Install FAQ
What does “DeepInfer install” give me?
An offline path to run DeepSeek-V4.1-Flash with DeepSeek’s open engine on supported hardware — not a full production serving shell.
When should I use vLLM instead?
When you lack SM100/Ascend 950, need a broader model zoo, or already standardize on vLLM/SGLang. Comparison: DeepInfer vs vLLM.
Where do FlashMLA / DeepGEMM fit?
They are optimized kernels listed in the README feature details — part of the engine once built, not separate installers. See FAQ.
Is Prefill a second install?
No. Prefill and Decode are phases of the same DeepInfer binary/engine.