DeepInfer hardware: SM100 & Ascend 950 today
Support matrix, benchmark topology, community RFCs for unsupported SKUs, and serving gaps — all linked to upstream. Prefill and Decode are two phases of the same engine.
Today support matrix
From the official README only. If your SKU is not listed, do not force a build.
| Accelerator | Vendor | Toolchain | Env | Status |
|---|---|---|---|---|
| SM100 GPUs | NVIDIA | CUDA 13 | DEEPINFER_ACCELERATOR=nvidia | Supported |
| Ascend 950 NPUs | Huawei | CANN 9.2 | DEEPINFER_ACCELERATOR=ascend | Supported |
| Ascend 910C / A3 | Huawei | CANN 9.1 (proposed) | — | Draft RFC only · #46 |
| SM90 / H20-3E | NVIDIA | — | — | Draft RFC only · #47 |
- Linux · GCC ≥ 13.0 · Python 3.12
- Model: DeepSeek-V4.1-Flash · default checkpoint
deepseek-ai/DeepSeek-V4.1-Flash-1017
Benchmark topology (same engine)
README performance charts: measured on 4× SM100 for prefill and 64× SM100 for decode — each within one scale-up domain. Reproduce via examples/sweep/run_sweep.sh. Prefill ≠ a separate product.
| Phase | Topology (README) | What it measures |
|---|---|---|
| Prefill | 4× SM100 · one scale-up domain | Non-cached input throughput vs latency |
| Decode | 64× SM100 · one scale-up domain | Output throughput vs per-user speed |
Community RFCs — unsupported today
Draft RFCs only. Do not claim 910C, SM90, or H20 are supported. Implementation has not landed; track the PRs.
| RFC | Target | Scope / blockers (from PR text) | Link |
|---|---|---|---|
| #46 | Ascend 910C / A3 | Offline-first single-server; V4.1-Flash W8A8 on CANN 9.1; online serving as a later phase. Documented today remains 950 + CANN 9.2. | PR #46 |
| #47 | SM90 · 4× H20-3E (141 GB) | Native FP4 checkpoint capacity ~499 GB of 564 GB; needs dequant-fused FP8 grouped GEMM (no FP4 tensor core on SM90); Hopper attention — FlashMLA dropped Hopper on 2026.09.30. | PR #47 |
Serving gaps (OSS release)
README lists what a complete inference service still needs — not included in this open-source release:
| Missing | Why it matters |
|---|---|
| KV cache system | Online multi-turn / high-QPS reuse |
| VE / Prefill / Decode disaggregation | Most engine functionality exists; full service glue is not released |
| DP load balancing | Replica balancing for both prefill and decode |
Skip checklist
- No SM100 and no Ascend 950 → skip install; watch RFCs or use hosted API
- Hosted Chat Completions is enough → skip self-host
- Need agents → DeepSeek Harness · dshbase.com
- Need protocols / prompts only → deepseek-recipe
- Need full online KV + PD service today → OSS does not ship that stack