背景：DeepSeek V4其核心突破在于全系标配 100 万 token（约 75 万汉字）的超长上下文窗口‌，并针对效率与性能进行了系统性优化。目前发布了两个版本，分别为DeepSeekV4-Pro(旗舰版)‌ 和 ‌‌DeepSeek V4-Flash(轻量版），下面以DeepSeek V4-Flash（轻量版284B）部署为例展开。

参考链接：

https://docs.vllm.ai/projects/ascend/zh-cn/v0.18.0/tutorials/models/DeepSeek-V4-Flash.html

硬件环境：单机Atlas 800 A2 (64G × 8) ，

模型下载链接：

https://www.modelscope.cn/models/Eco-Tech/DeepSeek-V4-Flash-w8a8-mtp

镜像下载链接：

https://quay.io/repository/ascend/vllm-ascend?tab=tags

下载完权重和镜像如下所示:

步骤一：加载镜像：

docker load -i quay.io_ascend_vllm-ascend.deepseekv4.tar

步骤二：起容器，Docker run，容器名字可以改为自己名字XX，注意加私有

export IMAGE=quay.io/ascend/vllm-ascend:deepseekv4

export NAME=XX_vllm-ascend

docker run --rm \

--name $NAME \

--net=host \

--shm-size=512g \

--device /dev/davinci0 \

--device /dev/davinci1 \

--device /dev/davinci2 \

--device /dev/davinci3 \

--device /dev/davinci4 \

--device /dev/davinci5 \

--device /dev/davinci6 \

--device /dev/davinci7 \

--device /dev/davinci_manager \

--device /dev/devmm_svm \

--device /dev/hisi_hdc \

--privileged=true \

-v /usr/local/dcmi:/usr/local/dcmi \

-v /usr/local/Ascend/driver/tools/hccn_tool:/usr/local/Ascend/driver/tools/hccn_tool \

-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \

-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \

-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \

-v /etc/ascend_install.info:/etc/ascend_install.info \

-v /etc/hccn.conf:/etc/hccn.conf \

-v /mnt/sfs_turbo/.cache:/root/.cache \

-v /XXX/:/XXX/ \

-it $IMAGE bash

最后XXX为镜像和权重挂在路径。

查看容器内部npu-smi info正常

步骤三：容器内部拉服务，修改模型权重路径

#!/usr/bin/bash

export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOAD

export OMP_PROC_BIND=false

export OMP_NUM_THREADS=8

export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True

export ACL_OP_INIT_MODE=1

export VLLM_ASCEND_ENABLE_FLASHCOMM1=1

export USE_MULTI_GROUPS_KV_CACHE=1

export TASK_QUEUE_ENABLE=1

export HCCL_OP_EXPANSION_MODE="AIV"

export HCCL_BUFFSIZE=512

export USE_MULTI_BLOCK_POOL=1

sysctl -w vm.swappiness=0

sysctl -w kernel.numa_balancing=0

sysctl kernel.sched_migration_cost_ns=50000

vllm serve /XXX/DeepSeek-V4-Flash-w8a8-mtp \

--safetensors-load-strategy 'prefetch' \

--max-model-len 135168 \

--max-num-batched-tokens 4096 \

--served-model-name ds \

--gpu-memory-utilization 0.92 \

--max-num-seqs 16 \

--data-parallel-size 1 \

--tensor-parallel-size 8 \

--enable-expert-parallel \

--quantization ascend \

 --port 7000 \

--block-size 128 \

--enable-chunked-prefill \

--enable-prefix-caching \

--tokenizer-mode deepseek_v4 \

--tool-call-parser deepseek_v4 \

--enable-auto-tool-choice \

--reasoning-parser deepseek_v4 \

--async-scheduling \

--additional-config '{"enable_cpu_binding":true,"multistream_overlap_shared_expert":false}' \

--compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY","cudagraph_capture_sizes":[2,4,6,8,10,12,14,16,18,20,22,24,32,36,40]}' \

--model-loader-extra-config '{"enable_multithread_load":true,"num_threads":16}' \

--speculative-config '{"num_speculative_tokens": 1,"method": "mtp"}'

容器拉起来过程截图：

容器拉起来后显存情况：

步骤四：验证服务是否正常(可选)

Model 用服务模型名字--served-model-name ds \，端口号是拉起服务的端口号：--port 7000 \

XXXX为部署主机IP地址。

结果如下：