基于vLLM-Ascend的GLM-5.1模型Atlas 800I A3单机混部部署实践

​作者​昇腾实战派​知识地图​https://blog.csdn.net/Lumos_Lovegood/article/details/161601003背景概述本文档将介绍基于vLLM-Ascend的GLM-5.1模型在Atlas 800I A3上的单机混部部署实践包括支持的特性、特性配置、环境信息以及性能测试典型case。基本信息软件版本设备信息组网形态总卡数数据格式0.18.0NPU: Atlas 800I A3-560T, HBM 128GCPU: Kunpeng 920 (80核-2900MHz)内存: 32根64G5200MHzOS: OpenEuler 22.03 LTS-SP4Atlas 800I A3单机8W8A8C16服务化配置低时延/高吞吐exportHCCL_OP_EXPANSION_MODEAIVexportOMP_PROC_BINDfalseexportOMP_NUM_THREADS1exportHCCL_BUFFSIZE200exportPYTORCH_NPU_ALLOC_CONFexpandable_segments:TrueexportVLLM_ASCEND_BALANCE_SCHEDULING1exportVLLM_ASCEND_ENABLE_MLAPO1vllm serve /mnt/weight/GLM-new-w8a8\--host0.0.0.0\--port8077\--data-parallel-size1\--tensor-parallel-size16\--enable-expert-parallel\--seed1024\--served-model-name glm-5\--max-num-seqs48\--max-model-len20480\--max-num-batched-tokens4096\--trust-remote-code\--gpu-memory-utilization0.95\--quantizationascend\--enable-chunked-prefill\--no-enable-prefix-caching\--async-scheduling\--additional-config{enable_npugraph_ex: true,fuse_muls_add:true,multistream_overlap_shared_expert:true}\--compilation-config{cudagraph_mode: FULL_DECODE_ONLY}\--speculative-config{num_speculative_tokens: 3, method: deepseek_mtp}典型测试用例平均输入平均输出并行策略上下文长度Prefix Cache命中率总请求数最大并发数请求频率(req/s)20482048MLADP1TP16204800164035001500MLADP1TP162048009624035001500MLADP1TP162048001640163841024MLADP1TP162048002050163841024MLADP1TP1620480041020482048MLADP2TP820480014436032768512MLADP2TP833792082032768512MLADP2TP8337920410测试命令参考aisbench官方测试指南。aisbench测试命令vllm-ascend社区官网特别声明以上配置均未开启Prefix Cache若实际生产环境需要使用该特性参考vLLM-Ascend社区参数指南开启–enable-prefix-caching