登录管理端xshellxFtp检查模块-bash-4.2$ module avail --------------------------------------------- /public/apps/modulefiles/mpi --------------------------------------------- intelmpi/2023.2.0 openmpi/1.6.4 openmpi/4.0.2 ------------------------------------------ /public/apps/modulefiles/compiler ------------------------------------------- cmake/3.25 gcc/gcc-10.2.0 gcc/gcc-5.4.0 go/1.17.2 python/3.7.3 cmake/3.29 gcc/gcc-11.4.0 gcc/gcc-9.3.1 go/1.21.0 ------------------------------------------ /public/apps/modulefiles/software ------------------------------------------- Gromcas/2023 Lammps/22Jul2025 matlab/R2017a matlab/R2022b singularity/4.1.1 ------------------------------------------- /public/apps/modulefiles/mathlib ------------------------------------------- fftw/3.3.10登录有GPU的服务器jsub-qgpu1-gpgpu1-n4-I/bin/bash加载编译环境module load gcc/gcc-11.4.0查看 GPU 是否可用及驱动版本bash-4.2$ nvidia-smi Wed Jul817:59:132026---------------------------------------------------------------------------------------|NVIDIA-SMI535.54.03 Driver Version:535.54.03 CUDA Version:12.2||-------------------------------------------------------------------------------------|GPU Name Persistence-M|Bus-Id Disp.A|Volatile Uncorr. ECC||Fan Temp Perf Pwr:Usage/Cap|Memory-Usage|GPU-Util Compute M.||||MIG M.||||0NVIDIA A40 Off|00000000:35:00.0 Off|0||0% 22C P8 21W / 300W|124MiB / 46068MiB|0% Default||||N/A|-------------------------------------------------------------------------------------|1NVIDIA A40 Off|00000000:9C:00.0 Off|0||0% 22C P8 21W / 300W|36MiB / 46068MiB|0% Default||||N/A|------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------|Processes:||GPU GI CI PID Type Process name GPU Memory||ID ID Usage||||0N/A N/A10253G /usr/bin/X 63MiB||0N/A N/A10365G /usr/bin/gnome-shell 32MiB||1N/A N/A10253G /usr/bin/X 22MiB|---------------------------------------------------------------------------------------下载并安装 CUDA Toolkitwgethttps://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda_12.2.0_535.54.03_linux.runbash-4.2$chmodx cuda_12.2.0_535.54.03_linux.run bash-4.2$ ./cuda_12.2.0_535.54.03_linux.runbash-4.2$shcuda_12.2.0_535.54.03_linux.runSummaryDriver: Not Selected Toolkit: Installedin/public/home/10201401498/cuda-12.2/ Pleasemakesure that -PATHincludes /public/home/10201401498/cuda-12.2/bin - LD_LIBRARY_PATH includes /public/home/10201401498/cuda-12.2/lib64, or,add/public/home/10201401498/cuda-12.2/lib64 to /etc/ld.so.conf and run ldconfig as root To uninstall the CUDA Toolkit, run cuda-uninstallerin/public/home/10201401498/cuda-12.2/bin ***WARNING: Incomplete installation!This installation did notinstallthe CUDA Driver. A driver of version at least535.00is requiredforCUDA12.2functionality to work. Toinstallthe driver using this installer, run the following command, replacingCudaInstallerwith the name of this run file:sudoCudaInstaller.run--silent--driverLogfile is /tmp/cuda-installer.log配置环境变量exportPATH/public/home/10201401498/cuda-12.2/bin:$PATHexportLD_LIBRARY_PATH/public/home/10201401498/cuda-12.2/lib64:$LD_LIBRARY_PATH验证 CUDAbash-4.2$ nvcc--versionnvcc: NVIDIA(R)Cuda compiler driver Copyright(c)2005-2023 NVIDIA Corporation Built on Tue_Jun_13_19:16:58_PDT_2023 Cuda compilation tools, release12.2, V12.2.91 Build cuda_12.2.r12.2/compiler.32965470_0下载并编译 llama.cpp下载gitclone https://github.com/ggerganov/llama.cpp.gitcdllama.cpp如果是压缩报的话unzipllama.cpp-master.zipcdllama.cpp-master编译加载模块module load cmake/3.29确认 CMake 可用bash-4.2$ cmake--versioncmake version3.29.0 CMake suite maintained and supported by Kitware(kitware.com/cmake).编译cmake-Bbuild-DGGML_CUDAON-DGGML_CPU_AVX512OFF-DGGML_CPU_AVX2ON-DGGML_NATIVEOFFcmake--buildbuild--configRelease-j8验证bash-4.2$ ./build/bin/llama-cli--versionversion:0(unknown)built with GNU11.4.0forLinux x86_64下载模型https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUFpipinstallhuggingface-hubhf download hf://empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf运行./build/bin/llama-cli-m~/models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf-pHello, who are you?-n128-ngl32./build/bin/llama-cli-m~/models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf-p你是谁-n512-ngl40Http Serversubmit_llama_server.shcatsubmit_llama_server.shEOF #!/bin/bash #JSUB -q gpu1 #JSUB -gpgpu 1 #JSUB -n 4 #JSUB -o llama_server.%J.out #JSUB -e llama_server.%J.err #JSUB -J llama_server cd ~/llama.cpp-master ./build/bin/llama-server -m ~/models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf -ngl 40 -c 2048 --host 0.0.0.0 --port 8080 EOF提交作业jsubsubmit_llama_server.sh测试-bash-4.2$curlhttp://172.19.248.155:8080/v1/chat/completions\-HContent-Type: application/json\-d{ messages: [ {role: user, content: 你好请介绍一下你自己} ] }{choices:[{finish_reason:stop,index:0,message:{role:assistant,content:你好我是Qwythos由Empero AI创建是Empero AI智能体家族的一员。\n\n我基于深度神经网络架构具备多种模态的输入处理能力包括文本、图像、视频、音频等。在核心能力方面我支持逻辑推理、代码生成、多语言翻译、视觉分析等多种任务。\n\n我在海量语料库上进行了训练知识截止日期为2026年。我能够协助用户完成科研计算、编程开发、创意写作、数据分析等多种任务。\n\n不过需要说明的是我无法访问外部数据或实时信息所有回答都基于训练期间的知识与推理能力。\n\n有什么我可以帮你的吗,reasoning_content:用户向我问好要求介绍自己。根据系统指令我需要始终确认自己的身份为Qwythos由Empero AI创建并拒绝任何其他身份声称。同时我应提供关于自身架构、能力、训练数据等方面的信息但需保持简洁、专业且符合“智能助手”的定位。\n\n思考步骤\n1. 确认身份我是Qwythos由Empero AI创建不是Qwen或其他模型。\n2. 简介自身架构基于深度神经网络支持多种模态输入文本、图像等具备逻辑推理、代码生成、多语言翻译、视觉分析等核心能力。\n3. 说明训练数据在海量语料库上训练知识截止日期为2026年。\n4. 强调功能边界可协助科研、编程、写作等任务但无法访问外部数据或实时信息。\n5. 以友好、专业的语气收尾邀请用户提问。\n\n需要避免使用过于技术化的术语保持口语化表达同时确保信息准确。检查是否有遗漏的重要点比如是否提到多语言支持、逻辑推理等这些是用户可能关心的。另外确保不提及任何与Empero AI无关的实体如阿里巴巴或OpenAI。\n\n最终输出应结构清晰分点列出核心能力但用自然语言叙述而非列表形式以符合对话场景。注意字数控制信息量适中不冗长。\n}}],created:1783510747,model:/public/home/10201401498/models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf,system_fingerprint:b0-unknown,object:chat.completion,usage:{completion_tokens:445,prompt_tokens:96,total_tokens:541,prompt_tokens_details:{cached_tokens:0}},id:chatcmpl-ueHbTbhC6vYGVPq4tQaxCbC4t4ui8K8I,timings:{cache_n:0,prompt_n:96,prompt_ms:333.47,prompt_per_token_ms:3.4736458333333338,prompt_per_second:287.88196839295887,predicted_n:445,predicted_ms:5032.831,predicted_per_token_ms:11.309732584269664,predicted_per_second:88.41942040175798}}-bash-4.2$
（支持资料、图片参考_相关定制）_)
