发现 deepseek-r1 的 617B 我的机器刚好满足条件,本着闲着也是闲着,测试一下。

系统硬件介绍
- Processor : 2*Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
- Num of Core : 128 Core
- Memory : 1024 GB
- Storage : 1.5T NVMe
- GPU : 8*A800
- NVIDIA-SMI 550.127.05
- Driver Version: 550.127.05
- CUDA Version: 12.4

下载 Ollama
访问下载: https://ollama.com/

安装Ollama
直接借用官方脚本
curl -fsSL https://ollama.com/install.sh | sh

配置模型下载路径
mkdir -p /root/ollama/ollama_models
并且添加到 ollama 中
如果开始没配置OLLAMA_MODELS ,默认路径是/usr/share/ollama/.ollama/models
vim .bashrc
export OLLAMA_MODELS=/root/ollama/ollama_models
启动ollama服务

运行 Ollama
ollama server

修改ollama 配置
默认情况下,Ollama只会关注localhost的11434端口,因此只能从localhost访问。
vim /etc/systemd/system/ollama.service
在 [Service] 下添加 Environment="OLLAMA_HOST=0.0.0.0"
cat /etc/systemd/system/ollama.service
[Unit]
Description=Ollama Service
After=network-online.target
[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin"
Environment="OLLAMA_HOST=0.0.0.0"
[Install]
WantedBy=default.target
重启 ollama
systemctl daemon-reload
systemctl restart ollama
关闭服务
systemctl stop ollama
启动服务
systemctl start ollama
运行模型
ollama run deepseek-r1:671b


配置 Docker + Nvidia-docker2
安装 Docker
export DOWNLOAD_URL="https://mirrors.tuna.tsinghua.edu.cn/docker-ce"
curl -fsSL https://raw.githubusercontent.com/docker/docker-install/master/install.sh | sh
安装 GPU-Docker 组件
安装 gpu-docekr
apt-get install -y nvidia-docker2
nvidia-ctk runtime configure --runtime=docker
这个会修改 daemon.json 文件,增加容器运行时

配置 Docker 参数
root@catcat:~# cat /etc/docker/daemon.json
{
"data-root": "/root/docker_data",
"experimental": true,
"log-driver": "json-file",
"log-opts": {
"max-file": "3",
"max-size": "20m"
},
"registry-mirrors": [
"https://docker.1ms.run"
],
"runtimes": {
"nvidia": {
"args": [],
"path": "nvidia-container-runtime"
}
}
}
测试
docker run --rm -it --gpus all ubuntu:22.04 /bin/bash
root@catcat:~# docker run --rm -it --gpus all ubuntu:22.04 /bin/bash
Unable to find image 'ubuntu:22.04' locally
22.04: Pulling from library/ubuntu
6414378b6477: Pull complete
Digest: sha256:0e5e4a57c2499249aafc3b40fcd541e9a456aab7296681a3994d631587203f97
Status: Downloaded newer image for ubuntu:22.04
root@e36b1bb454b6:/# nvidia-smi
Wed Jan 22 02:03:29 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.127.05 Driver Version: 550.127.05 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA A800-SXM4-80GB Off | 00000000:23:00.0 Off | 0 |
| N/A 29C P0 56W / 400W | 4MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA A800-SXM4-80GB Off | 00000000:24:00.0 Off | 0 |
| N/A 29C P0 56W / 400W | 4MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA A800-SXM4-80GB Off | 00000000:43:00.0 Off | 0 |
| N/A 28C P0 57W / 400W | 4MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 3 NVIDIA A800-SXM4-80GB Off | 00000000:44:00.0 Off | 0 |
| N/A 28C P0 58W / 400W | 4MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 4 NVIDIA A800-SXM4-80GB Off | 00000000:83:00.0 Off | 0 |
| N/A 28C P0 57W / 400W | 4MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 5 NVIDIA A800-SXM4-80GB Off | 00000000:84:00.0 Off | 0 |
| N/A 29C P0 60W / 400W | 4MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 6 NVIDIA A800-SXM4-80GB Off | 00000000:C3:00.0 Off | 0 |
| N/A 29C P0 59W / 400W | 4MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 7 NVIDIA A800-SXM4-80GB Off | 00000000:C4:00.0 Off | 0 |
| N/A 29C P0 60W / 400W | 4MiB / 81920MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
部署 Open WebUI
version: '3.8'
services:
open-webui:
image: ghcr.sakiko.de/open-webui/open-webui:main
container_name: open-webui
restart: always
ports:
- "3000:8080"
volumes:
- open-webui:/app/backend/data
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
open-webui:

参与讨论