Ubuntu 22.04+8*A800 Ollama 运行deepseek-r1

发现 deepseek-r1 的 617B 我的机器刚好满足条件，本着闲着也是闲着，测试一下。

系统硬件介绍#

Processor : 2*Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
_Num of Core : 128 Cor_e
Memory : 1024 GB
Storage : 1.5T NVMe
GPU : 8*A800
NVIDIA-SMI 550.127.05
Driver Version: 550.127.05
CUDA Version: 12.4

下载 Ollama#

访问下载： https://ollama.com/

安装Ollama#

直接借用官方脚本

1
curl -fsSL https://ollama.com/install.sh | sh

配置模型下载路径#

1
mkdir -p /root/ollama/ollama_models

并且添加到 ollama 中

如果开始没配置OLLAMA_MODELS ，默认路径是/usr/share/ollama/.ollama/models

1
vim .bashrc
2
export OLLAMA_MODELS=/root/ollama/ollama_models

启动ollama服务#

运行 Ollama#

ollama server

修改ollama 配置#

默认情况下，Ollama只会关注localhost的11434端口，因此只能从localhost访问。

1
vim /etc/systemd/system/ollama.service
2
在 [Service] 下添加  Environment="OLLAMA_HOST=0.0.0.0"
3
4
cat /etc/systemd/system/ollama.service
5
[Unit]
6
Description=Ollama Service
7
After=network-online.target
8
9
[Service]
10
ExecStart=/usr/local/bin/ollama serve
11
User=ollama
12
Group=ollama
13
Restart=always
14
RestartSec=3
15
Environment="PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin"
16
Environment="OLLAMA_HOST=0.0.0.0"
17
18
[Install]
19
WantedBy=default.target

重启 ollama#

1
systemctl daemon-reload
2
3
systemctl restart ollama
4
5
关闭服务
6
systemctl stop ollama
7
启动服务
8
systemctl start ollama

运行模型#

1
ollama run deepseek-r1:671b

配置 Docker + Nvidia-docker2#

安装 Docker#

1
export DOWNLOAD_URL="https://mirrors.tuna.tsinghua.edu.cn/docker-ce"
2
curl -fsSL https://raw.githubusercontent.com/docker/docker-install/master/install.sh | sh

安装 GPU-Docker 组件#

1
 安装 gpu-docekr
2

3
apt-get install -y nvidia-docker2
4
nvidia-ctk runtime configure --runtime=docker
5

6
这个会修改 daemon.json  文件，增加容器运行时

配置 Docker 参数#

1
root@catcat:~# cat /etc/docker/daemon.json
2
{
3
    "data-root": "/root/docker_data",
4
    "experimental": true,
5
    "log-driver": "json-file",
6
    "log-opts": {
7
        "max-file": "3",
8
        "max-size": "20m"
9
    },
10
    "registry-mirrors": [
11
        "https://docker.1ms.run"
12
    ],
13
    "runtimes": {
14
        "nvidia": {
15
            "args": [],
16
            "path": "nvidia-container-runtime"
17
        }
18
    }
19
}

测试#

1
docker run --rm -it --gpus all ubuntu:22.04 /bin/bash

1
root@catcat:~# docker run --rm -it --gpus all ubuntu:22.04 /bin/bash
2
Unable to find image 'ubuntu:22.04' locally
3
22.04: Pulling from library/ubuntu
4
6414378b6477: Pull complete
5
Digest: sha256:0e5e4a57c2499249aafc3b40fcd541e9a456aab7296681a3994d631587203f97
6
Status: Downloaded newer image for ubuntu:22.04
7
root@e36b1bb454b6:/# nvidia-smi
8
Wed Jan 22 02:03:29 2025
9
+-----------------------------------------------------------------------------------------+
10
| NVIDIA-SMI 550.127.05             Driver Version: 550.127.05     CUDA Version: 12.4     |
11
|-----------------------------------------+------------------------+----------------------+
12
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
13
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
14
|                                         |                        |               MIG M. |
15
|=========================================+========================+======================|
16
|   0  NVIDIA A800-SXM4-80GB          Off |   00000000:23:00.0 Off |                    0 |
17
| N/A   29C    P0             56W /  400W |       4MiB /  81920MiB |      0%      Default |
18
|                                         |                        |             Disabled |
19
+-----------------------------------------+------------------------+----------------------+
20
|   1  NVIDIA A800-SXM4-80GB          Off |   00000000:24:00.0 Off |                    0 |
21
| N/A   29C    P0             56W /  400W |       4MiB /  81920MiB |      0%      Default |
22
|                                         |                        |             Disabled |
23
+-----------------------------------------+------------------------+----------------------+
24
|   2  NVIDIA A800-SXM4-80GB          Off |   00000000:43:00.0 Off |                    0 |
25
| N/A   28C    P0             57W /  400W |       4MiB /  81920MiB |      0%      Default |
26
|                                         |                        |             Disabled |
27
+-----------------------------------------+------------------------+----------------------+
28
|   3  NVIDIA A800-SXM4-80GB          Off |   00000000:44:00.0 Off |                    0 |
29
| N/A   28C    P0             58W /  400W |       4MiB /  81920MiB |      0%      Default |
30
|                                         |                        |             Disabled |
31
+-----------------------------------------+------------------------+----------------------+
32
|   4  NVIDIA A800-SXM4-80GB          Off |   00000000:83:00.0 Off |                    0 |
33
| N/A   28C    P0             57W /  400W |       4MiB /  81920MiB |      0%      Default |
34
|                                         |                        |             Disabled |
35
+-----------------------------------------+------------------------+----------------------+
36
|   5  NVIDIA A800-SXM4-80GB          Off |   00000000:84:00.0 Off |                    0 |
37
| N/A   29C    P0             60W /  400W |       4MiB /  81920MiB |      0%      Default |
38
|                                         |                        |             Disabled |
39
+-----------------------------------------+------------------------+----------------------+
40
|   6  NVIDIA A800-SXM4-80GB          Off |   00000000:C3:00.0 Off |                    0 |
41
| N/A   29C    P0             59W /  400W |       4MiB /  81920MiB |      0%      Default |
42
|                                         |                        |             Disabled |
43
+-----------------------------------------+------------------------+----------------------+
44
|   7  NVIDIA A800-SXM4-80GB          Off |   00000000:C4:00.0 Off |                    0 |
45
| N/A   29C    P0             60W /  400W |       4MiB /  81920MiB |      0%      Default |
46
|                                         |                        |             Disabled |
47
+-----------------------------------------+------------------------+----------------------+
48

49
+-----------------------------------------------------------------------------------------+
50
| Processes:                                                                              |
51
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
52
|        ID   ID                                                               Usage      |
53
|=========================================================================================|
54
|  No running processes found                                                             |
55
+-----------------------------------------------------------------------------------------+

部署 Open WebUI #

1
version: '3.8'
2

3
services:
4
  open-webui:
5
    image: ghcr.sakiko.de/open-webui/open-webui:main
6
    container_name: open-webui
7
    restart: always
8
    ports:
9
      - "3000:8080"
10
    volumes:
11
      - open-webui:/app/backend/data
12
    extra_hosts:
13
      - "host.docker.internal:host-gateway"
14

15
volumes:
16
  open-webui:

Ubuntu 22.04+8*A800 Ollama 运行deepseek-r1

系统硬件介绍#

下载 Ollama#

安装Ollama#

配置模型下载路径#

启动ollama服务#

运行 Ollama#

修改ollama 配置#

重启 ollama#

运行模型#

配置 Docker + Nvidia-docker2#

安装 Docker#

安装 GPU-Docker 组件#

配置 Docker 参数#

测试#

部署 Open WebUI#

部署 Open WebUI #