Openstack VGPU配置

1
2
3
4
5
6
vim /etc/nova/nova.conf
[devices]
enabled_vgpu_types = nvidia-35
systemctl restart openstack-nova-compute
nova flavor-key flavor-id set resources:VGPU=1

1
2
3
4
UEFI VM 加入镜像属性
hw_machine_type=pc-q35-rhel7.6.0
hw_firmware_type=uefi
hw_video_model=qxl
1
2
3
4
5
6
用其它用户执行命令
su  -
su  - root
su  - root -c  "ls -l /root"
su  - oracle -c  "ulimit -aHS"
su  -s  /bin/sh  -c  "/usr/local/nginx/sbin/nginx"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
mdevctl types
“0000:01:00.0”这是显卡的物理地址，每个人的都可能不一样，我的是“0000:01:00.0”

“nvidia-156”这是VGPU的类型名

“Name: GRID P40-2B”这是名为“nvidia-156”VGPU类型的在虚拟机中显示的显卡名字

“Name: GRID P40-2B”中2B中的2代表显存容量，此类型的VGPU显存为2G

“Name: GRID P40-2B”中最后一个字母是VGPU的显卡功能标识

首先我们需要生成几组UUID，下面为大家准备了8组UUID。

f090c00f-338b-47b4-a419-700101cd70c5
cee34c0a-6144-4f3e-8bdb-51538a98ba80
692c4c12-0def-4aa8-87b4-732fd5ef43ea
bc4b8513-43f0-49d2-81b1-a4aa9fe35826
e46b8864-0271-41ce-86b4-d5df5579d332
6fb3f4a0-a17d-43e5-b4fd-1a52acc3e382
24642876-0a6b-4c50-a199-8bdeef5da6f1
e2736865-12c3-446d-9ec4-42d799a7dbed

选择合适自己的VGPU类型，我要创建两个全功的VGPU,因为我的卡是GTX1050只有2G显存，所以我的VGPU名字为“GRID P40-1Q”，对应的类型名为“nvidia-46”
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
git clone https://gitee.com/deskpool/vgpu_unlock.git
chmod -R +x vgpu_unlock
sed -i 's/#include "nv-time.h"/#include "nv-time.h"\n\n#include "\/root\/vgpu_unlock\/vgpu_unlock_hooks.c"/g' /usr/src/nvidia-460.32.04/nvidia/os-interface.c

echo "ldflags-y += -T /root/vgpu_unlock/kern.ld" >>/usr/src/nvidia-460.32.04/nvidia/nvidia.Kbuild
dkms remove -m nvidia -v 460.32.04 --all
dkms install -m nvidia -v 460.32.04

#下载 vgpu_unlock-rs 项目
git clone https://gitee.com/deskpool/vgpu_unlock-rs
cd vgpu_unlock-rs/
cargo build --release

#nvidia-vgpud 的unlock 服务
mkdir /etc/systemd/system/nvidia-vgpud.service.d
cat >>/etc/systemd/system/nvidia-vgpud.service.d/vgpu_unlock.conf <<EOF
[Service]
Environment=LD_PRELOAD=/root/vgpu_unlock-rs/target/release/libvgpu_unlock_rs.so
EOF

#nvidia-vgpu-mgr 的unlock服务
mkdir /etc/systemd/system/nvidia-vgpu-mgr.service.d
cat >>/etc/systemd/system/nvidia-vgpu-mgr.service.d/vgpu_unlock.conf <<EOF
[Service]
Environment=LD_PRELOAD=/root/vgpu_unlock-rs/target/release/libvgpu_unlock_rs.so
EOF

#描述文件，控制GPU的显示配置，以及cuda
mkdir /etc/vgpu_unlock
cat >>/etc/vgpu_unlock/profile_override.toml <<EOF
[profile.nvidia-55]
num_displays = 1
display_width = 1920
display_height = 1080
max_pixels = 2073600
cuda_enabled = 1
frl_enabled = 0
EOF

1
2
3
4
查看iommu是否开启
dmesg | grep -e DMAR | grep -e IOMMU
/boot/efi/EFI/centos/grub.cfg 文件添加引导参数 intel_iommu=on
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg