Ubuntu 18.04.1 LTS, lambdastack but no nvidia GPU

[tonyr]

Hello World!

I have collection of servers, most of which have GPUs but some have simple graphics cards which aren’t from nvidia.

Until I upgraded this week all was fine, my ansible installed the full lambdastack headless.

After upgrade+reboot, those without GPUs spend a lot of time in modprobe/systemd-udevd with continual:

root@code1:/home/tonyr# tail /var/log/syslog
Dec 21 21:40:10 code1 kernel: [18928.345252] PKCS#7 signature not signed with a trusted key
Dec 21 21:40:10 code1 kernel: [18928.356814] nvidia-nvlink: Nvlink Core is being initialized, major device number 237
Dec 21 21:40:10 code1 kernel: [18928.357062] NVRM: No NVIDIA graphics adapter found!
Dec 21 21:40:10 code1 kernel: [18928.357141] nvidia-nvlink: Unregistered the Nvlink Core, major device number 237
Dec 21 21:40:10 code1 systemd-udevd[2580]: Process ‘/sbin/modprobe nvidia-modeset’ failed with exit code 1.
Dec 21 21:40:10 code1 kernel: [18928.491969] PKCS#7 signature not signed with a trusted key
Dec 21 21:40:10 code1 kernel: [18928.503436] nvidia-nvlink: Nvlink Core is being initialized, major device number 237
Dec 21 21:40:10 code1 kernel: [18928.503674] NVRM: No NVIDIA graphics adapter found!
Dec 21 21:40:10 code1 kernel: [18928.503753] nvidia-nvlink: Unregistered the Nvlink Core, major device number 237
Dec 21 21:40:10 code1 systemd-udevd[2580]: Process ‘/sbin/modprobe nvidia-drm’ failed with exit code 1.

Anyone know the easiest work around to ‘/sbin/modprobe nvidia-modeset’ so that it’s not called or doesn’t give an error?

Tony

[tonyr]

systemd-udevd[2580]: Process ‘/sbin/modprobe

See https://askubuntu.com/questions/1035528/ubuntu-18-04-systemd-udevd-uses-high-cpu-conflict-with-nvidia-graphics specifically the zero voted sudo systemctl stop / start worked for me. Editing /lib/udev/rules.d/71-nvidia.rules is probably better.

[sabalaba]

@tonyr so this solved it for you?

sudo systemctl stop systemd-udevd systemd-udevd-kernel.socket systemd-udevd-control.socket

Just wanted to confirm what fixed your issue. And this is for a machine w/o an NVIDIA gpu, correct?

I think we have a CPU-only Lambda Stack you can install with:

LAMBDA_REPO=$(mktemp) && \
wget -O${LAMBDA_REPO} https://lambdalabs.com/static/misc/lambda-stack-repo.deb && \
sudo dpkg -i ${LAMBDA_REPO} && rm -f ${LAMBDA_REPO} && \
sudo apt-get update && sudo apt-get install -y lambda-stack

[tonyr]

Yes, thanks, that did it for me (on a machine without a NVIDIA GPU). However, I haven’t rebooted, it may be that the problem occurs on every reboot. If at all possible I like all my nodes to have the same software, that way I can compile for GPUs even if I can’t run on that node (all nodes are submit hosts to a GPU queue).