Article catalog

The reasons for error reporting are as follows: error reporting solution: delete the new kernel

Self built multi GPU servers can refer to https://blog.csdn.net/landian0531/article/details/120242839

Error reporting reason

The unexpected power failure caused the Ubuntu server to restart, and the container in docker could not be started through the docker PS - AQ | xargs - I {} docker start {} command

Errors are reported as follows:

gpu@gpu-workstation:~$ docker ps -aq | xargs -I {} docker start {}
Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #1:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: nvml error: driver not loaded: unknown
Error: failed to start containers: 485f0e25b37c

Error reporting solution: delete the new kernel

View the existing system kernel dpkg -- get selections | grep Linux

gpu@gpu-workstation:~$ dpkg --get-selections | grep linux
binutils-x86-64-linux-gnu                       install
console-setup-linux                             install
libnvpair1linux                                 install
libselinux1:amd64                               install
libuutil1linux                                  install
libzfs2linux                                    install
libzpool2linux                                  install
linux-base                                      install
linux-firmware                                  install
linux-generic                                   install
linux-headers-5.4.0-88                          install
linux-headers-5.4.0-88-generic                  hold
linux-headers-5.4.0-89                          install
linux-headers-5.4.0-89-generic                  install
linux-headers-generic                           install
linux-image-5.4.0-88-generic                    hold
linux-image-5.4.0-89-generic                    install
linux-image-generic                             install
linux-libc-dev:amd64                            install
linux-modules-5.4.0-88-generic                  hold
linux-modules-5.4.0-89-generic                  install
linux-modules-extra-5.4.0-88-generic            hold
linux-modules-extra-5.4.0-89-generic            install
util-linux                                      install
zfsutils-linux                                  install

It is found that 5.4.0-89 is automatically installed in the system. Delete the kernel through the sudo apt get purge linux-image-5.4.0-89-generic command
there is a prompt in the middle and select Cancel (Note: deleting the kernel is risky and needs your own consideration.)

After deletion, restart the server

gpu@gpu-workstation:~$ sudo apt-get purge linux-image-5.4.0-89-generic
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  amd64-microcode intel-microcode iucode-tool libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 libusbmuxd6 linux-headers-generic thermald upower usbmuxd
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  linux-image-unsigned-5.4.0-89-generic
Suggested packages:
  fdutils linux-doc | linux-source-5.4.0 linux-tools
The following packages will be REMOVED:
  linux-generic* linux-image-5.4.0-89-generic* linux-image-generic* linux-modules-extra-5.4.0-89-generic*
The following NEW packages will be installed:
  linux-image-unsigned-5.4.0-89-generic
0 upgraded, 1 newly installed, 4 to remove and 39 not upgraded.
Need to get 9,011 kB of archives.
After this operation, 202 MB disk space will be freed.
Do you want to continue?[Y/n] y
Get:1 http://ca.archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-unsigned-5.4.0-89-generic amd64 5.4.0-89.100 [9,011 kB]
Fetched 9,011 kB in 4s (2,522 kB/s)
(Reading database ... 113040 files and directories currently installed.)
Removing linux-generic (5.4.0.89.93) ...
Removing linux-image-generic (5.4.0.89.93) ...
Removing linux-modules-extra-5.4.0-89-generic (5.4.0-89.100) ...
Removing linux-image-5.4.0-89-generic (5.4.0-89.100) ...
W: Removing the running kernel
I: /boot/vmlinuz is now a symlink to vmlinuz-5.4.0-88-generic
I: /boot/initrd.img is now a symlink to initrd.img-5.4.0-88-generic
/etc/kernel/postrm.d/initramfs-tools:
update-initramfs: Deleting /boot/initrd.img-5.4.0-89-generic
/etc/kernel/postrm.d/zz-update-grub:
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.4.0-88-generic
Found initrd image: /boot/initrd.img-5.4.0-88-generic
Adding boot menu entry for UEFI Firmware Settings
done
Selecting previously unselected package linux-image-unsigned-5.4.0-89-generic.
(Reading database ... 107660 files and directories currently installed.)
Preparing to unpack .../linux-image-unsigned-5.4.0-89-generic_5.4.0-89.100_amd64.deb ...
Unpacking linux-image-unsigned-5.4.0-89-generic (5.4.0-89.100) ...
Setting up linux-image-unsigned-5.4.0-89-generic (5.4.0-89.100) ...
I: /boot/vmlinuz is now a symlink to vmlinuz-5.4.0-89-generic
I: /boot/initrd.img is now a symlink to initrd.img-5.4.0-89-generic
(Reading database ... 107663 files and directories currently installed.)
Purging configuration files for linux-modules-extra-5.4.0-89-generic (5.4.0-89.100) ...
Purging configuration files for linux-image-5.4.0-89-generic (5.4.0-89.100) ...
I: /boot/vmlinuz is now a symlink to vmlinuz-5.4.0-88-generic
I: /boot/initrd.img is now a symlink to initrd.img-5.4.0-88-generic
/var/lib/dpkg/info/linux-image-5.4.0-89-generic.postrm ... removing pending trigger
rmdir: failed to remove '/lib/modules/5.4.0-89-generic': Directory not empty
Processing triggers for linux-image-unsigned-5.4.0-89-generic (5.4.0-89.100) ...
gpu@gpu-workstation:~$