Failed to initialize NVML: Driver/library version mismatch

Technical Help
1

Hi,

I’m having this problem in our machine:
Failed to initialize NVML: Driver/library version mismatch

Here’s our folder structure:

anie@arthur1:~$ ls /usr/local | grep cuda
cuda
cuda-8.0

Here’s the lib64 for cuda folder:

anie@arthur1:~$ ls /usr/local/cuda/lib64
libcublas_device.a   libcufft.so           libcusolver.so.8.0.61  libnppicom.so.8.0.61  libnppist.so.8.0      libnvgraph_static.a
libcublas.so         libcufft.so.8.0       libcusolver_static.a   libnppidei.so         libnppist.so.8.0.61   libnvrtc-builtins.so
libcublas.so.8.0     libcufft.so.8.0.61    libcusparse.so         libnppidei.so.8.0     libnppisu.so          libnvrtc-builtins.so.8.0
libcublas.so.8.0.61  libcufft_static.a     libcusparse.so.8.0     libnppidei.so.8.0.61  libnppisu.so.8.0      libnvrtc-builtins.so.8.0.61
libcublas_static.a   libcufftw.so          libcusparse.so.8.0.61  libnppif.so           libnppisu.so.8.0.61   libnvrtc.so
libcudadevrt.a       libcufftw.so.8.0      libcusparse_static.a   libnppif.so.8.0       libnppitc.so          libnvrtc.so.8.0
libcudart.so         libcufftw.so.8.0.61   libnppc.so             libnppif.so.8.0.61    libnppitc.so.8.0      libnvrtc.so.8.0.61
libcudart.so.8.0     libcufftw_static.a    libnppc.so.8.0         libnppig.so           libnppitc.so.8.0.61   libnvToolsExt.so
libcudart.so.8.0.61  libcuinj64.so         libnppc.so.8.0.61      libnppig.so.8.0       libnpps.so            libnvToolsExt.so.1
libcudart_static.a   libcuinj64.so.8.0     libnppc_static.a       libnppig.so.8.0.61    libnpps.so.8.0        libnvToolsExt.so.1.0.0
libcudnn.so          libcuinj64.so.8.0.61  libnppial.so           libnppim.so           libnpps.so.8.0.61     libOpenCL.so
libcudnn.so.5        libculibos.a          libnppial.so.8.0       libnppim.so.8.0       libnpps_static.a      libOpenCL.so.1
libcudnn.so.5.1.10   libcurand.so          libnppial.so.8.0.61    libnppim.so.8.0.61    libnvblas.so          libOpenCL.so.1.0
libcudnn.so.6        libcurand.so.8.0      libnppicc.so           libnppi.so            libnvblas.so.8.0      libOpenCL.so.1.0.0
libcudnn.so.6.0.21   libcurand.so.8.0.61   libnppicc.so.8.0       libnppi.so.8.0        libnvblas.so.8.0.61   stubs
libcudnn.so.7        libcurand_static.a    libnppicc.so.8.0.61    libnppi.so.8.0.61     libnvgraph.so
libcudnn.so.7.0.5    libcusolver.so        libnppicom.so          libnppi_static.a      libnvgraph.so.8.0
libcudnn_static.a    libcusolver.so.8.0    libnppicom.so.8.0      libnppist.so          libnvgraph.so.8.0.61

I’m wondering if you guys have encountered this problem before, and what’s the best way to fix it…(there are many solutions all over the internet…)

1 Like
2

OK, first step, can you perform a hard reboot on your machine?

sudo reboot

If that doesn’t work, see here: CUDA NVIDIA NVML Driver/Library version mismatch - update-initramfs -u not working - old kernel version - Technical Help - DeepTalk - Deep Learning Community

3

Hi Stephen, the simple solution worked! Thank you!

1 Like