Potential Lambda Stack bugs

Wasn’t sure where to post bugs for the Lambda Stack, so I thought I’d post here.

After installing Lambda Stack using the commands from this blog post on Ubuntu 20.04 LTS, you end up with a virtualenv installation that is broken. Typing the virtualenv command into the terminal gives

ModuleNotFoundError: No module named 'virtualenv.seed.via_app_data'

I figured out that this was coming from the package python3-virtualenv. After removing it with

sudo apt-get purge python3-virtualenv

things started working again.

1 Like

Thank you! I will take a look and make sure it gets to the correct people.

I just tried it on a machine I installed this morning and did not see a problem.
virtualenv only comes back with the normal usage output. So I tried some other commands.

I will reinstall and try again on the workstation install [I was testing on the server install].

mark@lamda-dual:~$ sudo apt-get install python3-venv

mark@lamda-dual:~/python-virtual-environments$ dpkg --list | grep virtualenv
ii python3-virtualenv 20.0.17-1 all Python virtual environment creator
ii virtualenv 20.0.17-1 all Python virtual environment creator

mark@lamda-dual:~/python-virtual-environments$ virtualenv --version
virtualenv 20.0.17 from /usr/lib/python3/dist-packages/virtualenv/init.py

mark@lamda-dual:~/python-virtual-environments$ python3 -m venv env
mark@lamda-dual:~/python-virtual-environments$ source env/bin/activate

(env) mark@lamda-dual:~/python-virtual-environments$ deactivate
mark@lamda-dual:~/python-virtual-environments$

Mark

There’s always a chance that something other than lambda stack was causing the issue. I also installed pipenv so maybe that caused some kind of conflict?

Actually, I ended up having another issue with my lambda stack installation. Normally when I run a python script that uses tensorflow, I see an output like this:

2021-03-16 02:56:41.367083: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-03-16 02:56:41.367103: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2021-03-16 02:56:41.367111: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2021-03-16 02:56:41.367120: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-03-16 02:56:41.367128: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-03-16 02:56:41.367134: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-03-16 02:56:41.367141: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2021-03-16 02:56:41.367149: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8

However, when I ran it using the lambda stack, it was unable to find one of these files, and so GPU usage was disabled. I don’t remember which one was missing. Same thing happened when I tried to use tensorflow with a couple of the official NVIDIA docker images. Finally, I tried the google docker images and I haven’t had any issues since.

I saw a lot of people online saying that certain tensorflow versions are incompatible with certain CUDA versions and that this was often the cause of these kinds of errors. I was surprised to see lambda stack include tensorflow 2.3 and CUDA 11.1, which from what I’ve seen online, seem to be incompatible.

I’m sure your team has thought this all through, but I just wanted to share my experience in case it’s helpful.

1 Like

Yes, thank you for the update. I will test that also.