While I’m not much of a Jupyter user personally, it seems like get_ipython().system() does not respect your current virtual environment in the same way that you may expect. It’s not running a session with the virtual environment “activated” in the same way (meaning $PATH’s are not set to point to your Python3.11’s “pip” and what you’re finding is the system’s “pip”).
Quick Demonstration:
Notice how my Python3.11 session is seeing the system pip while virtual environment’s pip (which will install modules in the desired PythonPath) is different.
Fortunately, you can still install modules from within your Jupyter session without needing to hop back and forth between the terminal. While the commands issued to the system through ! aren’t looking at your virtual environment, you can just use the full path to your Python interpreter:
!/home/ubuntu/mikesVenv/bin/python -m pip install openai
then…
import openai
openai.__file__
# returns '/home/ubuntu/mikesVenv/lib/python3.11/site-packages/openai/__init__.py'
as we can see, the install and import works in our Python3.11 session and the module is installed in our desired Virtual Environment without needing to hop back and forth between an SSH or Terminal session and your notebook.
Quick screenshot demonstrating this with requests and pandas:
