We’re planning to use our Lambda Quad for training as part of one of our Google Cloud Dataflow pipelines. In this way, we have google-cloud as dependency, to be able to pull/push image blobs to GCS, together with apache-beam, to be able to actually run our TensorFlow-based models as Dataflow pipelines.
However, we are not able to import protobuf (in a Python 2.7 environment, because apache-beam unfortunately doesn’t support Python 3 yet):
Traceback (most recent call last):
File "floorplan.py", line 10, in <module>
import apache_beam as beam
File "/usr/local/lib/python2.7/dist-packages/apache_beam/__init__.py", line 91, in <module>
from apache_beam import coders
File "/usr/local/lib/python2.7/dist-packages/apache_beam/coders/__init__.py", line 19, in <module>
from apache_beam.coders.coders import *
File "/usr/local/lib/python2.7/dist-packages/apache_beam/coders/coders.py", line 28, in <module>
import google.protobuf.wrappers_pb2
ImportError: No module named protobuf.wrappers_pb2
We tried to pip install protobuf and google packages, but we got:
Requirement already satisfied: protobuf in /usr/lib/python2.7/dist-packages
When we try to import protobuf directly, it says that’s not installed.
I believe that is related with Lambda Stack and protobuf being installed from APT package python-protobuf.
Is there any workaround to satisfy apache-beam requirements for protobuf?
Thanks in advance.