Hi, I recently reinstalled everything on a Lambda workstation, to upgrade to Ubuntu 20.04 and installed the latest lambda-stack. But I’ve discovered that my Keras models no longer work, failing with
AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'
.
Since this seemed similar to other problems mentioned here, I’ve tried both upgrading protobuf and adding the lines
for device in tf.config.experimental.list_physical_devices('GPU'):
tf.config.experimental.set_memory_growth(device, True)
as suggested elsewhere. Neither have had any effect.
I’ve attached the full error output in case there’s anything useful in there
2020-11-02 10:39:36.652480: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-02 10:39:36.655510: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
Invalid MIT-MAGIC-COOKIE-1 keyUsing TensorFlow backend.
2020-11-02 10:39:38.028832: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2020-11-02 10:39:38.038570: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:1a:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2020-11-02 10:39:38.039058: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 1 with properties:
pciBusID: 0000:68:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s
2020-11-02 10:39:38.039076: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-02 10:39:38.041038: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-02 10:39:38.041774: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-02 10:39:38.041984: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-02 10:39:38.044095: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.11
2020-11-02 10:39:38.044564: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-02 10:39:38.044652: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-02 10:39:38.046506: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0, 1
Traceback (most recent call last):
File "cats.py", line 78, in <module>
x = Dense(intermediate_dim//32, activation='relu',
File "/usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py", line 75, in symbolic_fn_wrapper
return func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/keras/engine/base_layer.py", line 446, in __call__
self.assert_input_compatibility(inputs)
File "/usr/lib/python3/dist-packages/keras/engine/base_layer.py", line 310, in assert_input_compatibility
K.is_keras_tensor(x)
File "/usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py", line 695, in is_keras_tensor
if not is_tensor(x):
File "/usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py", line 703, in is_tensor
return isinstance(x, tf_ops._TensorLike) or tf_ops.is_dense_tensor_like(x)
AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'