Python 2.7 pytorch signal 137 killed "Killed" memory leak

1

Let’s say you’re running a pytorch script that seems to be crashing like this:

$ python test-pytorch.py
Killed
$ echo $?
137

Exit code 137 means that the process was killed with a signal value of 9 or SIGKILL. (137 - 128 = 9). That means your process was likely hogging system resources and was killed. The most likely situation is that you’re allocating too much memory / there’s a memory leak. I observed with htop that the memory allocation kept climbing up to the total system memory before the process was killed.

1 Like
2

Oddly enough switching to python 3 solved this problem. Maybe there’s a memory leak somewhere in the py2.7 version of pytorch.

python3 test-pytorch.py

runs perfectly fine.

1 Like