There is (on Lambda cloud) what I believe to be unwanted behavior in the allocation of disk sizes across instance types. I was installing some packages on an RTX 6000 instance when I suddenly ran out of disk space. I checked several tool outputs and the results are uniform.
/dev/vda1 29G 19G 10G 66% /
The RTX 6000 instances are stated to have 512GB of SSD space. Interestingly enough, I spun up an A6000 instance, and on these instances there appears to be more disk allocation than advertised (200GB):
/dev/vda1 497G 20G 478G 4% /
Other instance types such as 2xA100 however have as much disk space as they should.
I’m going to try installing conda in the shared storage, but it would be nice if someone could look into this.