Hi all,
I’m noticing that H100 instances seems to be much slower compared to A100 instance (not GPUs, but everything else).
For example, I launched two instances: an gpu_1x_a100_sxm4 in us-west-2 and gpu_1x_h100_pcie in us-west-3.
Downloading from an S3 bucket in AWS’s us-west-2 (into /dev/null) gives ~70MiB/s on A100 and only 35MiB/s 20-35MiB/s, and it’s pretty unstable.
Same with phoronix 7Z test. On A100 I get 131838 MIPS compression and 111464 MIPS decompression with <2% standard deviation.
On H100 it gives 134009 MIPS compression and 89066 MISP decompression, but the standard deviation is almost 7%. It starts at like 76480. And it all of the benchmarks I ran, the first few runs are always slow.
Subjectively, it feels like H100 instances are just less responsive too (take longer to boot, longer to pull my docker image from ECR, longer to decompress the docker image, etc).
Is this all expected?
The instances I used for the test were 46fcb5af702c433eb1f36e68da849db7 and c576ad0788374edfb1c85c355af4dbe6.