All About Rooflines
Definitions
Numbers
- H100 - roughly 9.9e14 bfloat16 FLOP/s
- TPU - 9.1e14 bfloat FLOP/s
- basically, order of mag ~ 1e15 FLOP / s
How to Scale Your Model: Chapter 1 Notes
Handwritten notes for Chapter 1 of How to Scale Your Model.
Loading preview...