layout title nav nav_order dropdown children page Sections true 8 true title permalink Part 0. Introduction /index title permalink Part 1. Intro to Rooflines /roofline title permalink Part 2. All About TPUs /tpus title permalink Part 3. Sharded Matmuls /sharding title permalink Part 4. Transformers /transformers title permalink Part 5. Training /training title permalink Part 6. Training LLaMA /applied-training title permalink Part 7. Inference /inference title permalink Part 8. Serving LLaMA /applied-inference title permalink Part 9. Profiling /profiling title permalink Part 10. All About JAX /jax-stuff title permalink Part 11. Conclusions /conclusion