LLM Study Diary #3: PyTorch

Continuation of the course...This lesson talks a lot related to pytorch. It talks about the tensors as the core building blocks for parameters, gradients, and optimizer states. And then he discusses floating-point representations, including FP32 (full precision), BF16 (brain float, often preferred f