Skip to main content

Module 6: Inference & Deployment

Optimizing LLMs for production deployment.

Learning Objectives​

  • ✅ Understand inference optimization techniques (KV cache, speculative decoding)
  • ✅ Learn quantization methods (INT8, INT4, GPTQ, AWQ)
  • ✅ Deploy models at scale with vLLM, TGI, and Triton

Chapters​

  1. Inference Optimization: KV cache, batching, speculative decoding
  2. Quantization: INT8, INT4, GPTQ, AWQ, bitsandbytes
  3. Serving at Scale: vLLM, TGI, load balancing, autoscaling