blog.ruahverce Vol · 2026 · Issue 013 — A field log of LLMs, RAG, and ML systems

Tag index

#FP8

01 entries
№001 llm-training-serving-foundations · 07

LLM Quantization: GPTQ·AWQ·SmoothQuant·FP8·KV Cache (7/10)

LLM 양자화를 weight-only, weight·activation, KV cache로 나누고 scale·zero point·granularity를 계산하며 GPTQ, AWQ, SmoothQuant, FP8, KIVI의 memory·kernel·품질을 설계합니다.

#LLM #Quantization #GPTQ #AWQ
Floating point tensor를 quantize해 weight-only, weight·activation, KV cache를 calibration, hardware kernel, 품질 평가로 연결하는 구조