Inference = Prefill + Decode

LLM Inference is split in 2 components:

PrefillDecodes

Prefill (Prompt Encoding)

Input prompt tokens = processed in parallel through entire Transformer stack.Characteristics:Compute-bound (large MatMuls, parallelizable).KV cache is built (for decode).

Decode (Autoregressive Generation)

Model generates one token at a time, conditioned on KV cache + past tokens. Characteristics:

Memory-bound (repeated KVcache lookups, smaller MatMul).Sequential dependency = less parallelism.D...