Only Accepted Tokens Count: An Energy Accounting of Speculative Decoding Under Latency Targets

Large language model serving is limited by facility power. Energy per token determines how many tokens a datacenter can serve, while a latency target bounds how quickly each user receives them. During decode, every weight is read from memory for every generated token. Batching amortizes this read ac