GPU Bit Tricks: LLM Kernel Arithmetic Optimizations

by David Spuler

GPU Bit LLM Kernel Arithmetic Optimizations: For LLM engineers writing device kernels in CUDA C++, this book covers all sorts of low-level bit tricks, arithmetic optimizations, and approximations to make things faster.

Highlights:
- Integer bitwise magic tricks
- Floating-point bit manipulation
- Approximations and other magic
- Mathematical manipulations
- Blackwell hardware exploitation

Table of Contents:
Part LLM Kernel Arithmetic 1. LLM Arithmetic Overview
2. Blackwell GPU Optimizations
3. Rubin and Feynman GPU Optimizations
4. Grace CPU Optimizations
5. Low-Bit Quantization
6. Block-Scaled Quantization (BSQ)
7. Bitshift Quantization
8. Zero-Multiplication Models

Part Integer Optimizations

9. Bitwise Operator Overview
10. Bitwise Intrinsics
11. Bit Fiddling Magic
12. Bit Data Structures

Part Floating-Point Optimizations 13. Floating-Point Bit Formats
14. Floating-Point Bit Magic
15. Fixed-Point Numbers
16. Block Floating Point (BFP)

Part Arithmetic Optimizations 17. Arithmetic Optimizations
18. Approximate Arithmetic
19. Math Tricks
20. BF16x9 Emulation
21. FP64 Emulation
22. Vector Magnitude Approximations
23. Integer Overflow, Saturation and Clamping

Part Branchless Arithmetic 24. Branchless Coding Techniques
25. Branchless Pitfalls
26. Branchless Conditional Tests

Part Advanced Optimizations 27. SWAR Techniques
28. Polynomial Approximation of Softmax
29. Polynomial Approximation in Tensor Cores
30. Advanced Number Systems

Appendix Long List of Low Latency Techniques
Appendix B. 500 LLM Inference Optimization Techniques
Appendix C. 200 CUDA C++ Optimization Techniques

474 pages, Kindle Edition

Published June 12, 2026