CPython Internals: Your Guide to the Python 3 Interpreter

by Anthony Shaw

Are there certain parts of Python that just seem like magic? Like how dictionaries are so much faster than looping over a list to find an item. How does a generator remember the state of the variables each time it yields a value and why do you never have to allocate memory like other languages? It turns out, CPython, the most popular Python runtime is written in human-readable C and Python code.This book will explain the concepts, ideas and technicalities of CPython.We’ll cover all the concepts behind the internals of CPython, how they work with visual explanations as you go.You’ll learn how • Read and navigate the source code• Compile CPython from source code• Navigate and comprehend the inner workings of concepts like lists, dictionaries, and generators• Run the test suite• Modify or upgrade components of the CPython library to contribute them to future versionsTake your time for each chapter and make sure you try out the demos and the interactive components. You’ll soon feel a sense of achievement that you grasp the core concepts of Python that can make you a better Python programmer.

Genres: Programming, Technology, Computer Science, Coding, Software

Kindle Edition

Published February 4, 2022

About the author

Anthony Shaw is a CPython contributor and a PyCon speaker. He has a B.Sc. in Computer Science and over 8 years of experience working with Python. Anthony is also a Fellow of the Python Software Foundation and member of the Open-Source Apache Foundation.

Reviews

[Omar Khaled · Rating 5 out of 5]

Good enough to get you started cruising through the project code and understand the overall architecture and why things the way they are like immutability of tuple/string structures. Really liked the parallelism and concurrency chapter and the explanation of the GIL.

[Booker · Rating 2 out of 5]

There's not much about CPython internals, and the small parts about internals are very shallow.

It's also not clear what the audience is. For example, it literally spends 4 pages explaining how a stack data structure works, which is explained in the first quarter of any Computer Science degree. But for actually interesting things, like the generational garbage collector, it only spends 2 pages.

Overall, it's inconsistent and it feels more like someone's notebook than a book. Very disappointing.

At least I learned a few things, like how code is parsed and converted to a Concrete Syntax Tree, and to an Abstract Syntax tree, and to a Control Flow Graph, and to Bytecode which determines which C functions will run.

[Joe Michalak · Rating 4 out of 5]

This is an excellent book that covers the inner workings of the most common implementation of Python, CPython. It is definitely helpful to have an understanding of C (and especially the compiler/compiling process). This book would be a great reference for learning how to write a new programming language in C; it is also naturally helpful for people who are interested in extending or improving a Python implementation. At a minimum, understanding the internals will help you be more thoughtful about writing performant Python code.

[Tung Son Do · Rating 4 out of 5]

Give the answers of how/why questions related to Python built-in types, compiler, memory management, paralleism/ concurrency.

[Ben Miller · Rating 5 out of 5]

Solid overview of the entire cpython program. Also fun to learn about lower level programming and interacting with the os.

[Carter · Rating 3 out of 5]

This is of some interest to me, since I wanted to understand on some level how interpreters work for modern production level languages. Thank you.

[Albert · Rating 2 out of 5]

Well structured, badly written.

[Kelly Graham · Rating 2 out of 5]

A good intro but light on hands on activities and long for what it is.

[pablo felgueres · Rating 4 out of 5]

As a good intro book, it covers a lot of ground without excessive detail.

It's a solid resource to understand how python is compiled and run, the relationship between C and python looking at actual code, and with good examples to cement knowledge.

There's plenty in this book for a motivated individual to get started and let their curiosity explore topics more deeply.