Reducing Go execution tracer overhead with frame pointer unwinding

Click for: original source

The Go Execution Tracer (aka runtime/trace) was designed to achieve low enough overhead to be usable on “a server in production serving live traffic”. This is achieved by writing events into per-P buffers, using RDTSC for timestamps, and encoding into a relatively efficient binary format. By Felix Geisendörfer.

Stack unwinding (aka stack walking) is part of the process for taking a stack trace. It involves iterating over all stack frames and collecting the return addresses (program counters) in each frame. It may also involve expanding this list if some of the program counters are part of inlined function calls.

CPU profile showing 94% of execution tracer overhead in gentraceback.

Source: https://blog.felixge.de/reducing-gos-execution-tracer-overhead-with-frame-pointer-unwinding/

So why is stack unwinding so expensive in Go? The short answer is because Go uses a form of asynchronous unwinding tables called gopclntab that require a relatively expensive lookup in order to traverse the stack frames. The gnarly details of this mechanism can be found in the gentraceback function. Follow the link to the full article to learn more!

[Read More]

Tags golang programming microservices cloud performance