How we saved 70k cores across 30 mission-critical services

Click for: original source

As part of Uber engineering’s wide efforts to reach profitability, recently our team was focused on reducing cost of compute capacity by improving efficiency. Some of the most impactful work was around GOGC optimization. In this blog we want to share our experience with a highly effective, low-risk, large-scale, semi-automated Go GC tuning mechanism. By Cristian Velazquez.

Uber’s tech stack is composed of thousands of microservices, backed by a cloud-native, scheduler-based infrastructure. Most of these services are written in Go. Our team, Maps Production Engineering, has previously played an instrumental role in significantly improving the efficiency of multiple Java services by tuning GC. At the beginning of 2021, we explored the possibilities of having a similar impact on Go-based services.

The article then explains:

  • GOGC tuner
  • Dynamic and diverse: One size does not fit all
  • A case for automation
  • Observability
  • Implementation
  • Impact

Garbage collection is one of the most elusive and underestimated performance influencers of an application. Go’s robust GC mechanism and simplified tuning, our diverse, large-scale Go services footprint, and a robust internal platform (Go, compute, observability) collectively allowed us to make such a large-scale impact. We expect to continue improving how we tune GC as the problem space itself is evolving, due to changes in the tech and our competency. Super interesting read!

[Read More]

Tags how-to microservices performance devops app-development agile