Today author is going to talk about why they made the unconventional decision to build thier logging system on top of Postgres, what worked well, what didn’t work well, and how they did it. By Pete Hunt.
The article captures:
- Framing the problem
- Don’t choose the right tool for the job
- Postgres as a message queue
- How we measured
- Scaling the database: archiving and rate limiting
- Dealing with failure
- Things that didn’t go well
- Future work
One of the big advantages of using a replicated, distributed message queue system like Kafka is its strong availability guarantees and ability to recover from failure. What we have found so far is that, because there are fewer moving parts than a large Kafka deployment, we likely have similar uptime with our single, rock-solid Postgres DB. Additionally, modern environments like AWS RDS allow for hot standbys and quick failovers to replicas, which means failures will often result in just a few seconds of downtime. Good read!
[Read More]