Palo Alto Networks eliminated the MQ layer for a project that correlates events in near real time, using an existing database instead of Kafka. By Cynthia Dunlop.
Global security provider Palo Alto Networks processes terabytes of network security events each day. It analyzes, correlates and responds to millions of events per second — many different types of events, using many different schemas, reported by many different sensors and data sources. One of its many challenges is understanding which of those events actually describe the same network “story” from different viewpoints.
The main points worth mentioning in the article:
- Evolving from Events to Stories
- Implementation 1: Relational database
- Implementation 2: NoSQL + message queue
- Implementation 3: NoSQL + Cloud-managed message queue
- Implementation 4: NoSQL (ScyllaDB), no message queue
According to authors the company reduced the operational complexity because tehy did not add another system — they actually removed a system [Kafka] from our deployment. Nice one!
[Read More]