Aggregator Leaf Tailer (ALT) is the data architecture favored by web-scale companies, like Facebook, LinkedIn, and Google, for its efficiency and scalability. In this blog post, I will describe the Aggregator Leaf Tailer architecture and its advantages for low-latency data processing and analytics. By Dhruba Borthakur.
The biggest advantage of the Lambda architecture is that data processing occurs when new data arrives in the system, but ironically this is its biggest weakness as well. Most processing in the Lambda architecture happens in the pipeline and not at query time. As most of the complex business logic is tied to the pipeline software, the application developer is unable to make quick changes to the application and has limited flexibility in the ways he or she can use the data. Having to maintain a pipeline just slows you down.
The article then goes about:
- Traditional Data Processing: Batch and Streaming
- Common Lambda Architectures: Kafka, Spark, and MongoDB/Elasticsearch
- Shortcomings of Lambda Architectures
- ALT: Real-Time Analytics Without Pipelines
- Advantages of ALT
- Why ALT Makes Sense Today
The ALT architecture enables the app developer or data scientist to run low-latency queries on raw data sets without any prior transformation. A large portion of the data transformation process can occur as part of the query itself. The ALT architecture clearly has the performance, scale, and efficiency to handle real-time use cases at some of the largest online companies. Excellent read!
[Read More]