Look at how Twitter handles its time series data ingestion challenges

Click for: original source

Ram Dagar is author of this overview on the time series topic. The components of time-series are as complex and sophisticated as the data itself. With increasing time, the data obtained increases and it doesn’t always mean that more data means more information but, larger sample avoids the error that due to random sampling.

According to Twitter’s software engineering team, the networking giant stores 1.5 petabytes of logical time series data, and handles 25K query requests per minute.

For social media platforms, the data handling chores get worse with their increasing popularity. The scale at which these firms operate requires customised in-built techniques. Twitter has done the same to solve their database challenges with MetricsDB.

In article you will learn:

  • What does MetricsDB offer
  • Key Takeaways

MetricsDB is multi-zone compliant. For storing mappings from partitions to servers, MetricsDB’s cluster manager uses HDFS. The Hadoop Distributed File System (HDFS) is a distributed file system designed to run on commodity hardware. To learn more, read the rest of the article!

[Read More]

Tags devops database machine-learning data-science software