Tag: Data and analytics
-
Look at how Twitter handles its time series data ingestion challenges
Posted on June 11, 2019, Level intermediate Resource Length short
Ram Dagar is author of this overview on the time series topic. The components of time-series are as complex and sophisticated as the data itself. With increasing time, the data obtained increases and it doesn't always mean that more data means more information but, larger sample avoids the error that due to random sampling.
Tags devops-and-ci-cd data-and-analytics ai-and-machine-learning software-engineering
-
How we use Apache Kafka and the Confluent Platform
Posted on June 4, 2019, Level intermediate Resource Length short
Jendrik Poloczek from confluent.io published this article about their experience building the core infrastructure to integrate, clean, and analyze blockchain data.
Tags business-and-emerging-tech data-and-analytics architecture-and-apis backend-development
-
Indexes bad practices
Posted on June 1, 2019, Level beginner Resource Length medium
In this article author will discuss secondary indexes common mistakes and bad practices. While the article is focused on MySQL and the InnoDB storage engine, most of this information applies to any DBMS. Published by Federico Razzoli.
Tags data-and-analytics backend-development miscellaneous cloud-and-infrastructure
-
Exposing MyRocks internals via system variables: data writing
Posted on May 31, 2019, Level intermediate Resource Length medium
Peter Sylvester wrote this blog post about new storage engine for MySQL that's rising in popularity. MyRocks, the log-structured merge-driven RocksDB engine in MySQL.
Tags data-and-analytics backend-development miscellaneous
-
Great engineer needs the liberal arts
Posted on May 23, 2019, Level beginner Resource Length medium
Thomas Betts wrote for infoq.com about how liberal arts eductaion can provide new insights and perspectives that shine a light on technical tasks for any software developer. E.g. empathy helps you know your audience and create great software that delights your customers.
Tags miscellaneous data-and-analytics leadership-and-career software-engineering
-
How to create histogram in Rlang
Posted on May 22, 2019, Level intermediate Resource Length short
In this article the author will show you how to create histogram in R using ggplot2 package. Written by Data Sharkie. When we get a new dataset for our analysis or research, often we would like to learn about the frequency of occurrence distribution of the variable of interest.
Tags data-and-analytics miscellaneous leadership-and-career
-
Introduction to message brokers -- Apache Kafka vs. RabbitMQ
Posted on May 8, 2019, Level intermediate Resource Length long
If you're looking for a message broker for your next project, read on to get an overview of to of the most popular open source solutions out there. Introduction by Vitaliy Samofal.
Tags devops-and-ci-cd architecture-and-apis data-and-analytics
-
How on-demand pricing slashed our DynamoDB bill by 90 percent
Posted on April 27, 2019, Level intermediate Resource Length short
Ashwin Bhat published this short article about their experience with a new pricing structure for DynamoDB called On-Demand Mode. With it, your capacity costs are strictly a function of how many capacity units you consume, rather than how many you would have provisioned in Provisioned Capacity Mode.
Tags cloud-and-infrastructure data-and-analytics
-
Building self-served ETL pipeline for third-party data ingestion
Posted on April 18, 2019, Level intermediate Resource Length medium
An article by Nikolaos Tsipas from Skyscanner with help of colleagues Omar Kooheji and Michael Okarimia about how to solve the puzzle when there is a need to import datasets from external sources, and make them available for querying. Examples of imported data include: analytics metrics, advertising data, and currency exchange rates, all of which are used by Skyscanner engineers and data scientists.
Tags data-and-analytics architecture-and-apis
-
Kafka-Streams -- road to autoscaling via Kubernetes
Posted on April 16, 2019, Level intermediate Resource Length long
This article is based on the talk Scale in / Scale out with Kafka-Streams and Kubernetes from Xebicon'18. Loïc Divad is the author of this article and he aims to present a few advantages that come with specific practices related to Kafka-Streams like containerization and orchestration, and especially autoscaling.
Tags devops-and-ci-cd architecture-and-apis data-and-analytics
-
Building stateful apps with serverless functions and Postgres
Posted on April 2, 2019, Level intermediate Resource Length medium
Tirumarai Selvan, Product Lead at Hasura, published this guide which explores the different ways to deal with application state when writing business logic in serverless functions. Author uses Postgres as the database when talking about these different approaches, and often use Hasura as one of the glue solutions between your serverless functions and your Postgres database.
Tags data-and-analytics architecture-and-apis cloud-and-infrastructure
-
Understanding microservices with tracing
Posted on March 27, 2019, Level intermediate Resource Length long
An interesting article on codacy blog from rtfpessoa. A typical system now is microservices-oriented, distributed and event-driven, and following its flow of information is a complicated endeavor. Moreover, instrumenting such a system to trace the flow of information has, at least historically, been an intricate task.
Tags devops-and-ci-cd architecture-and-apis data-and-analytics