Welcome to curated list of handpicked free online resources related to IT, cloud, Big Data, programming languages, Devops. Fresh news and community maintained list of links updated daily. Like what you see? [ Join our newsletter ]

Setting Google Analytics on a Jekyll website and implementing cookie consent collect

Categories

Tags analytics miscellaneous big-data web-development

It is very useful to have Google Analytics on your website to have an idea of how many (or how few) people are visiting. Using Google Analytics on a website requires a valid consent from visitors (Because of ePrivacy Directive and not the GDPR). By Coralie Collignon.

The article provides step by step instructions how to deal with cookie consent on a static site:

  • Implementing Google Analytics
    • Creating a Tracking ID on Google Analytics
    • Implementing Google Analytics in your Jekyll website
  • Collecting cookie consent
    • Not all cookie banners are compliant
    • Implementing a compliant cookie banner

You will find links to further resources. e.g. Jekyll Codex has a tutorial explaining how to have the correct cookie banner and provides the code to use in the cookie-consent.html. All the code is provided with explanation and enhancement journey documented. Good read!

[Read More]

Edge computing architecture and use cases

Categories

Tags cloud startups software-architecture cio apis iot

Benefits, challenges, and architectures when enterprises implement edge computing in Telecom and other industries. By Jason Gonzalez, Jason Hunt, Mathews Thomas, Ryan Anderson, Utpal Mangla @IBM.

The advent of 5G has made edge computing even more compelling, enabling significantly improved network capacity, lower latency, higher speeds, and increased efficiency. 5G promises data speeds in excess of 20 Gbps and the ability to connect over a million devices per square kilometer.

Developing new services that take advantage of emerging technologies like edge computing and 5G will help generate more revenue for many businesses today, but especially for telecommunications and media companies.

This article goes in depth over:

  • What is Edge Computing?
  • Edge Computing and 5G
  • Benefits and challenges of edge computing
  • Cross Industry Use Cases
    • Use case #1: Video Surveillance
    • Use case #2: Smart Cities
    • Use case #3: Connected Cars
    • Use case #4: Manufacturing
  • Overall Architecture
  • Sample implementation of an edge computing architecture

Edge computing architecture overview

Source: https://developer.ibm.com/depmodels/edge-computing/articles/edge-computing-architecture-and-use-cases/

In this brief overview of edge computing technology,authors have shown how edge computing is relevant to challenges faced by many industries, but especially the telecommunications industry. Perfect!

[Read More]

How data blending is leveling the playing field in online marketing

Categories

Tags startups cloud cio teams data-science

Online marketing today is about successfully manage your social media marketing, social media advertising, PPC ads, email campaigns - if you’re not careful, it can feel like throwing spaghetti at the wall just to see what works. Fortunately, data analysis and - more specifically - data blending can make your marketing decisions faster and easier to make, with more impact. By Zac Johnson.

Data blending is simply where you take data from multiple sources and combine them to give you one united dataset.

This is incredibly beneficial regardless of what data you’re looking at but can be even more so for online marketing.

The article then explains the following:

  • What is data blending?
  • The benefits of consolidating your marketing data
  • Finding the right tools

The article also briefly looks at the 2 tools you need to know about. Sisense and Google Data Studio. It’s more important than ever to start using data blending in your business and marketing efforts. Good read!

[Read More]

Rebuilding Linkerd's continuous integration (CI) with Kubernetes in Docker (kind) and GitHub Action

Categories

Tags cloud docker kubernetes containers devops

In mid-2019, the Linkerd project’s continuous integration (CI) took 45 minutes, all tests were serialized on a single Kubernetes cluster, and multi-hour backups were common. A migration onto one-off Kubernetes in Docker (kind) clusters and GitHub Actions got CI below 10 minutes, and made it parallelizable. By Andrew Seigner.

While the goal of this article is to detail an end user story of how to efficiently test Kubernetes applications in CI, some background on Linkerd is helpful. Linkerd is an open source service mesh, and a CNCF member project.

Linkerd Architecture

Source: https://buoyant.io/2020/09/16/linkerds-ci-kubernetes-in-docker-github-actions/

In the article the author captures following:

  • What is Linkerd?
  • Testing Linkerd
  • Prioritized CI requirements
  • CI tech evaluation
  • Lessons learned

Kind is a great tool, not only for CI but for local development as well. There are similar flavors of Kubernetes, such as Minikube and k3d. We selected kind because it’s used heavily by the Kubernetes community for testing Kubernetes itself. Also a huge shout out to the #kind channel in Kubernetes Slack. Nice one!2

[Read More]

A beginners guide to abstraction in Object-Oriented programming

Categories

Tags oop programming java javascript python software-architecture

Learn the basics of abstraction in object-oriented programming with example code and practice challenges. By Lee Nathan.

Along with inheritance, abstraction is an important concept in object-oriented programming. The theory is that every object should deliver simple and predictable results. Objects should also only share what needs to be shared.

The article dives into:

  • Abstraction keeps code and data hidden when appropriate
  • Abstraction is closely tied to encapsulation
  • Why classes should sometimes be abstract
  • What is abstract and what is concrete?
  • Abstraction practice
  • But what is the point of a class you can’t instantiate?

At first, abstract classes may seem unnecessary. After all, you can’t actually use them to make an object. And it’s not like they pass down their functions. Functions have to be rewritten in every child class. You will get code examples and link to the sandbox to practice abstraction with JavaScript. Good read!

[Read More]

AWS Lambda: Comparing Golang and Python

Categories

Tags golang python software-architecture serverless

Serverless functions are great for lightweight cloud architecture and rapid provisioning. However, sometimes serverless introduces additional complexity to the deployment process. Author compares Python and Go with respect to the ease of deployment when setting up a simple data factory on AWS Lambda. Written by Dick Abma.

Serverless functions are a great way to simplify your cloud stack and quickly produce new functionality. Serverless contributes to scalability, transferability, and flexibility of the entire stack. For author as a data engineer, this means that he can produce applications that will be understood by clients and can adapt to evolving requirements.

The article describes:

  • Case study - bunch of IoT devices add data to a database, which is accessible via a REST API
  • CI/CD: Platform compatibility
  • Python in AWS Lambda
  • Golang on AWS Lambda
  • AWS limits, concurrent requests and validation
  • Conclusion deserialization

The article covers more realistic setup with a Lambda Layer, in which you can place the needed packages. The layer is loaded by the function when it is invoked. Packages are thus removed from the function itself.

Serverless functions like AWS Lambda are great for lightweight cloud architecture. Though we should be careful not to introduce complexity in our CI/CD and Terraform code, which may outweigh the advantages of Lambda. Code is provided in this GitHub repository. Great read!

[Read More]

Mapping Moving clouds: How to stay on top of your ephemeral environments with Cartography

Categories

Tags cloud infosec cio devops software-architecture

Staying on top of ephemeral environments is a challenge many organizations face. This blog post describes the process we undertook at Thought Machine, a cloud-native company with environments spanning across multiple cloud providers, to identify a solution able to detect, identify, categorize, and visualize all the cloud assets being deployed in an organization. Written by Marco Lancini, Security Engineer.

If you look for common root causes of recent security breaches, you will see that a considerable amount of them can be traced back to assets that slipped out of the organization’s inventory management, or that have been misconfigured. “The scale of cloud infrastructure is essentially infinite”: the server fleet really has no practical limit since the cloud providers operate at enormous scale. We can then gather that, nowadays, one of the core problems many organizations are facing is to keep track of all their assets.

Cartography is described as: A Python tool that consolidates infrastructure assets and the relationships between them in an intuitive graph view powered by a Neo4j database. Cartography is not the only security graph tool out there, but it differentiates itself by being fully-featured yet generic and extensible enough to help make anyone better understand their risk exposure, regardless of what platforms they use.

High level multi-cloud auditing setup.

Source: https://thoughtmachine.net/blog/mapping-moving-cloudshow-to-stay-on-top-of-your-ephemeral-environments-with-cartography-mb6d8/

The article content is split into:

  • The challenges posed by ephemeral environments
  • Enter Cartography
  • Cartography’s value proposition
  • Real world setup
  • Multi-cloud auditing
  • Deployment on Kubernetes
  • Data consumption

In this blog post we saw what benefits Cartography could have on the security posture of your organization, and author walked through the process we undertook to deploy it in a multi-cloud environment, from inception to self-service dashboards for data consumption. But there is still much to do, and Cartography still has much more to offer. Great read!

[Read More]

InnoDB Data Locking – Part 1 Introduction

Categories

Tags cloud sql database devops software performance

In this blog series, we would like to introduce you gently to the topic on which we were working over last 2 years. Improving how InnoDB locks data (tables and rows) in order to provide illusion to clients that their queries are executed one after another, while in reality there is a lot of concurrency. By Kuba Łopuszański.

Do you remember Excel? You know, columns, and rows, perhaps in multiple tabs. And then you might have multiple files. You probably do not appreciate it consciously, but it is a nice feature, that whatever you put inside a cell stays there if you save. Perhaps sometimes you need to add a new row, or remove it expecting that to simply work, even if there were some formulas, like SUM(A1:A100) involving these rows, right? And have you ever had to collaborate on a single spreadsheet with another person? How did you go about not overwriting each others work?

In this post author will introduce following concepts:

  • Serializability of transactions (ability to explain states observed over time with a convincing story about relative order of parallel operations)
  • Amdahl’s law (limits of how much parallelism can help in presence of tasks which must be done one at a time)
  • Timeouts (for misbehaving lock owners, and to resolve deadlocks)
  • Reader-writer lock (shared/exclusive access rights)
  • Starvation (permanent inflow of readers starving a writer waiting for its turn) queueing (FIFO, or priority)
  • Read views (read-only snapshots which allow stale reads concurrent to new writes)
  • Wall clock time out of band communication and their possible inconsistency with serialization order
  • Deadlocks caused by escalation
  • Granularity of a lock (access right to everything vs. only the resources needed)

… and more. This is klong read with explaining the concepts rea;ted to locking on the Excel spreadsheet example. Enjoyable!

[Read More]

Zero downtime on MySQL schema change

Categories

Tags cloud sql database devops performance

Have you ever deployed a feature which interrupted your application because the schema change on the MySQL database has locked the table? Then I have good news for you! There are some tools like the Percona Toolkit which can apply the operation without a downtime. By Michi Lehr.

When you have a MySQL database and want to apply a scheme change on a table (e.g. an alter table), the table needs to be rebuild on some DDL (Data Definition Language) operations. This depends on the MySQL version and DDL operation.

The tool pt-online-schema-change of the Percona Toolkit will take another strategy when altering a table. It will create a new table and add triggers to the existing table so that changes will be copied to the new table. Then the existing data will be copied. After this, the tables will swapped, and the triggers will be dropped.

# Example of dropping a column.
pt-online-schema-change --alter "drop column my_column" D=my_database,t=my_column --alter-foreign-keys-method auto --execute

Pretty straightforward. You will also get the links to similar tools. Nice one.

[Read More]

How event driven systems work in commerce

Categories

Tags cloud management event-driven miscellaneous

The polling system acts like the impatient customer. In a commerce-related scenario, it keeps polling the system for new updates such as orders and payment authorizations while the event driven system relies on asynchronous event handlers to notify it of updates in the system. By Sajjad Heydari.

Event based systems in computer science go back to the 1950s where they were first designed to handle asynchronous events such as Input/Output. Edsger Wybe Dijkstra designed the first interrupt handler on the Electrologica X-1 that is still used in modern systems to this day. The modern day’s events are the same concept, only they are used in different environments.

The event driven system is easier to develop and more efficient, but it requires special infrastructure to work. The good news is that some software and platforms like Fabric support event driven systems. In commerce, these systems create more streamlined operations so customers get their orders faster.

The article then describes:

  • Example of an event driven system
  • Parts of an event driven system
    • The event
    • The event router
  • Benefits of event driven systems

In this post we provided an overview of how event driven systems streamline commerce. For a more detailed view of how events and event routers allow different services to communicate better in a commerce tech stack, check out thisvideo from AWS. Nice one!

[Read More]