Welcome to curated list of handpicked free online resources related to IT, cloud, Big Data, programming languages, Devops. Fresh news and community maintained list of links updated daily. Like what you see? [ Join our newsletter ]

Programming concurrency in C++

Categories

Tags web-development programming software-architecture distributed

The first in series of articles by Mehreen Tahir about programming C++ and introduce you to the features C++ offers in order to support concurrent programming. C++ was originally designed to support only single thread programming. In every application, there is one default thread.

Concurrent programming allows for the execution of multiple threads and thus, we can write highly efficient programs by taking advantage of any parallelism available in a computer system.

C++ version 11 acknowledged the existence of multi-threaded programs and the later standards also brought some improvements.

Following this article you will learn about:

  • Concurrency and concurrent programming
  • Concurrency and parallelization
  • Why even the need for concurrency?
  • The threading library in C++
  • Different ways of creating threads
  • Considerations for computation and / or IO intensive programs

A thread is basically a lightweight sub-process. It is an independent task running inside a program that can be started, interrupted, and stopped.

The article also has figures explaining the life cycle of a thread and example of code you can use for your own experiments with concurrent programming. Nice one!

[Read More]

Structuring applications in Go

Categories

Tags golang web-development programming software-architecture

An article by Liam Andrew Cura about his Gog journey. For him the hardest part of learning Go was in structuring his application.

Go doesn’t prescribe any particular project layout or application structure and Go’s conventions are mostly stylistic.

Author focuses on four patterns which can be tremendously helpful in creating architecture for Go application:

  • Don’t use global variables
  • Separate your binary from your application
  • Wrap types for application-specific context
  • Don’t go crazy with sub-packages

Go only has one requirement for packages: you can’t have cyclic dependencies. Author recommends to group related types and code together in each file. Also organize the most important type at the top of the file and add types in decreasing importance towards the bottom of the file.

Well written article with code examples and links to complementary resources and reading. Excellent!

[Read More]

Fully automated zero downtime deployments with Floating IPs on DigitalOcean Droplets

Categories

Tags devops docker cicd

Tutorial by Marco Ebbinghaus in which he explains how he has implemented a zero downtime deployment (blue-green like) of a small web application which is running on a DigitalOcean Droplet.

The cool thing is, that this deployment is fully automated. After pushing a code change to the web application code a CI/CD-Pipeline will be executed.

The article main parts:

  • Description of tech-stack
  • Creation of the web application (Spring Boot)
  • Adding web application in container
  • Creation of the infrastructure (as code) via Terraform
  • How to create the CI / CD pipeline

It also describes the way how to log into the docker registry. Also adding environment variables in the CI/CD-Settings in GitLab project. Detailed and exciting!

[Read More]

Managing analysis workflows in geospatial data science with GNU Make

Categories

Tags big-data machine-learning data-science miscellaneous python

Martí Bosch wrote this guide how to go about using Jupyter Notebooks while using iterative approach to both data analysis and software development. He will also explain how to avoid some bad practices. Many issues can be settled by choosing helpful file names, good organization, documentation and source control of the code.

The article then describes an example from geospatial data science: analysis of the spatiotemporal patterns of urbanization.

It deals with:

  • Computational workflow approach
  • Automating the workflow with GNU Make
  • Detecting completed targets
  • Pattern rules and abstracting our workflow from the data
  • Detecting edited source files

All above with detailed code examples and illustrated charts and animations. Link to GitHub repository is included, too.

The main advantage of Make is its flexibility, which allows it to precisely manage any kind of computational workflow, from the compilation of source files to the example above. Nice one!

[Read More]

Understanding stabilising experience replay for deep multi-agent reinforcement learning

Categories

Tags big-data machine-learning data-science miscellaneous

An article by Parnian Barekatain in which she describes some basic concepts in Reinforcement Learning. She also provides you with the link to Udacity’s free course on Deep Learning with Pytorch.

The blog consists of four parts:

  • Deep Neural Network for Single-Agent: Reinforcement Review, DQN and Replay Memory
  • Overview of multi-agent Reinforcement Learning
  • Deep Neural Network for multi-agent: Independent Q Learning (IQL) and challenges combing with multi-agent Reinforcement Learning
  • FingerPrinting

In 2015, DeepMind was able to successfully combine Deep Neural Network with Reinforcement Learning for a single-agent. Combining Deep Neural Network with Reinforcement Learning enabled AI for the first-time to surpass the performance of professional human players across many game scenarios.

This worked well fora single agent. However, when we have many agents, we cannot easily combine Deep Neural Networks with Reinforcement Learning, mainly because each agent constantly changes the dynamic of the environment and makes it really hard for other agents to learn what to do.

Read the rest of the article and watch explanation videos to understand this paper. Plenty of links to other resources to get you on track! Excellent read for any aspiring data scientist.

[Read More]

Single-Node Kubernetes on Raspberry Pi with MicroK8s and Ubuntu

Categories

Tags miscellaneous devops linux

Blog post by Canonical in which they do excellent job of explaining how to setup and run Ubuntu Server on a Raspberry Pi with MicroK8s on-top to provide a single-node Kubernetes host for development and testing.

Ubuntu Server 18.04 has been ported to run the Raspberry Pi 2/3 which means we can now run snaps and other Ubuntu applications on these devices with ease.

The post then describe in detail:

  • Hardware for the used deployment
  • How to obtain Ubuntu and flash MicroSD card
  • Initial configuration
  • MicroK8s basics & deploying a workload
  • Setting up remote access

Excellent guide with configuration which should work for future releases of Ubuntu Server and Kubernetes. Configuration code and screen grabs also included. Great read for any DevOps enthusiast!

[Read More]

Skaffold for local kubernetes development

Categories

Tags devops kubernetes software containers software-architecture

Straight to the point guide from Shane Lee aiming to get you running with kubernetes in no time. Great tutorial for anybody new to kubernetes.

Skaffold is fantastic for local development with kubernetes. You are able to test locally your changes without having to deploy remotely. This helps speed up local development and gives developers confidence in their changes.

Skaffold is a tool to develop containerized applications locally or remotely while deploying them on Kubernetes. It automatically builds and deploys your apps as you change your source code.

Skaffold primarily simplifies the build → deploy → refactor → repeat cycle.

The article then guides you through:

  • Skaffold main features
  • Local development with sample skaffold.yml file
  • Explaining local development workflow
  • Explaining Dockerfile

You will also get links to further reading and tools for kubernetes and skaffold exploration. Nice little gem!

[Read More]

MySQL high availability framework explained – part one

Categories

Tags sql database software devops

This is three-part blog series, in which author explains the details and functionality of a High Availability (HA) framework for MySQL hosting using MySQL semisynchronous replication and the Corosync plus Pacemaker stack. Published by Prasad Nagaraj VP Engineering at ScaleGrid.

The article dives into explaining what high availability means for the business. It describes components of HA framework and:

  • Redundancy in infrastructure & data
  • Failure detection & correction mechanism
  • Failover mechanism
  • Application / user redirection mechanism

Based on the above model, author uses the following HA framework for their MySQL hosting at ScaleGrid:

  • 3-Node Master-Slave setup using MySQL semisynchronous replication to provide infrastructure and data redundancy
  • Corosync plus Pacemaker stack to provide failure detection, correction, and failover mechanism
  • DNS mapping or Virtual IP component to provide the application and user redirection mechanism

Each component is then explained including schemas to depict the stack. Excellent!

[Read More]

Common API mistakes and how to avoid them

Categories

Tags javascript nodejs apis devops

Thomas Hunter II is author of this guide on hot topic how to avoid common API mistakes. The advice in this article applies to any API. However, some of the issues author considers are easier to encounter when the application is written in a dynamic language, such as JavaScript, versus a more static language, such as Java.

Be sure to follow the Robustness Principle wherever it may apply to your API. Quoting from Wikipedia, this principle is:

Be conservative in what you do, be liberal in what you accept from others.

The good practice mentioned in this article includes:

  • Be stingy with data
  • Represent upstream data as well-defined objects
  • Use forward compatible attribute naming
  • Normalize concepts and attributes
  • Use positive, “happy” names
  • Apply the Robustness Principle (e.g. in regard to HTTP headers)
  • Test all error conditions

Read the rest of the article to learn what advice to follow so you can be sure to avoid some of the most common pitfalls present in modern APIs. Nice one!

[Read More]

Three arguments for why you should write more (as developer)

Categories

Tags miscellaneous programming learning career

Posted by Marek Zaluski, this is a short list of arguments to persuade any developer to write more. He does not mean write more code. He means write more for humans.

There’s a huge benefit to getting good at writing as a developer. And all it takes it practice. You don’t have to be a good writer to write. In fact, it’s the opposite: you have to write a lot first, and then with time it becomes easier and easier.

The goal of practicing writing is not to write high-quality material. The goal is to get efficient and fast at getting your thoughts into text. It’s a skill that’s for you first, and for the reader second.

The arguments mentioned here:

  • Argument for the collaborators (boosting your value)
  • Argument for the competitors (stand out from the crowd)
  • Argument for the introverts (write for yourself, problem solving)

Start a blog. Write about your projects. Share what you’re learning with the world. These are powerful tactics for career advancement. This is how you get noticed and how you get your foot in the door at your dream company. Excellent advice!

[Read More]