Welcome to curated list of handpicked free online resources related to IT, cloud, Big Data, programming languages, Devops. Fresh news and community maintained list of links updated daily. Like what you see? [ Join our newsletter ]

Everything you will ever need to know about the IoT ecosystem in one blog post

Categories

Tags iot miscellaneous learning cloud

Speaking about any ecosystem, we speak about a complex system of interconnected components and the environment in which they exist and with which they interact. We can use the term IoT ecosystem instead of IoT system because IoT devices have no value without the environment they exist in. By @sumatosoft.

The article main parts are:

  • 7 Components of an IoT Ecosystem Chart
    • IoT devices
    • Security
    • Network
    • Gateway
    • The Cloud
    • Application
    • Users

‍IoT devices are the layer of sensors, actuators, and smart objects that gather information about the environment and measure physical parameters. It’s a rare case when the Internet of Things ecosystem features only one type of sensor or actuator. There are numerous types of sensors where every type has its sub-categories of sensors. By reading this article you will get a basic understanding of each part of IoT ecosystem. Nice one!

[Read More]

How to automate API testing with Azure Devops and Postman?

Categories

Tags devops azure cloud apis programming

I have been using Postman for a long time to test API requests from my local machine. I started to investigate during one project what kind of automated integration test possibilities could Postman offer. Actually it offered very nice features what I haven’t used before. Best finding was possibility to integrate everything very easily to Azure DevOps pipelines. By Kalle Marjokorpi.

The article then quickly explains:

  • How to configure Postman API requests?
    • Set Postman environment variables
    • Create a collection for your requests
    • Shared pre-request scripts in collection level
    • Test requests
  • Configure tests to be executed in Azure DevOps
    • Export test requests and environment variables
    • Remove Client Secret from environment JSON configuration
    • Create YAML template

Postman provides an easy and powerful way to create automated integration tests which can be executed in Azure DevOps. Author considered it really nice that you could write Postman test scripts with javascript. Test object model is easy to learn and there is a lot of good documentation available. At least for a simple integration cases this is a very good tool. Sharing test scripts between tests is difficult and one reason why you should consider other frameworks for very complex test cases. Yo u will also find links to further reading in this article. Good job!

[Read More]

Enabling transparent data encryption for Microsoft SQL with Vault

Categories

Tags database infosec cloud azure

Learn how HashiCorp Vault can help secure data in Microsoft SQL Server using a defense-in-depth encryption strategy. By Narayan Iyengar.

Data should be protected both at rest and while in transit. In-transit data protection is usually accomplished by establishing a mutual TLS channel between the two parties before data is transmitted. Protecting data at rest is also important, as database files and backups may be stolen or leaked. If data at rest is encrypted, it cannot be used without having access to decryption keys. This blog post will cover how HashiCorp Vault can help extend data protection for data at rest, specifically data that resides in your Microsoft SQL servers.

The article deals with:

  • What is transparent data encryption?
  • Why enable TDE?
  • Vault MS SQL TDE extensible key management (EKM) module

As highlighted above, protecting data at rest is a critical part of employing a defense-in-depth posture. Data at rest is protected by encrypting the data by using data encryption keys (DEKs). But working with encrypted data requires that the user or the application be aware that the data is protected by a DEK and that it must be decrypted using the correct DEK. This means that the application needs access to the encryption key (or knows how to get access to the encryption key). Good read!

[Read More]

What web frameworks solve and how to do without them

Categories

Tags frontend frameworks javascript app-development performance

In this article, Noam Rosenthal dives deep into a few technical features that are common across frameworks, and explains how some of the different frameworks implement them and what they cost. By Noam Rosenthal.

I was interested to see what are the commonalities and differences between the frameworks, what the web platform has to offer as a leaner alternative, and whether it’s sufficient. My objective is not to bash frameworks, but rather to understand the costs and benefits, to determine whether an alternative exists, and to see whether we can learn from it, even if we do decide to use a framework.

The article the dives on:

  • The frameworks (React, SolidJS, Svelte, Lit)
  • What frameworks solve
    • Declarative programming
    • Data binding
    • Reactivity
    • Logic
    • Component model
  • The cost
    • Bundle size
    • Build
    • Debugging
    • Upgrades

In the article author dived a bit deeper into understanding the core problems frameworks try to solve and how they go about solving them, focusing on data-binding, reactivity, conditionals and lists. We also looked at the cost. Good read!

[Read More]

Deep dive on Angular I18n with ngx-translate

Categories

Tags frontend angular nodejs javascript app-development

Learn how to internationalize Angular apps step by step with the help of ngx-translate, one of the most popular open-source libraries for i18n. By Ankit.

One of the most popular open-source i18n libraries, ngx-translate, lets you define translations for your app and switch between them dynamically. You can either use a service, directive, or pipe to handle the translated content. In this Angular 13 tutorial, we will learn how to use them all with the help of a small demo app.

For demonstration purposes, we will create a sample feedback form for Phrase, the most reliable software localization platform on the market, and launch our demo app in two different languages – English and German.

The article is very detailed and captures:

  • Why use ngx-translate instead of Angular i18n?
  • Configuring ngx-translate for an Angular app
  • Updating the AppModule
  • Creating a nav-bar component
  • Creating a model
  • Creating translation files
  • Using the translate pipe
  • Using the translate directive
  • Dynamic translation keys
  • Exploring TranslateService
  • Creating a Feedback Component

… and much more. There are Angular code examples with good explanation. Nice one!

[Read More]

Ray on databricks

Categories

Tags data-science python machine-learning big-data

Ray is an open-source project first developed at RISELab that makes it simple to scale any compute-intensive Python workload. With a rich set of libraries and integrations built on a flexible distributed execution framework, Ray brings new use cases and simplifies the development of custom distributed Python functions that would normally be complicated to create. By Stephen Offer.

The article then makes a good job explaining the following:

  • Why need another distributed framework on top of Spark?
  • A simple introduction to Ray architecture
  • Starting Ray on a Databricks cluster
  • Distributing Python UDFs
  • Reinforcement learning

Applications of reinforcement learning broadly consist of scenarios wherever a simulation is able to run, a cost function can be established, and the problem is complicated enough that hard-set logical rules or simpler heuristical models cannot be applied. The most famous cases of reinforcement learning are typically research-orientated with an emphasis on game-play such as AlphaGo, super-human level Atari agents, or simulated autonomous driving, but there are many real-world business use cases. Examples of recent applications are robotic manipulation control for factories, power consumption optimization, and even marketing and advertising recommendations. Good one!

[Read More]

Platform Engineering: Building AWS landing zone with Pulumi

Categories

Tags devops distributed aws containers

In this blog, I will cover some key concepts and architecture of Pulumi. We will be building and provisioning the AWS landing zone. By A B Vijay Kumar.

Author wonders if Terraform is the ultimate tech for IaC? Coming with an application developer background, and entering the full-stack cloud developer – learning another language and syntax is painful. Terraform is a DSL (domain-specific language) and requires significant effort to learn and master.

In this blog series, we will be building the following infrastructure on AWS and will run our applications in the EC2 instance. The article then covers:

  • Install Pulumi
  • Setup AWS CLI
  • Generate Pulumi project — boilerplate code for AWS in Python
  • Activate virtual environment and install dependencies
  • Set up public/private key infra
  • Write Infra code

The article contains a lot of Pulumi code with detailed explanation of every code block. Nicely done!

[Read More]

Controlling concurrency in distributed systems using AWS Step Functions

Categories

Tags programming distributed aws app-development code-refactoring containers

Managing concurrency in distributed systems can be challenging. In a monolithic application, you use familiar concepts such as in-memory locks to avoid overloading a database or prevent overwriting a customer record by two users at the same time. With a distributed system, where your application is dispersed across computing environments without shared memory, these methods are no longer available to you. By James Beswick.

Controlling concurrency in distributed systems using AWS Step Function

Source: https://aws.amazon.com/blogs/compute/controlling-concurrency-in-distributed-systems-using-aws-step-functions//

In this blog, you use AWS Step Functions to control concurrency in your distributed system. This helps you avoid overloading limited resources in your serverless data processing pipeline or reduce availability risk by controlling velocity in your IT automation workflows. With this sample application, you implement a distributed semaphore using AWS Step Functions and Amazon DynamoDB to control concurrent invocations of a function in AWS Lambda.

The article the explains:

  • Deploying the concurrency control application
  • Running the concurrency control application
  • How it works
  • Handling failure

To apply this concurrency control in your own application, replace the branch defined in the Do Work parallel state with a state machine definition that contains the business logic of your application. You also update the ConcurrentAccessLimit value template.yaml to specify the level of concurrency that your application requires. Nice one!

[Read More]

How using same programming language for IaC made AWS Step Function testable

Categories

Tags nosql cloud devops microservices tdd serverless

AWS Step Functions are difficult to test. I found an approach to testing that helped in one particular scenario. It relies on using the same programming language for both testing and infrastructure-as-code (IaC). I hope the idea helps others tame their Step Functions into a testable submission. By Graham Allan.

Instead of testing the Step Function by executing it, I took advantage of using TypeScript for both CDK and integration tests in the same repository. I defined constants in their own file. With this simple change, I can use the DynamoDB client to execute the query in a Jest test. I run the test from my machine. I can iterate on the query without worrying about deploying or executing the Step Function. Sure, I need a real DynamoDB table, but that is rarely modified.

The article main parts will explain:

  • Why are Step Functions hard to test?
  • What about integration testing a Step Function?
  • Adding a DynamoDB task to a Step Function
  • How the shared programming language enabled better tests

Step Functions are hard to test. If that explanation does not work for you, try this: Step Functions are difficult to test in the same way stored procedures on a database server are hard to test. Having a shared programming language across IaC and tests allowed a creative way to gain confidence in author’s system, with more maintainable tests. Excellent read!

[Read More]

Cloud trends show customers increasing investments in hybrid and multicloud

Categories

Tags cloud cio management miscellaneous

Across every industry and geography, companies are working hard to keep pace with evolving business needs and build on their existing digital investments. In my role leading the product team for the core of Azure, I spend a lot of time with customers learning what they need to be successful as they integrate cloud technologies into their business strategy to digitally transform. By Erin Chapple - Corporate Vice President, Azure Core PM & Design.

As many have experienced firsthand, the pandemic has pushed more businesses to adopt wider use of cloud computing technology. A new survey finds it’s also driving more companies to deploy a hybrid (mix of on-premises and one or more public clouds) or multicloud (multiple public clouds) approach. The survey, conducted by The Harris Poll and sponsored by Microsoft, found 86% of all respondents plan to increase investment in hybrid or multicloud environments, and 95% say those technologies have already been critical to their success.

The survey and our own observations both underscore how companies are emphasizing remote work, increasing demands for efficiency and ever-present competitive pressures to reduce costs. While the pandemic has already accelerated use of cloud technology, respondents shared they will continue to proactively and strategically invest in cloud, specifically hybrid and multicloud, in the next few years. Good read!

[Read More]