Welcome to curated list of handpicked free online resources related to IT, cloud, Big Data, programming languages, Devops. Fresh news and community maintained list of links updated daily. Like what you see? [ Join our newsletter ]

The digital path to growth – sensors and handling data

Categories

Tags leadership-and-career business-and-emerging-tech data-and-analytics

There are discussions being held today around hot commodity, “data”. Data has also been evolving at an unprecedented pace and affecting our lives directly with its mammoth influence on industries as varied as healthcare, education, banking and finance. By Australian Manufacturing Forum @aumanufacturing.

In the latest industrial revolution, Industry 4.0, we are monitoring, analysing and utilising unprecedented levels of data attained on the shop floor, thanks to the developments in the advanced manufacturing practices and techniques.

The article also mentions:

  • The development of data collection in smart manufacturing
  • The developments of data analysis in smart manufacturing
  • Three paradigms of data analysis
    • physical modeling
    • machine learning
    • deep learning

In data-driven manufacturing practices, the huge amounts of data gained from the plant floor, whether held on premise or in-cloud, will not stack up without proper interpretation through new analysis tools and methods.

The long-term planning approach, “Think big, start small,” can help manufacturers reap the benefits of technologies mentioned above, reviving local manufacturing industries to the 1960s levels of nearly 30 per cent of GDP, compared to today’s 5.6 per cent. Good read!

[Read More]

How to build an effective API strategy in 2021

Categories

Tags product-and-design leadership-and-career architecture-and-apis devops-and-ci-cd

Application Programming Interfaces (APIs) are the building blocks of modern digital ecosystems. They’ve collapsed data silos, allowing businesses to streamline their operations. By David Campbell.

APIs have become a strategic move that every business must adopt. That’s what’s happening. The growth of API traffic is witnessed across different industries.

The article content is focused on:

  • What are the benefits of having an effective API strategy?
  • Putting together a solid API strategy
    • Align the digital strategies
    • Align the organization’s culture and mission
    • Select the appropriate API platform and tools
    • Distribute clear up-to-date documentation
    • Create the appropriate API architecture and infrastructure
    • Set up a test API project
    • Begin creating a digital ecosystem
  • Bottom line

APIs are increasingly popular for good reasons. They boost collaboration, reduce the time to market for new products, promote innovation, enhance marketing strategies, and so on. Good read!

[Read More]

Test like you fly - intro

Categories

Tags product-and-design testing-and-quality devops-and-ci-cd architecture-and-apis leadership-and-career software-engineering

In contrast to all the other important test methodologies, “Test Like You Fly”, or TLYF for short, emphasizes testing to find fundamental flaws in a system that will prevent it from performing the mission. Most testing methodologies strive to confirm that requirements - the input to our designs - are being met by the system as written. By Tim Chambers.

TLYF is all about confirming that the system - as a whole - will operate in the environment it is designed to operate, the environment we typically refer to as “live” or “production”. It is vitally important to find what doesn’t perform as expected and to understand the reasons for this anomalous behavior, especially where such defects can degrade, cripple, or abruptly end a mission.

In our testing strategies we frequently attempt to isolate the component under test - large or small - from the rest of the system. We know what we want to test, and fine-grained or coarse, we test segments of the system. The larger the component, or the broader the test, the more we tend to try and prove it WILL work - work “as designed”.

TLYF comes at the system from the other direction. Testing Like You Fly is designed to as completely as possible drive the system as it will exist in production. It demonstrates that the mission of the system or application can achieve success, not that it merely meets requirements. All components are wired together and available in as real-world an environment as is possible. Read this article in full to get some very interesting insights!

[Read More]

Best practices for load balancing Kubernetes containers

Categories

Tags product-and-design devops-and-ci-cd architecture-and-apis leadership-and-career

Some organizations have hundreds of small containers across many different servers in different development, test, and production environments. This can be tricky to manage, which is why companies have turned to Kubernetes for container orchestration. By Craig Risi.

This has made Kubernetes not only a vital part of many development pipelines but also a central system and potential performance bottleneck that needs to be managed and balanced to ensure optimized performance.

The article deals with:

  • How to set up a load balancer on Kubernetes
  • Enable the readiness probe on a deployment
  • Enable the readiness probe on a deployment
  • Enable CPU/Memory requests and limits
  • Flag when RBAC rules change
  • Control the container images deployed into your cluster
  • Apply network policy to your deployments
  • Flag any service account changes
  • Adjusting POD toleration

Applying Security Groups policies to your VMs or your Kubernetes worker nodes is considered essential to security. We should do the same with Kubernetes workloads. And load balancing is essential to keeping your Kubernetes clusters operational and secure at a large scale.Good read!

[Read More]

How to undo (almost) anything with Git

Categories

Tags product-and-design cloud-and-infrastructure devops-and-ci-cd leadership-and-career

One of the most useful features of any version control system is the ability to “undo” your mistakes. In Git, “undo” can mean many slightly different things. By Joshua Wehner.

When you make a new commit, Git stores a snapshot of your repository at that specific moment in time; later, you can use Git to go back to an earlier version of your project.

The article will give you more info on how to:

  • Undo a “public” change
  • Fix the last commit message
  • Undo “local” changes
  • Reset “local” changes
  • Redo after undo “local”
  • Once more, with branching
  • Branch in time saves nine
  • Mass undo/redo
  • Fix an earlier commit
  • Stop tracking a tracked file

Each section has scenario with command(s) description and explanation what is happening behind the scenes. Older article but evergreen. Nice one!

[Read More]

Redis at the Edge with Cloudflare Workers

Categories

Tags product-and-design cloud-and-infrastructure architecture-and-apis data-and-analytics software-engineering

Computing at the Edge is one of the most exciting capabilities in recent years. CDN allows you to keep your files closer to your users. Edge computing allows you to run your applications closer to your users. This helps developers to build globally distributed, performant applications. By Enes Akar.

Cloudflare Workers is the leading product in this space right now. It gives you a serverless processing environment without cold starts. You leverage Cloudflare’s global network to minimize latency of your applications. You can write your functions in JavaScript, Rust, C and C++.

The article also deals with:

  • Comparing with Cloudflare Workers KV
  • Analytics at the Edge
  • Cloudflare workers code
  • Analytics tool code
  • Upstash edge roadmap

Similar to Serverless functions (AWS Lambda etc.), Cloudflare Workers are stateless. Unfortunately, most databases are not designed for serverless environments, they require persistent connections. We developed the REST API over Redis to enable serverless edge functions to access Upstash in the simplest and fastest way possible. Good read with link to the code used in the article.

[Read More]

Administrator's guide: What makes passwordless, dare we say it, phish-proof?

Categories

Tags product-and-design cloud-and-infrastructure security-and-privacy devops-and-ci-cd frontend-and-mobile

In some ways, the term “passwordless” is a misnomer. Yes, it’s a password-less authentication method, greatly streamlining the login experience, and while that’s a great incentive to use passwordless for logging in, it’s not an improvement in authentication security in and of itself. By Jeremy Erickson.

To prevent phishing, there are a few general properties that your authentication solution needs:

  • No Shared Secrets is the property that secrets are never shared and are always kept local to the authenticator device
  • Origin Binding is the property that the site you (as a user) are attempting to log in to must match the domain, or origin, of the site you’re actually on
  • Channel Binding is the property that the communication channel from the authenticator to the website must be strongly tied to the browser session attempting to authenticate

The article then looks in depth into how WebAuthn and FIDO2 implement these properties and provide a very robust resistance to phishing. Excellent!

[Read More]

10 common SQL operations to perform using Pandas

Categories

Tags data-and-analytics ai-and-machine-learning backend-development

One of the key features of being a data analyst is to query the data from files, databases etc to perform some data manipulation or visualisation and sometimes it’s much better if we can do it directly through code instead of looking into the database tables over and over again.. By Akash Mishra.

Pandas is a python library which can store query results in variables called “dataframes” and it helps us to perform data manipulations, visualisation and can convert our results back in the databases or files as write operations.

# In SQL:
SELECT name,roll_no,phone FROM student_details;

# In Pandas:
import pandas as pd

student_details = pd.read_csv('students.csv')
result = student_details(['name', 'roll_no', 'phone'])
result

This blog basically describes how one can use 10 basic SQL operations using the pandas library:

  • Selecting the data
  • Using aggregate functions
  • Order By clause
  • Group By clause
  • IN and NOT IN
  • Joins
  • Creating new column using existing ones
  • Selecting data conditionally

.. and more. You will also get 2 datasets to work with on your learning journey. This is just introduction to how one can execute basic SQL operation using pandas and there are many more operations of SQL which can be easily done using pandas. Nice one!

[Read More]

Storage performance for cloud native applications

Categories

Tags cloud-and-infrastructure devops-and-ci-cd leadership-and-career

We talk with our IT clients on a regular basis, and also conduct primary research into emerging IT trends. One of the leading trends we hear is the adoption and deployment of cloud native applications, including containers and Kubernetes, and the infrastructure changes required. By Russ Fellows.

The result of IT transformation to using cloud native applications has significant implications for on-premises as well as public cloud environments. With containers and Kubernetes comes new storage interfaces and storage system requirements. The topics of storage and data protection for container-based applications is one of the biggest concerns we’ve heard from our IT clients for several years.

The article further mentions:

  • Real world performance testing
  • Performance Comparison

There are many aspects to consider when evaluating storage options for cloud native and container environments. These include ease-of use, manageability, reliability, scalability and other criteria. Undoubtedly, performance is one critical metric that impacts application service levels, and also may have significant cost and architectural implications. Follow the link to the full article to get charts and other comparison data. Nice one!

[Read More]

How are client hints really useful for web performance

Categories

Tags architecture-and-apis devops-and-ci-cd software-engineering frontend-and-mobile

Client Hints are HTTP request header fields that a server can request from a client in order to get information about the client’s device, network, etc. The server can then determine the type of data to be displayed according to the client information received. By Hargunbeer Singh.

A server must tell the client that it supports client hints, this can be done using the Accept-CH header. When a client that supports client hints receives the Accept-CH header it can append client hint headers that match the advertised field-values to subsequent requests.

Client hints can also be specified in HTML using the <meta> tag with http-equiv attribute.

The client hints which determine which resources are sent in responses should be included in the Vary header. This ensures that a different resource is cached for every different value of the hint header. Usually, client hints like Width and DPR are specified in the Vary header. Interesting read!

[Read More]