Welcome to curated list of handpicked free online resources related to IT, cloud, Big Data, programming languages, Devops. Fresh news and community maintained list of links updated daily. Like what you see? [ Join our newsletter ]

How we shipped PostgreSQL 14 on Azure within one day of its release

Categories

Tags devops how-to learning azure database cloud

In this blog post, you’ll first get a glimpse into some of our favorite features in Postgres 14. These include connection scaling, faster VACUUM, and improvements to crash recovery times. By Ozgun Erdogan.

With each new PostgreSQL release, there can be breaking changes with any of the above integration points. The process of making extensions compatible with Postgres versions is incorporating changes to these integration points. For example, with PostgreSQL 14, the utility hook’s signature changed to include a new argument. So, we had to incorporate this change, as shown below. You can also read the complete set of changes for Postgres 14 integration in this pull request.

The article the reads about:

  • Favorite new features in PostgreSQL 14
  • Making Citus & other extensions compatible with PostgreSQL 14
  • Hyperscale (Citus) – Releasing a new PostgreSQL version

The first best practice is the separation of responsibilities between Hyperscale (Citus)’s control plane and data plane. In our architecture, the control plane is responsible for the business logic for managing Postgres/Citus databases. This logic includes periodic health checks, high availability and failover, backup and restore, read replicas, regular maintenance operations, and others. The data plane is solely responsible for running the database. As such, the data plane contains almost nothing else other than stock PostgreSQL and its extensions.

In summary, authors were excited to announce general availability (GA) for Postgres 14 on Azure, within one day of the official Postgres 14 release. Good read!

[Read More]

12 critical resources to help you learn DevSecOps

Categories

Tags learning devops infosec containers

Whether you are a seasoned DevOps engineer who wants to branch out to DevSecOps, or you’re just starting your career and defining your direction, there has perhaps never been a better time to enter the DevSecOps career field or learn about cloud native security. By Maor Goldberg.

Indeed, as Gartner notes, there was already a shortage of skilled cybersecurity professionals — a field that includes DevSecOps — as of 2019, and “the global pandemic has further escalated this situation.” That’s due especially to the surge in cybersecurity attacks during 2020 and into 2021, which has made companies more eager than ever to hire engineers who understand DevSecOps and cloud native security.

The article then reads about:

  • How do you learn DevSecOps?
  • Learning resources for DevSecOps and cloud native security
    • Book: Cloud Native Security
    • Book: Hands-On Security in DevOps
    • Book: The DevOps Handbook
    • Book: Container Security
    • Book: Securing DevOps
    • Podcast: “The DevSecOps Talks”
    • Courses: KubeAcademy

… and more. There are many ways to learn DevSecOps and cloud native security. Whether you prefer books, courses, podcasts or a combination thereof, you’ll find a DevSecOps learning resource tailored to your preferences. Good read!

[Read More]

How to handle data duplication in data-heavy Kubernetes environments

Categories

Tags data-science devops how-to learning big-data

It’s convenient to create a copy of your application with a copy of its state for each team. For example, you might want a separate database copy to test some significant schema changes or develop other disruptive operations like bulk insert/delete/update… By Augustinas Stirbis.

Duplicating data takes a lot of time. That’s because you need first to download all the data from a source block storage provider to compute and then send it back to a storage provider again. There’s a lot of network traffic and CPU/RAM used in this process. Hardware acceleration by offloading certain expensive operations to dedicated hardware is always a huge performance boost. It reduces the time required to complete an operation by orders of magnitude.

The article then covers:

  • Volume Snapshots to the rescue
  • Solution? Creating a Golden Snapshot externally
  • High-level plan for preparing the Golden Snapshot
  • High-level plan for cloning data for each team
    • Step 1: Identify disk
    • Step 2: Prepare your golden source
    • Step 3: Get your Disk Snapshot ID
    • Step 4: Create a development environment for each team

You will also get a loads of screenshots and config yaml files to go with this article. At the end of the tutorial you have Golden Snapshot, which is immutable data. Each team will get a copy of this data, and team members can modify it as they see fit, given that a new EBS/persistent disk will be created for each team. Good read!

[Read More]

How to deploy a machine learning model with FastAPI, Docker and Github Actions

Categories

Tags machine-learning learning big-data devops agile

You’ve just trained a model and you’re happy with it because it performs well on your local cross-validation. Now is the time to put this model in production so that other teams within your organization can consume it and embed it in their applications. By Ahmed Besbes.

The tutorial also covers:

  • Introduction to production machine learning and APIs
  • A quick overview of FastAPI features
  • Using FastAPI and SpaCy to build an inference API
  • Packaging the API with Docker and docker-compose
  • Deploying the API to AWS EC2 and automating the process with a Github Actions CI/CD

Broadly speaking and without going into many details, putting a model in production is a process in which a model is integrated into an existing IT environment and made available to other teams to use and consume. You will also get links to further resources and reading. Good read!

[Read More]

A new language for digital transformation

Categories

Tags management machine-learning learning software cio agile

Organizations still struggle to advance digital transformation. A common language that transcends technology could be the key to strategic transformation. By Rich Nanda, Ragu Gurumurthy, Sam Roddick, Deborah Golden.

From disruptors and disruptive tech to pandemics, political unrest, and climate change, winning the future depends on adaptation. To survive and thrive, leaders should determine how to maintain a competitive advantage and enable an ability to win in a way that doesn’t just withstand change but embraces it to generate new strategic possibilities.

The article then dives into:

  • Framing the digital transformation conversation
  • Five imperatives to drive digital transformation
  • Putting the imperatives to work: Align strategy and drive transformation
  • Change, compete, win: Getting to value with the five imperatives
  • A language for today’s transformation and tomorrow’s

Our digital imperatives can enable organizations to drive transformations that align to their overarching ambition while remaining open to future strategy changes. They acknowledge the importance of AI, cloud, and cybersecurity today but leave room to evolve toward “horizon next” technologies, avoiding the trap of leaping at every shiny new technology. Ultimately, they help design-adaptive business processes and technology architectures (modular “capability stacks”) that embrace constant change and reconfiguration in the face of ongoing disruption and risk with the goal of compatibility for multiple possible futures. Excellent read!

[Read More]

How to monitor packet loss and latency in the cloud

Categories

Tags monitoring software how-to devops cloud

NetOps teams have quickly learned the benefits of hosting applications in the cloud. But before they migrated or adopted a few SaaS applications, they knew in the back of their minds that monitoring performance would be difficult. A tiny voice was asking, “How will we monitor packet loss and connection latency, hop-by-hop, when using cloud applications?”. By Kevin Woods, Head of Product Marketing.

The article will walk you through the following:

  • Packet loss causes problems with cloud applications
  • What causes packet loss?
  • Detecting packet loss
    • Using ping on the command prompt
    • By polling all the SNMP devices on the network
    • Packet capture
    • TCP traceroute
  • How to monitor packet loss and latency in the cloud
  • Establishing baselines and setting thresholds for packet loss and latency

Typically, poor connections at the physical layer, such as bad cables or connectors, are the problem. Congestion in the form of high-connection utilization or an overworked router in the path is another common source of trouble. Good read!

[Read More]

Authenticated boot and disk encryption on Linux

Categories

Tags linux software how-to infosec

Linux has been supporting Full Disk Encryption (FDE) and technologies such as UEFI SecureBoot and TPMs for a long time. However, the way they are set up by most distributions is not as secure as they should be, and in some ways quite frankly weird. In fact, right now, your data is probably more secure if stored on current ChromeOS, Android, Windows or MacOS devices, than it is on typical Linux distributions. By Pid Eins.

The article content covers:

  • The strange state of authenticated boot and disk encryption on generic Linux distributions
  • The basic technologies
    • LUKS/dm-crypt/cryptsetup
    • UEFI SecureBoot
    • TPMs
  • How Linux distributions use these technologies
  • Attack scenarios
  • Are we safe?
  • Can we do better?
  • In detail

The most basic attack scenario to focus on is probably that you want to be reasonably sure that if someone steals your laptop that contains all your data then this data remains confidential. Because distributions set up disk encryption the way they do, and only bind it to a user password, an attacker can easily duplicate the disk, and then attempt to brute force your password.

Article also points reader in the direction how we can do full disk encryption better on general Linux distro. Every single component of the boot process and OS needs to be authenticated, i.e. all of shim (done), boot loader (done), kernel (done), initrd (missing so far), OS binary resources (missing so far), OS configuration and state (missing so far), the user’s home (missing so far). Very interesting read.

[Read More]

How containment in React can improve your code

Categories

Tags react javascript web-development app-development

Component composition is undoubtedly where React shines. Being able to freely compose larger views from smaller, flexible building blocks is great. But you have to be careful because it’s easy to get it wrong (or at least suboptimal). By Tomasz Gil.

Containment is a concept where components don’t know their children ahead of time. They are simply containers for other elements. This is especially common for components like Sidebar or Dialog that represent generic “boxes”.

The article then dives into example component code and explains:

  • Blurred responsibility
  • Longer path to understanding the code
  • Poor performance
  • Hard to extend and reuse
  • What’s containment in React

Simply by moving the component up we solved all problems we’ve identified - made the code easier to reason about, more flexible, and performant. Composition is a really powerful mechanism, but you have to be cautious to do it right. Good read!

[Read More]

Building your own Data Science infrastructure for Deep Learning

Categories

Tags big-data learning data-science miscellaneous how-to

Build your own machine and install KNIME, Jupyter-Notebook and Tableau to be fully equipped for all data science and deep learning tasks. By Dennis Ganzaroli, Data Scientist and Head of Report & Data-Management in a big Telco in Switzerland.

Do you want to get started with data science but lack the appropriate infrastructure or are you already a professional but still have knowledge gaps in deep learning? Then you have two options: Rent a virtual machine from a cloud provider like Amazon, Microsoft Azure, Google Cloud or similar. Or build your own physical machine and install the right software.

The author tried both options but in the end the decision to build his own rig was the better one, and these are the reasons why:

  • Costs savings
  • More power and resources
  • Your machine can also be used for other tasks

A study from Bizon-Tech shows that a pre-build using 1 GPU is up to 10 times cheaper and those with 4 GPUs are up to 21 times cheaper within 1 year compared to web-based services. And when it comes to storage capacity, prices for web services go through the roof above a certain size.

The article also covers:

  • Choosing the right system and software
  • Building the machine
  • Installation of the software

We should strongly go for a NVIDIA graphics card since all current state of the art frameworks (be it Keras, TensorFlow, PyTorch or any other library) fully support NVIDIA’s CUDA SDK, a software library for interfacing to GPUs. Another important point are Tensor Cores. Tensor Cores accelerate matrix operations, which are foundational to deep learning, and perform mixed-precision matrix multiply and accumulate calculations in a single operation. Excellent source of information for personal computing builders!

[Read More]

Understanding enterprise messaging APIs and protocols

Categories

Tags apis ibm devops queues

In this article we’ll explore the world of messaging APIs and protocols. We’ll identify why developers depend on them and focus on the capabilities of some of the most commonly used messaging APIs and protocols. By Callum Jackson, Richard J. Coppen.

A protocol is just a set of rules that computers follow, allowing them to interact with each other, and with the outside world, in some predictable, deterministic way. At the lowest level, multiple protocols are being employed right now to transmit this text to your browser. Most developers never need to worry about this low-level stuff, and most of the time it is completely transparent to the end user. This is only because both ends have agreed – or maybe assumed – that a particular convention is being followed to share data. As in this case, your browser was expecting HTTP or HTTPS protocol over a TCP/IP socket to deliver some HTML-encoded data that creates some human-readable web content.

The article also mentions:

  • Enterprise messaging
  • Enterprise messaging protocols
  • Choosing your enterprise messaging APIs and protocols
    • Proprietary Protocols
    • MQTT
    • AMQP
    • STOMP

In this article we explained the difference between messaging APIs, protocols, and clients. Easy read!

[Read More]