Welcome to curated list of handpicked free online resources related to IT, cloud, Big Data, programming languages, Devops. Fresh news and community maintained list of links updated daily. Like what you see? [ Join our newsletter ]

How to program an IoT device in Elixir using Nerves?

Categories

Tags iot elixir servers performance distributed apis

Elixir is a great choice language for IoT devices due to its fault-tolerant nature, but also because of the ability to handle a lot of concurrent I/O operations thanks to the BEAM scheduler running under the hood. By Artur Ziętkiewicz.

In the article author discusses:

  • Minimum requirements
  • Configuration
  • Setting up an Over-The-Air updating
  • Adding web interface with Phoenix
  • Other UI libraries

Nerves is a platform that comes with a set of tooling that simplifies the process of creation and maintenance of an Elixir app on IoT devices. Software built using it are already battle-tested and working on production. Nerves supports most common boards (for example, every type of Raspberry Pi), but there’s one restriction: it has to be able to run Linux, that’s because Nerves actually runs your Elixir program on a lightweight Linux distro. Nice one!

[Read More]

Technology leader insights from the PwC Pulse Survey

Categories

Tags miscellaneous cio management

There are always some unknowns in business, and right now those include how exactly the pandemic will play out, what the outlook for corporate taxes will be and when supply chain disruptions will ease. By @PwC.

The main points form the survey:

  • Technology and teaming will drive growth in 2022
  • Tax leaders and tech leaders are the new dynamic duo
  • Reinvention, cloud and data top the CIO agenda for 2022
  • AI leads innovation while emerging tech convergence drives the metaverse

AI tops the list of those they see as very important to the company’s products and services strategy (61%). Next on their list is digital identity (57%), 5G (56%) and IoT (54%). In fact, the ultimate tech convergence is reflected in the metaverse—which 46% of CIOs say will be very important to their offerings. While each of the technologies on our list is notable and can underpin a company’s innovation strategy, in reality many of these will be used in combination with one another. Good read!

[Read More]

Uber freight carrier metrics with near-real-time analytics

Categories

Tags data-science apache event-driven messaging distributed devops

Uber Freight has been around since 2016 and is dedicated to provide a platform to seamlessly connect shippers with carriers. We’re simplifying the lives of trucking companies by providing a platform for carriers to browse through all available shipment opportunities with upfront pricing and book with the tap of a button, and making the fulfillment process more scalable and efficient. By Ujwala Tulshigiri, Yeqing Lu, Ting Chen, Branden Colen.

The article describes how Uber Freight developed the Carrier Scorecard to show the carriers several metrics, including engagement with the app, on-time pickup/delivery, tracking automation, and late cancellations. By showing this information in near real time on the Carrier App, they are able to provide feedback to carriers in real time and set ourselves apart from most of our competitors in the industry:

  • Backend requirements
  • Potential solutions considered
  • Final system design
    • Kafka
    • Flink
    • Pinot
  • Data schema and challenges
  • Flink stateful stream processor
  • Hybrid Pinot table
  • Golang GRPC service
  • Impact

Apache Pinot provides rich index optimization techniques like inverted, Star-tree, JSON, sorted column and many other indexes to accelerate the query performance. For example, the Star-tree pre-aggregation index can speed up the queries to summarize the average wait time at facilities. The fast query made it an interactive experience for carriers to check out the wait time on a carrier app before they book a shipment.

Providing reliable service to shippers is critical for Uber Freight in order to gain their trust. Because carriers’ performance could significantly impact reliability of Freight’s service, we need to be transparent with carriers about the level to which we are holding them accountable, providing them with a clear view of how well they are performing and, if needed, where they can improve. Excellent read!

[Read More]

How to prevent Scala 3 compiler regressions with Community Build

Categories

Tags scala programming akka java web-development

We always assume the language compiler’s or interpreter’s impeccability when writing code in any programming language. Just as any other program, it might contain inconsistencies with its specification or subtle bugs. By Wojciech Mazur.

The article then dives into:

  • How does the compiler team test Scala?
  • Increasing the coverage of the Scala 3 Community Build
  • How does Open Community Build work?
  • Infrastructure and general flow
  • How to create a build plan
  • Execute the plan
  • Collecting the results of projects compiled with a new version like Scala 3
  • Impact of the Scala 3 Open Community Build

In author’s biew: We believe the Scala 3 compiler version should be treated as any other external dependency. We should always try to use the latest version whenever possible. The latest versions of Scala should improve performance or fix security vulnerabilities. Basically, bumping up the version shouldn’t be associated with troubleshooting, especially in terms of Scala’s minor versions. Good read!

[Read More]

GitHub CI/CD pipeline design choices for deploying Terraform code to GCP data domain

Categories

Tags devops cicd kubernetes containers

We have built our GCP data platform using Data Mesh principles. Each line-of-business (LoB) roughly translates to a domain on GCP. Each domain will have 3 environments, Dev, Non-Prod and Prod, and each environment will have 3 GCP projects/zones. By Bharath N.

Author has come up with multiple designs for CI/CD deployment pipelines:

  • Design 1: 3 environments, 3 branches and 3 code bases in GitHub
  • Design 2: 3 environments, 1 branch and 1 code base in GitHub
  • Design 3: 3 environments, 2 branches and 2 code bases in GitHub
  • Design 4: 3 environments, 1 branch and 1 code base in GitHub

Each designed solution contains explanation about advantages and downsides. Good read!

[Read More]

Why you absolutely must document your no-code software

Categories

Tags software-architecture big-data bots programming data-science devops

Here’s a conceptual framework to help keep your no-code and low-code effort simple, but comprehensive. By Joe Procopio.

The main points captured in the blog post:

  • Yes, you need to document code you didn’t write
  • Create a single, central document
  • Document before you act
  • Be verbose but direct
  • Document not just what you did, but why
  • Be judicious about what makes a release
  • Don’t rely on the platform’s versioning

Some platforms allow you to create save points and restore to previous versions like I mentioned above, but most don’t do it well, and some don’t do it at all. Make your own backup copies of text, scripts you’ve modified, settings you’ve changed, anything that would require a lot of rethinking to to re-do. Good read!

[Read More]

How to build low-code mixed-reality apps in Power Apps, AI Builder and HoloLens

Categories

Tags software-architecture big-data bots management data-science software

Off-the-shelf tools and components like Power Apps, AI Builder and HoloLens let you deliver your own industrial metaverse at a low cost. Microsoft’s low-code Power Platform is a key piece of its business software suite. By sitting between Microsoft 365’s productivity tools and the Dynamics 365 line-of-business applications, it’s a way to build out custom workflows and add your own user experiences. By Simon Bisson.

There are three ways to build Power Apps applications: using Canvas apps to drag and drop controls onto a design surface, attaching code to them, or using model-driven apps to go straight from data to code.

The article also describes:

  • Power Platform in the metaverse
  • Adding mixed reality to a Power Apps application
  • Bringing Power Apps, device sensors and AI together
  • Building the future of application development with Power Apps

Building complex mixed-reality apps like this used to be hard, requiring a lot of developer resources. By using Power Apps, businesses can take advantage of its ability to quickly connect together different applications, using device cameras and cloud services along with simple workflows. Nice one!

[Read More]

7 best UX case study generators (and how to use them)

Categories

Tags ux web-development management miscellaneous learning

If you’re looking for UX case study ideas, topics or challenges, these UX case study generators can help you create portfolio pieces and improve your UX/UI skills. Here are some of the best ones and the pros and cons of using them. By designerup.co.

The article main sections and generators mentioned:

  • UX Challenge
  • Sharpen
  • Designercize
  • UX Tools Challenges
  • Uplabs Challenges
  • 100Daysofproductdesign
  • DesignerUp
  • UX case study usage
  • Pros and cons of using UX case study generators

Keep in mind that these generated UX case study projects and prompts are not considered ‘real-world or real- work experience’ by employers, but rather self-directed projects. They are a good first step for new UX designers to start practicing their problem solving skills on their own and refining their UX design process, but you’ll have to go the extra mile if you want to use them in your portfolio to impress during interviews. Good read!

[Read More]

Hosting ASP.NET Core web api with AWS Lambda – Truly serverless REST APIs

Categories

Tags bots apis programming learning cloud devops

In this article, we will learn about hosting ASP.NET Core Web API with AWS Lambda in a rather simple-to-follow manner. It is going to be as simple as developing a .NET 6 Web API as you would normally do using Controllers or Minimal APIs, and running some CLI commands which will deploy your API as Lambda Function to AWS Lambda super fast! By Mukesh Murugan.

The article provides a deep dive into serverless with dotnet:

  • Pros & cons of hosting ASP.NET Core Web API with AWS Lambda
  • When to Host ASP.NET Core Web API with AWS Lambda?
  • Prerequisites
  • Hosting ASP.NET Core Web API with AWS Lambda
    • Creating the .NET 6 Web API project
    • Installing the required Lambda package & service registration
    • Deploying the AWS Lambda with CLI
    • Lambda function URLs
    • Testing
    • Adding environment variables to the AWS Lambda
    • AWS Lambda CLI functions
    • Storing Lambda configurations in aws-lambda-tools-defaults.json
  • DevOps Enhancement for CI/CD

Being serverless, it costs nothing when there are no incoming requests, and also it costs close to nothing even when there are requests coming in, thanks to the generous Free Tier of AWS! Good read!

[Read More]

Delta Lake and MinIO for multi-cloud data lakes

Categories

Tags cio big-data cloud database data-science

Delta Lake is an open-source storage framework that is used to build data lakes on top of object storage in a Lakehouse architecture. Delta Lake supports ACID transactions, scalable metadata handling and unified streaming and batch data processing. By Matt Sarrel.

Delta Lake was designed to combine the transactional reliability of databases with the horizontal scalability of data lakes. Delta Lake was built to support OLAP-style workloads with an ACID table storage layer over cloud native object stores such as MinIO.

The main focus of this article is on:

  • Delta Lake files
  • ACID Transactions with Delta Lake
  • Delta Lake and MinIO tutorial
  • Create a test Delta Lake table on MinIO
  • MinIO and Delta Lake for high-performance ACID transactions on data lakes

The combination of MinIO and Delta Lake enables enterprises to have a multi-cloud data lake that serves as a consolidated single source of truth. The ability to query and update Delta Lake tables provides enterprises with rich insights into their businesses and customers. Various groups access Delta Lake tables for their own analytics or machine learning initiatives, knowing that their work is secure and the data timely. Good read!

[Read More]