CodeIsGo.com

A good few years back, I looked into building a Scala webapp using Java’s little-known provided HTTP server rather than one of the many frameworks already out there during my 10% time at Springer Nature. I made a mistake here - to avoid those frameworks I built my own framework to route requests within a webapp. I called it Sommelier, and it is now hidden away in a private github repo as really, we don’t need another of those. Maybe we don’t need any of them at all. By Jim Kinsey.

Building a Scala app with the intention of not using any of those frameworks with their obscure syntax and unnecessary concepts and then going on to build it with SBT would be kind of perverse. I could use Li Haoyi’s excellent Mill, and I have been doing both in professional and personal projects, but given the nature of this project I wanted to see what I could do without.

The article main points:

• Building a Scala app without a build tool
• Testing a Scala app without a test framework
• Handling HTTP in a Scala app without a web framework
• Frontend in a Scala webapp without a framework
• Database access in a Scala app without an ORM / DB framework
• Dependency injection in a Scala app without a DI framework
• Performance of a frameworkless Scala app

Interesting observations were found in performance space. The average cold build time for the “unframeworkless” version using SBT was 178.86s, over 3 minutes, compared to 7.08s frameworkless. Understanding the kinds of problems the frameworks and libraries we use every day are designed to solve, and the challenges faced in solving them, should help us understand how to work with them better ourselves. The potential causes of unexpected behaviour become clearer - and by understanding these fundamentals without reference to any framework, the learnings can be applied across frameworks. It helps to understand the fundamentals. Good read!

Successful data-driven companies like Uber, Facebook and Amazon rely on real-time analytics. Personalizing customer experiences for e-commerce, managing fleets and supply chains, and automating internal operations all require instant insights on the freshest data. By Dhruba Borthakur.

One of the technologies I founded was open source RocksDB, the high-performance key-value engine used by MySQL, Apache Kafka and CockroachDB. RocksDB’s data format is a mutable data format, which means that you can update, overwrite or delete individual fields in a record. It’s also the embedded storage engine at Rockset, a real-time analytics database I founded with fully mutable indexes.

The article contains good information on:

• Differences between mutable and immutable data
• The historic usefulness of immutability
• The problems with immutable data
• Mutability aids machine learning
• How mutability enables real-time analytics

To sum up, mutability is key for today’s real-time analytics because event streams can be incomplete or out of order. When that happens, a database will need to correct and backfill missing and erroneous data. To ensure high performance, low cost, error-free queries and developer efficiency, your database must support mutability.

More recently I have been experimenting with different monetization strategies for the apps so users can unlock features or just leave a tip. For the longest time I kept things simple with a “non-consumable” one-time purchase. Over the holiday break I decided to take things a step further and dip my toes into the world of subscriptions. By James Montemagno.

One of my goals of this was to not use any backend servers or accounts at all as I didn’t want to add additional costs to management. That leads to even more complexity, but for me it was worth it, so let’s get into it!

The article provides details on the following:

• Setting up subscriptions
• Purchasing subscriptions
• Handling subscription renewals and cancellations
• Verifying & restoring a subscription
• Receipt validation
• Testing subscriptions
• In-App Subscriptions SaaS

In-app subscriptions are pretty neat as they offer a lot of flexibility over a single IAP or up-front app purchase. If you have great features and want to help your long-term development it is a great option to generate revenue. Nice one!

However, even though network automation has become increasingly popular, most organizations are still managing their network infrastructure manually by a CLI or GUI. Why is this? By Sean Cavanaugh.

This manual CLI work often means that network engineers are reactive and constantly drowning with break-fix network issues because of manual mis-configurations, or the inability to implement change quickly and efficiently.

Because network engineers are so busy firefighting in their day job, they don’t have time to look at a new activity like automating, even though automation will save them time and money in the long run. I fundamentally believe that network automation is not an all or nothing situation. You need to adopt network automation in small increments so you can earn yourself, and your team more time. In other words, start small and think big. I put together this video to help network engineers brainstorm five great use cases for network automation. This list is not exhaustive; there are many more use-cases that you can do with network automation, but this is meant to give network administrators an idea of what is possible.

The video post contains following sections:

• Configuration backup and restore
• Infrastructure awareness
• Scoped configuration management
• Operational state validation
• Automated netops

The article also contains links to further resources and reading. Whether you are beginning your automation journey or a seasoned veteran, there are a variety of resources to enhance your automation knowledge. Nice one!

Everyone knows that using a Kubernetes Load Balancer is a challenge. Back in the day when I was responsible for network operations, the load balancer management was under my team’s responsibility too. By Alex Saroyan.

Public cloud providers offer a great deal of convenience by automatically provisioning and configuring the services needed by the application.

All major public cloud providers offer on-demand load balancer services that you use for exposing your Kubernetes applications to the public Internet. Or you are using an Ingress controller and your Ingress controller is using the on-demand load balancer for exposing it’s public TCP port towards the Internet.

In the article you will learn:

• What user experience are cloud practitioners expecting from Layer-4 load balancers?
• How can we get this cloud-like experience in a private cloud, on-prem?
• Where is the most practical place for the Layer-4 load balancer function in my architecture?
• Why are network gateways great places for Layer-4 load balancing functions?
• What are the options (suitable for Kubernetes)?
• Practical, cloud-like network gateway with native Kubernetes integration

Linux machine + optionally DPDK / SmartNIC acceleration can make a great network gateway with Layer-4 load balancer and native integration with Kubernetes. Almost any common Linux distribution has the minimum building blocks i.e. Layer-2, Layer-3, Layer-4 network functionality. Good read!

Test-driven development (TDD) is a popular software development methodology that aims to reduce defects and improve the ease of maintaining code. In this post, you learn how to apply TDD to write queries and expressions with the Fauna Query Language (FQL). By Ron Callahan.

TDD begins with writing a single failing test. For example, if you are implementing a Split() function that accepts a string and a delimiter and returns an array of strings “split” at each occurrence of the delimiter, you would test that Split(“127.0.0.1”, “.”) is equal to [“127”, “0”, “0”, “1”]. When you first run your test suite, the test fails, because you have not yet implemented the Split() function.

The tutorial reads about:

• TDD overview
• Setting up your environment
• Configuring Jest
• Testing partial FQL expressions
• Preparing your test harness
• Create a single failing test
• Implementation

To test your knowledge, try implementing tests for the remaining two specifications. Once you have those tests in place and passing, try to refactor the implementation of Split(). Notice how TDD allows you to safely refactor your code without introducing regressions. Interesting!

In this post, we start to delve into DevOps toolsets, specifically Microsoft Azure DevOps Services. This is the third in a series of blog posts about DevOps. By Ron Callahan.

Just some of the benefits of a DevOps toolset include the standardization and automation of development processes, improved collaboration within and among the teams, consolidated code repositories, work-item tracking, automated testing, and release pipelines.

Tools (or one person) will not automatically transform an IT department into a DevOps shop, but a department that has adopted the culture and organization to follow DevOps practices WILL benefit from a suite of DevOps tools as they mature.

The article then deals with:

• Azure Boards
• Azure Pipelines
• Azure Repositories
• Azure Test Plans
• Azure Artifacts

Faster development cycles allow businesses to push out new features faster, allowing them to be more agile in responding to their competition and to new requests from customers. Furthermore, a tightly integrated CI/CD platform means less downtime – and less downtime equals more revenue! Good read!

One of the advantages of using the cloud is the ability to scale rapidly. On-demand scalability can eliminate the need to overbuy capacity that is only required for peak times. By factioninc.com.

Data egress is when data leaves a network and goes to an external location. If you’re using the cloud, data egress occurs whenever your applications write data out to your network or whenever you repatriate data back to your on-premises environment. While cloud providers usually do not charge to transfer data into the cloud (“ingress”), they do charge for data egress in most situations. Another charge related to data egress fees are data transfer fees, which may be assessed when moving data between regions or availability zones within the same cloud provider.

The article pays attention to:

• What is data egress?
• Egress fees can hinder innovation
• How are egress fees calculated?
• Best practices to reduce or eliminate egress fees

Data egress fees can vary considerably. Each cloud has its own egress fee structure. Use clouds with higher egress fees for only the workloads that require the capabilities of that specific cloud. workloads that demand it. When services are comparable, always choose the less expensive option. While an older article it is a good read!

Whether you’re a data engineer, a data scientist, a software developer, or someone else working in the field of software and data - it’s very likely that you have implemented an ETL pipeline before. By Stefan Sprenger.

ETL stands for Extract, Transform, and Load. These three steps are applied to move data from one datastore to another one. First, data are extracted from a data source. Second, data are transformed in preparation for the data sink. Third, data are loaded into a data sink. Examples are moving data from a transactional database system to a data warehouse or syncing a cloud storage with an API.

The article content is split into:

• What are real-time ETL pipelines?
• What are the benefits of real-time ETL pipelines?
• How to implement real-time ETL with Apache Kafka

The open-source community provides most essentials for getting up and running. You can use open-source Kafka Connect connectors, like Debezium, for integrating Kafka with external systems, implement transformations in Kafka Streams, or even implement operations spanning multiple rows, such as joins or aggregations, with Kafka. Good read!

Monads are notorious in the programming world for their use in the Haskell programming language and for being hard to grasp. There’s even a joke that writing a “monad tutorial” is a rite of passage for new Haskellers, and been described as pointless. By Chris Done.

One of the Haskell designers3 in the 90s just came up with a class/interface that worked for all of these. As he was into category theory, he “monad”. The types also sort of match the theory if you squint hard enough.

You can re-use your intuition from existing common place chaining of things in other popular languages:

• Async chains (JS)
• Parser combinator chains (Rust, JS)
• Optional or erroneous value chains (TypeScript, Rust)
• Continuation passing style (you can do this in Lisp and JS)
• Cartesian products/SQL (C#’s LINQ)

Monad is the name of the class for “and_then”,5 defined in a sensible way, with some laws for how it should behave predictably, and then a bunch of library code works on anything that implements “and_then”. Apart from F# or Haskell (or descendants), no other language embraces the abstraction with syntax so it’s hard to find a good explanation without them. It’s like explaining Lisp macros without using a Lisp, the explanation tends to be awkward and unconvincing.

Haskellers don’t like to throw exceptions, or use mutation, and functions can’t return early, etc. Suddenly Monad and syntactic sugar for it looks pretty attractive to them. Nice one!