This guide demonstrates how to set up layer 4 load balancing across multiple AKS clusters using Azure Fleet Manager. We’ll create two AKS clusters in different regions (East US and West US), configure Virtual Network (VNet) peering between them, and deploy a demo application using Fleet Manager. The process covers AKS cluster setup, VNet peering, Fleet Manager configuration, and application deployment across regions.
While AKS does NOT provide access to the cluster's managed control plane, it does provide access to the control plane component logs via diagnostic settings. The easiest option to persist and search this data is to send it directly to Azure Log Analytics, however there is a LOT of data in those logs, which makes it cost prohibitive in Log Analytics. Alternatively, you can send all the data to an Azure Storage Account, but then searching and alerting can be challenging.
To address the above challenge, one option is to stream the data to Azure Event Hub, which then gives you the option to use Azure Stream Analytics to filter out events that you deem important and then just store the rest in cheaper storage (ex. Azure Storage) for potential future diagnostic needs.
The Kubernetes AI Toolchain Operator, also known as Project KAITO, is an open-source solution to simplify the deployment of inference models in a Kubernetes cluster. In particular, the focus is on simplifying the operation of the most popular models available (ex. Falcon, Mistral and Llama2).
KAITO provides operators to manage validation of the requested model against the requested nodepool hardware, deployment of the nodepool and the deployment of the model itself along with a REST endpoint to reach the model.
When working with secrets in an application running in Kubernetes, you can use native Kubernetes secrets, however there are limitations in the security of those secrets. A better practice is to use a secure vault, like Azure Key Vault.
Azure Key Vault can be accessed via a direct SDK call, as demonstrated in our previous Workload Identity post. However, in some cases you may not have the option to use the SDK, like in cases where you dont have access to source code. In those cases you may prefer to load secrets directly into an environment variable or a file. In these cases, the Azure Key Vault CSI driver is here to save the day.
A network virtual appliance (NVA) is a virtual machine that performs network functions such as firewalling. In this post, I will walk you through how to use a Linux VM as an NVA in Azure and route traffic to an endpoint running on Azure Kubernetes Service (AKS). I will cover two scenarios: one where the traffic goes through an internal load balancer, and another where the traffic goes directly to a pod.
This assumes you already have a Linux VM and an AKS Cluster created - both in their own VNET. The cluster and the Linux VM need to be VNET peered already. Here is a quick start on VNET peering if you need a refresher.
The following walkthrough shows how you can using Azure Workload Identity with the AKS Workload Identity add-on along with MSAL to access an Azure Blob Storage Account.
The following walk-through shows how to use the External DNS project to monitor for services with DNS annotation to automatically create DNS records in an Azure Private Zone. The system user authorized to update the private zone will be an Azure Managed Identity, using AKS Workload Identity support.
Right after Kubernetes 1.21, the way Capabilities(7) worked in Kubernetes changed. At that version, a change in the upstream code enforced that Capabilities will only work when runAsUser is set to 0 - meaning root. This is somewhat counterintuitive to what most of us would expect but code goes into the reasoning and how to work with Capabilities after 1.21.
This is part one of a two part post on container image signing and runtime verification. In this post, we'll walk through the notation project and its ability to sign container images, using the Notary project specification. In the next post, we'll walk through setting up gatekeeper and ratify to perform policy based runtime verification of images.
NOTE: The manifest files used in this post can be found here
In the prior post, we ran through using the notation cli tool to sign images in Azure Container Registry. If you havent gone through that post, I recommend you start there at Part 1 - Image Signing with Notation