The Kubernetes AI Toolchain Operator, also known as Project KAITO, is an open-source solution to simplify the deployment of inference models in a Kubernetes cluster. In particular, the focus is on simplifying the operation of the most popular models available (ex. Falcon, Mistral and Llama2).

KAITO provides operators to manage validation of the requested model against the requested nodepool hardware, deployment of the nodepool and the deployment of the model itself along with a REST endpoint to reach the model.

When working with secrets in an application running in Kubernetes, you can use native Kubernetes secrets, however there are limitations in the security of those secrets. A better practice is to use a secure vault, like Azure Key Vault.

Azure Key Vault can be accessed via a direct SDK call, as demonstrated in our previous Workload Identity post. However, in some cases you may not have the option to use the SDK, like in cases where you dont have access to source code. In those cases you may prefer to load secrets directly into an environment variable or a file. In these cases, the Azure Key Vault CSI driver is here to save the day.

A network virtual appliance (NVA) is a virtual machine that performs network functions such as firewalling. In this post, I will walk you through how to use a Linux VM as an NVA in Azure and route traffic to an endpoint running on Azure Kubernetes Service (AKS). I will cover two scenarios: one where the traffic goes through an internal load balancer, and another where the traffic goes directly to a pod.

This assumes you already have a Linux VM and an AKS Cluster created - both in their own VNET. The cluster and the Linux VM need to be VNET peered already. Here is a quick start on VNET peering if you need a refresher.

The following walkthrough shows how you can using Azure Workload Identity with the AKS Workload Identity add-on along with MSAL to access an Azure Blob Storage Account.

The following walk-through shows how to use the External DNS project to monitor for services with DNS annotation to automatically create DNS records in an Azure Private Zone. The system user authorized to update the private zone will be an Azure Managed Identity, using AKS Workload Identity support.

Introduction​

Right after Kubernetes 1.21, the way Capabilities(7) worked in Kubernetes changed. At that version, a change in the upstream code enforced that Capabilities will only work when runAsUser is set to 0 - meaning root. This is somewhat counterintuitive to what most of us would expect but code goes into the reasoning and how to work with Capabilities after 1.21.

Introduction​

This is part one of a two part post on container image signing and runtime verification. In this post, we'll walk through the notation project and its ability to sign container images, using the Notary project specification. In the next post, we'll walk through setting up gatekeeper and ratify to perform policy based runtime verification of images.

Introduction​

NOTE: The manifest files used in this post can be found here

In the prior post, we ran through using the notation cli tool to sign images in Azure Container Registry. If you havent gone through that post, I recommend you start there at Part 1 - Image Signing with Notation

Introduction​

In this walkthrough we'll use the Gatekeeper project and AKS Policy to create a policy that resticts the host name on a Kubernetes Ingress. The host names used for validation will be provided via parameters on the Azure Policy assignment.

The following provides guidance on the minimum roles needed by an AKS user to get their credentials and interact with a namespace we'll create called 'sample-app'.