4 posts tagged with "Ray"

Running AI workloads on Kubernetes often requires platform teams to assemble and maintain multiple open-source projects, custom scripts, and operational tools. This includes the glue between those components: submission scripts, queue wrappers, health checks, and result retrieval. TauGrid brings these pieces together into a single platform, so platform teams operate one stack instead of many, and researchers submit workloads without learning Kubernetes.

We are delighted to announce the open-source release of TauGrid. TauGrid combines the tau CLI, Kueue queueing, KubeRay orchestration, GPU-node health monitoring, and observability in a single Kubernetes-native stack. It supports workflows spanning data preparation, distributed training, fine-tuning, and inference.

Instead of building and maintaining these components and the integration between them separately, TauGrid provides a single Helm install with clear ownership boundaries. Platform teams get a unified way to operate the stack, while researchers get one CLI to submit workloads, observe their progress, recover from failures, and retrieve results without interacting with Kubernetes directly.

The project is MIT-licensed and available at github.com/Azure/taugrid, with container images and Helm charts published to Microsoft Container Registry. Full documentation is available at azure.github.io/taugrid.

A few weeks ago at Microsoft Build, the public preview of Anyscale on Azure was announced. If you're not familiar, Anyscale on Azure is a managed platform for running distributed AI/ML workloads with Ray on AKS. It's an Azure Native integration, a co-engineered effort between Anyscale and Microsoft, that deploys an operator onto your AKS cluster and integrates with Microsoft Entra ID for single sign-on (SSO).

The official quickstart walks you through deploying Anyscale on Azure using the Azure CLI and Azure Portal with Envoy Gateway for ingress. That's a solid starting point, but if you're like me, you want everything in Terraform so your infrastructure is repeatable, version-controlled, and easy to tear down.

In this post, we'll walk through a Terraform configuration that deploys the full Anyscale on Azure stack, swapping out Envoy Gateway for AKS managed Gateway API with Istio-based app routing. With this approach, AKS handles the gateway lifecycle and you skip the step of installing and configuring Envoy Gateway entirely.

This post focuses on running Anyscale's managed Ray service on AKS, using the Anyscale Runtime (formerly RayTurbo) for an optimized Ray experience. For open-source Ray on AKS, see our Ray on AKS overview.

Ray is an open-source distributed compute framework for scaling Python and AI workloads from a laptop to clusters with thousands of nodes. Anyscale provides a managed ML/AI platform and an optimized Ray runtime with better scalability, observability, and operability than running open-source KubeRay—including intelligent autoscaling, enhanced monitoring, and fault-tolerant training.

As part of Microsoft and Anyscale's strategic collaboration to deliver distributed AI/ML Azure-native computing at scale, we've been working closely with Anyscale to enhance the production-readiness of Ray workloads on Azure Kubernetes Service (AKS) in three critical areas:

• Elastic scalability through multi-cluster multi-region capacity aggregation
• Data persistence with unified storage across ML/AI development and operation lifecycle
• Operational simplicity through automated credential management with service principal

Whether you're fine-tuning models with DeepSpeed or LLaMA-Factory or deploying inference endpoints for LLMs ranging from small to large-scale reasoning models, Anyscale on AKS delivers a production-grade ML/AI platform that scales with your needs.

We've released new guidance for running Ray on AKS!