3 posts tagged with "Ray"

A few weeks ago at Microsoft Build, the public preview of Anyscale on Azure was announced. If you're not familiar, Anyscale on Azure is a managed platform for running distributed AI/ML workloads with Ray on AKS. It's an Azure Native integration, a co-engineered effort between Anyscale and Microsoft, that deploys an operator onto your AKS cluster and integrates with Microsoft Entra ID for single sign-on (SSO).

The official quickstart walks you through deploying Anyscale on Azure using the Azure CLI and Azure Portal with Envoy Gateway for ingress. That's a solid starting point, but if you're like me, you want everything in Terraform so your infrastructure is repeatable, version-controlled, and easy to tear down.

In this post, we'll walk through a Terraform configuration that deploys the full Anyscale on Azure stack, swapping out Envoy Gateway for AKS managed Gateway API with Istio-based app routing. With this approach, AKS handles the gateway lifecycle and you skip the step of installing and configuring Envoy Gateway entirely.

This post focuses on running Anyscale's managed Ray service on AKS, using the Anyscale Runtime (formerly RayTurbo) for an optimized Ray experience. For open-source Ray on AKS, see our Ray on AKS overview.

Ray is an open-source distributed compute framework for scaling Python and AI workloads from a laptop to clusters with thousands of nodes. Anyscale provides a managed ML/AI platform and an optimized Ray runtime with better scalability, observability, and operability than running open-source KubeRay—including intelligent autoscaling, enhanced monitoring, and fault-tolerant training.

As part of Microsoft and Anyscale's strategic collaboration to deliver distributed AI/ML Azure-native computing at scale, we've been working closely with Anyscale to enhance the production-readiness of Ray workloads on Azure Kubernetes Service (AKS) in three critical areas:

• Elastic scalability through multi-cluster multi-region capacity aggregation
• Data persistence with unified storage across ML/AI development and operation lifecycle
• Operational simplicity through automated credential management with service principal

Whether you're fine-tuning models with DeepSpeed or LLaMA-Factory or deploying inference endpoints for LLMs ranging from small to large-scale reasoning models, Anyscale on AKS delivers a production-grade ML/AI platform that scales with your needs.

We've released new guidance for running Ray on AKS!