Introducing New Ray Capabilities on SageMaker HyperPod

Amazon SageMaker HyperPod has introduced new capabilities that integrate Ray, an open-source framework for distributed Python workloads, with purpose-built infrastructure for large-scale machine learning on Amazon EKS. This integration simplifies the management of Ray clusters, allowing data scientists to create, monitor, and connect to clusters directly from SageMaker Studio without the need for manual Kubernetes configuration.

The new features include the ability to open the Ray Dashboard and observability dashboards through Amazon Managed Grafana, submit distributed jobs remotely, and attach Ray clusters to JupyterLab or Code Editor workspaces for interactive development. These improvements are designed to streamline workflows, reduce setup complexity, and enhance operational visibility.

Furthermore, SageMaker HyperPod enhances training resilience by providing automatic node recovery, hung job detection, and tiered checkpointing, which together minimize downtime and speed up recovery from hardware failures or stalled jobs. Ray Serve workloads for model deployment and inference are also supported, with features like tiered KV cache to reduce latency for long-context requests.

All these capabilities are compatible with open-source KubeRay and standard Ray APIs, ensuring existing scripts and workflows can be used without modification. The integration is available across all AWS regions where SageMaker HyperPod EKS is supported, and detailed setup instructions are provided in the official documentation.