Deepgram Enhances Amazon SageMaker AI Observability with New Metrics Capabilities

Deepgram has expanded its support for self-hosted speech AI on Amazon SageMaker AI by adding two key features that enhance observability and transparency. These capabilities enable users to monitor billing, feature usage, and engine behavior without leaving their AWS environment.

The first feature, Deepgram Enhanced Metrics, provides detailed billing and usage data directly into Amazon CloudWatch. This includes metrics such as billable inference units, audio processing duration, and character counts for text-to-speech, all without requiring additional agents or permissions. These metrics help users reconcile their AWS bills with actual traffic and understand feature utilization.

The second feature involves engine-level and per-GPU visibility through Prometheus and OpenTelemetry support. Deepgram containers now serve a Prometheus metrics endpoint, and SageMaker AI's detailed observability runs an AWS-managed OpenTelemetry Collector on each instance. This setup allows for real-time, granular monitoring of GPU utilization, engine capacity, and other system metrics, accessible via PromQL queries in CloudWatch or Grafana.

Both features operate within the existing network isolation constraints of AWS Marketplace deployments, ensuring security and compliance. They are enabled by default for new endpoints and can be activated or customized through simple configuration changes.

These enhancements aim to close the gap between vendor telemetry and user monitoring, providing comprehensive insights into speech AI deployments. They support capacity planning, cost management, and feature analysis, making self-hosted AI on SageMaker more transparent and manageable.

Deepgram models, including Nova, Flux, and Aura-2, are available on SageMaker AI with a 14-day free trial. While deploying these models incurs AWS infrastructure costs, the new metrics features are provided at no additional charge, offering a more detailed and integrated monitoring experience for AI developers and operators.