Build real-time voice applications with vLLM-Omni on SageMaker AI – Part 1

What happened

This Part 1 tutorial deploys Qwen3-TTS and streams speech through a Gradio application. AWS provides deployment guidance for broadly adopted serving frameworks such as vLLM and SGLang . Amazon Web Services is a cloud computing company based in Seattle.

Deploy a text-to-speech model on Amazon SageMaker AI with the AWS vLLM-Omni Deep Learning Container and stream generated speech over a persistent bidirectional connection. Voice agents, interactive learning applications, accessibility tools, and customer service assistants need to respond without long silent pauses.

In this tutorial, you deploy a text-to-speech (TTS) model on Amazon SageMaker AI that can start playing speech before it finishes generating the full response. AWS Deep Learning Containers provide Docker images with deep learning frameworks and dependencies for training and inference on AWS.

Key facts

• AWS — provides: deployment guidance for broadly adopted serving frameworks such as vLLM and SGLang