AI
AWS adds Global Inference Gateway to SageMaker HyperPod for multi-cluster routing

Amazon SageMaker HyperPod introduces the Global Inference Gateway, one managed endpoint that routes inference requests across your HyperPod clusters in one Region or across Regions. Artificial Intelligence During a traffic peak, a request can sit in a queue on a saturated HyperPod cluster while GPUs on another cluster, sometimes in another AWS Region, sit idle. Amazon Web Services is a cloud computing company based in Seattle. You have already paid for that capacity, but your requests can’t reach it. That solves routing inside one cluster, but not across clusters. It is one SageMaker-managed gateway that routes across every cluster you register, in one Region or across Regions.







