WORLDTECH NEWS Global technology intelligence.Contact

Using NVIDIA Run:ai GPU Fractioning on Amazon EKS

Top view of NVIDIA GTX 1080 and RTX 2080 graphics cards used in advanced computer setups.
Illustrative photo.Photo by Nana Dua on Pexels

What happened

NVIDIA Run:ai GPU (a chip built for many calculations at once, used for graphics and AI) Fractioning builds on MPS with Kubernetes-integrated scheduling, per-Pod memory contracts, enforcement, and metrics. NVIDIA Run:ai GPU Fractioning adds Kubernetes scheduling integration, enforced GPU-memory limits, and per-Pod metrics to software-based sharing. NVIDIA is a semiconductor company based in Santa Clara, and its products and services include graphics processing units and software.

Learn how to run two memory-bounded Kubernetes Pods on a single NVIDIA A10G GPU using open-source NVIDIA Run:ai GPU Fractioning on Amazon EKS. This walkthrough builds a custom Amazon Linux 2023 image, installs KAI Scheduler and the GPU Fractioning Operator, and validates per-Pod GPU-memory contracts with enforcement and metrics.

GPU sharing is not new: Multi-Instance GPU (MIG), time-slicing, and Multi-Process Service (MPS) address different isolation and utilization needs. NVIDIA Run:ai is AI workload orchestration software for Kubernetes.

Its GPU Fractioning Operator can help several teams share one or two GPUs on Amazon Elastic Kubernetes Service (Amazon EKS) . Artificial Intelligence GPU sharing is not new: Multi-Instance GPU (MIG), time-slicing, and Multi-Process Service (MPS) address different isolation and utilization needs.

Notebooks, model evaluations, and small inference (running a trained model to get an answer, rather than training it) services can leave teams waiting even when GPU memory remains available. In this walkthrough’s baseline, the NVIDIA device plugin exposes the single GPU as one nvidia.com/gpu resource, with GPU sharing disabled.

Two Pods each requesting nvidia.com/gpu: 1 cannot run concurrently on that node, even when their combined GPU-memory demand fits. The approach applies to NVIDIA GPUs that meet the GPU Fractioning Operator’s hardware and software requirements, including GPUs without MIG.

Sources & evidence