AI Infra Engineer – SRE (Kubernetes)

Job Category:
Software Engineering
Job Type:
Full Time
Job Location:
Hybrid Remote
About The Role
We are a fast-growing AI infrastructure company building cutting-edge GPU cloud platforms and high-performance inference solutions that empower AI developers, startups, and enterprises worldwide. As we scale our global operations, we are looking for a skilled and hands-on
AI Infra Engineer – SRE (Kubernetes)
to join our Global Infrastructure team.
Role Overview
This is a critical hands-on position focused on the reliability, performance, and operational excellence of large-scale, high-performance AI/ML GPU clusters in our data centers. As an AI Infra Engineer – SRE (Kubernetes), you will design, operate, and optimize Kubernetes-based infrastructure to ensure maximum uptime, efficiency, and scalability for demanding AI workloads. You will bring deep expertise in system-level troubleshooting, GPU cluster management, and automation to keep our platforms running at peak performance.
Key Responsibilities

• Design, build, and maintain scalable, production-grade AI/ML infrastructure using Kubernetes.
• Proactively monitor GPU cluster health, performance, and utilization across compute, accelerators, storage, and networking layers, performing root-cause analysis and resolution.
• Develop and implement automation for infrastructure provisioning, configuration, and ongoing management.
• Own the complete GPU node lifecycle — including provisioning, dynamic scaling, maintenance, decommissioning, and zero-downtime upgrades of GPU-enabled nodes in Kubernetes environments.
• Build and improve CI/CD pipelines for reliable infrastructure deployment and orchestration.
• Enforce security best practices, compliance standards, and operational excellence across the infrastructure stack.
• Lead incident response and post-incident improvements for issues related to GPUs, CPUs, high-speed storage, and networks.
• Manage end-to-end customer GPU resource provisioning — from request intake and configuration to onboarding, troubleshooting, and support — ensuring high levels of customer satisfaction.
• Stay up to date with the latest GPU hardware, software, and orchestration technologies, integrating relevant advancements into our platforms.
• Be available for occasional regional or international travel to data center locations as required.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related technical field.
• 3+ years of practical experience in data center operations, infrastructure engineering, or site reliability engineering.
• Strong background in infrastructure automation using tools such as Terraform and Ansible.
• Deep hands-on experience with Kubernetes in large-scale environments, including:

• NVIDIA GPU Operator for GPU driver management, device plugins, container toolkit, and monitoring (DCGM).
• NVIDIA Network Operator for high-performance networking, RDMA, and GPUDirect support.
• CNI (Container Network Interface) and CSI (Container Storage Interface) plugins tailored for AI/ML workloads.
• Integration with job schedulers such as Slurm in Kubernetes clusters.

• Proficiency in Linux system administration and scripting (Python, Bash).
• Experience with observability stacks including Prometheus, Grafana, and Loki.
• Solid understanding of GPU architecture, NVIDIA CUDA, NCCL, and AI/ML frameworks is a strong plus.
• Excellent troubleshooting skills with the ability to analyze complex system logs and performance metrics.
• Strong communication and collaboration skills to work effectively with engineering and operations teams.