GenAI Platform Engineer
Role summary
This GenAI Platform Engineer role is crucial for operationalizing large language models (LLMs) and retrieval-augmented generation (RAG) systems within production environments supporting critical US federal missions. The engineer will ensure AI services are reliable, compliant with strict standards, and cost-effective, directly impacting mission success by maintaining high levels of latency, safety, and data grounding. Responsibilities include deploying and maintaining AI services, implementing guardrails, defining and monitoring SLIs/SLOs, leading incident response, optimizing system performance, managing costs with FinOps principles, and developing reusable platform components like SDKs and IaC modules. Proficiency in Python, production system management, and vector search technologies is required.
- About Our Client:
This organization supports the US federal government by strengthening national security and improving public services across defense, national security, public safety, civilian, and military health sectors. With a workforce of over 13,000 professionals, the organization focuses on transforming technology and innovation into operational solutions that enhance government missions. It operates within a larger global technology company and is recognized for fostering a collaborative and supportive community that values employee growth and development. The organization emphasizes delivering reliable, private, and safe technology applications for confidential federal programs, prioritizing impact and operational excellence.
- About the Opportunity:
The GenAI Platform Engineer role focuses on operationalizing large language models and retrieval-augmented generation (RAG) systems into production environments that serve critical federal missions. The position is responsible for ensuring AI services are reliable, governed by strict compliance standards, and cost-effective. This role directly impacts mission success by maintaining high standards for latency, safety, and data grounding while supporting multiple teams through reusable platform components. The engineer will improve system observability, performance, and security to enable seamless AI integration in sensitive and secure environments.
- Responsibilities:
• Deploy and maintain AI services with minimal hallucination errors
• Implement guardrails including prompt/version management, policy filtering, and role-based access
• Define and monitor service level indicators and objectives for quality, latency, safety, and cost
• Lead incident response, on-call duties, and postmortem analysis to reduce downtime
• Optimize system performance using caching, batching, and autoscaling techniques
• Track and manage usage and costs applying FinOps principles
• Develop reusable SDKs, CI/CD templates, and infrastructure as code modules
• Apply information retrieval metrics and queueing theory to improve system reliability and evaluation
- Requirements:
• Experience managing production systems end-to-end in integration, deployment, and observability
• Proficiency in Python programming
• Knowledge of retrieval/vector search technologies such as pgvector, Milvus, or OpenSearch
• Ability to ground AI responses in enterprise data securely and reliably
• Strong understanding of SLIs, SLOs, and data-driven reliability improvements
• Effective communication skills across engineering, product management, and security teams
• US Citizenship required
Nice to Have:
• Experience with cloud AI services or on-prem inference stacks
• Background in AI evaluation, safety testing, or A/B experimentation
• Knowledge of FinOps for AI usage and budget management
• Experience in regulated environments with FedRAMP-like controls and ATO processes
• Contributions to frameworks or open-source tools and mentoring experience
• Advanced degree preferred but not required
- Pay Range and Compensation Package:
• The pay range for certain US states and cities is $100,200—$203,400 USD
• Compensation varies by location, role, skills, and experience
Equal Opportunity Statement: Our client is an equal opportunity employer. They celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, or national origin.
Note:
RemoteHunter is not the Employer of Record (EOR) for this role. Our purpose in this opportunity is to connect exceptional candidates with leading employers. We help job seekers worldwide discover roles that match their goals and guide them to complete their full application directly through the hiring company’s career page or ATS.
