High Trail logo
High Trail Verified
Financial Services, Investment Management, Hedge Fund

Staff AI Engineer

United StatesOnsiteFull TimeStaff$180,000–$220,000 /yrPosted 2 months ago

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Role summary

Our client seeks a Staff AI Engineer to lead the design and evolution of AI agents, with a strong emphasis on voice systems. This role involves building and operating AI agents reliably in production, owning agent behavior end-to-end from orchestration and tool use to evaluation and failure handling. You will architect complex, non-deterministic systems for consistency, observability, and scalability, while also defining best practices and mentoring engineers. The ideal candidate has 7+ years of software engineering experience, with a proven track record in production AI agent development and operation, including expertise in orchestration, tool use, control flow, LLM behavior, and evaluation systems.

About the Role

Our client is hiring a Staff AI Engineer to lead the design and evolution of AI agents, with a focus on voice systems.

This role is centered on building and operating AI agents that perform reliably in real-world environments. You’ll own agent behavior end-to-end — from orchestration and tool use to latency, evaluation, and failure handling.

You’ll be working on complex, non-deterministic systems, designing the architecture needed to make them consistent, observable, and scalable in production.

What You’ll Do

  • Design and own AI agent architecture (orchestration, tool use, control flow)
  • Build systems that enable agents to operate reliably across real-world interactions
  • Develop and improve evaluation frameworks for agent quality and behavior
  • Debug and resolve complex production issues (edge cases, failures, regressions)
  • Optimize performance across the stack (latency, cost, reliability)
  • Define best practices for building and operating AI agents
  • Mentor engineers and elevate AI systems design across the team

What We’re Looking For

  • 7+ years of software engineering experience
  • Experience building and operating AI agents or conversational systems in production
  • Agent orchestration and tool use
  • Multi-step workflows and control flow
  • LLM behavior in real-world settings
  • Experience designing evaluation systems for AI quality
  • Strong debugging skills across complex, non-deterministic systems

Nice to Have

  • Experience with voice systems (speech-to-text, text-to-speech, real-time pipelines)
  • Experience building customer support agents or similar systems
  • Experience optimizing low-latency / real-time AI systems
  • Experience working across multiple model providers or inference stacks
Ready to apply?
You'll be redirected to High Trail's application page.

Similar roles