RiseMe
STN logo
STN Verified
Digital Media, Sports Technology, Advertising Technology

Platform Reliability (Horizon AI)

RemoteRemoteFull TimePosted 17 days ago

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Required skills

  • distributed systems
  • reliability
  • caching
  • queuing
  • circuit breaking
  • backpressure
  • timeouts
  • idempotency
  • Python
  • containerized environments

Nice to have

  • durable workflow orchestration
  • blue/green deployment
  • canary deployment at scale
  • performance profiling
  • Python services

Role summary

Horizon AI is seeking a Staff Platform Reliability Engineer to ensure the dependability and scalability of their AI platform. This role focuses on the reliability of Horizon AI's software, working closely with the Principal Engineer. Responsibilities include building caching layers, implementing event streaming, hardening agent dispatch with health checks and observability, developing platform observability (metrics, tracing, alerting), contributing to orchestration migration and infrastructure-as-code, and improving deployment practices like canary releases and traffic shifting. The ideal candidate has production experience with distributed systems reliability, Python, containerized environments, Kubernetes, event streaming, and deep observability.

Platform Reliability

Level: Staff (MTS 4) · Reports to: Chief AI Officer, later the Senior Engineering Manager

The role. Make our own platform dependable and scalable, working with the Principal Engineer. This is reliability engineering for Horizon AI’s software rather than for the GPU fleet, though the two meet when our agents are the ones being trusted with fleet actions.

You will

  • Build the caching layer that stops every agent tool call from hitting external systems directly

  • Implement event streaming so services stop depending on synchronous HTTP chains

  • Harden agent dispatch: health checks, circuit breakers, retry semantics, and observability that distinguishes failure from degradation

  • Build observability of our own platform: service-level metrics, tracing across services, and alerting we act on

  • Contribute to the orchestration migration and to infrastructure-as-code

  • Improve deployment practice: canary, traffic shifting, and real rollback

You have

  • Production experience making distributed systems reliable: caching, queuing, circuit breaking, backpressure, timeouts, idempotency

  • Comfort in Python and containerized environments, with real exposure to Kubernetes or equivalent orchestration

  • Experience with event streaming and with the failure modes each option brings - Observability depth: metrics, distributed tracing, and the judgment to alert on symptoms rather than causes

  • A record of shipping reliability improvements with measurable before and after

Nice to have: durable workflow orchestration, blue/green and canary deployment at scale, performance profiling of Python services

Ready to apply?
You'll be redirected to STN's application page.

Similar roles

See how RiseMe works

Match, tailor, and prep so you send fewer applications and land more interviews.