Senior Software Engineer

### Who you are
- We welcome candidates at various experience levels for this role
- During the interview process, candidates will be assessed for the appropriate level, and offers will align with that level, which may differ from the one in this posting
- Strong C++ engineer and comfortable working in both low-level environments and distributed systems design
- Experience building atop observability platforms such as Prometheus, OpenTelemetry, Grafana, ClickHouse, or similar technologies
- Solid understanding of data structures for manipulating large volumes of data
- Familiarity with SQL databases, with time-series databases a plus
- Curious about networking and communication across large clusters and comfortable reasoning from first principles while challenging industry conventions

### What the job involves
- Architect, implement, and maintain TT-Telemetry, our C++-based service for collecting and exporting device-level metrics
- Interface with internal engineering teams to build a deep understanding of Tenstorrent’s architecture and identify and surface useful metrics
- Design efficient built-in web GUIs for observing device- and cluster-level state, diagnosing problems, and monitoring utilization
- Design ingestion pipelines for industry standard telemetry systems (e.g., Prometheus)
- Help define the long-term architecture of Tenstorrent’s distributed telemetry stack
- What You Will Learn:
- How large-scale AI clusters are architected from the networking layer up
- The performance characteristics of custom AI hardware and RISC-V processors at scale
- How telemetry and observability considerations impact the design of next-gen AI accelerators
- How to design and architect a world-class telemetry and observability platform from the ground up