Artificial Intelligence

Senior Product Manager, Inference

San Francisco, CAOnsiteFull TimeSenior$250,000–$300,000 /yrPosted 2 months ago

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Required skills

Model APIs
Developer Experience
APIs
developer tools
API specifications
prototypes
AI development tools
voice applications
batch inference
multimodal APIs

Nice to have

FlashAttention
Hyena
FlexGen
RedPajama

Role summary

Together AI is seeking a Senior Product Manager to own Model APIs and Developer Experience, focusing on making their open-weight models easy to adopt and reliable for developers and agents. This hands-on role involves defining product direction for chat, multimodal, voice, and batch inference APIs, addressing gaps with proprietary providers, and enhancing ecosystem compatibility. The ideal candidate will have experience with APIs and developer tools, strong product judgment, a willingness to engage technically, and a track record of building trust and driving iteration through experimentation. The role requires close collaboration with engineering and research, customer engagement, and clear roadmap communication.

About the Role

Inference is a critical part of Together AI’s vision and growth strategy. Developers and enterprises use Together to run leading open models across coding agents, AI-native applications, enterprise workflows, and large-scale production workloads.

We’re looking for a Senior Product Manager to shape how customers consume and build on Together inference.

As inference becomes a larger part of our business, the product surface is becoming more complex. Workloads span different tenancy, capacity, performance, and modality needs; agentic applications are changing how inference is consumed; and customers increasingly expect deep infrastructure capabilities to translate into predictable, easy-to-use products.

This role owns the product layer between Together’s inference infrastructure and the developers and enterprises consuming it. You’ll help define how our portfolio of inference products fits together, where Together should differentiate, how customers move from experimentation to production, and how the developer experience evolves as workloads become more sophisticated.

This is a hands-on technical product role. You’ll use our APIs and SDKs, study real workloads, test integrations, and work closely with inference platform engineers and researchers to turn infrastructure and model capabilities into clear product decisions and customer experiences.

Responsibilities

Shape the inference product strategy. Define how Together’s portfolio should serve different customer needs across tenancy, capacity, performance, workload patterns, and modalities.

Own the developer experience around inference. Improve our APIs, SDKs, compatibility, and integrations across applications, coding agents, frameworks, and developer tools.

Build for emerging workloads and modalities. Understand how agentic, voice, multimodal, batch, and other workload patterns are changing what customers need from inference infrastructure, and translate those shifts into product direction.

Turn infrastructure and model capabilities into customer-facing products. Work closely with inference platform engineers and researchers to understand capabilities and constraints across performance, routing, caching, reliability, capacity, and model behavior, and translate them into the right abstractions, guarantees, and controls for customers.

Connect product decisions to business outcomes. Shape pricing and packaging alongside Finance and GTM, and use adoption, inference consumption, utilization, unit economics, and customer expansion to prioritize where we invest.

Stay close to customers and the product. Talk with developers and enterprise teams, inspect usage patterns, test workflows yourself, and use that context to form a clear point of view on what we should build and what we should not.

Drive products from idea to production. Partner closely with engineering, research, sales, partnerships, finance, and other teams to make tradeoffs, align around priorities, launch, and iterate.

Requirements

Strong product judgment and the ability to take an ambiguous technical problem, identify what matters, make tradeoffs, and develop a clear point of view.

Meaningful experience building or owning APIs, developer tools, infrastructure, cloud platforms, data systems, AI products, or another technically deep product.

Strong technical depth and curiosity. You can work credibly with engineers and researchers, reason about system behavior and constraints, and go deeper when the product problem requires it.

A hands-on operating style. You’re comfortable testing APIs and SDKs, using developer tools, reading technical documentation or code when useful, and building lightweight prototypes or analyses to sharpen your thinking.

Strong developer and customer empathy. You care about what a system actually feels like to integrate with and operate in production, not just whether the underlying capability exists.

A track record of making clear product decisions and shipping in fast-moving environments where the right answer is not always obvious.

Strong communication and cross-functional leadership, with the ability to earn trust across engineering, research, customers, and business teams.

Experience with LLM inference, model serving, distributed systems, GPU infrastructure, agents, or developer-facing AI products is helpful, but not required. We care most about your ability to learn technical systems quickly, get close to the product, and develop strong judgment.

About Together AI

Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.

Compensation

We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $250 - 300k + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Application Policy

Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at https://www.together.ai/privacy.

Ready to apply?

You'll be redirected to Together AI's application page.