Artificial Intelligence

Research Intern, Frontier Speculative Decoding (Winter 2027)

San Francisco, CAOnsiteInternshipJunior / Entry-level$58,000–$70,000 /yrPosted 21 days ago

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Required skills

Master's or Ph.D. degree
Electrical Engineering
Machine Learning fundamentals
Deep Learning fundamentals
training large models
fine-tuning large models
distributed training frameworks
Megatron-LM
FSDP
modern model architectures

Nice to have

speculative decoding
efficient inference
efficient machine learning
post-training
reinforcement learning
LLM serving engines

Role summary

Together AI is seeking a Research Intern for its Frontier Speculative Decoding team. This role focuses on developing and optimizing algorithms for faster large language model generation without compromising output quality. Interns will research new architectures, training methods, and adaptation strategies for speculative decoding, working with large models on GPU clusters and testing against real-world serving workloads. Responsibilities include prototyping, training, fine-tuning, experimentation, and communicating findings. The ideal candidate is pursuing a Master's or Ph.D. in a related field with strong ML/DL fundamentals and experience in distributed training frameworks.

About the Role

The Speculative Decoding team at Together AI builds the draft models and algorithms that let large language models generate faster without changing their outputs. Our speculators run in production on frontier open models such as DeepSeek, Kimi and GLM. Our work sits at the intersection of model training, inference algorithms and serving systems, and we co-design all three to lower the cost and latency of generation.

As a research intern, you will dive into open questions in speculative decoding: new drafter architectures and verification schemes, knowledge distillation and training recipes that align draft and target models, and drafters that adapt as traffic shifts. You will train and fine-tune draft models at scale on dedicated GPU clusters, and test your ideas against realistic serving workloads, including long-context and agentic traffic.

Projects aim to answer important research questions in speculative decoding, enabling faster generation on frontier models, drafters that keep pace with changing workloads, and robust, reproducible evaluation under real traffic.

Past Research

Speculative decoding research by Together researchers, interns and academic collaborators includes:

Speculative Speculative Decoding (ICLR 2026)

When RL Meets Adaptive Speculative Training: A Unified Training-Serving System (ICML 2026)

Beat the Long Tail: Distribution-Aware Speculative Decoding for RL Training (MLSys 2026)

Responsibilities

Research and prototype new efficient speculative decoding algorithms and draft model architectures

Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses.

Implement optimized version for serving engines to show efficacy in real world workloads

Communicate the plans, progress, and results of projects to the broader team

Share findings through technical write-ups and blog posts, and help bring successful ideas into production

Requirements

Currently pursuing a Master's or Ph.D. degree in Computer Science, Electrical Engineering, or a related field

Strong knowledge of Machine Learning and Deep Learning fundamentals

Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP)

Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention

Preferred Qualifications

Prior research experience in speculative decoding, efficient inference, or efficient machine learning

Publications at leading conferences in machine learning or systems (e.g., NeurIPS, ICML, ICLR, MLSys)

Experience with knowledge distillation, post-training, or reinforcement learning

Familiarity with LLM serving engines (e.g., vLLM, SGLang, TensorRT-LLM)

Contributions to open-source machine learning projects

About Together AI

Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.

Internship Program Details

Our internship program runs 12 to 14 weeks, giving you the opportunity to work alongside industry-leading engineers and researchers across multiple teams. This cohort's internship dates span January 4th to April 9th.

Compensation

We offer competitive compensation, housing stipends, and other competitive benefits. The estimated US hourly rate for this role is $58 - $70 an hour. Our hourly rates are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Application Policy

Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.

Equal Opportunity

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

For more information about our privacy policy, please visit: https://www.together.ai/privacy

Ready to apply?

You'll be redirected to Together AI's application page.