LiteLLM Verified
AI/ML, Software Development, Developer Tools
Backend LLM Engineer
San Francisco, California, United StatesRemoteFull TimeEntry-level (exp-based)$120,000–$180,000 /yrPosted 2 months agoHidden Gem · YC Startup
Role summary
LiteLLM, a rapidly growing open-source LLM Gateway with 34K+ GitHub stars and $6M ARR, is seeking a Backend LLM Engineer. This role focuses on unifying API formats across core LLMs like OpenAI, Gemini, and Anthropic, working directly with the CEO and CTO. Responsibilities include developing transformations for API requests, building provider-agnostic functionality, and implementing features like cost tracking and logging. The ideal candidate has 1-2 years of backend/full-stack experience with production systems, familiarity with LLM APIs (especially OpenAI), and a passion for open source. The tech stack includes Python, FastAPI, Redis, and Postgres.
### TLDR
LiteLLM is an **open-source LLM Gateway with 34K+ stars on GitHub** and trusted by companies like **NASA, Rocket Money, Samsara, Lemonade, and Adobe.** We’re rapidly expanding and seeking our 6th Engineer focused on owning ‘excellence’ for unified API’s across Core LLM’s (openai/gemini/anthropic models).
### What is LiteLLM
LiteLLM provides an **open source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic) in the OpenAI format**
We just hit **$6M ARR** and have raised a **$1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund.** You can find more information on our [website](https://www.litellm.ai/), [Github](https://github.com/BerriAI/litellm) and [Technical Documentation.](https://docs.litellm.ai/docs/)
**Why do companies use LiteLLM enterprise**
Companies use LiteLLM Enterprise once they put LiteLLM into production and need enterprise features like Prometheus metrics (production monitoring) and need to give LLM access to a large number of people with SSO (secure sign on) or JWT (JSON Web Tokens)
### What you will be working on
**Skills:** Python, LLM APIs, FastAPI, High-throughput/low-latency
As the Backend LLM Engineer, you'll be responsible for ensuring LiteLLM unifies the format for calling LLM APIs in the broader OpenAI + Anthropic spec. This involves writing transformations to convert API requests from OpenAI/Anthropic spec to various LLM provider formats, building provider-agnostic unification functionality (e.g. session management across non-openai models for /v1/responses API, etc.). You'll work directly with the CEO and CTO on critical projects including:
* Adding support for Anthropic and Bedrock Anthropic 'thinking' parameter
* Handling provider-specific quirks like OpenAI o-1 streaming limitations
* Maintaining ‘excellent’ unified API’s across /v1/messages, /v1/responses, /chat/completions for OpenAI/Gemini/Anthropic models across Azure, OpenAI API, Bedrock Invoke, Bedrock Converse, Vertex AI, Google AI Studio
* Implementing cost tracking and logging for Anthropic API
### **What is our tech stack**
The tech stack includes Python, FastAPI, Redis, Postgres.
### **Who we are looking for**
* 1-2 years of backend/full-stack experience with production systems
* Passion for open source and user engagement
* Experience working with the OpenAI api (understand the difference between /chat/completions and /responses, and can speak to API-specific nuances)
* Strong work ethic and ability to thrive in small teams
* Eagerness to talk to users and help solve real problems
LiteLLM is an **open-source LLM Gateway with 34K+ stars on GitHub** and trusted by companies like **NASA, Rocket Money, Samsara, Lemonade, and Adobe.** We’re rapidly expanding and seeking our 6th Engineer focused on owning ‘excellence’ for unified API’s across Core LLM’s (openai/gemini/anthropic models).
### What is LiteLLM
LiteLLM provides an **open source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic) in the OpenAI format**
We just hit **$6M ARR** and have raised a **$1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund.** You can find more information on our [website](https://www.litellm.ai/), [Github](https://github.com/BerriAI/litellm) and [Technical Documentation.](https://docs.litellm.ai/docs/)
**Why do companies use LiteLLM enterprise**
Companies use LiteLLM Enterprise once they put LiteLLM into production and need enterprise features like Prometheus metrics (production monitoring) and need to give LLM access to a large number of people with SSO (secure sign on) or JWT (JSON Web Tokens)
### What you will be working on
**Skills:** Python, LLM APIs, FastAPI, High-throughput/low-latency
As the Backend LLM Engineer, you'll be responsible for ensuring LiteLLM unifies the format for calling LLM APIs in the broader OpenAI + Anthropic spec. This involves writing transformations to convert API requests from OpenAI/Anthropic spec to various LLM provider formats, building provider-agnostic unification functionality (e.g. session management across non-openai models for /v1/responses API, etc.). You'll work directly with the CEO and CTO on critical projects including:
* Adding support for Anthropic and Bedrock Anthropic 'thinking' parameter
* Handling provider-specific quirks like OpenAI o-1 streaming limitations
* Maintaining ‘excellent’ unified API’s across /v1/messages, /v1/responses, /chat/completions for OpenAI/Gemini/Anthropic models across Azure, OpenAI API, Bedrock Invoke, Bedrock Converse, Vertex AI, Google AI Studio
* Implementing cost tracking and logging for Anthropic API
### **What is our tech stack**
The tech stack includes Python, FastAPI, Redis, Postgres.
### **Who we are looking for**
* 1-2 years of backend/full-stack experience with production systems
* Passion for open source and user engagement
* Experience working with the OpenAI api (understand the difference between /chat/completions and /responses, and can speak to API-specific nuances)
* Strong work ethic and ability to thrive in small teams
* Eagerness to talk to users and help solve real problems