
Generative AI Engineer
Role summary
This role focuses on building production-grade Generative AI (GenAI) solutions, with a strong emphasis on Retrieval-Augmented Generation (RAG) pipelines and advanced prompt engineering techniques. The engineer will be responsible for implementing robust evaluation and observability systems, including setting up ground truth, monitoring metrics, and utilizing LLM-as-judge approaches. Proficiency in LLM frameworks like LangChain or LlamaIndex, and experience with cloud LLM providers (Azure OpenAI, AWS Bedrock, Vertex AI) are essential. The role also involves workflow orchestration using tools like Airflow or Dagster, and ensuring security and privacy through PII handling and RBAC. Strong Python expertise is required to deliver a production-ready system with measurable quality improvements and reduced hallucinations.
The project is focused on building production-grade GenAI solutions with emphasis on:
• RAG pipelines leveraging LLMs
• Prompt engineering (system/tool prompts, function calling, versioning with evals)
• Evaluation & observability (ground truth setup, confusion metrics, LLM-as-judge with human review, cost & latency monitoring)
• Retrieval strategies & prompt patterns (context management, hallucination mitigation)
• LangChain/LlamaIndex (or equivalent) proficiency
• Cloud LLM providers (Azure OpenAI, AWS Bedrock, Vertex AI)
• Workflow orchestration (Airflow, Dagster)
• Security & privacy (PII handling, RBAC)
• Python expertise with strong practical experience
Success criteria:
• A production-ready prompt + RAG pipeline system with measurable quality lift (accuracy/confusion metrics) and reduced hallucinations, backed by a repeatable evaluation harness.
Similar roles
- Generative AI EngineerMitre Media · Fort Meade, Maryland, United States · Hybrid
Generative AI EngineerJ2B Global LLC · Mississauga, Ontario, Canada · Onsite- Generative AI EngineerCyber Space Technologies LLC · United States · Remote
- Senior Generative AI EngineerVLink Inc · United States · Remote
- Generative AI EngineerJAMY INTERACTIVE · Santa Clarita, California, United States · Onsite