Generative AI Data Analyst
Role summary
This fully remote role seeks a Generative AI Data Analyst to support a critical machine learning project. The analyst will be responsible for creating high-quality prompts and responses, leading labeling initiatives, and developing guidelines for LLM datasets. Key tasks include performing LLM annotation and evaluation, identifying model issues like hallucinations, and training teams on best practices for LLM development. The ideal candidate is a strong communicator with native-level English, attention to detail, and enthusiasm for AI technologies. A college degree or equivalent experience in relevant fields like Linguistics or Creative Writing, along with domain knowledge, is preferred.
### Who you are
- The ideal candidate is a strong communicator with native-level U.S. English, experienced in working with data and comfortable training teams on best practices for LLM development
- This position is fully remote and suited for someone motivated to work with cutting-edge AI technologies
- Hands-on experience performing data annotation or evaluation tasks (e.g., labeling, ranking, scoring, or tagging LLM outputs)
- Native or near-native English with excellent writing skills
- Strong attention to detail and ability to follow guidelines consistently
- Self-driven, motivated and enthusiastic to work on state-of-art machine learning tools
- 4 year Accredited College degree or equivalent experience
- College Degree or experience in Linguistics, English Literature, Creative Writing, Journalism, and domain knowledge (Law/Medical/Math/Coding/etc.)
- Experience working in annotation platforms or structured labeling environments is a plus
- Deep understanding of Large Language Models/RLHF
- Experience in labeling/tagging of frames/tasks/prompts to prepare for DNN
- QA/testing experience
### What the job involves
- We are seeking a Generative AI Analyst to support a high-impact machine learning project
- This role focuses on creating high-quality prompts and responses across diverse topics, leading labeling initiatives with internal and external partners, and developing clear guidelines to ensure consistency and accuracy in large language model datasets
- Hours: 40 hours weekly
- Start date: April 2026
- Creatively writing prompts and responses to a variety of diverse topics
- Perform LLM annotation and evaluation tasks (ranking, scoring, labeling, tagging)
- Evaluate model outputs for accuracy, relevance, and instruction-following
- Identify and document issues such as hallucinations and inconsistencies
- Participating in and/or supporting labeling workflows, including hands-on annotation and collaboration with internal or external teams
- Training teams on best practices for creating Large Language Models/Data sets