AI and Data Engineer
Role summary
This role focuses on developing and refining prompts for a Generative AI Platform, collaborating with data scientists, ML engineers, and product managers. Responsibilities include researching leading AI models like Perplexity and ChatGPT, designing effective prompts, conducting rigorous testing, and maintaining prompt documentation. The engineer will build tools to streamline prompt engineering, stay updated on AI advancements, and integrate prompt capabilities into the platform. A Master's degree in a related field, extensive NLP and prompt engineering experience, and proficiency in Python are required. Familiarity with AI ethics and production deployment is also valued.
Responsibilities
- Collaborate with cross-functional teams, including data scientists, machine learning engineers, and product managers, to understand the specific use cases and requirements for the Generative AI Platform.
- Research and analyze the capabilities, limitations, and best practices of leading Generative AI models, such as Perplexity, ChatGPT / Azure AI/ , and NVIDIA's language models.
- Develop and refine prompts that effectively leverage the strengths of these AI models to generate content that is accurate, coherent, and aligned with the desired tone, style, and purpose.
- Conduct extensive testing and evaluation of prompts, iterating on the design to optimize for performance, safety, and ethical considerations.
- Maintain a comprehensive prompt library and documentation, ensuring prompt reusability and consistency across the platform.
- Develop tools and frameworks to streamline the prompt engineering process, including prompt templates, prompt scoring metrics, and automated prompt generation capabilities.
- Stay up-to-date with the latest advancements in Generative AI, natural language processing, and prompt engineering best practices, and continuously incorporate new techniques and insights into the platform.
- Collaborate with the product and engineering teams to integrate prompt engineering capabilities into the Generative AI Platform, ensuring seamless user experience and scalability.
- Provide training and support to platform users, educating them on effective prompt design and usage.
Qualifications
- Master's degree in Computer Science, Linguistics, or a related field. (PhD preferred)
- Extensive experience in natural language processing, text generation, and prompt engineering, with a deep understanding of the underlying principles and techniques.
- Proficiency in programming languages such as Python, Jupyter Notebooks, and Markdown, with the ability to write clean, maintainable, and efficient code.
- Strong analytical and problem-solving skills, with the ability to think critically and creatively to address complex challenges.
- Excellent communication and collaboration skills, with the ability to work effectively in a cross-functional team.
- Familiarity with leading Generative AI models, such as Perplexity, Azure GPT, and NVIDIA's language models, and their respective capabilities and limitations.
- Understanding of ethical considerations in AI development, including bias, fairness, and safety.
- Experience in developing and deploying AI-powered applications in a production environment is a plus.
Pay: $92,199.96 - $111,036.52 per year
Work Location: In person
