Software Engineering Evaluator
Role summary
This is a remote, contract role for a Software Engineering Evaluator. The position requires a minimum of 10 hours per week, with up to 40 hours, and requires partial PST overlap. The core responsibilities include curating and building code examples, evaluating AI-generated code for various quality metrics, designing automated verification mechanisms, assessing AI capabilities across the software development lifecycle, and collaborating with research and cross-functional teams. Requirements include strong professional software engineering experience, continuous full-time experience at a top-tier product company, expertise in full-stack development and production-grade system design, and a deep understanding of software architecture, debugging, and code quality assessment. Excellent communication skills are also necessary.
About the job
Position: Software Engineering Evaluator
Type: Contract
Compensation: $150/hr
Location: Remote
Commitment: Minimum 10 hours/week, up to 40 hours/week (partial PST overlap required)
Role Responsibilities
•Curate, build, and correct real-world code examples across multiple programming languages.
•Evaluate AI-generated code for efficiency, scalability, reliability, and correctness.
•Design automated verification mechanisms to validate software engineering solutions.
•Assess AI capabilities across the full software development lifecycle.
•Collaborate with researchers and cross-functional teams to improve AI-driven coding systems.
Requirements
•Strong professional software engineering experience.
•Continuous full-time experience at a top-tier product company.
•Strong expertise in full-stack development and production-grade system design.
•Deep understanding of software architecture, debugging, and code quality assessment.
•Excellent written and verbal communication skills.
•Access to a desktop or laptop computer
Application Process (45 min)
•Upload resume
•Automated coding exercise
•Interview
Desired Skills and Experience
- Software Development