Senior Bash Software Engineer (AI Evaluation)
Role summary
This role focuses on a structured AI evaluation initiative to enhance the reliability and reasoning accuracy of conversational systems, particularly in systems engineering and automation. The Senior Bash Software Engineer will review, execute, and validate AI-generated Bash scripts, identifying errors, inefficiencies, and edge cases. Responsibilities include providing detailed feedback on script correctness, clarity, and operational safety, performing fact-checking, and applying standardized evaluation frameworks. The ideal candidate possesses strong Bash and shell scripting expertise, deep Unix/Linux knowledge, and experience with AI models in coding workflows, aiming to improve system performance through precise technical assessment.
About the Role
A structured AI evaluation initiative focused on improving the reliability and reasoning accuracy of conversational systems in systems engineering and automation contexts. The work emphasizes how models generate, interpret, and explain shell-based workflows, scripting logic, and system-level operations.
This opportunity is ideal for experienced Bash and shell scripting professionals with strong problem-solving ability and a deep understanding of Unix/Linux environments. It suits individuals who can independently validate scripts, identify subtle issues, and assess the quality of technical explanations.
The work involves reviewing AI-generated Bash scripts, executing and validating outputs, and providing structured feedback on correctness and clarity, where precision and consistency are essential to improving system performance.
What You'll Do
- Evaluate AI-generated responses to scripting and systems engineering tasks
- Execute and validate Bash scripts to ensure correctness and reliability
- Identify logical errors, inefficiencies, and edge case failures in shell workflows
- Annotate outputs with structured feedback on strengths and weaknesses
- Assess script readability, maintainability, and operational safety
- Perform fact-checking using reliable technical documentation and references
- Apply standardized evaluation frameworks and scoring criteria
- Ensure outputs align with expected system behavior and engineering standards
Requirements
- 5+ years of professional experience in software engineering, systems engineering, or related fields
- Strong expertise in Bash and shell scripting
- Deep familiarity with Unix/Linux command-line environments
- Ability to solve medium to hard algorithmic and scripting problems independently
- Experience writing, executing, and debugging production-level scripts
- Understanding of system processes, file systems, networking basics, and automation workflows
- High attention to detail in reviewing technical reasoning and outputs
- Fluent English communication skills
- Experience using LLMs in coding or scripting workflows and understanding their limitations
- Ability to follow structured evaluation frameworks and guidelines
- Bachelor’s degree or higher in Computer Science or related discipline
- Experience contributing to open-source projects with accepted contributions
- Familiarity with additional programming or scripting languages (preferred)
- Experience in model evaluation, RLHF, or data annotation (preferred)
- Background in DevOps, infrastructure automation, or CI/CD systems (preferred)
- Experience reviewing scripts in production environments (preferred)
- Ability to explain complex system behaviors and scripting concepts clearly (preferred)