Robotics

ML, Evals & Benchmarks

Redwood City, CAFull TimePosted 23 days ago

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Required skills

evaluation strategy
trustworthy
reproducible
checkpoint
sim-to-real correlation
internal benchmark
external baselines
published results
continuous evaluation
model training

Nice to have

robots
policies
harnesses
task suites
regression infrastructure
demo

Role summary

We are seeking an engineer to develop and own the evaluation systems for our robot policies, moving from research to production. This role involves building the infrastructure for rigorous and fast evaluation, ensuring trustworthiness and reproducibility of results. You will establish sim-to-real correlation, manage internal benchmarks, and make evaluation cost-effective for continuous runs. The ideal candidate has experience building evaluation, validation, or benchmarking systems, fluency in ML training and evaluation workflows, strong software engineering skills, and practical statistics for experiment design. Experience in robotics, simulation environments, and evaluating specific policy types is a plus.

About the Role

We're looking for an engineer to own how we know whether our robots are actually getting better. As we move from research to production, we need evaluation that's rigorous enough to trust and fast enough to keep up. You'll build the harnesses, task suites, and regression infrastructure that turn "the demo looked good" into a number we can act on.

What You'll Do

Own the evaluation strategy for our policies in simulation and on real hardware.
Make evaluation results trustworthy and reproducible; the same evaluation, run by different people on different days, gives the same answer.
Ensure a checkpoint that improves one behavior can't silently break another, and that releases are gated on it.
Establish sim-to-real correlation, so we know which simulation results we can trust.
Own the internal benchmark our models are measured against, and keep results comparable as it evolves.
Know where we stand against the state of the art, benchmarked against external baselines and published results.
Make evaluation cheap enough to run continuously, without burning robot hours on every checkpoint.
Give the company clear answers to "is this checkpoint better than the last one" and "what is it failing at."

What We're Looking For

Built evaluation, validation, or benchmarking systems that a team made real decisions from.
Fluency with model training and evaluation workflows: comfortable reading a training curve, reasoning about generalization, and telling a real improvement from an overfit one.
Strong software engineering and programming skills.
Practical statistics for experiment design: sample sizing, significance, and knowing when a result is noise.
Demonstrated skepticism: a track record of finding the flaw in a result that everyone else was ready to accept.

Bonus Points

Experience in robotics: manipulation, locomotion, or mobile autonomy.
Experience with simulation environments (Isaac, MuJoCo, Gazebo) and sim-to-real transfer.
Experience evaluating imitation-learned or RL policies specifically.

Ready to apply?

You'll be redirected to Genki Robotics's application page.