Software Development, Digital Agency, Consulting, IT Services

Staff ML Engineer

RemoteRemoteFull TimeStaffPosted 1 month ago

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Required skills

PyTorch
SageMaker
AWS Trainium
Python
distributed training
hardware acceleration
compilation
accelerator
SageMaker pipelines
production engineering

Nice to have

AWS Inferentia
Neuron SDK

Role summary

Robots & Pencils is seeking a Staff ML Engineer to build production AI systems for enterprise clients, focusing on training and operating models on Amazon SageMaker using AWS Trainium. This role requires deep PyTorch expertise, comfort with hardware-level compilation and debugging on custom accelerators, and experience building end-to-end, cost-aware training pipelines. You will translate complex training requests into working production systems, optimize distributed training runs, and collaborate directly with client teams. Strong Python fundamentals and production engineering experience are essential. Experience with AWS Trainium or Inferentia is a plus.

Robots & Pencils is an AWS Partner building production AI systems for enterprise clients who need real engineering, not proofs of concept that never ship. We work forward-deployed, embedded directly with client teams, solving the problems that are too new or too specialized for a typical vendor relationship to handle. The RoleWe're looking for an engineer who can operate and train models on Amazon SageMaker running on AWS Trainium, AWS's custom silicon built specifically for large-scale model training. This isn't a role where you call an API and wait. You'll be walking up the stack: understanding what a training request actually looks like at the Trainium hardware and compiler level, then carrying that understanding all the way up through PyTorch training code and into a production SageMaker pipeline.PyTorch is the backbone of this work. If you know the framework deeply and you're comfortable reasoning about how your code actually behaves on custom accelerator hardware rather than treating it as a black box, this role is built around that skill set specifically. What You'll Do

Train and operate models on Amazon SageMaker with AWS Trainium as the underlying compute

Write and optimize PyTorch training code with a real understanding of how it compiles and executes on Trainium (NeuronCore architecture, compiler behavior, memory and throughput tradeoffs)

Diagnose training run issues that show up specifically because of the hardware, not just the model, distinguishing a data or code problem from a compiler or device-level one

Translate a request for "a Trainium job" into an actual working, cost-aware training pipeline, end to end

Tune distributed training runs for throughput and cost on SageMaker's training infrastructure

Work directly with client and internal engineering teams to scope and deliver real production training workloads, not experiments that stay in a notebook

What You'll Bring

Strong, hands-on PyTorch experience, ideally including distributed or multi-device training

Production experience with Amazon SageMaker for training and/or inference

Comfort working close to the hardware layer: you understand device-specific compilation and can debug issues that are actually about the accelerator, not just the model

AWS Trainium or Inferentia (Neuron SDK) experience is a strong plus; if you don't have it yet but have deep PyTorch and a track record of picking up new hardware targets fast, we want to talk to you

Solid Python fundamentals and comfort operating in a client-facing, production engineering environment

Ready to apply?

You'll be redirected to Robots & Pencils's application page.

Staff ML Engineer

Is this role right for you?

Required skills

Nice to have

Role summary

Similar roles

See how RiseMe works