Machine Learning Engineer — AI Architecture Research
Required skills
- AI Architecture Research
- neural network architectures
- Transformers
- recurrent models
- hybrid models
- long-context systems
- scaling laws
- memory mechanisms
- compute trade-offs
- PyTorch
Nice to have
- non-Transformer architectures
- RNN variants
- SSMs
- large-scale training
- custom training loops
- inference optimization
Role summary
We are seeking a Machine Learning Engineer specializing in AI architecture research to design, prototype, and validate next-generation model architectures. This role involves working at the intersection of research and production, transforming novel ideas into scalable, real-world systems. The ideal candidate will challenge architectural assumptions, experiment with new model designs, and explore beyond standard Transformer approaches. Responsibilities include researching and developing new neural network architectures, designing and conducting architecture-level experiments, prototyping models end-to-end, collaborating with engineers for deployability, analyzing model behavior, and extending cutting-edge research.
About the Role
We’re looking for a Machine Learning Engineer focused on AI architecture research to help design, prototype, and validate next-generation model architectures. You’ll work at the intersection of research and production — turning new ideas into scalable, real-world systems.
This role is ideal for someone who enjoys questioning architectural assumptions, experimenting with novel model designs, and pushing beyond standard Transformer-style approaches.
What You’ll Work On
Research and develop new neural network architectures (e.g. alternatives or extensions to Transformers, recurrent / hybrid models, long-context systems)
Design and run architecture-level experiments (scaling laws, memory mechanisms, compute trade-offs)
Prototype models end-to-end — from research code to training-ready implementations
Collaborate with inference and systems engineers to ensure architectures are deployable and efficient
Analyze model behavior, failure modes, and inductive biases
Read, reproduce, and extend cutting-edge research papers
Contribute to internal research notes, benchmarks, and open-source efforts (where applicable)
What We’re Looking For
Strong background in machine learning fundamentals and deep learning
Hands-on experience implementing model architectures from scratch
-
Solid understanding of:
Attention mechanisms, RNNs, state-space models, or hybrid architectures
Training dynamics, scaling behavior, and optimization
Memory, latency, and compute constraints at the model level
Comfortable working in PyTorch or JAX
Ability to move fluidly between theory, experimentation, and engineering
Clear communicator who can explain architectural trade-offs
Nice to Have
Experience with non-Transformer architectures (RNN variants, SSMs, long-context models)
Background in research-driven startups or open-source ML projects
Experience with large-scale training or custom training loops
Publications, preprints, or notable research contributions
Familiarity with inference optimization and deployment constraints
Why Join
Work on core model architecture, not just fine-tuning
Direct influence on the technical direction of a Series-A company
Small, high-caliber team with fast feedback loops
Opportunity to ship research into production
Competitive compensation + meaningful equity
Originally posted on Himalayas
Similar roles
- Founding Sales EngineerFeatherless AI · Remote · Hybrid
- Content MarketerFeatherless AI · Remote · Remote
- Senior Machine Learning EngineerCheckr · Remote · Remote
Staff Machine Learning Engineer - Team Lead (Active Secret Clearance)Striveworks · Remote · Remote- Machine Learning Engineer - New GradSambaNova Systems · Austin, TX · Onsite
- Principal Machine Learning EngineerPhysicsX · New York, NY · Onsite
