GCP Cloud Engineer
Role summary
This Lead Cloud Engineer role focuses on designing, deploying, and managing cloud-native platforms, primarily on Google Cloud Platform (GCP) and OpenShift, to support containerized Spark applications and statistical risk models. The engineer will architect and enhance platforms, drive modernization, and oversee large-scale data processing workflows. Key responsibilities include developing Python-based microservices, implementing robust CI/CD pipelines, ensuring system reliability through monitoring and troubleshooting, and adhering to security and compliance standards. The role involves cross-functional collaboration and technical mentorship, requiring strong leadership and deep expertise in distributed systems, cloud technologies, and big data processing.
Title: Software Engineer 4 (Lead Cloud Engineer)
Duration: 12 Months
Type: W2 Only
Location: Charlotte, NC – Hybrid Role
Key Responsibilities
Platform & Architecture Leadership
- Lead design, deployment, and management of OpenShift clusters or GCP projects supporting containerized Spark applications
- Architect and enhance the platform used for executing statistical risk models
- Drive platform modernization and cloud adoption initiatives
Distributed Data Processing
- Architect and oversee large-scale data processing workflows using Apache Spark
- Optimize Spark jobs for performance and cost efficiency using Kubernetes orchestration and auto-scaling
- Oversee integration of Spark with data sources (Kafka, S3/GCS, databases, data lakes)
Application & Tooling Development
- Lead development of Python-based microservices and backend tools (Django preferred) to support statisticians and data users
- Ensure scalability, usability, and reliability of tools and platform features
Automation & CI/CD
- Design and govern CI/CD pipelines in GitHub Actions, Sonar, Helm, Harness, etc.
- Implement automation across deployments, environment management, and testing
Monitoring & Reliability
- Monitor and tune cluster health, performance, and resource allocation using Prometheus, Grafana, and GCP tools
- Troubleshoot distributed and cloud-native system issues
Security & Compliance
- Ensure solutions follow enterprise security requirements
- Implement RBAC, encryption, and secure coding patterns
Cross-Functional Leadership
- Partner with data scientists, platform engineering, DevOps, and architecture teams
- Provide technical guidance and mentorship to junior and senior engineers
- Lead production deployments and root-cause analysis efforts
Qualifications:
Experience
- 5–7+ years with Apache Spark for big data processing
- 3+ years with Python/Django backend development
- 3+ years building tools or platforms for high-volume data users
- 3–5+ years working with Google Cloud or equivalent cloud platforms
- 3+ years managing OpenShift/Kubernetes containerized workloads
- Experience supporting data-heavy financial or regulated environments preferred
Technical Skills:
- Proficiency in Spark frameworks (PySpark, Scala, or Java)
- Strong experience with Docker/Kubernetes concepts
- Hands-on OpenShift administration
- Experience with conda environments
- Strong coding in Python, Scala, or Java
- CI/CD exposure: GitHub Actions, Helm, Harness, Sonar
- Strong debugging skills across distributed systems
- Knowledge of GCP services (GCS, IAM, GKE, Cloud Run, etc.)
Education
- Bachelor’s degree in Computer Science, Engineering, or related discipline
Role Level
- Lead Engineer: 8–10+ years with demonstrated leadership of complex platforms
Similar roles
GCP Cloud EngineerQuantiphi · Canada · Remote
Lead GCP Cloud EngineerLIGHTFEATHER IO LLC · Alexandria, Virginia, United States · Onsite
Lead GCP Cloud EngineerCollective Health · Plano, Texas, United States · Hybrid
GCP Cloud EngineerCollective Health · Texas, United States · Hybrid
GCP Cloud EngineerLIGHTFEATHER IO LLC · Alexandria, Virginia, United States · Onsite