Cloud Engineering Manager (GCP)
Role summary
This Cloud Engineering Manager role focuses on operationalizing and managing a large-scale enterprise lakehouse architecture on Google Cloud Platform (GCP) using Databricks. The manager will be responsible for supporting the delivery and implementation of the lakehouse, establishing best practices for data engineering, and driving data and cost optimizations. Key responsibilities include defining standards for Delta Lake usage, supporting FinOps activities for cost control, and leading efforts in security, governance, and data ingestion patterns. The role requires extensive experience with big data platforms, cloud environments (GCP, Azure), and Databricks capabilities, along with strong leadership and communication skills to guide teams and stakeholders.
Operational Lakehouse Strategy, Operations & Platform Management Management level experience for senior big data platform management (10+ years)
Experience working across different functional (application, infrastructure, security, compliance / audit, operations and business domains
Strong communication and organizational skills
Support delivery and management of the enterprise lakehouse architecture and implementation on large-scale cloud data platforms (Databricks)
Experience with Databricks usage in hyperscaler environments (Azure, Google Cloud Platform and Azure)
Support and lead implementation of best practices standards for SQL/PySpark development and usage
Standardize data using industry frameworks to ensure IT-related data alignment (infrastructure-related information, infrastructure capacity, security-related, application runtime data, IT monitoring-related information, and additional meta-data)
Support and provide best practices on data mapping
Establish multi-zone / Medallion architecture to drive data and cost optimizations: Bronze (raw telemetry) Silver (cleaned/normalized) Gold (aggregated/KPIs)
Design for 500TB+/day ingestion scale
Define standards for: Delta Lake usage including Delta Tables / DLT Table optimization (Z-ordering, partitioning) Data lifecycle management User workflows and use cases across various areas including line of business and IT
Knowledge of various Databricks capabilities including data engineering tools, Mosaic (AI/ML tools), Autoloader, Unity Catalog, Delta Tables / DLT, query builder, workspace - schema - table structures, Autoloader, LakeFlow, Genie, DataBricks Workflows / Jobs and additional Databricks components
Support FinOps (usage and capabilities cost controls) related activities including management and optimizations of compute, storage and DBU usage
Support Unity Catalog buildout including IAM and RBAC Support and lead expertise
Support user-related best practices including use cases across various stakeholder roles, governance, user support, SLO / SLA development, predictive alerting and anomaly detection
Support pattern development and optimizations for data ingestion including streaming, batch and incremental
Knowledge and expertise in various data pipeline approaches and platforms to ensure data quality, data optimizations and reductions, ETL functions, data protection and high throughput and low latency
Support and provide expertise on semantic models Support schema
For applications and inquiries, contact: hirings@openkyber.com