Quantum World Technologies Inc. Verified
IT Services, Staffing & Consulting, Software Development
Senior Data Engineer (Hadoop + GCP Dataproc)
Toronto, Ontario, CanadaHybridFull TimeSeniorPosted 1 month ago
Compensation estimateAI
See base, equity, bonus, and total comp estimates for this role — free, no credit card.
Sign up to see compensation estimatePosition: Senior Data Engineer (Hadoop + GCP Dataproc)
Location : Toronto
Mode of Work : Hybrid
Permanent Full Time
Job Description ::
Required Qualifications:
- 6–10+ years of experience working within the Hadoop ecosystem, with deep expertise in Hive, Hive Metastore, and GCP Dataproc.
- Strong hands-on experience with Dataproc Serverless, BigQuery, Google Cloud Storage (GCS), Cloud Composer, and Cloud Logging/Monitoring.
- Solid understanding of table formats (Parquet, ORC), partitioning/bucketing strategies, and query performance optimization.
- Proven experience migrating Hive datasets, tables, and SQL queries to BigQuery, including handling syntax differences, functions, UDFs, and performance tuning.
- Strong knowledge of Security/IAM best practices, service accounts, and network/security controls; ability to operate within strict organization-level policies.
