Senior Data Engineer (Hadoop + GCP Dataproc)

Position Name – Senior Data Engineer (Hadoop + GCP Dataproc)

Type of hiring – Fulltime

Location – Toronto, ON (4 days a week)

Job Description:

We are looking for an experienced
Senior Data Engineer
with strong expertise in the
Hadoop ecosystem
and
Google Cloud Platform, particularly GCP Dataproc
. The ideal candidate will have hands-on experience in modernizing data platforms, optimizing large-scale data processing workloads, and migrating
Hive-based
workloads to
BigQuery
.

Required Qualifications:

• 6-10+ Years of experience working within the Hadoop ecosystem, with deep expertise in Hive, Hive Metastore, and GCP Dataproc.
• Strong hands-on experience with Dataproc Serverless, BigQuery, Google Cloud Storage (GCS), Cloud Composer, and Cloud Logging/Monitoring.
• Solid understanding of table formats (Parquet, ORC), partitioning/bucketing strategies, and query performance optimization.
• Proven experience migrating Hive datasets, tables, and SQL queries to BigQuery, including handling syntax differences, functions, UDFs, and performance tuning.
• Strong knowledge of Security/IAM best practices, service accounts, and network/security controls; ability to operate within strict organization-level policies.

Nice-to-Have Skills:

• Familiarity with Dataproc Metastore vs. standalone Hive Metastore, and experience with Glue or other metadata/catalog services.
• Exposure to data quality frameworks and lineage tools (e.g., OpenLineage, Collibra).
• A FinOps mindset, including experience managing quotas, reservations, and cost governance for Dataproc and BigQuery workloads