
Big Data Engineer
Role summary
We are seeking a Big Data Engineer with overall 8+ years of experience, including 7-10 years in Big Data/Hadoop. The role requires strong expertise in the Hadoop ecosystem (HDFS, Hive, Spark, MapReduce), data ingestion using Sqoop, and programming skills in Python, Scala, or Java. You will also need advanced SQL knowledge, experience with data lake architecture, distributed computing, ETL/ELT pipeline design, cloud migration strategies, data governance tools, and CI/CD practices. Preferred skills include experience with cloud platforms like GCP (Dataproc, BigQuery, Cloud Storage) and workflow orchestration tools such as Airflow or Cloud Composer.
Technical skills requirements
The candidate must demonstrate proficiency in,
· Strong expertise in Hadoop ecosystem – HDFS, Hive, Spark(PySpark/Scala),MapReduce
· Hands-on experience with Sqoop for data ingestion
· Strong programming skills in Python / Scala / Java
· Advanced knowledge of SQL and query optimization
· Experience in data lake architecture and maintenance
· Strong understanding of distributed computing and parallel processing
· Experience in ETL/ELT pipeline design and optimization
· Exposure to cloud migration strategies and readiness planning
· Experience with data governance tools and frameworks
· Knowledge of CI/CD pipelines and DevOps practices
Nice-to-have skills
· Experience with cloud platforms (GCP preferred – Dataproc, BigQuery, Cloud Storage)
· Familiarity with workflow orchestration tools (Airflow / Cloud Composer)
Qualifications
- Overall 8 + years with 7-10 years of relevant work experience in Big Data/Hadoop
- B.Tech., M.Tech. or MCA degree from a reputed university
Similar roles
- Big Data EngineerMercor · New York, New York, United States · Remote
- Big Data EngineerMercor · Toronto, Ontario, Canada · Remote
Big Data EngineerTMX Group · Toronto, Ontario, Canada · Onsite- Big Data EngineerMercor · Montreal, Quebec, Canada · Remote
- Senior Big Data EngineerJobs via Dice · Tysons, Virginia, United States · Onsite