Data Engineering Lead

New York, New York, United StatesOnsiteFull Time$135,000–$150,000 /yrPosted 25 days agoVisa sponsorship available

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Role summary

Seeking a Data Engineering Lead with 4-15 years of experience to build and manage data platforms. Responsibilities include developing ETL/ELT pipelines, managing cloud data services on Azure/AWS/GCP, and implementing CI/CD practices. Requires strong skills in Python, SQL, Spark, Kafka, Snowflake, and Databricks. Familiarity with Prefect/Airflow, dbt, and Docker is essential. Preferred qualifications include experience with APIM, LLM frameworks like LangChain, data observability tools, and financial data controls.

You’ll need to have:

Bachelor’s or advanced degree in Computer Science, Data Engineering, or a related field.

4–15 years of professional data engineering experience.

Strong Python, SQL, and Spark (PySpark) skills, and/or Kafka.

Snowflake (Snowpipe, Tasks, Streams) as a complementary warehouse.

Databricks (Delta formats, workflows, cataloging) or equivalent Spark platforms.

Hands-on experience building ETL/ELT with Prefect (or Airflow), dbt, Spark, and/or Kafka.

Experience onboarding datasets to cloud data platforms (storage, compute, security, governance).

Familiarity with Azure/AWS/GCP data services (e.g., S3/ADLS/GCS; Redshift/BigQuery; Glue/ADF).

Git-based workflows CI/CD and containerization with Docker (Kubernetes a plus).

It is a bonus to have:

Advanced APIM practices (custom policies, OAuth2/JWT, mTLS, private endpoints) and Azure AD integration.

Integrating datasets into MCP tools/providers for LLM/agent applications; familiarity with frameworks such as LangChain or LlamaIndex.

Data observability/quality tools (e.g., Great Expectations, Monte Carlo, Datafold) and strong lineage practices.

Exposure to financial datasets and controls (PII handling, encryption, masking).

TCS Employee Benefits Summary:

Discretionary Annual Incentive.

Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support: Maternal & Parental Leaves.

Insurance Options: Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.

Time Off: Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Salary Range $135,000-$150,000 a year

Location

New York, NY

Job Function

TECHNOLOGY

Role

Engineer

Job Id

413596

Desired Skills

Python | Spark | Tableau

Salary Range

$135,000-$150,000 a year

Desired Candidate Profile

Qualifications : BACHELOR OF COMPUTER SCIENCE

Ready to apply?

You'll be redirected to Tata Consultancy Services's application page.

Is this role right for you?

Role summary

Similar roles