Senior Data Engineer

Oakland, California, United StatesOnsiteContractSeniorPosted 2 months ago

Is this role right for you?

Upload your resume and get a skill-by-skill breakdown — see exactly where you match, where you're close, and what to highlight. Not a mystery percentage.

Get a tailored resume highlighting what this role needs.

Role summary

We are seeking a Senior Data Engineer to design and develop ontologies and robust data pipelines. This role involves implementing and maintaining ETL processes, optimizing cloud-based data storage and processing solutions, and ensuring data integrity and security. You will collaborate with data analysts and scientists to deliver high-quality data solutions, monitor and troubleshoot data pipelines, and use Python and PySpark for ETL and dataset derivation. A Bachelor's degree in a related field or equivalent experience, 5+ years of data engineering experience, strong SQL, and proficiency in Python and PySpark are required.

Responsibilities
Job Description:

Work closely with Subject Matter Experts (SMEs) to design and develop the ontology (data model) and data pipelines.
Design, implement, and maintain robust data pipelines and ETL processes.
Develop and optimize cloud-based data storage and processing solutions.
Collaborate with data analysts and data scientists to understand data requirements and deliver high-quality data solutions.
Ensure data integrity and security across all data workflows and storage solutions.
Monitor and troubleshoot data pipelines, addressing any issues promptly to ensure the smooth flow of data.
End to End data pipeline development – responsible for using Python and Pyspark to perform ETL and derive new datasets necessary for business applications,

Minimum Qualifications

Bachelor's Degree in Computer Science, Engineering, or related field or related experience.
5+ years of experience as a data engineer.
Hands-on experience with building data pipelines
Strong SQL skills and experience working with large datasets and complex data structures.
Proficient in Python, and Pyspark
Experience in Typescript (preferred) or Javascript
Excellent problem-solving and analytical skills with a strong attention to detail.

Preferred Qualifications

Experience with Palantir Foundry application development and data visualization tools
Knowledge with commercial visualization tools such as Tableau or Power BI.
Experience in Typescript (preferred) or Javascript
Databases – familiarity with common relational database models and proprietary instantiations, such as SAP, Salesforce etc.
Git – knowledge of version control / collaboration workflows and best practices

Agile – familiarity with agile and iterative working methodology and rapid user feedback gathering concepts

UX design – knowledge of best practices and applications

Skills: pipelines,python,etl,data,cloud,design

Ready to apply?

You'll be redirected to TheCorporate's application page.

Is this role right for you?

Role summary

Similar roles