Senior Data Engineer

Responsibilities:

• Design and build high performance, scalable data engineering solutions that meet needs of millions of Anywhere’s independent agents, brokers, as well as their home buyers and sellers;
• Design and develop data ingestion pipelines for batch and real-time streaming of data from in-house OLTP systems and third-party data;
• Collaborate with other Data Engineers at Anywhere for the build out of Next Generation Data Ingestion and Data Enrichment Pipelines;
• Work with Data Engineering Team to design and develop a Data Lake to store and process terabytes of industry data;
• Design and develop ETL pipelines to process data in Data Lake for use with APIs, reporting, analytics and related AI and data science;
• Develop ETL data pipelines to build Enterprise Data Models for Property, Agent, Broker, office and other master partners and related clients and entities;
• Design and develop CI/CD process for continuous delivery in AWS Cloud;
• Design, develop, and test robust, scalable data platform components; and
• Partner with a variety of data and delivery teams within the Anywhere organization, including product engineers to understand their data pipeline needs to assist them with coming up with innovative data engineering solutions.

Qualifications:

• Must have a Bachelor’s degree in Computer Science, Information Technology or related field, plus five (5) years of experience in the job offered or in any occupation that includes the required experience and skills. Experience must include:
• 5 years of experience working with Scala or Java;
• 5 years of experience designing Real-Time Streaming Data Pipelines utilizing Apache Spark or Apache Flink and Apache Kafka;
• 5 years of development experience in Apache Spark for distributed data processing;
• 5 years of experience with AWS services including EC2, ECS, S3, EMR and Lambda;
• 5 years of experience in SQL databases, including PostgreSQL, MySQL, or Oracle, and NoSQL databases, including MongoDB or DynamoDB;
• 5 years of experience with Continuous Integration & Deployment (CI/CD) processes utilizing tools including AWS CodePipeline, Gitlab, or Jenkins; and
• 5 years of experience in serving in a lead role for onshore and offshore development teams, providing code reviews, architectural guidance, and technical leadership.

#LI-DNI