Senior Data Engineer

JOB SUMMARY:
The Data Engineer (Streaming/ETL) will support the design, development, and deployment of data ingestion and processing pipelines for advanced analytics platforms. This role is critical to enabling real-time and batch data integration across multiple sources, including operational systems, telemetry streams, and structured/unstructured datasets.

The position focuses on building scalable data pipelines, supporting instrumentation data ingestion, and enabling downstream analytics, machine learning, and decision support capabilities within a cloud-native environment. The Data Engineer will work closely with data scientists, software engineers, and solution architects to ensure data is accessible, reliable, and optimized for analytics and AI/ML workloads.

RESPONSIBILITIES:

• Data Ingestion & Pipeline Development: ETL/ELT pipelines, data ingestion frameworks, data connectors and APIs, data stream ingestion, formatting for analytics
• Streaming & Real-Time Data Processing: data buffering, processing strategies, event-based data processing, analytics workflow integration, optimization
• Data Integration & Architecture Support: multi-source integration collaboration, integration across various environments, building unified data ingestion frameworks
• Data Quality & Governance: data validation, integrity checks, monitoring ingestion within pipelines, data quality frameworks, verification processes, track data lineage, ensure traceability, enforce data handling and security practices
• Collaboration & Stakeholder Support: work with data scientists to prepare datasets for ML/analytics, support dashboarding/visualization efforts, participate in info gathering and technical discussions, support cross-functional teams in integration
• Research, Development, & Continuous Improvement: evaluate new tools and technologies, improve performance/scalability/reliability, assist in prototyping, document data architecture/pipelines/integration processes

REQUIRED Knowledge, Skills, and Abilities:

• Technical Expertise: Strong proficiency in Python for data engineering workflows, experience with SQL and relational databases (PostgreSQL, MySQL, etc.), familiarity with data lake architectures (S3 or similar), and experience building APIs or working with REST-based integrations.
• Data Engineering Skills: Experience building ETL/ELT pipelines for large-scale data processing, strong understanding of data ingestion patterns and pipeline design, and experience working with structured and unstructured data sources.
• Streaming & Data Processing Skills: Familiarity with streaming or event-driven architectures (Kafka, Kinesis, etc.), experience handling high-volume data ingestion and processing workflows, and understanding of real-time versus batch processing tradeoffs.
• Cloud & Infrastructure Skills: Experience working in cloud environments (AWS preferred: EC2, S3, ECS, Lambda, etc.), familiarity with containerized environments (Docker, Kubernetes is a plus), and understanding of scalable and distributed data systems.
• Collaboration & Communication Skills: Ability to work with cross-functional teams including data scientists and software engineers, strong written and verbal communication, and ability to translate technical concepts into understandable solutions.
• Problem Solving: Ability to design efficient data pipelines and troubleshoot performance issues, experience identifying data integration challenges and proposing solutions, and comfortable working in fast-paced, evolving environments.
• Experience with Specific Technologies: Python (pandas, PySpark, or similar data processing tools), SQL/relational databases, data streaming tools (Kafka, Kinesis, or equivalent), cloud services (AWS preferred), ETL tools or frameworks (custom or commercial), and data lakes (S3, Delta Lake, or similar).

CLEARANCE:

• Active Secret clearance (or ability to obtain and maintain one).

EDUCATION:

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related technical field

TRAVEL:

• Up to 20% travel may be expected with this position

PHYSICAL REQUIREMENTS:

• Ability to work independently with minimal guidance and lift up to 25 pounds.

LOCATION:

• Hybrid remote in Washington, DC (preferred proximity to Northern Virginia or Hampton Roads area)

BENEFITS OFFERED:

A comprehensive and generous benefits package is offered. The benefits package includes medical, dental, and vision insurance for the employee and/or families. There is a 401(k) program offered with a company match. Also included is included basic life insurance plus short- and long-term disability for the employee.

The company is committed to non-discrimination and equal employment opportunity. All qualified applicants will receive consideration for employment without discrimination based on disability, protected veteran status or any other characteristics protected by law.

Job Type: Full-time

Benefits:

• 401(k)
• 401(k) matching
• Dental insurance
• Health insurance
• Life insurance
• Paid time off
• Vision insurance

Work Location: Hybrid remote in Washington, DC 20546