Location: Remote
Duration: 12 months with potential extension
The Data Engineer will support the development, optimization, and governance of enterprise data pipelines and integration workflows. This role involves designing scalable data architecture, collaborating with cross-functional teams, and ensuring high-quality, secure, and reliable data operations.
Assess feasibility and technical requirements for integrating source systems into a centralized Data Lake.
Collaborate with internal teams and external partners on data specifications and recurring ingestion pipelines.
Build, optimize, and maintain ETL workflows for multiple datasets including clinical, registry, and vital records data.
Design scalable data workflows that ensure high data quality, identity resolution, and integrity across systems.
Implement data governance, observability, and lineage tracking mechanisms for all data pipelines.
Mentor engineering staff, support testing efforts, and enforce best practices in orchestration, architecture, and coding standards.
Prepare documentation and effectively communicate technical solutions to both technical and non-technical stakeholders.
3+ years experience in data engineering and/or data architecture.
2+ years experience with Python for ETL, automation, and API integration (pandas, requests, etc.).
2+ years hands-on SQL experience including stored procedures and performance tuning (Oracle, SQL Server, MySQL preferred).
1+ year experience with orchestration tools (e.g., Prefect, Airflow).
1+ year experience with cloud platforms (Azure, AWS, or GCP) including data onboarding/migration.
1+ year exposure to Data Lake / Medallion architecture (bronze, silver, gold).
2+ years of experience delivering clear written documentation and collaborative technical communication.
5+ years overall experience in data engineering roles.
Experience integrating or developing REST/JSON or XML-based APIs.
Familiarity with CI/CD pipelines (GitHub Actions, Azure DevOps, etc.).
Exposure to Infrastructure as Code tools (Terraform, CloudFormation).
Experience with metadata and governance tools (Atlan, OpenMetadata, Collibra).
Background with public health, healthcare datasets, or PHI/PII workflows.
Familiarity with SAS or R workflows supporting analytics teams.
Experience with additional SQL platforms such as Postgres, Snowflake, Redshift, or BigQuery.
Knowledge of data quality frameworks (Great Expectations, Deequ).
Experience with real-time/streaming technologies (Kafka, Spark Streaming).
Familiarity with big data frameworks (Spark, Hadoop).
Understanding of data security and compliance frameworks (HIPAA, SOC 2, etc.).
Experience working within Agile/Scrum environments.