Develop automated scripts to validate source-to-target data movements, ensuring schema consistency, completeness, and timeliness.
Build robust, reusable testing utilities using Python, native AWS services that integrate seamlessly into DataOps workflows.
Set up continuous data drift monitoring, schema validation, and data quality dashboards to proactively catch pipeline failures.
Partner with data engineers, BI teams, and software engineers to translate business rules into strict data quality KPIs and SLAs.
Key Responsibilities
Develop automated scripts to validate source-to-target data movements, ensuring schema consistency, completeness, and timeliness.
Build robust, reusable testing utilities using Python, native AWS services that integrate seamlessly into DataOps workflows.
Set up continuous data drift monitoring, schema validation, and data quality dashboards to proactively catch pipeline failures.
Partner with data engineers, BI teams, and software engineers to translate business rules into strict data quality KPIs and SLAs.
Skill Requirements
Deep proficiency in AWS data and storage services like Amazon S3, AWS Glue, Amazon Redshift, and AWS Lambda.
Advanced proficiency in Python for building automation frameworks and orchestrating data tests.
Experience with orchestration tools (e.g., Apache Airflow), data transformation, and data quality monitoring frameworks (e.g., Amazon Deequ).
Strong command of SQL and UNIX/Linux scripting for querying and analyzing large-scale, distributed datasets
Other Requirements
Certifications: Candidate with AWS Certified Data Engineer – Associate - Highly preferred
Apply for this role
Complete the form below. When you submit, you’ll be asked to complete a quick Polyguard Trust Check on your phone, and your application is only sent once verification passes.